How to Train Your Antivirus: RL-based Hardening through the Problem Space

Author: Davy Preuveneers / KU Leuven

Ilias Tsingenopoulos successfully presented his paper, “How to Train Your Antivirus: RL-based Hardening through the Problem Space,” at the 27th International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2024). The prestigious event took place from September 30 to October 2, 2024, at the historic Palazzo della Salute in Padua, Italy.

The joint work – co-authored by Ilias Tsingenopoulos (KU Leuven), Jacopo Cortellazzi (King’s College London), Branislav Bosansky (Gen Digital), Simone Aonzo (EURECOM), Davy Preuveneers (KU Leuven), Wouter Joosen, (KU Leuven), Fabio Pierazzi (King’s College London) and Lorenzo Cavallaro (University College London) –  addresses a critical vulnerability in machine learning-based malware detection systems, which are susceptible to evasion and spurious correlations, particularly when analyzing dynamic reports. His research focuses on strengthening the machine learning module of a widely-used commercial antivirus against adversarial malware.

Traditional adversarial training techniques, while effective in improving robustness in many domains, encounter significant challenges when applied to malware detection. Specifically, gradient-based transformations often fail to map back to feasible programs, rendering this defensive strategy ineffective.

To overcome this, Tsingenopoulos et al. introduced a novel Reinforcement Learning (RL) approach for generating adversarial examples, a key step in adversarially training the antivirus model. Unlike traditional methods, this approach ensures that only feasible modifications are applied, effectively bypassing the problem of inverse mapping. Moreover, the RL-based framework allows for theoretical guarantees on the robustness of the model against a clearly defined set of adversarial capabilities.

The paper’s theoretical investigation was complemented with empirical results, demonstrating that after a few iterations of adversarial retraining, the model consistently achieved a 0% attack success rate, confirming its enhanced resilience to evasion tactics.

Tsingenopoulos’ et al. work has garnered significant attention at RAID 2024, as it offers a promising solution to harden ML-driven malware detection systems, paving the way for more secure antivirus technologies.