Awesome Reinforcement Learning for Cyber Security

A curated list of resources dedicated to reinforcement learning applied to cyber security. Note that the list includes only work that uses reinforcement learning, general machine learning methods applied to cyber security are not included in this list.

For other related curated lists, see :

• Awesome Machine Learning for Cyber Security
• Awesome Adversarial Machine Learning

Table of Contents

• RL-Environments
• Papers
• Books
• Blogposts
• Talks
• Miscellaneous

↑ Environments

gym-idsgame

gym-idsgame An Abstract Cyber Security Simulation and Markov Game for OpenAI Gym. Paper: (2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play

CyberBattleSim (Microsoft)

CyberBattleSim CyberBattleSim is an experimentation research platform to investigate the interaction of automated agents operating in a simulated abstract enterprise network environment. The simulation provides a high-level abstraction of computer networks and cyber security concepts. Its Python-based Open AI Gym interface allows for the training of automated agents using reinforcement learning algorithms. Blogpost: (2021) Gamifying machine learning for stronger security and AI models

gym-malware

gym-malware Malware Env for OpenAI Gym Paper: (2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning

malware-rl

malware-rl Extended and Updated `gym_malware` which supports recent LIEF versionS and an enhanced collection of models (EMBER, MalConv and SOREL-20M) Paper: (2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning

gym-flipit

gym-flipit Gym environment for FLIPIT: The Game of "Stealthy Takeover" invented by Marten van Dijk, Ari Juels, Alina Oprea, and Ronald L. Rivest. Paper: (2019) QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game

gym-threat-defense

gym-threat-defense Gym environment for the environment described in the paper: (2019) Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs

gym-nasim

gym-nasim Thesis: (2018) Autonomous Penetration Testing using Reinforcement Learning

gym-optimal-intrusion-response

gym-optimal-intrusion-response An OpenAI Gym interface to a MDP/Markov Game model for optimal intrusion response of a realistic infrastructure simulated using system traces. Paper: (2021) Learning Intrusion Prevention Policies through Optimal Stopping

sql_env

sql_env Paper: (2021) SQL Injections and Reinforcement Learning: An Empirical Evaluation of the Role of Action Structure

cage-challenge

cage-challenge-1 The first Cyber Autonomos Gym for Experimentation (CAGE) challenge environment released at the 1st International Workshop on Adaptive Cyber Defense held as part of the 2021 International Joint Conference on Artificial Intelligence (IJCAI).

cage-challenge-2 The second Cyber Autonomous Gym for Experimentation (CAGE) challenge environment announced at the AAAI-22 Workshop on Artificial Intelligence for Cyber Security Workshop (AICS).

cage-challenge-3 The third Cyber Autonomous Gym for Experimentation (CAGE) challenge environment.

ATMoS

ATMoS Paper: (2020) ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning

MAB-Malware

MAB-malware Paper: (2022) MAB-Malware: A Reinforcement Learning Framework for Attacking Static Malware Classifiers

ASAP

Autonomous Security Analysis and Penetration Testing framework (ASAP) Paper: (2020) Autonomous Security Analysis and Penetration Testing

Yawning Titan

Yawning Titan Yawning Titan is an abstract, highly flexible, cyber security simulator that is capable of simulating a range of cyber security scenarios. Paper: (2022) Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation

Cyborg

(2020) ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning (2020) Modeling Penetration Testing with Reinforcement Learning Using Capture-the-Flag Challenges: Trade-offs between Model-free Learning and A Priori Knowledge (2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play (2020) AFRL: Adaptive federated reinforcement learning for intelligent jamming defense in FANET (2020) Reinforcement Learning for Efficient Network Penetration Testing (2020) The Agent Web Model -- Modelling web hacking for reinforcement learning (2020) Stochastic Dynamic Information Flow Tracking Game using Supervised Learning for Detecting Advanced Persistent Threats (2020) Reinforcement Learning Based PHY Authentication for VANETs (2020) Deep Reinforcement Learning for Cybersecurity Assessment of Wind Integrated Power Systems (2020) Smart Security Audit: Reinforcement Learning with a Deep Neural Network Approximator (2020) Quickest Detection of Advanced Persistent Threats: A Semi-Markov Game Approach (2020) Distributed Reinforcement Learning for Cyber-Physical System With Multiple Remote State Estimation Under DoS Attacker (2020) Secure Crowdsensing in 5G Internet of Vehicles: When Deep Reinforcement Learning Meets Blockchain (2020) Deep Reinforcement Learning based Intrusion Detection System for Cloud Infrastructure (2020) Application of deep reinforcement learning to intrusion detection for supervised problems (2019) A dynamic games approach to proactive defense strategies against Advanced Persistent Threats in cyber-physical systems (2019) Deep Q-Learning and Particle Swarm Optimization for Bot Detection in Online Social Networks (2019) Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learning (2019) Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control (2019) Study of Learning of Power Grid Defense Strategy in Adversarial Stage Game (2019) Learning to Cope with Adversarial Attacks (2019) Learning Distributed Cooperative Policies for Security Games via Deep Reinforcement Learning (2019) An Efficient Reinforcement Learning-Based Botnet Detection approach (2019) Strategic Learning for Active, Adaptive, and Autonomous Cyber Defense (2019) QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game (2019) Solving Cyber Alert Allocation Markov Games with Deep Reinforcement Learning (2019) Adaptive Honeypot Engagement Through Reinforcement Learning of Semi-Markov Decision Processes (2019) Detecting Phishing Websites through Deep Reinforcement Learning (2019) Adversarial Deep Reinforcement Learning based Adaptive Moving Target Defense (2019) Autonomous Penetration Testing using Reinforcement Learning (2019) A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution (2019) Automating Penetration Testing using Reinforcement Learning (2019) Reinforcement Learning-Based DoS Mitigation in Software Defined Networks (2019) Adversarial attack and defense in reinforcement learning-from AI security view (2019) A Learning-Based Solution for an Adversarial Repeated Game in Cyber–Physical Power Systems (2019) Empowering Reinforcement Learning on Big Sensed Data for Intrusion Detection (2019) Cyber-Attack Recovery Strategy for Smart Grid Based on Deep Reinforcement Learning (2019) Deep Reinforcement Learning for Partially Observable Data Poisoning Attack in Crowdsensing Systems (2018) Simulating SQL Injection Vulnerability Exploitation Using Q-Learning Reinforcement Learning Agents (2018) Security in Mobile Edge Caching with Reinforcement Learning (2018) Detection of online phishing email using dynamic evolving neural network based on reinforcement learning (2018) A reinforcement learning approach for attack graph analysis (2018) Reinforcement Learning for Autonomous Defence in Software-Defined Networking (2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning (2018) Autonomic Computer Network Defence Using Risk State and Reinforcement Learning (2018) Reinforcement Learning for Intelligent Penetration Testing (2018) Autonomous Intelligent Cyber-defense Agent (AICA) Reference Architecture (2018) Deep reinforecement learning based optimal defense for cyber-physical system in presence of unknown cyber-attack (2018) Adversarial Reinforcement Learning for Observer Design in Autonomous Systems under Cyber Attacks (2018) Machine learning for autonomous cyber defense (2018) Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach (2018) Deep Reinforcement Learning based Smart Mitigation of DDoS Flooding in Software-Defined Networks (2018) UAV Relay in VANETs Against Smart Jamming With Reinforcement Learning (2018) A Game-Theoretical Approach to Cyber-Security of Critical Infrastructures Based on Multi-Agent Reinforcement Learning (2018) Security in Mobile Edge Caching with Reinforcement Learning (2018) Robotics CTF (RCTF), a playground for robot hacking (2018) An IRL Approach for Cyber-Physical Attack Intention Prediction and Recovery (2018) QRASSH - A Self-Adaptive SSH Honeypot Driven by Q-Learning (2018) Using Reinforcement Learning to Conceal Honeypot Functionality (2018) Improving adaptive honeypot functionality with efficient reinforcement learning parameters for automated malware (2018) Enhancing Machine Learning Based Malware Detection Model by Reinforcement Learning (2017) Network Defense Strategy Selection with Reinforcement Learning and Pareto Optimization (2017) Adversarial Reinforcement Learning in a Cyber Security Simulation (2017) Detecting Stealthy Botnets in a Resource-Constrained Environment using Reinforcement Learning (2017) Q-learning Based Vulnerability Analysis of Smart Grid against Sequential Topology Attacks (2017) Multi-agent Reinforcement Learning Based Cognitive Anti-jamming (2017) Reinforcement Learning Based Mobile Offloading for Cloud-Based Malware Detection (2017) A Secure Mobile Crowdsensing Game With Deep Reinforcement Learning (2017) Online Algorithms for Adaptive Cyber Defense on Bayesian Attack Graphs (2016) Markov Security Games: Learning in Spatial Security Problems (2016) Dynamic Scheduling of Cybersecurity Analysts for Minimizing Risk Using Reinforcement Learning (2016) Balancing Security and Performance for Agility in Dynamic Threat Environments (2016) Reinforcement Learning Based Anti-jamming with Wideband Autonomous Cognitive Radios (2016) PHY-Layer Spoofing Detection With Reinforcement Learning in Wireless Networks (2015) Application of reinforcement learning for security enhancement incognitive radio networks (2015) Power control with reinforcement learning in cooperative cognitive radio networks against jamming (2015) Game Theory with Learning for Cyber Security Monitoring (2015) Spoofing Detection with Reinforcement Learning in Wireless Networks (2015) Mobile Cloud Offloading for Malware Detections with Learning (2014) Reinforcement Learning Algorithms for Adaptive Cyber Defense against Heartbleed (2014) Cooperative game theoretic approach using fuzzy Q-learning for detecting and preventing intrusions in wireless sensor networks (2014) Q-Learning: From Computer Network Security to Software Security (2013) Multiagent Router Throttling: Decentralized Coordinated Response Against DDoS Attacks (2013) Hybrid Learning in Stochastic Games and Its Application in Network Security (2013) Competing Mobile Network Game: Embracing Antijamming and Jamming Strategies with Reinforcement Learning (2012) Intrusion Detection System using Log Files and Reinforcement Learning (2012) Anti-jamming in Cognitive Radio Networks Using Reinforcement Learning Algorithms (2011) An Anti-jamming Strategy for Channel Access in Cognitive Radio Networks (2011) Distributed strategic learning with application to network security (2010) Dynamic policy-based IDS configuration (2008) Reinforcement Learning for Vulnerability Assessment in Peer-to-Peer Networks (2007) Defending DDoS Attacks Using Hidden Markov Models and Cooperative Reinforcement Learning (2006) An intrusion detection game with limited observations (2005) A Reinforcement Learning Approach for Host-Based Intrusion Detection Using Sequences of System Calls (2005) Multi-agent reinforcement learning for intrusion detection (2000) Next Generation Intrusion Detection: Autonomous Reinforcement Learning of Network Attacks PhD Theses (2022) Anomaly Detection in Competitive Multiplayer Games (2022) Secure Automated and Autonomous Systems (2014) Distributed Reinforcement Learning for Network Intrusion Response (2009) Multi-Agent Reinforcement Learning for Intrusion Detection Master Theses (2022) Self-Play Reinforcement Learning for Finding Intrusion Prevention Strategies (2022) Reinforcement Learning-aided Dynamic Analysis of Evasive Malware (2021) Intrusion Detection Based on Reinforcement Learning (2021) Bayesian Reinforcement Learning Methods for Network Intrusion Prevention (2019) Learning to Hack (2018) Autonomous Penetration Testing using Reinforcement Learning (2018) Analysis of Network Intrusion Detection System with Machine Learning Algorithms (Deep Reinforcement Learning Algorithm) Bachelor Theses (2022) Simulating Network Lateral Movements through the CyberBattleSim Web Platform (2018) Autonomous Penetration Testing using Reinforcement Learning Posters (2022) Autonomous Network Defence using Reinforcement Learning (2022) Intrusion Prevention through Optimal Stopping (2022) Intrusion Prevention through Optimal Stopping (2021) Learning Intrusion Prevention Policies through Optimal Stopping ↑ Books (2021) Game Theory and Machine Learning for Cyber Security (Chapter 5 on RL) (2019) Reinforcement Learning for Cyber-Physical Systems with Cybersecurity Case Studies (2010) Network Security: A Decision and Game-Theoretic Approach ↑ Blogposts (2021) Gamifying machine learning for stronger security and AI models (2021) Automating Cyber-Security With Reinforcement Learning (2021) Towards a method for computing effective intrusion prevention policies using reinforcement learning ↑ Talks (2022) CNSM 2022, Adapting Security Policies in Dynamic IT Environments - Hammar & Stadler (2022) Self-Learning Systems for Cyber Defense (2022) Poster: Study of Intelligent Cyber Range Simulation using Reinforcement Learning (2022) Paper Study - Survey of Reinforcement Learning for Cyber Security (2022) The Role of Artificial Intelligence in Cyber-Defence | AI & Cybersecurity | Vincent Lenders (2022) Learning Security Strategies through Game Play and Optimal Stopping - Hammar & Stadler (2022) Reinforcement Learning for Complex Security Games and Beyond (2022) NOMS22 Demo - A System for Interactive Examination of Learned Security Policies - Hammar & Stadler (2022) Reinforcement Learning Applications: Cyber Security (2021) Artificial Intelligence Applications to Cybersecurity (AI ATAC) Prize Challenges I&II (2021) NordSec 2021 - SQL Injections and Reinforcement Learning (2021) Deep hierarchical reinforcement agents for automated penetration testing (2021) USENIX Security '21 - SyzVegas: Beating Kernel Fuzzing Odds with Reinforcement Learning (2021) CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems (2021) Simulating a Logistics Enterprise Using an Asymmetrical Wargame Simulation with Soar Reinforcement Learning and Coevolutionary Algorithms (2021) Incorporating Deception into CyberBattleSim for Autonomous Defense (2021) CybORG: A Gym for the Development of Autonomous Cyber Agents (2021) Defending the Cyber Front with AI - CyCon 2021 (2021) Informing Autonomous Deception Systems with Cyber Expert Performance Data (2021) ACD 2021 Keynote - Prof. George Cybenko - Attrition in Adaptive Cyber Defense (2021) Reinforcement learning approaches on intusion detection (2021) Applying Deep Reinforcement Learning (DRL) in a Cyber Wargaming Engine (2021) Automated Penetration Testing using Reinforcement Learning (2021) Training an Autonomous Pentester with Deep RL (2021) Learning Intrusion Prevention Policies Through Optimal Stopping (2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play (2020) Autonomous Security Analysis and Penetration Testing (ASAP) - Ankur Chowdhary (2020) Autonomous Security Analysis and Penetration Testing: A reinforcement learning approach. (2020) Artificial Intelligence based Autonomous Penetration Testing. (2019) Cost-Efficient Malware Detection Using Deep Reinforcement Learning (2019) Trying to Make Meterpreter into an Adversarial Example (2019) A Reinforcement Learning Framework for Smart, Secure, and Efficient Cyber-Physical Autonomy (2019) Adaptive Honeypot Engagement through Reinforcement Learning of Semi-Markov Decision Processes (2018) Autonomous Cyber Defense: AI and the Immune System Approach (2018) Bonware to the Rescue: the Future Autonomous Cyber Defense Agents | Dr Alexander Kott | CAMLIS 2018 (2018) CSIAC Webinar - Learning to Win: Making the Case for Autonomous Cyber Security Solutions ↑ Miscellaneous (2022) AISec '22: 15th ACM Workshop on Artificial Intelligence and Security (2022) ICML Workshop on Machine Learning for Cybersecurity (2022) AAAI Workshop on Artificial Intelligence for Cyber Security (AICS) (2022) ECMLPKDD Workshop on Machine Learning for Cybersecurity(MLCS) (2021) IJCAI First International Workshop on Adaptive Cyber Defense (2021) ICONIP Workshop on Artificial Intelligence for Cyber Security (AICS) (2021) ECMLPKDD Workshop on Machine Learning for Cybersecurity (MLCS) (2021) Self-Learning AI (2021) AI/ML for Cybersecurity: Challenges, Solutions, and Novel Ideas at SIAM Data Mining 2021 (2020) ECMLPKDD Workshop on Machine Learning for Cybersecurity(MLCS) (2020) Self-Learning Systems for Cyber Defense (2020) Workshop on Artificial Intelligence for Cyber Security (AICS) (2019) ECMLPKDD Workshop on Machine Learning for Cybersecurity (MLCS) (2019) Workshop on Artificial Intelligence for Cyber Security (AICS) Contribute Contributions are very welcome. Please use Github issues and pull requests. List of Contributors License LICENSE Creative Commons (C) 2022