{"id":13499427,"url":"https://github.com/endgameinc/gym-malware","last_synced_at":"2026-01-19T12:36:38.646Z","repository":{"id":23190785,"uuid":"98222506","full_name":"endgameinc/gym-malware","owner":"endgameinc","description":null,"archived":false,"fork":false,"pushed_at":"2022-11-21T22:04:05.000Z","size":570,"stargazers_count":622,"open_issues_count":13,"forks_count":165,"subscribers_count":49,"default_branch":"master","last_synced_at":"2025-03-29T04:31:44.390Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/endgameinc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-24T18:33:29.000Z","updated_at":"2025-03-20T23:00:40.000Z","dependencies_parsed_at":"2023-01-13T22:54:40.619Z","dependency_job_id":null,"html_url":"https://github.com/endgameinc/gym-malware","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/endgameinc/gym-malware","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fgym-malware","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fgym-malware/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fgym-malware/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fgym-malware/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/endgameinc","download_url":"https://codeload.github.com/endgameinc/gym-malware/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fgym-malware/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28567938,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T08:53:44.001Z","status":"ssl_error","status_checked_at":"2026-01-19T08:52:40.245Z","response_time":67,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T22:00:33.067Z","updated_at":"2026-01-19T12:36:38.620Z","avatar_url":"https://github.com/endgameinc.png","language":"Python","funding_links":[],"categories":[":wrench: Tools","[▲](#keywords) Code","Using AI for Pentesting","Adversarial Machine Learning","Attack Techniques \u0026 Red Teaming","Tools \u0026 Frameworks"],"sub_categories":["Before 2000","Prediction","Adversarial ML \u0026 Classical Models","Learning Environments"],"readme":"# Malware Env for OpenAI Gym\r\n**************************\r\nCiting\r\n======\r\n\r\nIf you use this code in a publication please cite the following [paper](https://arxiv.org/abs/1801.08917):\r\n\r\n```\r\n\r\nHyrum S. Anderson, Anant Kharkar, Bobby Filar, David Evans, Phil Roth, \"Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning\", in ArXiv e-prints. Jan. 2018.\r\n\r\n@ARTICLE{anderson2018learning,\r\n  author={Anderson, Hyrum S and Kharkar, Anant and Filar, Bobby and Evans, David and Roth, Phil},\r\n  title={Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning},\r\n  journal={arXiv preprint arXiv:1801.08917},\r\n  archivePrefix = \"arXiv\",\r\n  eprint = {1801.08917},\r\n  primaryClass = \"cs.CR\",\r\n  keywords = {Computer Science - Cryptography and Security},\r\n  year = 2018,\r\n  month = jan,\r\n  adsurl = {http://adsabs.harvard.edu/abs/2018arXiv180108917A},\r\n}\r\n```\r\n\r\n\r\n**This is a malware manipulation environment for OpenAI's ``gym``.** \r\n[OpenAI Gym](https://gym.openai.com/) is a toolkit for developing and comparing reinforcement \r\nlearning algorithms. This makes it possible to write agents that learn \r\nto manipulate PE files (e.g., malware) to achieve some objective \r\n(e.g., bypass AV) based on a reward provided by taking specific manipulation\r\nactions.\r\n\r\nObjective\r\n======\r\nCreate an AI that learns through reinforcement learning which functionality-preserving transformations to make on a malware sample to break through / bypass machine learning static-analysis malware detection.\r\n\r\n![Breakout](https://github.com/matthiasplappert/keras-rl/raw/master/assets/breakout.gif?raw=true\r\n\"Breakout\")\r\n\r\nBasics\r\n======\r\n\r\nThere are two basic concepts in reinforcement learning: the environment (in our case, the malware sample) and the agent (namely, the algorithm used to change the environment). The agent sends `actions` to the environment, and the environment replies with `observations` and `rewards` (that is, a score).\r\n\r\nThis repo provides an environment for manipulating PE files and providing rewards that are based around bypassing AV.  An agent can be deployed that have already been written for the rich ``gym`` framework.  For example\r\n\r\n* https://github.com/pfnet/chainerrl [recommended]\r\n* https://github.com/matthiasplappert/keras-rl\r\n \r\nSetup\r\n=====\r\nThe EvadeRL framework is built on Python3.6 we recommend first creating a virtualenv (details can be found [here]) with Python3.6 then performing the following actions ensure you have the correct python libraries:\r\n\r\n[here]: https://docs.python.org/3/tutorial/venv.html\r\n```sh\r\npip install -r requirements.txt\r\n```\r\n\r\nEvadeRL also leverages a Library to Instrument Executable Formats aptly named [LIEF]. It allows our agent to modify the binary on-the-fly. To add it to your virtualenv just ```pip install``` one of their pre-built packages. Examples below:\r\n\r\n[LIEF]: https://github.com/lief-project/LIEF\r\n\r\nLinux\r\n```\r\npip install https://github.com/lief-project/LIEF/releases/download/0.7.0/linux_lief-0.7.0_py3.6.tar.gz\r\n```\r\n\r\nOSX\r\n```\r\npip install https://github.com/lief-project/LIEF/releases/download/0.7.0/osx_lief-0.7.0_py3.6.tar.gz\r\n```\r\n\r\nOnce completed ensure you've moved malware samples into the \r\n```\r\ngym_malware/gym_malware/envs/utils/samples/\r\n```\r\n\r\nIf you are unsure where to acquire malware samples see the **Data Acquisition** section below. If you have samples in the correct directory you can check to see if your environment is correctly setup by running :\r\n\r\n```\r\npython test_agent_chainer.py\r\n```\r\n\r\nNote that if you are using Anaconda, you may need to\r\n```\r\nconda install libgcc\r\n```\r\nin order for LIEF to operate properly.\r\n\r\nData Acquisition\r\n=====\r\nIf you have a VirusTotal API key, you may download samples to the `gym_malware/gym_malware/envs/utils/samples/` using the Python script `download_samples.py`.\r\n\r\nGym-Malware Environment\r\n====\r\nEvadeRL pits a reinforcement agent against the malware environment consisting of the following components:\r\n\r\n* Action Space\r\n* Independent Malware Classifier\r\n* OpenAI framework malware environment (aka gym-malware)\r\n \r\nAction Space\r\n----\r\nThe moves or actions that can be performed on a malware sample in our environment consist of the following binary manipulations:\r\n* append_zero\r\n* append_random_ascii\r\n* append_random_bytes\r\n* remove_signature\r\n* upx_pack\r\n* upx_unpack\r\n* change_section_names_from_list\r\n* change_section_names_to random\r\n* modify_export\r\n* remove_debug\r\n* break_optional_header_checksum\r\n\r\nThe agent will randomly select these actions in an attempt to bypass the classifier (info on default classifier below). Over time, the agent learns which combinations lead to the highest rewards, or learns a policy (*like an optimal plan of attack for any given observation*).\r\n\r\nIndependent Classifier\r\n----\r\nIncluded as a default model is a [gradient boosted decision trees model] trained on 50k malicious and 50k benign samples with the following features extracted:\r\n* Byte-level data (e.g. histogram and entropy)\r\n* Header\r\n* Section\r\n* Import/Exports\r\n\r\n\r\n[gradient boosted decision trees model]: http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html\r\n\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fendgameinc%2Fgym-malware","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fendgameinc%2Fgym-malware","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fendgameinc%2Fgym-malware/lists"}