{"id":15113736,"url":"https://github.com/sentinelbyte/detection_as_code","last_synced_at":"2025-04-05T18:16:01.867Z","repository":{"id":254816300,"uuid":"847622489","full_name":"SentinelByte/Detection_as_Code","owner":"SentinelByte","description":"Detection as Code is a methodology consist of a set of principles that use code and automation to implement and manage threat detection capabilities.","archived":false,"fork":false,"pushed_at":"2024-08-26T10:00:57.000Z","size":54,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T18:16:00.244Z","etag":null,"topics":["bash","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SentinelByte.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-26T08:14:21.000Z","updated_at":"2024-08-31T15:14:35.000Z","dependencies_parsed_at":"2024-08-26T10:48:02.977Z","dependency_job_id":"632c84b3-933f-4184-a2c3-14644bb50379","html_url":"https://github.com/SentinelByte/Detection_as_Code","commit_stats":null,"previous_names":["sentinelbyte/detection_as_code"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SentinelByte%2FDetection_as_Code","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SentinelByte%2FDetection_as_Code/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SentinelByte%2FDetection_as_Code/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SentinelByte%2FDetection_as_Code/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SentinelByte","download_url":"https://codeload.github.com/SentinelByte/Detection_as_Code/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247378140,"owners_count":20929296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","python"],"created_at":"2024-09-26T01:22:54.467Z","updated_at":"2025-04-05T18:16:01.847Z","avatar_url":"https://github.com/SentinelByte.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Detection as Code (ELK Stack)\n\nWelcome to the **Detection as Code Pipeline** repository! This project aims to streamline and enhance threat detection through a robust, scalable pipeline built on the principles of \"detection as code.\" This repository provides a comprehensive solution for creating, testing, and deploying detection rules in a systematic and automated manner. ***This Repo focus with ELK stack !!!***.\n\n\n## 📋 Overview\n\n***This Repo focus with ELK stack !!!***.\nThe Detection as Code Pipeline is designed to bridge the gap between detection engineering and development practices. It enables security teams to define, test, and manage detection rules as code, ensuring that these rules are both effective and maintainable. This approach brings the benefits of version control, automated testing, and continuous integration to the realm of security detection.\n\n\n## 🚀 Features\n\n- **Detection as Code**: Define and manage your detection rules in code, making them easily versioned and reviewed.\n- **Automated Testing**: Run automated tests on your detection rules to ensure their accuracy and effectiveness before deployment.\n- **Continuous Integration**: Integrate with CI/CD pipelines to automate the deployment and updating of detection rules.\n- **Scalability**: Designed to scale with your organization's needs, handling complex rule sets and large datasets efficiently.\n- **Documentation and Examples**: Comprehensive documentation and example configurations to get you started quickly.\n\n\n## 🛠️ Getting Started\n\n### Prerequisites\n\nBefore you start, ensure you have the following installed:\n- [Python](https://www.python.org/) (version 3.8 or higher)\n- [Docker](https://www.docker.com/) (for containerized environments)\n- [Requests](https://pypi.org/project/requests/)\n- [Elastic_Cloud](https://www.elastic.co/guide/en/security/current/security-apis.html)\n- [elasticsearch - Elasticsearch]\n- [elastic_transport - RequestsHttpNode]\n\n\n### Installation and Usage\n\n1. ***Clone the Repository***\n\n   ```\n   git clone https://github.com/SentinelByte/Detection_as_Code.git\n   cd detection-as-code\n   ```\n\n2. ***Store the create_json.py for custom runs***\n\n   Create a dedicated VM (Virtual Machine) on your prefered cloud provider (GCP/AWS/Azure/etc.) and store the create_json.py code.\n   \n   Altrernatively, you can use a local machine (Note! just make sure you have a proper allowed connection to the ELK SaaS and API Endpoints).\n\n   Upon detection rule creation, you will trigger manually this code and it will take you through the process of a JSON file creation, that will be used later for the detection rule creation.\n\n3. ***Set up a push job to your CICD tool***\n\n   To run the code and create the detection rule, push ths json file to your CICD tool (Jenkins/ GitLab CI/CD/ Circle CI/ etc).\n   \n   The job should fetch a JSON file created from the crate_json.py code.\n   \n   3.1. Set up a cron job/ bash/ or other method to push the json file created from step 2 to your Github account.\n\n   If you don't want to use a github account, fill free to use any other solution for that.\n\n   You can even choose to push the json file directly to a cicd tool you choose (Jenkins/ GitLab CI/CD/ Circle CI/ etc.)\n\n   3.2. setup permissions - chmod +x ~/push_and_archive.sh\n\n   3.3 Open crontab and create the cron job:\n   \n   ```\n   crontab -e\n   0 * * * * /bin/bash ~/push_to_github.sh\n   ```\n   \n   ** Note! You can adjust the cron job interval to your needs.\n   \n5. ***Set Up CI Environment***\n   \n   Use Github Actions/ Juenkins/ etc.\n   If needed, create a virtual environment and install dependencies:\n\n   ```\n   python -m venv venv\n   source venv/bin/activate\n   pip install -r requirements.txt\n   ```\n\n7. ***Configure the Pipeline***\n\n   Update the configuration files located in the `config` directory.\n   you will need to run the following:\n   - create_rule.py\n   - check_query_main.py\n  \n     ** Make sure you upload the following also:\n      - validate_qury.py\n      - check_query.py\n\n9. ***Run the Pipeline***\n\n   Start the pipeline using Docker:\n\n   ```\n   docker-compose up\n   ```\n\n   Or run locally:\n\n   ```\n   python3 craft_json.py\n   python3 create_rule.py\n   python3 check_query_main.py\n   ```\n\n\n## 🧪 Testing\n\nYou can run this code locally and see if everything works.\nMake sure you have connection between your local machine to the Elastic endpint.\nTo ensure your detection rules are functioning as expected, run the following one by one:\n1. craft_json.py\n2. create_rile.py\n3. check_query_main.py\n\n\n\n## 📝 Documentation\n\nFor more detailed information on how to use and customize the Detection as Code refere to the comments within the code.\n\n\n## 🤝 Acknowledgements\n\nProvide ideas \u0026 inspiration for this project: https://medium.com/threatpunter/from-soup-to-nuts-building-a-detection-as-code-pipeline-28945015fc38\n\n© SentinelByte | dancohvax\n\nHappy detecting! 🚀🔍\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsentinelbyte%2Fdetection_as_code","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsentinelbyte%2Fdetection_as_code","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsentinelbyte%2Fdetection_as_code/lists"}