{"id":19054416,"url":"https://github.com/datadog/threatest","last_synced_at":"2025-04-06T06:11:26.214Z","repository":{"id":56743335,"uuid":"504052004","full_name":"DataDog/threatest","owner":"DataDog","description":"Threatest is a CLI and Go framework for end-to-end testing threat detection rules.","archived":false,"fork":false,"pushed_at":"2023-12-19T21:27:54.000Z","size":390,"stargazers_count":328,"open_issues_count":10,"forks_count":23,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-03-30T05:05:48.264Z","etag":null,"topics":["continuous-security","detection-engineering","security-automation","threat-detection"],"latest_commit_sha":null,"homepage":"https://securitylabs.datadoghq.com/articles/threatest-end-to-end-testing-threat-detection/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataDog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-16T07:16:51.000Z","updated_at":"2025-03-19T13:46:40.000Z","dependencies_parsed_at":"2024-06-18T22:44:32.748Z","dependency_job_id":"066f15fb-4c96-4422-9f53-1f2b4b7f86ec","html_url":"https://github.com/DataDog/threatest","commit_stats":{"total_commits":86,"total_committers":8,"mean_commits":10.75,"dds":"0.17441860465116277","last_synced_commit":"dbffdf994753b509b6c5e6bb1da91932d1531e54"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fthreatest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fthreatest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fthreatest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fthreatest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataDog","download_url":"https://codeload.github.com/DataDog/threatest/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247441053,"owners_count":20939239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-security","detection-engineering","security-automation","threat-detection"],"created_at":"2024-11-08T23:38:18.916Z","updated_at":"2025-04-06T06:11:26.191Z","avatar_url":"https://github.com/DataDog.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Threatest\n\n![unit tests](https://github.com/DataDog/threatest/actions/workflows/test.yml/badge.svg)\n![static analysis](https://github.com/DataDog/threatest/actions/workflows/static-analysis.yml/badge.svg)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./logo.png\" alt=\"Threatest\" width=\"300\" /\u003e\n\u003c/p\u003e\n\nThreatest is a CLI and Go framework for testing threat detection end-to-end.\n\nThreatest allows you to **detonate** an attack technique, and verify that the alert you expect was generated in your favorite security platform.\n\nRead the announcement blog post: https://securitylabs.datadoghq.com/articles/threatest-end-to-end-testing-threat-detection/\n\n## Concepts\n\n### Detonators\n\nA **detonator** describes how and where an attack technique is executed.\n\nSupported detonators:\n* Local command execution\n* SSH command execution\n* Stratus Red Team\n* AWS CLI detonator\n* AWS detonator (programmatic only, does not work with the CLI)\n\n### Alert matchers\n\nAn **alert matcher** is a platform-specific integration that can check if an expected alert was triggered.\n\nSupported alert matchers:\n* Datadog security signals\n\n### Detonation and alert correlation\n\nEach detonation is assigned a UUID. This UUID is reflected in the detonation and used to ensure that the matched alert corresponds exactly to this detonation.\n\nThe way this is done depends on the detonator; for instance, Stratus Red Team and the AWS Detonator inject it in the user-agent; the SSH detonator uses a parent process containing the UUID.\n\n## Usage\n\n### Through the CLI\n\nThreatest comes with a CLI that you can use to run test scenarios described as YAML, following a specific [schema](./schemas/threatest.schema.json). You can configure this schema in your editor to benefit from in-IDE linting and autocompletion (see [documentation for VSCode](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml#associating-a-schema-to-a-glob-pattern-via-yaml.schemas) using the [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) extension).\n\nInstall the CLI by downloading a [binary release](https://github.com/DataDog/threatest/releases) or with Homebrew:\n\n```\nbrew tap datadog/threatest https://github.com/datadog/threatest\nbrew install datadog/threatest/threatest\n```\n\nSample usage:\n\n```bash\n$ threatest lint scenarios.threatest.yaml\nAll 6 scenarios are syntaxically valid\n\n# Local detonation\n$ threatest run local-scenarios.threatest.yaml\n\n# Remote detonation over SSH\n$ threatest run scenarios.threatest.yaml --ssh-host test-box --ssh-username vagrant\n\n# Alternatively, specify SSH parameters from environment variables\n$ export THREATEST_SSH_HOST=test-box\n$ export THREATEST_SSH_USERNAME=vagrant\n$ threatest run scenarios.threatest.yaml\n```\n\n**Sample scenario definition files**\n\n* Detonating over SSH\n\n```yaml\nscenarios:\n  # Remote detonation over SSH\n  # Note: SSH configuration is provided using the --ssh-host, --ssh-username and --ssh-keyfile CLI arguments\n  - name: curl metadata service\n    detonate:\n      remoteDetonator:\n        commands: [\"curl http://169.254.169.254 --connect-timeout 1\"]\n    expectations:\n      - timeout: 1m\n        datadogSecuritySignal:\n          name: \"Network utility accessed cloud metadata service\"\n          severity: medium\n```\n\n* Detonating using Stratus Red Team\n\n```yaml\nscenarios:\n  # Stratus Red Team detonation\n  # Note: You must be authenticated to the relevant cloud provider before running it\n  # The example below is equivalent to manually running \"stratus detonate aws.exfiltration.ec2-security-group-open-port-22-ingress\"\n  - name: opening a security group to the Internet\n    detonate:\n      stratusRedTeamDetonator:\n        attackTechnique: aws.exfiltration.ec2-security-group-open-port-22-ingress\n    expectations:\n      - timeout: 15m\n        datadogSecuritySignal:\n          name: \"Potential administrative port open to the world via AWS security group\"\n```\n\n\n* Detonating using AWS CLI commands\n\n```yaml\nscenarios:\n  # AWS CLI detonation\n  # Note: You must be authenticated to AWS before running it and have the AWS CLI installed\n  - name: opening a security group to the Internet\n    detonate:\n      awsCliDetonator:\n        script: |\n          set -e\n          \n          # Setup\n          vpc=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 --query Vpc.VpcId --output text)\n          sg=$(aws ec2 create-security-group --group-name sample-sg --description \"Test security group\" --vpc-id $vpc --query GroupId --output text)\n          \n          # Open security group\n          aws ec2 authorize-security-group-ingress --group-id $sg --protocol tcp --port 22 --cidr 0.0.0.0/0\n          \n          # Cleanup\n          aws ec2 delete-security-group --group-id $sg\n          aws ec2 delete-vpc --vpc-id $vpc\n    expectations:\n      - timeout: 15m\n        datadogSecuritySignal:\n          name: \"Potential administrative port open to the world via AWS security group\"\n```\n\n\nYou can output the test results to a JSON file:\n\n```\n$ threatest run scenarios.threatest.yaml --output test-results.json\n$ cat test-results.json\n[\n  {\n    \"description\": \"change user password\",\n    \"isSuccess\": true,\n    \"errorMessage\": \"\",\n    \"durationSeconds\": 22.046627348,\n    \"timeDetonated\": \"2022-11-15T22:26:14.182844+01:00\"\n  },\n  {\n    \"description\": \"adding an SSH key\",\n    \"isSuccess\": true,\n    \"errorMessage\": \"\",\n    \"durationSeconds\": 23.604699625,\n    \"timeDetonated\": \"2022-11-15T22:26:14.182832+01:00\"\n  },\n  {\n    \"description\": \"change user password\",\n    \"isSuccess\": false,\n    \"errorMessage\": \"At least one scenario failed:\\n\\nchange user password returned: change user password: 1 assertions did not pass\\n =\\u003e Did not find Datadog security signal 'bar'\\n\",\n    \"durationSeconds\": 3.505294235,\n    \"timeDetonated\": \"2022-11-15T22:26:36.229349+01:00\"\n  }\n]\n```\n\nBy default, scenarios are run with a maximum parallelism of 5. You can increase this setting using the `--parallelism` argument.\nNote that when using remote SSH detonators, each scenario running establishes a new SSH connection.\n\n### Using Threatest programmatically\n\nSee [examples](./examples) for complete programmatic usage example.\n\n#### Testing Datadog Cloud SIEM signals triggered by Stratus Red Team\n\n```go\nthreatest := Threatest()\n\nthreatest.Scenario(\"AWS console login\").\n  WhenDetonating(StratusRedTeamTechnique(\"aws.initial-access.console-login-without-mfa\")).\n  Expect(DatadogSecuritySignal(\"AWS Console login without MFA\").WithSeverity(\"medium\")).\n  WithTimeout(15 * time.Minute)\n\nassert.NoError(t, threatest.Run())\n```\n\n### Testing Datadog Cloud Workload Security signals triggered by running commands over SSH\n\n```go\nssh, _ := NewSSHCommandExecutor(\"test-box\", \"\", \"\")\n\nthreatest := Threatest()\n\nthreatest.Scenario(\"curl to metadata service\").\n  WhenDetonating(NewCommandDetonator(ssh, \"curl http://169.254.169.254 --connect-timeout 1\")).\n  Expect(DatadogSecuritySignal(\"EC2 Instance Metadata Service Accessed via Network Utility\"))\n\nassert.NoError(t, threatest.Run())\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadog%2Fthreatest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatadog%2Fthreatest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadog%2Fthreatest/lists"}