{"id":20683354,"url":"https://github.com/oisf/suricata-verify","last_synced_at":"2025-04-09T19:19:13.141Z","repository":{"id":32157988,"uuid":"130185529","full_name":"OISF/suricata-verify","owner":"OISF","description":"Suricata Verification Tests - Testing Suricata Output","archived":false,"fork":false,"pushed_at":"2025-04-08T12:23:40.000Z","size":78510,"stargazers_count":105,"open_issues_count":29,"forks_count":96,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-09T19:18:54.928Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OISF.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-19T08:40:55.000Z","updated_at":"2025-04-08T12:23:44.000Z","dependencies_parsed_at":"2024-05-29T17:40:43.245Z","dependency_job_id":null,"html_url":"https://github.com/OISF/suricata-verify","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OISF%2Fsuricata-verify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OISF%2Fsuricata-verify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OISF%2Fsuricata-verify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OISF%2Fsuricata-verify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OISF","download_url":"https://codeload.github.com/OISF/suricata-verify/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248094991,"owners_count":21046770,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T22:16:21.675Z","updated_at":"2025-04-09T19:19:13.130Z","avatar_url":"https://github.com/OISF.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Suricata Verification Tests\n\nThese are tests that run Suricata with a specific configuration and/or\ninputs and verify the outputs.\n\n## Running All Tests\n\nFrom your Suricata source directory run:\n\n```\n../path/to/suricata-verify/run.py\n```\n\nOr to run a single test:\n```\n../path/to/suricata-tests/run.py TEST-NAME\n```\n\n## Adding a New Test\n\n- Create a directory that is the name of the new test.\n\n  If you want a test to be run in IPS mode, add `ips` to the test name.\n  This will make the `--simulate-ips` command-line argument be passed when\n  the test is run.\n\n- Copy a single pcap file into the test directory. It must end in\n  \".pcap\".\n\n  This is enough for a basic test that will run Suricata over the pcap\n  testing for a successful exit code.\n\n- Optional: Create a suricata.yaml in the test directory.\n\n    Its usually OK to just add the bits of YAML required to enable\n    features for the test.\n\n\tIf the test directory does not include a suricata.yaml, the one\n    found in your build directory will be used.\n\n- Add any rules required to ${dir}/test.rules.\n\n- Add a *test.yaml* descriptor file to add further control to your\n  tests such as restricting features required for the test, and\n  validating output.\n\n## Example test.yaml\n\n```\nrequires:\n\n  # Require a minimum version of Suricata.\n  min-version: 8.0.0\n\n  # Require that the Suricata version be less than a version.\n  lt-version: 8\n\n  # Test is only for this version. For example, 4.0 would match any 4.0\n  # release, but 4.0.3 would only match 4.0.3.\n  version: 7.0.6\n\n  # Test is only for the listed OS. For example, the following would make\n  # a test run only on Linux.\n  os: linux\n\n  # Test is only for the liste architecture. For example, following would\n  # make a test run only on x86_64. Other values can be amd64, i386, etc.\n  arch: x86_64\n\n  # Require the presence of specific features.\n  features:\n    # Restrict the test to builds with HAVE_LUA.\n    - HAVE_LUA\n\n  # Require that one or more files exist in the Suricata source directory\n  files:\n\t- src/detect-ipaddr.c\n\n  # Don't require a pcap file to be present. By default a test will be skipped\n  # if there is no pcap file in the test directory. Not applicable if a\n  # command is provided.\n  pcap: false\n\n  # Run the script and only continue with the test if the script exists\n  # successfully.\n  script:\n    - command1\n    - command2\n    - ...\n\n  # Require the output of a Python expression to be true. For example,\n  # this will run on all platforms other than win32.\n  lambda: \"sys.platform != win32\"\n\nskip:\n  # Skip a test if a feature is present, with a message that is logged.\n  - feature: HAVE_LUA\n    msg: eve dns v1 not supported by rust\n\n# Add additional arguments to Suricata.\nargs:\n  - --set stream.reassembly.depth=0\n\n# Override the default command. This is also an example of how it can\n# be broken up over multiple lines for readability. If providing the command\n# all arguments must be provided as part of the command.\ncommand: |\n  ${SRCDIR}/src/suricata -T -c ${TEST_DIR}/suricata.yaml -vvv \\\n      -l ${TEST_DIR}/output --set default-rule-path=\"${TEST_DIR}\"\n\n# Retry a test 3 more times on failure. Some tests are subject to\n# timing errors on CI systems and this can help filter out the noise\n# of tests that fail in such environments. By default, tests are only\n# run once.\nretry: 3\n\n# Execute Suricata with the test parameters this many times. All checks will\n# done after each iteration.\ncount: 10\n\npre-check: |\n  # Some script to run before running checks.\n  cp eve.json eve.json.bak\n\n# Provide a pcap filename. A falsey value like false or empty is equivalent to setting\n# \"pcap: false\" in the requires section.\npcap: input.pcap\n\n# Test for a specific exit code. By default a test will fail if the\n# exit code is anything other than 0, however sometimes we may be\n# testing for failure.\nexit-code: 1\n\nchecks:\n\n  # A verification filter that is run over the eve.json. Multiple\n  # filters may exist and all must pass for the test to pass.\n  - filter:\n\n      # Requires that apply just to this check. Has all the same options\n      # as the test level requires above.\n      requires:\n        features:\n          - HTTP2_DECOMPRESSION\n\n      # The number of records this filter should match.\n      count: 1\n\n      # The fields to match on.\n      match:\n        # Example match on event_type:\n        event_type: alert\n\n        # Example match on array item:\n        alert.metadata.tag[0]: \"tag1\"\n\n        # Example match on the length of an array.\n        alert.metadata.tag.__len: 3\n\n        # Check that a field exists:\n        has-key: alert.rule\n\n        # Check that a field does not exist:\n        not-has-key: flow\n\n  - filter:\n\t  # Use a filename other than eve.json\n\t  filename: suricata.json\n\t  count: 1\n\t  match:\n\t    # Find a substring in a field\n\t\tengine.message.__find: script failed\n\n        # Check if a string starts with an expected value\n        engine.message.__startswith: \"This is the start of the string\"\n\n        # Check if a string ends with an expected value\n        engine.message.__endswith: \"the end of a string\"\n\n        # Check if a string is contained within a JSON list.\n        # Eg. \"ftp\":{\"reply\":[\"Opening BINARY mode data connection for temp.txt (1164 bytes).\",\"Transfer complete.\"], }\n        ftp.reply.__contains: 'Transfer complete.'\n\n  - shell:\n      # A simple shell check. If the command exits with a non-0 exit code the\n      # check will fail. The script is run in the output directory of the\n      # test.\n      args: grep \"GPL ATTACK_RESPONSE\" fast.log\n\n  - shell:\n      # A shell check that also tests the output of the command.\n      args: cat fast.log | wc -l | xargs\n      expect: 1\n\n  - file-compare:\n      # A check that compares two files\n      filename: datasets.csv\n      expected: expected/datasets.csv\n```\n\n## Adding a new test the automated way: createst\n\nCreatest is a script to create a test directory with test.yaml for a given PCAP.\nThis needs to be run from a valid Suricata source directory.\n\n### Usage\n```\nusage: createst.py [-h] [--rules \u003crules\u003e] [--output-path \u003coutput-path\u003e]\n                   [--eventtype-only] [--allow-events [ALLOW_EVENTS]] [--strictcsums]\n                   [--midstream] [--min-version \u003cmin-version\u003e] [--version \u003cadd-version\u003e]\n                   [--cfg \u003cpath-to-suricata.yaml\u003e] [--features \u003cfeatures\u003e]\n                   \u003ctest-name\u003e \u003cpcap-file\u003e\n\nCreate tests with a given PCAP. Execute the script from a valid Suricata source\ndirectory.\n\npositional arguments:\n  \u003ctest-name\u003e           Name of the test folder\n  \u003cpcap-file\u003e           Path to the PCAP file\n\noptions:\n  -h, --help            show this help message and exit\n  --rules \u003crules\u003e       Path to rule file\n  --output-path \u003coutput-path\u003e\n                        Path to the folder where generated test.yaml should be put\n  --eventtype-only      Create filter blocks based on event types only\n                        This means the subfields of the event in the eve log\n                        will not be added to the test.yaml file\n  --allow-events [ALLOW_EVENTS]\n                        Create filter blocks for the specified events\n                        Events must be comma-separated only\n                        This means that just the events listed will be checked\n                        against in the test\n  --strictcsums         Strictly validate checksum\n  --midstream           Allow midstream session pickups\n  --min-version \u003cmin-version\u003e\n                        Adds a global minimum required version\n  --version \u003cadd-version\u003e\n                        Adds a global suricata version\n  --cfg \u003cpath-to-suricata.yaml\u003e\n                        Adds a suricata.yaml to the test\n  --features \u003cfeatures\u003e\n                        Adds specified features\n```\n\n### Examples\n\nThe only mandatory arguments for ``createst.py`` are the test name and the pcap\nfile. These examples show how some of the optional arguments can be used.\n\n#### Example 1\n\nCreate a Suricata-verify test named ``test-01`` that runs over a pcap file called\n``input.pcap`` and that requires strict checksums, filters only on the event-types\nand uses no Suricata rules:\n```\n../suricata-verify/createst.py --strictcsums --eventtype-only test-01 input.pcap\n```\n\n#### Example 2\n\nCreate a Suricata-verify test named ``test-02`` that runs over a pcap file called\n``input.pcap``, only checks for ``http``, ``alert`` and ``flow`` events, and\nuses a rules file located in another test in the same suricata-verify folder.\nIt also doesn't require strict checksums and will run only for versions 6 and\nnewer:\n```\n../suricata-verify/createst.py --min-version 6 --allow-events http,alert,flow \\\n--rules ../suricata-verify/tests/no-payload-output/test.rules test-02 input.pcap\n```\n\n#### Add Required Features\n\n```\n../suricata-verify/createst.py --features HAVE_LUA\n```\n\n```\n../suricata-verify/createst.py --features HAVE_LUA,AF_PACKET\n```\n\nThe features are taken from the `Features:` line in `suricata\n--build-info`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foisf%2Fsuricata-verify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foisf%2Fsuricata-verify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foisf%2Fsuricata-verify/lists"}