{"id":20603822,"url":"https://github.com/owasp-modsecurity/mrts","last_synced_at":"2025-08-04T06:14:02.446Z","repository":{"id":247662354,"uuid":"818949563","full_name":"owasp-modsecurity/MRTS","owner":"owasp-modsecurity","description":"ModSecurity Regression Test set","archived":false,"fork":false,"pushed_at":"2025-06-01T20:23:25.000Z","size":157,"stargazers_count":4,"open_issues_count":5,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-06-02T04:34:11.790Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/owasp-modsecurity.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-06-23T10:41:51.000Z","updated_at":"2025-06-01T20:23:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"a1737aa4-f72f-433e-97fc-7b1f592ffcd9","html_url":"https://github.com/owasp-modsecurity/MRTS","commit_stats":null,"previous_names":["owasp-modsecurity/mrts"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/owasp-modsecurity/MRTS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owasp-modsecurity%2FMRTS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owasp-modsecurity%2FMRTS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owasp-modsecurity%2FMRTS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owasp-modsecurity%2FMRTS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/owasp-modsecurity","download_url":"https://codeload.github.com/owasp-modsecurity/MRTS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/owasp-modsecurity%2FMRTS/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268657458,"owners_count":24285507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-04T02:00:09.867Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T09:18:46.639Z","updated_at":"2025-08-04T06:14:02.429Z","avatar_url":"https://github.com/owasp-modsecurity.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MRTS\n\nMRTS is a utility that helps you create rule sets and their tests for [ModSecurity](https://github.com/owasp-modsecurity/ModSecurity) or ModSecurity compliant engines (eg. [Coraza](https://github.com/corazawaf/coraza/)) for regression testing. The format of the test cases is compatible with [go-ftw](https://github.com/coreruleset/go-ftw/).\n\nPlease note that this project is in very beta state.\n\n## Goals\n\nThe goals of this project:\n* create as many rules as possible for ModSecurity to test its behavior\n* create as many tests as possible for each rule\n\nModSecurity uses its rules [targets](https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-%28v2.x%29#user-content-Variables), [operators](https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-%28v2.x%29#user-content-Operators), [transformations](https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-%28v2.x%29#transformation-functions) (special actions) and so many other components. It is necessary to test their behavior.\n\nNote, that [libmodsecurity3](https://github.com/owasp-modsecurity/ModSecurity/tree/v3/master) has a [regression test framework](https://github.com/owasp-modsecurity/ModSecurity/tree/v3/master/test) with several [test cases](https://github.com/owasp-modsecurity/ModSecurity/tree/v3/master/test/test-cases/regression), but it tests only the library, not the embedded state. For example we don't know anything about behavior of [Nginx connector](https://github.com/owasp-modsecurity/ModSecurity-nginx).\n\nWith the generated rules and tests we can check the operation of [mod-security2](https://github.com/owasp-modsecurity/ModSecurity/tree/v2/master) and [Nginx-connector](https://github.com/owasp-modsecurity/ModSecurity-nginx).\n\nThe generated rules can help in the quality assurance of these engines, e.g. after sending pull requests, we can verify that the change did not change the expected behavior.\n\n## Idea\n\nThe idea is to generate rules to see what happens to a particular component. It's not as trivial as it seems at first glance. Consider there are 5 phases - can we be sure of behaviors are same in each phases? Variables can be collections, every combinations of collections works as we need? Do you want to check the variable against multiple operator? With multiple operator arguments?\n\nIt's easy to see that the number of possible combinations can be infinite. It could be too much efford to write a rule for every possible format - and a test case too.\n\nInstead of doing this, we can make a description about the object, and expand the possible combinations and their test cases.\n\nRules are generated based on templates. You can define as many templates as you want, and you can apply them for each rule description.\n\nThe operation is very simple: create one or more configuration files, and run the generator script with those files. the format of the files is some structured data (YAML, JSON) which can be human readable (and writable). Generator will produce rules with combination of given:\n* target + colkeys (collection keys) (eg. `ARGS:arg1`, `ARGS:arg2`, `ARGS:arg1|ARGS:arg2`)\n* operators (you can pass multiple operators)\n* operator arguments - also can pass several arguments\n* phases - it depends on your choose, in which phases you want to check the target\n\n\n## API\n\nThe framework has an API that describes which keywords can be used for the description. To avoid unwanted typing, there are several global settings that are derived in each case.\n\nThe syntax of API can be YAML or JSON.\n\n### Global keywords\n\nEvery global settings should be put under the `global` keyword, eg:\n\n```yaml\nglobal:\n  version: MRTS/0.1\n  baseid: 100000\n```\n\nYou can place `global` keywords in every file, each subsequent occurrence will overwrite the previous one. The files are processed in ABC order, later overwriting does not change the previous settings.\n\n#### global\n\nThis keyword shows that the next block contains global settings.\n\n#### version\n\n`version` shows the current version of framework and can appear as constant in templates (see later).\n\n#### baseid\n\n`baseid` defines the first `id` what a rule can use. Inside the generator increments that for every rule, and that variable is avaluable as `${CURRID}$` (see later).\n\n#### default_operator\n\nThis global variable defines the default operator for rules. You can overwrite it at every case, moreover you can add more operators for every case. But if you don't want to type, the `operator` member can be omitted.\n\nSyntax:\n```yaml\nglobal:\n  default_operator: \"@rx\"\n```\n\n#### templates\n\n`templates` defines a list of templates. Each item in the list is a `template` block - see [template](#Template) section.\n\n#### default_tests_phase_methods\n\nThis keyword describes an object. Each keys of the object is a phase value, and the value is the method what you prefer to send the request during the test (with `go-ftw`). In `phase:1` we prefer to use `GET` method, in case of each other the `POST`. Example:\n\n```yaml\nglobal:\n  default_tests_phase_methods\n  - 1: get\n  - 2: post\n  - 3: post\n  - 4: post\n  - 5: post\n```\n\n### Template\n\nYou can create one or more template which can be used for generated rules. A template object has two other named objects: `name` and `template`.\n\n`name` must be a unique name, and `template` is a text with the rule definition. This definition can contain macros - see [macros](#macros) section.\n\nAn example for `templates`:\n\n```yaml\n  - name: \"SecRule for TARGETS\"\n    template: |\n      SecRule ${TARGET}$ \"${OPERATOR}$ ${OPARG}$\" \\\n          \"id:${CURRID}$,\\\n          phase:${PHASE}$,\\\n          deny,\\\n          t:none,\\\n          log,\\\n          msg:'%{MATCHED_VAR_NAME} was caught in phase:${PHASE}$',\\\n          ver:'${VERSION}$'\"\n```\n\nAs you can see the template macros are delimited by `${...}$`.\n\n### macros\n\nMarcos are coming from the definition. That can be from the unique definition or if there no such variable, then from the globals.\n\nAvaliable macros:\n\n* `${TARGET}$` the variable name when you want to check the SecRule's variable\n* `${OPERATOR}$` is the used operator; it must be placed with the leading `@`, eg. `@rx`.\n* `${OPARG}$` is the argument of the operator in the rule\n* `${CURRID}$` is the incremented `id`, which guaranties that every generated rule will have a unique `id`\n* `${PHASE}$` is the current phase in the list that you define in the definition file (see later its syntax)\n* `${VERSION}$` is the `VERSION`, see above\n* `${ACTIONS}$` is the actions you want in the rule \n* `${DIRECTIVES}$` is the additional directives you want next to the rule \n\nPlease note that `%{MATCHED_VAR_NAME}` is not a tool macro, but the ModSecurity's macro. You can use them where you want.\n\n## Definition\n\nIn a definition file there also many keywords are available. See an example then expand the meanings:\n\n```yaml\ntarget: null\nrulefile: MRTS_001_INIT.conf\ntestfile: null\nobjects:\n- object: secaction\n  actions:\n    id: 10001\n    phase: 1\n    pass: null\n    nolog: null\n    msg: \"'Initial settings'\"\n    ctl: ruleEngine=DetectionOnly\n- object: secrule\n  target: REQUEST_HEADERS:X-MRTS-Test\n  operator: '@rx ^.*$'\n  actions:\n    id: 10002\n    phase: 1\n    pass: null\n    t: none\n    log: null\n    msg: \"'%{MATCHED_VAR}'\"\n```\n\nor\n\n```yaml\ntarget: ARGS_COMBINED_SIZE\nrulefile: MRTS_003_ARGS_COMBINED_SIZE.conf\ntestfile: MRTS_003_ARGS_COMBINED_SIZE.yaml\ntemplates:\n- SecRule for TARGETS\ncolkey:\n- - ''\noperator:\n- '@lt'\noparg:\n- 2\nactions:\n  - action:\n      - status:404\ndirectives:\n  - directive: \n      - SecAction \"id:${CURRID}$,phase:${PHASE}$, pass, setenv:'123=abc'\"\ntestdata:\n  phase_methods:\n    1: get\n    2: post\n    3: post\n    4: post\n  targets:\n    - target: 2\n      test:\n        data:\n          foo: attack\n    - target: arg1\n      test:\n        data:\n          arg1: attack\n    - target: arg2\n      test:\n        data:\n          arg2: attack\n```\n\n* `target` - defines the variable name what you want to test; it can be null, but then you must define the expected rules or actions under the `object` block\n* `rulefile` - the name of generated file; the path will be passed as cli argument, you should define here the relative path\n* `testfile` - the name of generated test file; can be null if you don't want to make tests against rules. The path here also will be passed as cli argument.\n* `objects` - a list type item, you can order the `object` which describes a `SecRule` or a `SecAction`. This is necessary because there are some special rules/actions, which can't described as regular rule. The first example generates the file `MRTS_001_INIT.conf` with a `SecAction` and a `SecRule`:\n\n```\nSecAction \\\n    \"id:10001,\\\n    phase:1,\\\n    pass,\\\n    nolog,\\\n    msg:'Initial settings',\\\n    ctl:ruleEngine=DetectionOnly\"\n\nSecRule REQUEST_HEADERS:X-MRTS-Test \"@rx ^.*$\"\\\n    \"id:10002,\\\n    phase:1,\\\n    pass,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR}'\"\n```\n\nThese are necessary for `go-ftw`.\n\n* `templates` - you can list the name of templates what you want to apply\n* `colkey` - list collection keys what you want to test; note that each item in the list is a list too! See this example:\n```\ncolkey:\n- - ''\n- - arg1\n- - arg1\n  - arg2\n- - /^arg_.*$/\n```\n\nwill produce: `[[''], ['arg1'], ['arg2'], ['arg1', 'arg2'], ['/^arg_.*$']]`. This will generate rules with targets:\n\n```\nSecRule ARGS\nSecRule ARGS:arg1\nSecRule ARGS:arg1|ARGS:arg2\nSecRule ARGS:/^arg_.*$/\n```\n* `operator` - list of used operators\n* `oparg` - list of used operator arguments\n* `actions` - list of used actions - see [actions](#actions) section \n* `directives` - list of used directives - see [directives](#directives) section\n* `testdata` - list of expected test cases - see [testdata](#testdata) section\n\n### actions\n\n`actions` are defined for the `${ACTIONS}$` macro. See this example:\n\n```yaml\nactions:\n  - action:\n      - setvar:ABC=1\n      - auditlog\n      - status:400\n  - action:\n      - setvar:XYZ=2\n      - status:500\n```\nEach `action` field contains a list of actions to be included in a SecRule/SecAction. Every `action` list will be used to generate different combinations of rules.\n\nThe above example used with this template:\n\n```yaml\n    template: |\n      SecRule ${TARGET}$ \"${OPERATOR}$ ${OPARG}$\" \\\n          \"id:${CURRID}$,\\\n          phase:${PHASE}$,\\\n          deny,\\\n          t:none,\\\n          log,\\\n          msg:'%{MATCHED_VAR_NAME} was caught in phase:${PHASE}$',\\\n          ver:'${VERSION}$',\\\n          ${ACTIONS}$\"\n```\n\nwould produce the following rules:\n\n```yaml\nSecRule ARGS \"@contains attack\" \\\n    \"id:100000,\\\n    phase:2,\\\n    deny,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR_NAME} was caught in phase:2',\\\n    ver:'MRTS/0.1',\\\n    setvar:ABC=1,\\\n    auditlog,\\\n    status:400\"\n\nSecRule ARGS \"@contains attack\" \\\n    \"id:100001,\\\n    phase:2,\\\n    deny,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR_NAME} was caught in phase:2',\\\n    ver:'MRTS/0.1',\\\n    setvar:XYZ=2,\\\n    status:500\"\n```\n\n### directives\n\n`directives` are defined for the `${DIRECTIVES}$` macro. See this example:\n```yaml\ndirectives:\n  - directive:\n      - SecAction \"id:${CURRID}$,phase:${PHASE}$, pass, setenv:'123=abc'\"\n      - SecAction \"id:${CURRID}$,phase:${PHASE}$, pass, setenv:'456=def'\"\n  - directive:\n      - SecAction \"id:${CURRID}$,phase:${PHASE}$, pass, setenv:'789=xyz'\"\n```\nEach `directive` field contains a list of directives to be included in a template. Every `directive` list will be used to generate different combinations of rules. Macros are available and will be replaced with the current combination's value, except for macro `${CURRID}$` that is instead incremented at each substitution to guarantee a unique id per SecRule/SecAction.\n\nThe above example used with this template:\n\n```yaml\n    template: |\n      SecRule ${TARGET}$ \"${OPERATOR}$ ${OPARG}$\" \\\n          \"id:${CURRID}$,\\\n          phase:${PHASE}$,\\\n          deny,\\\n          t:none,\\\n          log,\\\n          msg:'%{MATCHED_VAR_NAME} was caught in phase:${PHASE}$',\\\n          ver:'${VERSION}$'\"\n\n      ${DIRECTIVES}$\n```\nwould produce the following rules:\n```yaml\nSecRule ARGS \"@contains attack\" \\\n    \"id:100000,\\\n    phase:2,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR_NAME} was caught in phase:2',\\\n    ver:'MRTS/0.1'\"\n\nSecAction \"id:100001,phase:2, pass, setenv:'123=abc'\"\nSecAction \"id:100002,phase:2, pass, setenv:'456=def'\"\n\nSecRule ARGS \"@contains attack\" \\\n    \"id:100003,\\\n    phase:2,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR_NAME} was caught in phase:2',\\\n    ver:'MRTS/0.1'\"\n\nSecAction \"id:100004,phase:2, pass, setenv:'789=xyz'\"\n```\n\n### testdata\n\n`testdata` is a keyword in the definition file. Here you can list the necessary test case definitions. A testdata item can contain two member:\n\n* `phase_methods` - where you can overwrite the [default_tests_phase_methods](#default_tests_phase_methods) - this keyword is optional\n* `targets` - here you can define the posible collection keys that can occurres in generated rules\n\n#### test case definition\n\nLet's see a test case definition example:\n\n```yaml\n  targets:\n    - target: ''\n      test:\n        data:\n          foo: attack\n    - target: arg1\n      test:\n        data:\n          arg1: attack\n```\n\nAs it described above, `targets` is list of tests. A test case contains two keywords:\n* `target` - describes the collection key which used at the rule (can be empty: ``)\n* `test` - is an object\n\nThe `test` object can contains these keywords:\n* `data` - which can be a single string or a key:value pair\n* `input` - a structure which overrides the test case in predefined structure\n\nNote, that the `go-ftw` test structure is hard-coded in the script, the `input` overwrites that structure.\n\nThe given example above contains two test cases: one if the collection key is empty, and another one if the collection key is the `arg1` - see the generated rules example above. You **must** give at least one test for each used collection keys at the rules definition!\n\nHere are some examples for test cases:\n\n```yaml\n  targets:\n    - target: ''\n      test:\n        data:\n          foo: attack\n    - target: ''\n      test:\n        data:\n          arg1: attack\n```\n\nThis will generate two test cases for empty collection key with data: `foo=attack` and `arg1=attack`.\n\n```yaml\n    - target: ''\n      test:\n        data:\n          foo: attack\n    - target: arg1\n      test:\n        data:\n          arg1: attack\n```\n\nThis will generate one test for empty collection key and one for the collection key `arg1`. The data for the first case will be `foo=attack` and `arg1=attack` for the second.\n\n```yaml\n  targets:\n    - target: '/*'\n      test:\n        data:\n          \u003clevel1\u003e\u003clevel2\u003efoo\u003c/level2\u003e\u003clevel2\u003ebar\u003c/level2\u003e\u003c/level1\u003e\n        input:\n          headers:\n            - name: Content-Type\n              value: application/xml\n```\n\nThis will generate a test case for collection key `/*` (usually used for `XML`), the data will be the given `XML` string, and the test add an extra header for `go-ftw` test.\n\n#### Encoded request\n\nThe field `input.encoded_request` allows defining a whole request encoded in base64. When running the test, the request is decoded into bytes and sent verbatim as the input for this test case. This allows sending malformed requests. Using this field will override all other fields related to the request.\n    \n```yaml\n    targets:\n        - target: ''\n          test:\n            data: null\n            input:\n              encoded_request: R0VUIC8gSFRUUC8xLjENCkhvc3Q6IGxvY2FsaG9zdA0KDQo=\n```\n\n#### Uri\n\nThe field `input.uri` allows defining the uri used for the request manually. This is in particular useful for using the `/reflect` endpoint of [albedo](https://github.com/coreruleset/albedo) which allows defining what the server response should be from within the body of the post request that was sent.\n\n```yaml\n  targets:\n    - target: ''\n      test:\n        data: '{\"status\": 201, \"body\": \"\u003chtml\u003ereflected-token\u003c/html\u003e\"}'\n        input:\n          headers:\n            - name: Content-Type\n              value: application/json\n          uri: '/reflect'\n        output:\n          status: 201\n          response_contains: \"reflected-token\"\n```\n\n### Constants\nThe yaml schema has a mechanism to handle global and local constants.\n\n~~~yaml\nglobal:\n  default_constants:\n    one: 1\n    TWO: 2\n    two_in_list:\n      - 2\n    FOO_IN_DICT:\n      foo: attack\n...\n\nconstants:\n  HEADERS_IN_DICTIONARY:\n    headers:\n      - name: test\n        value: test\n      - name: one\n        value: ~{one}~\n      - name: 2\n        value: ~{TWO}~\n  template_in_list:\n    - SecRule for TARGETS\n    - Template with constants\n  one: one\n~~~\n\nGlobal constants are defined under the `global.default_constants` field. They are accessible across files and are reset whenever a new `global` field is defined.\n\nLocal constants are defined under a `constants` field at the root of a file. They are only accessible in the file they are defined in.\n\n#### Syntax\nConstants are defined as key-value pairs where:\n\n~~~yaml\nNAME: VALUE\n~~~\n\nThe name is used for referencing the constant and the value is used for the substitution. Referencing a constant can be done inside the value of any other key in the API. References use the `~{...}~` separators like so:\n\n~~~yaml\n~{NAME}~\n~~~\n\nVariable names can be lower or upper case and are case sensitive.\n\n#### Properties\n\nConstants can be yaml scalars, lists, or dictionaries:\n\n~~~yaml\nscalar: 1\nlist:\n  - 1\ndictionary:\n  1: 1\n~~~\n\nConstants can reference other constants in their values:\n\n~~~yaml\nheaders:\n  - name: one\n    value: ~{one}~\n  - name: two\n    value: ~{TWO}~\n~~~\n\nLocal constants with the same name as global constants have precedence in their local scope:\n~~~yaml\nglobal:\n  default_constants:\n    ONE: 1\n...\nconstants:\n  ONE: one\n...\nkey: ~{ONE}~  # substituted by 'one'\n~~~\nValues can contain multiple references, such as in templates:\n\n~~~yaml\n  - name: \"Template with constants\"\n    template: |\n      SecRule ~{target}~ \"${OPERATOR}$ ${OPARG}$\" \\\n          \"id:${CURRID}$,\\\n          phase:${PHASE}$,\\\n          deny,\\\n          t:~{None}~,\\\n          log,\\\n          msg:'%{MATCHED_VAR_NAME} was caught in phase:${PHASE}$',\\\n          ver:'~{VERSION}~'\"\n~~~\n\n### Output / additional checks\n\nBy default, the generator will produce checks for tests with `go-ftw`'s `expect_ids` field using the current rule id as parameter. If the associated rule matches and it's id put in the log, the test will pass.\n\nTo use additional check methods, the `output` field can be used to redefine this default behavior:\n    \n```yaml\n    targets:\n        - target:\n          test:\n            data:\n              foo: attack\n            output:\n              status: 200\n```\n\nThis will use the `status` check. The available checks are ([as of version 2.1.1 of the `go-ftw` yaml schema specs](https://github.com/coreruleset/ftw-tests-schema/blob/main/spec/v2.1.1/ftw.md)):\n* `status` - the expected HTTP status code\n* `response_contains` - a regex match on the response\n* `[no_]log_contains` - a string match on the log\n* `log.[no_]expect_ids` - a list of expected rule ids in the log\n* `log.[no_]match_regex` - a regex match on the log\n* `expect_error` - expect an error from the waf\n\nFor a full syntax of:\n```yaml\n          output:\n            status: 200\n            response_contains: HTTP/1.1\n            log_contains: nothing\n            no_log_contains: everything\n            log:\n                expect_ids:\n                    - 123456\n                no_expect_ids:\n                    - 123456\n                match_regex: id[:\\s\"]*123456\n                no_match_regex: id[:\\s\"]*123456\n            expect_error: true\n```\n\nTo combine the default check on the current rule id with additional checks, the `expect_ids` field must be used in conjunction with the `output` field:\n```yaml\n          output:\n            status: 200\n            log:\n                expect_ids: []\n``` \n\nThis way, the status check will be used in addition to the default rule id check.\n\nExact properties, syntax, available checks and parameters are dependent on the used version of `go-ftw`. The generator will simply replace what is defined under the `output` field in the corresponding field of the generated test case.\n\n As described for `go-ftw`,  [if any of the checks fail the test will fail](https://github.com/coreruleset/go-ftw?tab=readme-ov-file#how-log-parsing-works).\n\n### Before(-each) and After(-each) rule generation additions\n\nContent defined in test configuration files can be added around the default template generation.\n\n~~~yaml\ntemplates:\n- SecRule for TARGETS\n\ngeneration:\n  before: |\n    # STRING BEFORE ALL\n    SecAction \"id:${CURRID}$,phase:2, pass, setenv:'before=123'\"\n  after: |\n    # STRING AFTER ALL\n    SecAction \"id:${CURRID}$,phase:2 pass, setenv:'after=789'\"\n  before_each: |\n      # STRING BEFORE EACH\n      SecAction \"id:${CURRID}$,phase:${PHASE}$, pass, setenv:'before_each=456'\"\n  after_each: |\n      # STRING AFTER EACH\n      SecAction \"id:${CURRID}$,phase:${PHASE}$, pass, setenv:'after_each=456'\"\n...\n~~~\n\nUnder the `generation` section, four options exist to surround the generated configurations for the test:\n- `before` Add the content at the beginning of the generated rule file, before any generation of the template.\n- `after` Add the content at the end of the generated rule file, after any generation of the template.\n- `before_each` Add the content before each generated rule using the template.\n- `after_each` Add the content after each generated rule using the template.\n\nEach section can use the `${CURRID}$` macro to guarantee a unique id to each SecRule/SecAction. `before_each` and `after_each` sections can use all other macros used in template generation.\n\nThe above example would generate the following rules:\n\n```\n# STRING BEFORE ALL\nSecAction \"id:100013,phase:2, pass, setenv:'before=123'\"\n\n# STRING BEFORE EACH\nSecAction \"id:100014,phase:2, pass, setenv:'before_each=456'\"\n\nSecRule ARGS:arg1 \"@contains attack\" \\\n    \"id:100015,\\\n    phase:2,\\\n    deny,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR_NAME} was caught in phase:2',\\\n    ver:'MRTS/0.1'\"\n\n# STRING AFTER EACH\nSecAction \"id:100016,phase:2, pass, setenv:'after_each=456'\"\n\n# STRING BEFORE EACH\nSecAction \"id:100017,phase:2, pass, setenv:'before_each=456'\"\n\nSecRule ARGS:arg2 \"@contains attack\" \\\n    \"id:100018,\\\n    phase:2,\\\n    deny,\\\n    t:none,\\\n    log,\\\n    msg:'%{MATCHED_VAR_NAME} was caught in phase:2',\\\n    ver:'MRTS/0.1'\"\n\n# STRING AFTER EACH\nSecAction \"id:100019,phase:2, pass, setenv:'after_each=456'\"\n\n# STRING AFTER ALL\nSecAction \"id:100020,phase:2 pass, setenv:'after=789'\"\n```\n\n## Run the tool\n### Run the entire toolchain\n\n#### Required:\n* [albedo](https://github.com/coreruleset/albedo)\n* [go-ftw](https://github.com/coreruleset/go-ftw)\n* apache2 with the modsecurity and proxy modules for using the `apache2_ubuntu` + ModSecurity V2 infrastructure\n\nTo run the tests on a provided configuration, run the tool:\n\n~~~bash\n$ ./mrts/mrts.py\nusage: mrts.py [-h] -i /path/to/infra/ -r /path/to/mrts/*.yaml -e /path/to/mrts/rules/ -t /path/to/mrts/tests/ [-c]\n               [-f /path/to/mrts/ftw.mrts.config.yaml] [-v]\nmrts.py: error: the following arguments are required: -i/--infrastructure, -r/--rulesdef, -e/--expdir, -t/--testdir\n~~~\n\nAs you can see there are few command line arguments.\n* `-i` - WAF infrastructure files\n* `-r` - rules definition files\n* `-e` - export directory where rules will be written\n* `-t` - export test directory where tests will be written\n* `-c` - clean previously generated rule and test files\n* `-f` - `go-ftw` custom configuration file, if you don't want to use the default file provided in the infrastructure directory\n* `-v` - verbose output\n\nFor running without a custom `go-ftw` configuration, run the `mrts.py` script from the root directory of the project (or else provide a ftw configuration file with a correct relative path).\n\n~~~bash\n$ ./mrts/mrts.py -i config_infra/apache2_ubuntu/ -r config_tests/ -e generated/rules/ -t generated/tests/regression/tests/\nGenerate rules and tests\nLaunch backend\nLaunch infrastructure\nExecuting test set...\n🎉🎉🎉 Success: test set passed\nBackend shutdown\nInfrastructure shutdown\nMRTS completed\n~~~\n\n### Rule and test generation\n\nTo generate the rules and their tests, run the tool:\n\n```bash\n$ ./generate-rules.py \nusage: generate-rules.py [-h] -r [/path/to/mrts/*.yaml ...] -e /path/to/mrts/rules/ -t /path/to/mrts/tests/\ngenerate-rules.py: error: the following arguments are required: -r/--rulesdef, -e/--expdir, -t/--testdir\n```\n\nAs you can see there are few command line arguments.\n\n* `-r` - rules' definition files\n* `-e` - export directory where rules will be written\n* `-t` - export test directory where tests will be written\n\n```bash\n$ ./mrts/generate-rules.py -r config_tests/*.yaml -e generated/rules/ -t generated/tests/regression/tests/\n```\n\nOnce generated, rules need to be added to your ModSecurity configuration file.\n\nChange `m̀rts.load` with your absolute path to the generated rules:\n```\nInclude /Absolute/Path/To/MRTS/generated/rules/*.conf\n```\nIn `modsecurity.conf` include your absolute path to `mrts.load`:\n\n```\n...\n\nInclude /Absolute/Path/To/MRTS/mrts.load\n```\nDon't forget to restart your server each time you generate new rules.\n\nIf you finished the generation and configuration process, you can download `go-ftw` and run it.\n\nFor more info about `go-ftw` please see its [README](https://github.com/coreruleset/go-ftw/) or CRS's [excellent documentation](https://coreruleset.org/docs/development/testing/).\n\nHere is an example:\n\n```bash\n$ cat .ftw.apache-mrts.yaml \n---\nlogfile: '/var/log/apache2/error.log'\nlogmarkerheadername: 'X-MRTS-TEST'\nlogtype:\n  name: 'apache'\n  timeregex:  '\\[([A-Z][a-z]{2} [A-z][a-z]{2} \\d{1,2} \\d{1,2}\\:\\d{1,2}\\:\\d{1,2}\\.\\d+? \\d{4})\\]'\n  timeformat: 'ddd MMM DD HH:mm:ss.S YYYY'\n\n\n$ ./go-ftw run --config .ftw.apache-mrts.yaml -d generated/tests/regression/tests/\n🛠️ Starting tests!\n🚀 Running go-ftw!\n👉 executing tests in file MRTS_002_ARGS_A-GET.yaml\n\trunning 100000-1: ✔ passed in 4.03606ms (RTT 51.456201ms)\n\trunning 100000-2: ✔ passed in 2.100709ms (RTT 50.672271ms)\n\trunning 100000-3: ✔ passed in 3.572981ms (RTT 51.673479ms)\n\trunning 100000-4: ✔ passed in 2.85232ms (RTT 50.599715ms)\n👉 executing tests in file MRTS_002_ARGS_A-GET.yaml\n\trunning 100001-1: ✔ passed in 2.102875ms (RTT 50.515938ms)\n\trunning 100001-2: ✔ passed in 2.188394ms (RTT 50.675838ms)\n\trunning 100001-3: ✔ passed in 2.013156ms (RTT 50.583605ms)\n\trunning 100001-4: ✔ passed in 1.750534ms (RTT 50.572695ms)\n...\n```\n\n## Check the state of covered variables\n\nWhen you finished the build process, you can check which variables (and later the other entities) are covered by the generated rule set.\n\nYou should type:\n\n```bash\n$ cd mrts/collect_rules\n\n$ ./collect-rules.py \nusage: collect-rules.py [-h] -r [/path/to/mrts/*.conf ...]\ncollect-rules.py: error: the following arguments are required: -r/--rules\n```\n\nAs you can see here are also a mandatory argument, the path of generated rules.\n\n```bash\n$ ./collect-rules.py -r ../../generated/rules/*.conf\nConfig file: ../../generated/rules/MRTS_001_INIT.conf\n Parsing ok.\nConfig file: ../../generated/rules/MRTS_002_ARGS.conf\n Parsing ok.\nConfig file: ../../generated/rules/MRTS_003_ARGS_COMBINED_SIZE.conf\n Parsing ok.\nConfig file: ../../generated/rules/MRTS_004_ARGS_GET.conf\n Parsing ok.\nConfig file: ../../generated/rules/MRTS_005_ARGS_GET_NAMES.conf\n Parsing ok.\nConfig file: ../../generated/rules/MRTS_110_XML.conf\n Parsing ok.\n\n=====\nCovered TARGETs: REQUEST_HEADERS, ARGS, ARGS_COMBINED_SIZE, ARGS_GET, ARGS_GET_NAMES, XML\n\nUNCOVERED TARGETs: ARGS_NAMES, ARGS_POST, ARGS_POST_NAMES, ...\n```\n\nBased on the output, we actually covered 6 targets, so there are lot of works to cover all variables.\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fowasp-modsecurity%2Fmrts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fowasp-modsecurity%2Fmrts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fowasp-modsecurity%2Fmrts/lists"}