{"id":31572161,"url":"https://github.com/nigelhorne/app-test-generator","last_synced_at":"2025-10-05T13:49:31.109Z","repository":{"id":304247616,"uuid":"1018231096","full_name":"nigelhorne/App-Test-Generator","owner":"nigelhorne","description":"Test Corpus \u0026 Fuzzer","archived":false,"fork":false,"pushed_at":"2025-09-27T20:59:10.000Z","size":89,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-09-27T22:23:46.085Z","etag":null,"topics":["perl","perl5","test","test-automation","testing"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nigelhorne.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-11T20:56:38.000Z","updated_at":"2025-09-27T20:59:14.000Z","dependencies_parsed_at":"2025-09-27T22:23:46.426Z","dependency_job_id":null,"html_url":"https://github.com/nigelhorne/App-Test-Generator","commit_stats":null,"previous_names":["nigelhorne/test-corpus","nigelhorne/app-test-generator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nigelhorne/App-Test-Generator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nigelhorne%2FApp-Test-Generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nigelhorne%2FApp-Test-Generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nigelhorne%2FApp-Test-Generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nigelhorne%2FApp-Test-Generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nigelhorne","download_url":"https://codeload.github.com/nigelhorne/App-Test-Generator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nigelhorne%2FApp-Test-Generator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278464273,"owners_count":25991177,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["perl","perl5","test","test-automation","testing"],"created_at":"2025-10-05T13:49:29.952Z","updated_at":"2025-10-05T13:49:31.103Z","avatar_url":"https://github.com/nigelhorne.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NAME\n\nApp::Test::Generator - Generate fuzz and corpus-driven test harnesses\n\n# SYNOPSIS\n\nFrom the command line:\n\n    fuzz-harness-generator t/conf/add.conf \u003e t/add_fuzz.t\n\nFrom Perl:\n\n    use App::Test::Generator qw(generate);\n\n    # Generate to STDOUT\n    App::Test::Generator::generate(\"t/conf/add.conf\");\n\n    # Generate directly to a file\n    App::Test::Generator::generate(\"t/conf/add.conf\", \"t/add_fuzz.t\");\n\n# OVERVIEW\n\nThis module takes a formal input/output specification for a routine or\nmethod and automatically generates test cases. In effect, it allows you\nto easily add comprehensive black-box tests in addition to the more\ncommon white-box tests typically written for CPAN modules and other\nsubroutines.\n\nThe generated tests combine:\n\n- Random fuzzing based on input types\n- Deterministic edge cases for min/max constraints\n- Static corpus tests defined in Perl or YAML\n\nThis approach strengthens your test suite by probing both expected and\nunexpected inputs, helping you to catch boundary errors, invalid data\nhandling, and regressions without manually writing every case.\n\n# DESCRIPTION\n\nThis module implements the logic behind [fuzz-harness-generator](https://metacpan.org/pod/fuzz-harness-generator).\nIt parses configuration files (fuzz and/or corpus YAML), and\nproduces a ready-to-run `.t` test script using [Test::Most](https://metacpan.org/pod/Test%3A%3AMost).\n\nIt reads configuration files (Perl `.conf` with `our` variables,\nand optional YAML corpus files), and generates a [Test::Most](https://metacpan.org/pod/Test%3A%3AMost)-based\nfuzzing harness in `t/fuzz.t`.\n\nGenerates `t/fuzz.t` combining:\n\n- Randomized fuzzing of inputs (with edge cases)\n- Optional static corpus tests from Perl `%cases` or YAML file (`yaml_cases` key)\n- Functional or OO mode (via `$new`)\n- Reproducible runs via `$seed` and configurable iterations via `$iterations`\n\n## EDGE CASE GENERATION\n\nIn addition to purely random fuzz cases, the harness generates\ndeterministic edge cases for parameters that declare `min`, `max`,\n`len`, or `len` in their schema definitions.\n\nFor each constraint, three edge cases are added:\n\n- Just inside the allowable range\n\n    This case should succeed, since it lies strictly within the bounds.\n\n- Exactly on the boundary\n\n    This case should succeed, since it meets the constraint exactly.\n\n- Just outside the boundary\n\n    This case is annotated with `_STATUS = 'DIES'` in the corpus and\n    should cause the harness to fail validation or croak.\n\nSupported constraint types:\n\n- `number`, `integer`\n\n    Uses numeric values one below, equal to, and one above the boundary.\n\n- `string`\n\n    Uses strings of lengths one below, equal to, and one above the boundary\n    (minimum length = `len`, maximum length = `len`).\n\n- `arrayref`\n\n    Uses references to arrays of lengths one below, equal to, and one above the boundary\n    (minimum length = `len`, maximum length = `len`).\n\n- `hashref`\n\n    Uses hashes with key counts one below, equal to, and one above the\n    boundary (`min` = minimum number of keys, `max` = maximum number\n    of keys).\n\n- `memberof` - optional arrayref of allowed values for a parameter:\n\n        our %input = (\n            status =\u003e { type =\u003e 'string', memberof =\u003e [ 'ok', 'error', 'pending' ] },\n            level =\u003e { type =\u003e 'integer', memberof =\u003e [ 1, 2, 3 ] },\n        );\n\n    The generator will automatically create test cases for each allowed value (inside the member list),\n    and at least one value outside the list (which should die, `_STATUS = 'DIES'`).\n    This works for strings, integers, and numbers.\n\n- `boolean` - automatic boundary tests for boolean fields\n\n        our %input = (\n            flag =\u003e { type =\u003e 'boolean' },\n        );\n\n    The generator will automatically create test cases for 0 and 1, and optionally invalid values that should trigger `_STATUS = 'DIES'`.\n\nThese edge cases are inserted automatically, in addition to the random\nfuzzing inputs, so each run will reliably probe boundary conditions\nwithout relying solely on randomness.\n\n# CONFIGURATION\n\nThe configuration file is a Perl file that should set variables with `our`.\nExample: the generator expects your config to use `our %input`, `our $function`, etc.\n\nRecognized items:\n\n- `%input` - input params with keys =\u003e type/optional specs:\n\n            our %input = (\n                    name =\u003e { type =\u003e 'string', optional =\u003e 0 },\n                    age =\u003e { type =\u003e 'integer', optional =\u003e 1 },\n            );\n\n    Supported basic types used by the fuzzer: `string`, `integer`, `number`, `boolean`, `arrayref`, `hashref`.\n    (You can add more types; they will default to `undef` unless extended.)\n\n- `%output` - output param types for Return::Set checking:\n\n            our %output = (\n                    type =\u003e 'string'\n            );\n\n    If the output hash contains the key \\_STATUS, and if that key is set to DIES,\n    the routine should die with the given arguments; otherwise, it should live.\n    If it's set to WARNS,\n    the routine should warn with the given arguments\n\n- `$module` - module name (optional).\n\n    If omitted, the generator will guess from the config filename:\n    `My-Widget.conf` -\u003e `My::Widget`.\n\n- `$function` - function/method to test (defaults to `run`).\n- `$new` - optional hashref of args to pass to the module's constructor (object mode):\n\n            our $new = { api_key =\u003e 'ABC123', verbose =\u003e 1 };\n\n    To ensure new is called with no arguments, you still need to defined new, thus:\n\n        our $new = '';\n\n- `%cases` - optional Perl static corpus (expected =\u003e \\[ args... \\]):\n\n        our %cases = (\n          'ok'   =\u003e [ 'ping' ],\n          'error'=\u003e [ '' ],\n        );\n\n- `$yaml_cases` - optional path to a YAML file with the same shape as `%cases`.\n- `$seed` - optional integer. When provided, the generated `t/fuzz.t` will call `srand($seed)` so fuzz runs are reproducible.\n- `$iterations` - optional integer controlling how many fuzz iterations to perform (default 50).\n- `%edge_cases` - optional hash mapping parameter names to arrayrefs of extra values to inject:\n\n            our %edge_cases = (\n                    name =\u003e [ '', 'a' x 1024, \\\"\\x{263A}\" ],\n                    age  =\u003e [ -1, 0, 99999999 ],\n            );\n\n    (Values can be strings or numbers; strings will be properly quoted.)\n\n- `%type_edge_cases` - optional hash mapping types to arrayrefs of extra values to try for any field of that type:\n\n            our %type_edge_cases = (\n                    string  =\u003e [ '', ' ', \"\\t\", \"\\n\", \"\\0\", 'long' x 1024, chr(0x1F600) ],\n                    number  =\u003e [ 0, 1.0, -1.0, 1e308, -1e308, 1e-308, -1e-308, 'NaN', 'Infinity' ],\n                    integer =\u003e [ 0, 1, -1, 2**31-1, -(2**31), 2**63-1, -(2**63) ],\n            );\n\n# EXAMPLES\n\n## Math::Simple::add()\n\nFunctional fuzz + Perl corpus + seed:\n\n    our $module = 'Math::Simple';\n    our $function = 'add';\n    our %input = ( a =\u003e { type =\u003e 'integer' }, b =\u003e { type =\u003e 'integer' } );\n    our %output = ( type =\u003e 'integer' );\n    our %cases = (\n      '3'     =\u003e [1, 2],\n      '0'     =\u003e [0, 0],\n      '-1'    =\u003e [-2, 1],\n      '_STATUS:DIES'  =\u003e [ 'a', 'b' ],     # non-numeric args should die\n      '_STATUS:WARNS' =\u003e [ undef, undef ], # undef args should warn\n    );\n    our $seed = 12345;\n    our $iterations = 100;\n\n## Adding YAML file to generate tests\n\nOO fuzz + YAML corpus + edge cases:\n\n        our %input = ( query =\u003e { type =\u003e 'string' } );\n        our %output = ( type =\u003e 'string' );\n        our $function = 'search';\n        our $new = { api_key =\u003e 'ABC123' };\n        our $yaml_cases = 't/corpus.yml';\n        our %edge_cases = ( query =\u003e [ '', '    ', '\u003cscript\u003e' ] );\n        our %type_edge_cases = ( string =\u003e [ \\\"\\\\0\", \"\\x{FFFD}\" ] );\n        our $seed = 999;\n\n### YAML Corpus Example (t/corpus.yml)\n\nA YAML mapping of expected -\u003e args array:\n\n        \"success\":\n          - \"Alice\"\n          - 30\n        \"failure\":\n          - \"Bob\"\n\n## Example with arrayref + hashref\n\n    our %input = (\n      tags   =\u003e { type =\u003e 'arrayref', optional =\u003e 1 },\n      config =\u003e { type =\u003e 'hashref' },\n    );\n    our %output = ( type =\u003e 'hashref' );\n\n## Example with memberof\n\n    our %input = (\n        status =\u003e { type =\u003e 'string', memberof =\u003e [ 'ok', 'error', 'pending' ] },\n    );\n    our %output = ( type =\u003e 'string' );\n\nThis will generate fuzz cases for 'ok', 'error', 'pending', and one invalid string that should die.\n\n# OUTPUT\n\nBy default, writes `t/fuzz.t`.\nThe generated test:\n\n- Seeds RNG (if configured) for reproducible fuzz runs\n- Uses edge cases (per-field and per-type) with configurable probability\n- Runs `$iterations` fuzz cases plus appended edge-case runs\n- Validates inputs with Params::Get / Params::Validate::Strict\n- Validates outputs with [Return::Set](https://metacpan.org/pod/Return%3A%3ASet)\n- Runs static `is(... )` corpus tests from Perl and/or YAML corpus\n\n# NOTES\n\n- The conf file must use `our` declarations so variables are visible to the generator via `require`.\n\n# SEE ALSO\n\n- Test coverage report: [https://nigelhorne.github.io/App-Test-Generator/coverage/](https://nigelhorne.github.io/App-Test-Generator/coverage/)\n- [Test::Most](https://metacpan.org/pod/Test%3A%3AMost), [Params::Get](https://metacpan.org/pod/Params%3A%3AGet), [Params::Validate::Strict](https://metacpan.org/pod/Params%3A%3AValidate%3A%3AStrict), [Return::Set](https://metacpan.org/pod/Return%3A%3ASet), [YAML::XS](https://metacpan.org/pod/YAML%3A%3AXS)\n\n# AUTHOR\n\nNigel Horne, `\u003cnjh at nigelhorne.com\u003e`\n\nPortions of this module's design and documentation were created with the\nassistance of [ChatGPT](https://openai.com/) (GPT-5), with final curation\nand authorship by Nigel Horne.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnigelhorne%2Fapp-test-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnigelhorne%2Fapp-test-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnigelhorne%2Fapp-test-generator/lists"}