{"id":15529055,"url":"https://github.com/alexander-schranz/markdown-based-api-testing","last_synced_at":"2025-04-23T12:35:25.863Z","repository":{"id":40476819,"uuid":"461609556","full_name":"alexander-schranz/markdown-based-api-testing","owner":"alexander-schranz","description":"An article about adopting markdown files as source of truth for testing your API with the usage of php matcher library","archived":false,"fork":false,"pushed_at":"2022-05-05T22:26:59.000Z","size":685,"stargazers_count":8,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-29T23:41:17.854Z","etag":null,"topics":["alexander-schranz-article","api","api-testing","json","phpunit","rest","symfony","tests"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alexander-schranz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-02-20T20:34:54.000Z","updated_at":"2024-11-21T00:23:19.000Z","dependencies_parsed_at":"2022-08-09T21:31:30.835Z","dependency_job_id":null,"html_url":"https://github.com/alexander-schranz/markdown-based-api-testing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexander-schranz%2Fmarkdown-based-api-testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexander-schranz%2Fmarkdown-based-api-testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexander-schranz%2Fmarkdown-based-api-testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexander-schranz%2Fmarkdown-based-api-testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alexander-schranz","download_url":"https://codeload.github.com/alexander-schranz/markdown-based-api-testing/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250435940,"owners_count":21430379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alexander-schranz-article","api","api-testing","json","phpunit","rest","symfony","tests"],"created_at":"2024-10-02T11:16:06.792Z","updated_at":"2025-04-23T12:35:25.843Z","avatar_url":"https://github.com/alexander-schranz.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Testing APIs using Markdown and PHP Matcher\n\nMy experience is that almost every API test follow the same workflow. \nFixtures are loaded in setupBeforeClass or in a bootstrap file of\nyour test framework. A request will be sent and the response should match.\n\nIn the last projects I have matched the content against an own `.json` \nfile for my tests. To do that I need to adopt the request in my\nTest file but the expected result is in another own `.json` file, which make the maintainance a little bit difficult because I need to find\nthe correct `.json` file for the correct test case.\n\nAfter I stumbled over the test cases in the [rectorphp](https://github.com/rectorphp/rector-src/pull/1668/files) library,\nI was thinking about how I could adopt this also for API tests. I wanted to get my API\ntests into a format which remove the whole boilerplate of creating the tests\nand also have the request and response in the same file.\n\n## Creating a framework independent format\n\nFirst I thought about moving all into a json format like the following:\n\n```json\n{\n    \"request\": {\n        \"method\": \"POST\",\n        \"uri\": \"/api/examples/1\",\n        \"headers\": {\n            \"Accept\": \"application/json\"\n        },\n        \"content\": {\n            \"title\": \"Test\"\n        }\n    },\n    \"response\": {\n        \"statusCode\": 200,\n        \"headers\": {\n            \"Accept\": \"application/json\"\n        },\n        \"content\": {\n            \"id\": \"@integer@\",\n            \"title\": \"Test\"\n        }\n    }\n}\n```\n\nBut I think that JSON is not very readable and has the\ndisadvantage that we can not add any comment to it.\n\nSo I had the idea to use a markdown flavoured format for this:\n\n~~~markdown\n# Request\n\n```http request\nPOST /api/examples\nAccept: application/json\n\n{\n    \"title\": \"Test\"\n}\n```\n\n---\n\n# Response\n\n```http request\nHTTP/1.1 200 OK\nContent-Type: application/json\n\n{\n    \"id\": \"@integer@\",\n    \"title\": \"Test\"\n}\n```\n~~~\n\nThe format is really great because it matches\nthe HTTP protocol nevertheless it has one downside. \nCompared to JSON, autocompletion and code highlighting are missing in the IDE.\nTo fix this, I split the content into an own code block of the markdown file:\n\n~~~markdown\n# Request\n\n```http request\nPOST /api/examples\nAccept: application/json\n```\n\n```json\n{\n    \"title\": \"Test\"\n}\n```\n\n---\n\n# Response\n\n```http request\nHTTP/1.1 200 OK\nContent-Type: application/json\n```\n\n```json\n{\n    \"id\": \"1\",\n    \"title\": \"Test\"\n}\n```\n~~~\n\nThere it is, the format how we are able to write tests.\nNow its time to use it.\n\n## Creating a basic test case\n\nTo create our basic test case we first need a method to read our markdown files.\nWe will use the [symfony/finder](https://symfony.com/doc/current/components/finder.html)\ncomponent to read only the files we need.\n\n```php\n\u003c?php\n\nnamespace App\\Tests\\Functional;\n\nuse Symfony\\Bundle\\FrameworkBundle\\Test\\WebTestCase;\nuse Symfony\\Component\\Finder\\Finder;\n\nabstract class AbstractApiTest extends WebTestCase\n{\n    /**\n     * @return \\Generator\u003c\\SplFileInfo\u003e\n     */\n    protected function yieldFilesFromDirectory(string $directory): \\Generator\n    {\n        $finder = new Finder();\n        $finder-\u003ein($directory)\n            -\u003eignoreVCS(true)\n            -\u003efiles()\n            -\u003ename('*.md');\n\n        foreach ($finder as $file) {\n            if ($file instanceof \\SplFileInfo) {\n                yield [$file];\n            }\n        }\n    }\n}\n```\n\nNow we need to add a method to parse the markdown file and make our expected\nassertions:\n\n```php\nprotected function doTestFileInfo(\\SplFileInfo $fileInfo): void\n{\n    // arrange\n    $content = \\file_get_contents($fileInfo-\u003egetPathname());\n\n    [$input, $output] = \\explode(\"\\n---\\n\", $content);\n\n    [$method, $uri, $headers, $content] = $this-\u003eparseRequest($input);\n    [$expectedServerProtocol, $expectedStatusCode, $expectedHeaders, $expectedContent] = $this-\u003eparseResponse($output);\n\n    /** @var array\u003cstring, string\u003e $server */\n    $server = [];\n    foreach ($headers as $key =\u003e $value) {\n        $servers['HTTP_' . strtoupper(str_replace('-', '_', $key))] = $value;\n    }\n\n    // act\n    $client = $this-\u003ecreateClient();\n    $client-\u003erequest($method, $uri, [], [], $server, $content);\n\n    // assert\n    $response = $client-\u003egetResponse();\n\n    $this-\u003eassertSame($expectedServerProtocol, $response-\u003egetProtocolVersion());\n    $this-\u003eassertSame($expectedStatusCode, $response-\u003egetStatusCode());\n    foreach ($expectedHeaders as $headerName =\u003e $expectedHeaderValue) {\n        $this-\u003eassertSame($expectedHeaderValue, $response-\u003eheaders-\u003eget($headerName));\n    }\n    $this-\u003eassertSame($expectedContent, $response-\u003egetContent());\n}\n```\n\nTo get the information I need, I parsed the markdown file with a regex.\n\n\u003cdetails\u003e\n    \u003csummary\u003eparseRequest Function\u003c/summary\u003e\n\n```php\n/**\n * @return array{\n *     0: string,\n *     1: string,\n *     2: array\u003cstring, string\u003e,\n *     3: string,\n * }\n */\nprivate function parseRequest(string $input): array\n{\n    preg_match('/```http request\\n(\\w+) (.+)\\n([^```]*)```([^```]*```\\w+\\n([^```]*)```)?/', $input, $matches);\n    $method = $matches[1];\n    $uri = $matches[2];\n    $headerString = $matches[3];\n    $content = $matches[5] ?? '';\n\n    $headers = [];\n    $headerParts = array_filter(explode(\"\\n\", $headerString));\n    foreach ($headerParts as $headerPart) {\n        [$headerName, $headerValue] = \\explode(':', $headerPart, 2);\n        $headers[trim($headerName)] = trim($headerValue);\n    }\n\n    return [$method, $uri, $headers, $content];\n}\n```    \n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eparseResponse Function\u003c/summary\u003e\n\n```php\n/**\n * @return array{\n *     0: string,\n *     1: int,\n *     2: array\u003cstring, string\u003e,\n *     3: string,\n * }\n */\nprivate function parseResponse(string $output): array\n{\n    preg_match('/```http request\\n\\w+\\/(\\d+.\\d+) (\\d+) \\w+\\n([^```]*)```([^```]*```\\w+\\n([^```]*)```)?/', $output, $matches);\n    $serverProtocol = $matches[1];\n    $statusCode = (int) $matches[2];\n    $headerString = $matches[3];\n    $content = $matches[5] ?? '';\n\n    $headers = [];\n    $headerParts = array_filter(explode(\"\\n\", $headerString));\n    foreach ($headerParts as $headerPart) {\n        [$headerName, $headerValue] = \\explode(':', $headerPart, 2);\n        $headers[trim($headerName)] = trim($headerValue);\n    }\n\n    return [$serverProtocol, $statusCode, $headers, $content];\n}\n```    \n\n\u003c/details\u003e\n\nAnd then used the parsed data to send the request over the symfony\nweb test case. The parsed data of the response is used for the assertions\nagainst the response object. We are matching here all relevant data\nthe Protocol Version, Status Code, Headers and Content.\n\nNow we can use our `AbstractApiTest` in our own Test by testing our rest endpoint:\n\n```php\n\u003c?php\n\nnamespace App\\Tests\\Functional\\Controller;\n\nuse App\\Tests\\Functional\\AbstractApiTest;\n\nclass ExampleControllerTest extends AbstractApiTest\n{\n    /**\n     * @dataProvider provideData()\n     */\n    public function testFixtures(\\SplFileInfo $fileInfo): void\n    {\n        $this-\u003edoTestFileInfo($fileInfo);\n    }\n\n    /**\n     * @return \\Generator\u003c\\SplFileInfo\u003e\n     */\n    public function provideData(): \\Generator\n    {\n        return $this-\u003eyieldFilesFromDirectory(__DIR__ . '/fixtures');\n    }\n}\n```\n\nIn the testMethod itself or in the `setupBeforeClass` class you even could still\nmake sure that you are loading some database fixtures which are required for your test case.\n\nNow we can create our test cases in our markdown files like the followings:\n\n\u003cp\u003e\n\u003ca href=\"tests/Functional/Controller/fixtures/example_get.md\"\u003e\n    \u003cimg src=\"pictures/get-test.png\" alt=\"Get Test Case\" style=\"width: 420px\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp\u003e\n\u003ca href=\"tests/Functional/Controller/fixtures/example_post.md\"\u003e\n    \u003cimg src=\"pictures/post-test.png\" alt=\"Post Test Case\" style=\"width: 420px\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\nI think it is really nice as it forces the developer to work in \na format based on the HTTP Protocol Standard.\n\n## Fix problem with dynamic response content\n\nWhile it is hard when you example work with date times,\nauto generated ids to exact match the response against a json. \nThere is a solution for this by using the [coduo php matcher](https://github.com/coduo/php-matcher).\n\nThis library allows you to write something like:\n\n```json\n{\n    \"id\": \"@integer@\",\n    \"text\": \"@string@.startsWith('Test')\"\n}\n```\n\nTo use expression or type matching instead of exact matching.\n\nSo instead of:\n\n```php\n$this-\u003eassertSame($expectedContent, $response-\u003egetContent());\n```\n\n```php\n$this-\u003eassertMatchesPattern($expectedContent, $response-\u003egetContent());\n```\n\nFrom the `Coduo\\PHPMatcher\\PHPUnit\\PHPMatcherAssertions` Trait.\n\nThis way we can use [all coduo patterns](https://github.com/coduo/php-matcher#available-patterns)\ninside our response content.\n\n## Optimizing the output format in PHPUnit\n\nI personally prefer to use the [`nunomaduro\\collision`](https://github.com/nunomaduro/collision) package\nas a printer for my PHPUnit tests. As it output the tests \nin a nice readable way:\n\n![Picture of default output](pictures/output-default.png)\n\nStill it does not indicate here a lot but with a little change in our\nAPI test case by using a key in our test generator this way:\n\n```diff\n-yield [$file];\n+yield \\str_replace(\\getcwd() . '/', '', $file-\u003egetPathname()) =\u003e [$file];\n```\n\nIt will output the test in the following format:\n\n![Picture of collision output with set key](pictures/output-formatted.png)\n\nNow the nice thing is as we are using the path from `getcwd()` (current directory)\nwe are able to directly click in our terminal on the `.md` file path to open the\ntest case file and adopt it to our needs when required.\n\nBy using the filename as key it is also possible to filter by the specific\ntest case via phpunit:\n\n```bash\nvendor/bin/phpunt --filter=\"example_post.md\"\n```\n\nThis way only the `example_post.md` is exectued.\n\n## Conclusion\n\nWith the usage of a more general format we got rid of a lot of boilerplate\ncode for our api tests. With the usage of markdown files we even make it possible\nto better document our test cases as we could add test for them.\nAlso I like to use the response and request format of the official http standard.\n\nThe tests could also be ported to another framework if the framework of your\napplication is changing or even to another language. You just would need\nto reimplement your base test case again.\n\nAlso if you don't want to use markdown files I recommend using the\n[coduo/php-matcher](https://github.com/coduo/php-matcher) instead of manual\nmatching your response data. Because your tests should also fail if you are\nadding a new key to your response object. So you can be sure that new added keys\nare also added to your test cases.\n\nIf you want to test it yourself feel free to clone this repository and run and adopt its tests:\n\n```bash\ngit clone https://github.com/alexander-schranz/markdown-based-api-testing\ncomposer install\nvendor/bin/phpunit\n```\n\nTell me what you think about this way of testing your api. \nI am also interessted in existing api test frameworks that work in a similar way.\nAttend the discussion about this on [Twitter](https://twitter.com/alex_s_/status/1495503991429554181).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexander-schranz%2Fmarkdown-based-api-testing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexander-schranz%2Fmarkdown-based-api-testing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexander-schranz%2Fmarkdown-based-api-testing/lists"}