{"id":13739315,"url":"https://github.com/michaelo/sapt","last_synced_at":"2025-08-17T16:22:42.646Z","repository":{"id":143445458,"uuid":"406947343","full_name":"michaelo/sapt","owner":"michaelo","description":"Simple file-oriented API-testing tool","archived":false,"fork":false,"pushed_at":"2024-02-01T15:46:05.000Z","size":1431,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-15T09:42:57.638Z","etag":null,"topics":["api","cli","curl","testing","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/michaelo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-09-15T23:01:03.000Z","updated_at":"2024-09-26T13:01:01.000Z","dependencies_parsed_at":"2024-02-01T17:41:11.928Z","dependency_job_id":null,"html_url":"https://github.com/michaelo/sapt","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelo%2Fsapt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelo%2Fsapt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelo%2Fsapt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelo%2Fsapt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/michaelo","download_url":"https://codeload.github.com/michaelo/sapt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253135463,"owners_count":21859647,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","cli","curl","testing","zig"],"created_at":"2024-08-03T04:00:32.280Z","updated_at":"2025-05-08T19:33:55.691Z","avatar_url":"https://github.com/michaelo.png","language":"Zig","funding_links":[],"categories":["Applications"],"sub_categories":[],"readme":"sapt - A simple tool for API testing\r\n==============\r\n*Att: I'm testing different intro-texts to test what communicates the intention the best*\r\n\r\nsapt aims to be a simple tool to help with API-testing and similar use cases. It focuses on making it easy for developers to compose, organize and perform tests/requests in an open, reliable and source-control friendly way.\r\n\r\nsapt *\"is not\"* a full-fledged GUI-based do-anything tool, but rather a focused command line utility.\r\n\r\n\r\nUsage: Basic\r\n-------------\r\n\r\nsapt requires you to organise your requests in individual files. Those files may be gathered in folders to create test suites.\r\n    \r\ntestsuite/01-mytest.pi contents:\r\n    \r\n    \u003e GET https://api.warnme.no/api/status\r\n    \u003c 200\r\n\r\nRunning a single test:\r\n\r\n    % sapt testsuite/01-mytest.pi\r\n    1/1 testsuite/01-mytest.pi                                : OK\r\n\r\nHelp:\r\n\r\n    % sapt -h\r\n    sapt v1.0.0 - Simple API Tester\r\n\r\n    Usage: sapt [arguments] [file1.pi file2.pi ... fileN.pi]\r\n\r\n    Examples:\r\n    sapt api_is_healthy.pi\r\n    sapt testsuite01/\r\n    sapt -b=myplaybook.book\r\n    sapt -i=generaldefs/.env testsuite01/\r\n\r\n    Arguments:\r\n        --colors=auto|on|off Set wether to attempt to use colored output or not\r\n        --delay=NN          Delay execution of each consecutive step with NN ms\r\n    -e, --early-quit        Abort upon first non-successful test\r\n    -h, --help              Show this help and exit\r\n        --help-format       Show details regarding file formats and exit\r\n    -i=file,\r\n        --initial-vars=file Provide file with variable-definitions made available\r\n                            to all tests\r\n        --insecure          Don't verify SSL certificates\r\n    -m, --multithread       Activates multithreading - relevant for repeated\r\n                            tests via playbooks\r\n    -p, --pretty            Try to format response data based on Content-Type.\r\n                            Naive support for JSON, XML and HTML\r\n    -b=file,\r\n        --playbook=file     Read tests to perform from playbook-file -- if set,\r\n                            ignores other tests passed as arguments\r\n    -d, --show-response     Show response data. Even if -s.\r\n    -s, --silent            Silent. Suppresses output. Overrules verbose.\r\n    -v, --verbose           Verbose output\r\n        --verbose-curl      Verbose output from libcurl\r\n        --version           Show version and exit\r\n\r\n    -DKEY=VALUE             Define variable, similar to .env-files. Can be set\r\n                            multiple times\r\n\r\n\r\nsapt can take multiple arguments, both files and folders. The entire input-set will be sorted alphanumerically when passing a folder, thus you can dictate the order of execution by making sure the names of the scripts reflects the order:\r\n\r\n* suite/01-auth.pi\r\n* suite/02-post-entry.pi\r\n* suite/03-get-posted-entry.pi\r\n* suite/04-delete-posted-entry.pi\r\n\r\n*Note: playbooks provides a way to override this.*\r\n\r\n\u003c!--\r\n\r\nWhy oh why\r\n----------------\r\n\r\n    You: Why should I use this tool?\r\n    \r\n    Me: Good question, I'm glad you asked!\r\n        There's a good chance you shouldn't. There are several well established\r\n        alternatives you should consider instead. E.g. Postman, httpier, or \r\n        perhaps cURL.\r\n    \r\n    You: ... ok? ...\r\n    \r\n    Me: But, if you should find those tools either to heavy to run, too\r\n        unpredictable with regards to where your data may be stored, or simply\r\n        just too slow or complex to run or compose tests for, sapt might be of\r\n        interest.\r\n    \r\n    You: Go on...\r\n\r\n    Me: sapt is a lightweight tool, both with regards to runtime requirements,\r\n        as well as its feature set. It also provides you with full control of\r\n        your own data. See \"Design goals\" further down, or\r\n        \"docs/COMPARISONS.md\" to see what sapt focuses on. \r\n\r\n    You: I've tried it and: (pick a choice)\r\n        a) I loved it\r\n            Me: Awesome!\r\n        b) I hated it\r\n            Me: No worries. Take care!\r\n--\u003e\r\n\r\n\r\nUsage: Complex\r\n----------------\r\n\r\nAssuming you first have to get an authorization code from an auth-endpoint, which you will need in other tests.\r\n\r\nLet's say you have the following files:\r\n\r\n* myservice/.env\r\n* myservice/01-auth.pi\r\n* myservice/02-get-data.pi\r\n\r\n### Set up variables: myservice/.env\r\n\r\n    OIDC_USERNAME=myoidcclient\r\n    OIDC_PASSWORD=sup3rs3cr3t\r\n    USERNAME=someuser\r\n    PASSWORD=supersecret42\r\n\r\n### Get the auth-token: myservice/01-auth.pi\r\n\r\n    \u003e POST https://my.service/api/auth\r\n    Authorization: Basic {{base64enc({{OIDC_USERNAME}}:{{OIDC_PASSWORD}})}}\r\n    Content-Type: application/x-www-form-urlencoded\r\n    -\r\n    grant_type=password\u0026username={{USERNAME}}\u0026password={{PASSWORD}}\u0026scope=openid%20profile\r\n    \u003c 200\r\n    id_token=\"id_token\":\"()\"\r\n\r\nProvided that the auth-endpoint will return something like this:\r\n\r\n    {\"access_token\": \"...\", \"id_token\":\"...\", \"...\", ...}\r\n\r\n... the test will then set the id_token-variable, allowing it to be referred in subsequent tests.\r\n\r\n\r\n### Get data from service using data from previous test: myservice/02-get-data.pi\r\n    \r\n    \u003e GET https://my.app/api/entry\r\n    Cookie: SecurityToken={{id_token}}\r\n    \u003c 200\r\n\r\n\r\n### Finally, Run the testsuite\r\n\r\n    sapt myservice\r\n\r\nOutput:\r\n\r\n    1: myservice/01-auth.pi                                 :OK (HTTP 200)\r\n    2: myservice/02-get-data.pi                             :OK (HTTP 200)\r\n    ------------------\r\n    2/2 OK\r\n    ------------------\r\n    FINISHED - total time: 0.189s\r\n\r\n*Tips: You can add -v or -d for more detailed output*\r\n\r\nUsage: playbook\r\n-----------\r\nmyplay.book contents:\r\n\r\n    # Run this request 1 time\r\n    myproj/auth.pi\r\n    # Run this request 100 times\r\n    myproj/api_get.pi * 100\r\n\r\nTests shall be run in the order declared in the playbook. Each test may be followed by a number indicating the number of times it shall be performed.\r\n\r\nRunning the playbook:\r\n\r\n    sapt -b=myplay.book\r\n\r\nPlaybooks resolves paths relative to its own location.\r\n\r\nOutput:\r\n\r\n    1/5: myproj/auth.pi                                         : OK (HTTP 200 - OK)\r\n    time: 256ms\r\n    2/5: myproj/api_get.pi                                      : OK (HTTP 200 - OK)\r\n    100 iterations. 100 OK, 0 Error\r\n    time: 1050ms/100 iterations [83ms-215ms] avg:105ms\r\n    ------------------\r\n    2/2 OK\r\n    ------------------\r\n\r\n\r\nBuild:\r\n------------\r\nThe tool is written in [zig](https://ziglang.org/) v0.9.0, and depends on [libcurl](https://curl.se/libcurl/).\r\n\r\nPrerequisites:\r\n* [zig is installed](https://ziglang.org/download/) and available in path. Development is done on 0.9.0\r\n* [libcurl is installed](https://curl.se/download.html) and library and headers are available in either path or through pkg-config.\r\n\r\nGet source:\r\n\r\n    git clone https://github.com/michaelo/sapt\r\n    cd sapt\r\n\r\n\r\nDevelopment build/run:\r\n\r\n    zig build run\r\n\r\nRun all tests:\r\n\r\n    zig build test\r\n\r\nInstall:\r\n\r\n    zig build install --prefix-exe-dir /usr/local/bin\r\n\r\n*... or other path to put the executable to be in path.*\r\n\r\n\r\nDesign goals:\r\n------------\r\n* Only you should own and control your data - e.g. any version control and data sharing is up to you.\r\n* Tests should be easily written and arranged - no hidden headers or such\r\n* Support easy use of secrets and common variables/definitions\r\n* Tests should live alongside the artifacts they tests\r\n\r\n\r\nTerminology:\r\n------\r\n\r\n\u003ctable\u003e\r\n    \u003cthead\u003e\r\n        \u003ctr\u003e\r\n        \u003cth\u003eTerm\u003c/th\u003e\r\n        \u003cth\u003eDescription\u003c/th\u003e\r\n        \u003c/tr\u003e\r\n    \u003c/thead\u003e\r\n    \u003ctbody\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003etest\u003c/td\u003e\r\n            \u003ctd\u003ethe particular file/request to be processed. A test can result in \"OK\" or \"ERROR\"\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eplaybook\u003c/td\u003e\r\n            \u003ctd\u003ea particular recipe of tests, their order and other parameters to be executed in a particular fashion\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eextraction-expression\u003c/td\u003e\r\n            \u003ctd\u003eThe mechanism which allows one to extract data from responses according to a given expression and store the results in a variable for use in following tests. E.g. extract an auth-token to use in protected requests.\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n    \u003c/tobdy\u003e\r\n\u003c/table\u003e\r\n\r\nLimitations:\r\n------\r\nDue in part to the efforts to both having a clear understanding of the RAM-usage, as well as keeping the heap-usage low and controlled, a set of discrete limitations are currently characteristic for the tool. I will very likely revise a lot of these decisions going forward - but here they are:\r\n\r\n\u003ctable\u003e\r\n    \u003cthead\u003e\r\n        \u003ctr\u003e\r\n            \u003cth\u003eWhat\u003c/th\u003e\r\n            \u003cth\u003eLimitation\u003c/th\u003e\r\n        \u003c/tr\u003e\r\n    \u003c/thead\u003e\r\n    \u003ctbody\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd colspan=\"2\"\u003eGeneral limitations\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax number of tests to process in a given run\u003c/td\u003e\r\n            \u003ctd\u003e128\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax length for any file path\u003c/td\u003e\r\n            \u003ctd\u003e1024\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax repeats for single test\u003c/td\u003e\r\n            \u003ctd\u003e1000\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd colspan=\"2\"\u003eTest-specific parameters\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax number of headers\u003c/td\u003e\r\n            \u003ctd\u003e32\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax length for a given HTTP-header\u003c/td\u003e\r\n            \u003ctd\u003e8K\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax number of variables+functions in a given test\u003c/td\u003e\r\n            \u003ctd\u003e64\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax length of a function response\u003c/td\u003e\r\n            \u003ctd\u003e1024\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax length of a variable key\u003c/td\u003e\r\n            \u003ctd\u003e128\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax length of a variable value\u003c/td\u003e\r\n            \u003ctd\u003e8K\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax URL length\u003c/td\u003e\r\n            \u003ctd\u003e2048\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eMax size of payload\u003c/td\u003e\r\n            \u003ctd\u003e1M\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n        \u003ctr\u003e\r\n            \u003ctd\u003eExtraction-expressions: order of precedence\u003c/td\u003e\r\n            \u003ctd\u003esapt will first attempt to match extraction entries with response body first, response headers second.\u003c/td\u003e\r\n        \u003c/tr\u003e\r\n    \u003c/tbody\u003e \r\n\u003c/table\u003e\r\n\r\n\r\n\r\n\r\nTest-file specification:\r\n--------\r\n\r\n    \u003cinput section\u003e\r\n    \u003coptional set of headers\u003e\r\n    \u003coptional payload section\u003e\r\n    \u003cresponse section\u003e\r\n    \u003coptional set of variable extraction expressions\u003e\r\n\r\nComments:\r\n\r\n    # Comment - must be start of line. Can be used everywhere except of in payload-section\r\n\r\nVariables:\r\n\r\n    {{my_var}}\r\n\r\nFunctions:\r\n\r\nConvenience-functions are (to be) implemented. The argument can consist of other variables.\r\n\r\n    {{base64enc(string)}}\r\n\r\nExample:\r\n\r\n    Authorization: basic {{base64enc({{username}}:{{password}})}}\r\n\r\nSupported functions:\r\n\r\n* base64enc(string) - base64-encoding\r\n* env(string) - lookup variables from OS-environment\r\n* *TODO: urlencode(string)*\r\n* *TODO: base64dec(string) ?*\r\n\r\nInput section:\r\n\r\n    # '\u003e' marks start of 'input'-section, followed by HTTP verb and URL\r\n    \u003e POST https://some.where/\r\n    # List of HTTP-headers, optional\r\n    Content-Type: application/json\r\n    Accept: application/json\r\n\r\nPayload section, optional:\r\n\r\n    # '-' marks start of payload-section, and it goes until output-section\r\n    -\r\n    {\"some\":\"data}\r\n\r\n*TBD: Implement support for injecting files? If so: allow arbitrary sizes.*\r\n\r\nOutput section:\r\n\r\n    # \u003cExpected HTTP code\u003e [optional string to check response for it to be considered successful]\r\n    \u003c 200 optional text\r\n\r\n    # HTTP-code '0' is \"don't care\"\r\n    \u003c 0\r\n\r\nSet of variable extraction expressions, optional:\r\n\r\n    # Key=expression\r\n    #   expression format: \u003ctext-pre\u003e\u003cgroup-indicator\u003e\u003ctext-post\u003e\r\n    # Expression shall consists of a string representing the output, with '()' indicating the part to extract\r\n    AUTH_TOKEN=\"token\":\"()\"\r\n\r\n*TBD: Might support more regex-like expressions to control e.g. character groups and such.*\r\n\r\nExit codes\r\n-------------\r\nsapt is also usable in e.g. cron jobs to monitor availability of a service.\r\n\r\nThis is a tentative list of exit codes currently implemented in sapt:\r\n\r\n* 0: OK\r\n* 1: Something went wrong processing the input set\r\n* 2: One or more tests failed\r\n\r\n\r\nUse cases\r\n-------------\r\nThis sections aims to provide a set of example use cases for which sapt can be useful. This is not an exchaustive list, but please let me know if any other use cases are found:\r\n\r\n* Test / explore the behaviour of an API / web service\r\n* Describe and verify desired behviour when developing a service - e.g. TDD-like process\r\n* Monitor a set of relevant endpoints to quickly debug which - if any - one fails\r\n* Cron-job to monitor a service\r\n* Load-testing\r\n\r\n\r\n\r\nTODO, somewhat ordered:\r\n------------\r\n*Att! The points here are not changes that we strictly commit to, but an organic list of things to consider. Higher up = more likely*\r\n\r\n* Separate automagic handling between .env-file explicitly and implicitly passed (through folder)? Explicit should perhaps ignore folder-rules? \r\n* Determine if current solution where variables can't be overwritten is a good idea or not.\r\n* Libs/deps handling:\r\n    * Credits: Determine all deps we need to ship.\r\n        * libcurl for all platforms - license added to CREDITS\r\n        * zlib for Windows - license added to CREDITS\r\n        * OpenSSL? Other curl-deps?\r\n    * Look into staticly linking all deps - the absolute best would be a single, self-contained executable\r\n    * Get proper version-control of which dynamic libraries we depend on/provide.\r\n* Dev: Set up automatic builds/cross-builds for Win10 x64, Linux x64, macOS (x64 and Arm)\r\n    * Clean up lib-handling. Currently we e.g. have libcurl stored as libcurl.dll and curl.dll due to some linkage-discrepencies for Windows/vcpkg. Can we simply vendor it?\r\n* Append current git hash (short) for debug-builds. Need to figure out how to detect this at compile-time and inject it properly.\r\n* Due to this being an explorative project while learning Zig, there are inconsistencies regarding memory-handling. This must be cleaned up and verified.\r\n* Code quality - especially in main.zig - is quite crap at this point.\r\n* More advanced sequence options?:\r\n    * Specify setup/teardown-tests to be run before/after test/suite/playbook?\r\n        * --suite-setup, --suite-teardown, --step-setup, --step-teardown?\r\n        * What about playbooks?\r\n    * Specify option to only re-run specific steps of a sequence? --steps=1,3,5-7.\r\n* Provide better stats for repeats. We currently have min, max and avg/mean time. Could median or something mode-like be as useful or more? A plot would be nice here.\r\n    * Also ensure that total time for entire test set also prints the accumulated value of each step without the delay set by --delay\r\n* Describe/explore how/if --delay shall affect playbooks. Currently: it doesn't. Playbooks should be self-contained, so we'd need an in-playbook variant of the functionality\r\n* Implement support to do step-by-step tests by e.g. requiring user to press enter between each test?\r\n* Store responses? E.g. 'sapt mysuite/ --store-responses=./out/' creates ./out/mysuite/01-test1.pi.out etc\r\n* Playbooks:\r\n    * TBD: What shall the semantics be regarding response data and variable-extraction when we have multiple repetitions? Makes no sense perhaps, so either have \"last result matters\", \"undefined behaviour\" or \"unsupported\". Wait for proper use cases.\r\n* Test/verify safety of string lengths: parsing\r\n* Support both keeping variables between (default) as well as explicitly allowing sandboxing (flag) of tests?\r\n* TBD: Shall we support \"repeats\" in test-files as well, not only in playbooks?\r\n* Actively limit the set of protocols we allow. We currently just forward the URL however it is to CURL. If we staticly link, build a custom variant with only the feature set we need.\r\n* Finish basic syntax highligh ruleset for the test-files\r\n* Dev: Test feature flags based on comptime-parsing a feature-file\r\n\r\n\r\nFeature-exploration AKA Maybe-TODO:\r\n-------------\r\n* Support handling encrypted variables?\r\n* Support list of curl-commands as alternative output?\r\n* Performant, light-weight GUI (optional)? Plotting performance for stress tests and such.\r\n* Support response-time as test-prereq? Perhaps in playlist (low pri)\r\n* TBD: Possibility to set \"verbose\" only for a specific test? Using e.g. the test-sequence-number?\r\n* TBD: Actually parse response data - which also can allow us to more precisely extract data semantically\r\n\r\n\r\nThanks / attributions:\r\n--------\r\n* zig - an interesting language of which this project is my first deliverable\r\n* libcurl - the workhorse\r\n\r\n*See ATTRIBUTIONS.md for licenses.*\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelo%2Fsapt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichaelo%2Fsapt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelo%2Fsapt/lists"}