Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/phux/apijc
A tool to automate regression testing by comparing responses of URLs on two different domains.
https://github.com/phux/apijc
api automated-testing json regression-testing
Last synced: 4 days ago
JSON representation
A tool to automate regression testing by comparing responses of URLs on two different domains.
- Host: GitHub
- URL: https://github.com/phux/apijc
- Owner: phux
- Created: 2023-12-01T00:35:13.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2023-12-07T12:03:58.000Z (12 months ago)
- Last Synced: 2023-12-07T22:43:40.917Z (12 months ago)
- Topics: api, automated-testing, json, regression-testing
- Language: Go
- Homepage:
- Size: 46.9 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# API JSON Compare (apijc)
`apijc` fetches and compares the status codes and json responses of
user-defined paths on two domains and reports any differences.
- [API JSON Compare (apijc)](#api-json-compare-apijc)
- [Features](#features)
- [Installation](#installation)
- [Binary](#binary)
- [Golang](#golang)
- [Usage](#usage)
- [Quickstart](#quickstart)
- [Example output](#example-output)
- [Configuration](#configuration)
- [CLI Flags](#cli-flags)
- [urlFile](#urlfile)
- [targets](#targets)
- [sequentialTargets](#sequentialtargets)
- [Structure](#structure)
- [Path expansion](#path-expansion)
- [requestBody vs requestBodyFile](#requestbody-vs-requestbodyfile)
- [urlFile Example](#urlfile-example)
- [rateLimit](#ratelimit)
- [headerFile](#headerfile)
- [headerFile Example](#headerfile-example)
- [Precedence](#precedence)
- [Output](#output)
- [stdout](#stdout)
- [outputFile](#outputfile)
- [outputFile Example](#outputfile-example)
- [Exit codes](#exit-codes)
- [TODOs](#todos)
## Features
- Diff comparison of response bodies from both domains
- Sequential request chains. See [sequentialTargets](#sequentialtargets) below
- Check for expected status codes
- Path expansion of
- Lists (example: `/foo/{1,2,3}/bar`)
- Numerical ranges (example: `/foo/{1-100}/bar`)
- Mixed list and ranges (example: `/foo/{1,3-5,99,200-400}`)
- See [Path expansion](#path-expansion) below
- Rate limiting
- Load headers from file
- Specify header key-value pairs globally or per domain
- Custom headers per url target
- Write errors/mismatches to stdout or file
## Installation
### Binary
1. Download the binary for your architecture from the
[Releases](https://github.com/phux/apijc/releases) page.
2. Put it into a directory in your `$PATH`
### Golang
```sh
go install github.com/phux/apijc@latest
```
## Usage
```sh
apijc \
--baseDomain "" \
--newDomain "" \
--urlFile \
--headerFile \ # optional
--rateLimit 100 \ # optional
--outputFile # optional
```
### Quickstart
1. Install - see [Installation](#installation)
2. Create a urlFile JSON - see [urlFile](#urlfile) - and setup up at least one target
Minimal `urlFile` example:
```json
{
"targets": [
{
"relativePath": "/some/relative/path",
"httpMethod": "GET",
"expectedStatusCode": 200
}
]
}
```
3. execute `apijc`
```sh
apijc \
--baseDomain "" \
--newDomain "" \
--urlFile path/to/your/urlFile.json
```
Note: if `--rateLimit` is not passed to `apijc`, the default rate limit is 1 request per second.
## Example output
```sh
$ apijc --baseDomain http://localhost:8080 \
--newDomain http://localhost:8081 \
--urlFile .testdata/urlfile_example.json \
--rateLimit 1000
Starting with rate limit: 1000.000000/second
2023/12/06 22:09:35 Checking GET /v1/example
2023/12/06 22:09:35 Success: GET /v1/example (checked 1 of 1 paths)
2023/12/06 22:09:35 Checking GET /v1/{1-100}
2023/12/06 22:09:36 Success: GET /v1/{1-100} (checked 100 of 100 paths)
2023/12/06 22:09:36 Checking GET /v1/@1-3@
2023/12/06 22:09:36 Success: GET /v1/@1-3@ (checked 3 of 3 paths)
2023/12/06 22:09:36 Checking GET /v1/expected_jsonmissmatch
2023/12/06 22:09:36 ERROR: GET /v1/expected_jsonmissmatch (checked 1 of 1 paths)
2023/12/06 22:09:36 Checking POST /v1/example
2023/12/06 22:09:36 Success: POST /v1/example (checked 1 of 1 paths)
2023/12/06 22:09:36 Checking sequential group: First POST, then GET
2023/12/06 22:09:36 Success: POST /v1/sequential_post (checked 1 of 1 paths)
2023/12/06 22:09:36 Success: GET /v1/sequential_get (checked 1 of 1 paths)
2023/12/06 22:09:36 Done. Checked 108 of 108 paths
2023/12/06 22:09:36 Findings:
2023/12/06 22:09:36 /v1/expected_jsonmissmatch
Error: JSON mismatch
Diff: @ ["foo"]
- "baz"
+ "bar"
2023/12/06 22:09:36 Finished - 1 findings
exit status 1
```
## Configuration
### CLI Flags
| Flag | Required | Description | Default |
| ---------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| baseDomain | yes | The first domain to make all requests to | - |
| newDomain | yes | The second domain to make all requests to | - |
| urlFile | yes | Path to JSON file containing target URL paths, HTTP method, ...
See [urlFile](#urlfile) | - |
| headerFile | no | Path to JSON file containing global and/or per-domain header key-value pairs that will be set on each request. See [headerFile](#headerfile) | - |
| rateLimit | no | Requests per second (float).
See [rateLimit](#ratelimit) | 1 |
| outputFile | no | Path to store findings in JSON format. See [outputFile](#outputfile) | - |
### urlFile
The `urlFile` defines the relative paths that will be requested and compared on
both domains. It contains `targets` and/or `sequentialTargets`.
#### targets
Standalone requests are defined in the `targets` key of the `urlFile`. Each
target will be requested on both, `baseDomain` and `newDomain`.
#### sequentialTargets
In the `sequentialTargets` key chains of consecutive calls can be defined.
Example: first make a POST request to create an entity, then
make a GET request to fetch the created entity.
The steps for a sequential target group are:
1. make request to target 1 on `baseDomain`
2. compare actual status code vs `expectedStatusCode`
3. make request to target 1 on `newDomain`
4. compare actual status code vs `expectedStatusCode`
5. compare response bodies
6. make request to target 2 on `baseDomain`
7. compare actual status code vs `expectedStatusCode`
8. make request to target 2 on `newDomain`
9. compare actual status code vs `expectedStatusCode`
10. compare response bodies
#### Structure
```json
{
"targets": [
{
"relativePath": "",
"httpMethod": "",
"expectedStatusCode": ,
"requestBody": "",
"requestBodyFile": "",
"requestHeaders": { // optional
"": ""
},
"patternPrefix": "",
"patternSuffix": ""
}
],
"sequentialTargets": {
"Some name for the sequence, example: Create Order, then fetch Order": [
{
"relativePath": "/first/path",
"httpMethod": "",
"expectedStatusCode": 201
},
{
"relativePath": "/second/path",
"httpMethod": "",
"expectedStatusCode": 200
}
]
}
}
```
#### Path expansion
`relativePath` can contain lists and/or ranges to quickly define multiple
targets at once.
Expansions are triggered for everything between a configurable `patternPrefix`
(default: `{`) and a `patternSuffix` (default: `}`) on each target.
List items are separated by `,` (comma).
Numerical ranges can be defined by `-` (dash).
Example:
```json
"relativePath": "/foo/{bar,3-5}"
```
This will translate to requesting and checking 4 paths:
- `/foo/bar` <-- from list item `bar`
- `/foo/3` <-- from range `3-5`
- `/foo/4` <-- from range `3-5`
- `/foo/5` <-- from range `3-5`
Note: a path can also define multiple expansions, like `/foo/{1-2}/bar/{a,b}`.
This path will result in 4 paths in total:
- `/foo/1/bar/a`
- `/foo/1/bar/b`
- `/foo/2/bar/a`
- `/foo/2/bar/b`
#### requestBody vs requestBodyFile
The `urlFile` can contain two different exclusive keys to specify the request body to a target: `requestBody` and `requestBodyFile`.
`requestBody` contains an escaped JSON string to be sent as the body.
Example:
```json
"requestBody": "{\"a\":\"b\"}",
```
`requestBodyFile` contains a path to a JSON file containing the body to be sent
for the target. This is helpful if the request body to be sent is bigger and avoids escaping hell.
Example:
```json
"requestBodyFile": ".testdata/request_body.json"
```
#### urlFile Example
```json
{
"targets": [
{
"relativePath": "/v1/example",
"httpMethod": "GET",
"expectedStatusCode": 200
},
{
"relativePath": "/v1/{1-100}",
"httpMethod": "GET",
"expectedStatusCode": 200
},
{
"relativePath": "/v1/example",
"httpMethod": "POST",
"expectedStatusCode": 201,
"requestBody": "{\"a\":\"b\"}",
"requestHeaders": {
"Content-Type": "application/json"
},
"patternPrefix": "{",
"patternSuffix": "}"
},
{
"relativePath": "/v1/post_with_body_file",
"httpMethod": "POST",
"expectedStatusCode": 201,
"requestBodyFile": ".testdata/request_body.json"
}
],
"sequentialTargets": {
"First POST, then GET": [
{
"relativePath": "/v1/sequential_post",
"httpMethod": "POST",
"expectedStatusCode": 201,
"requestBody": "{\"a\":\"b\"}"
},
{
"relativePath": "/v1/sequential_get",
"httpMethod": "GET",
"expectedStatusCode": 200
}
]
}
}
```
### rateLimit
Sometimes it's necessary to limit the rate with which the tool makes requests to the configured domains.
The flag `--rateLimit` allows to configure the rate.
Must be `int` or float.
The number defines the allowed requests per seconds.
Examples:
- `--rateLimit=1`: 1 request per second (default)
- `--rateLimit=0.5`: 1 request per 2 seconds
- `--rateLimit=10`: 10 requests per second
### headerFile
The `headerFile` allows to define key-value pairs in the `global` key that will be set on each
request, in addition to the static `requestHeaders` defined on each target in
the `urlFile`.
Additionally, it is possible to set per-domain header key-value pairs that will
be set on each request to the particular domain (`baseDomain|newDomain`).
This is helpful for example if you need to set different `Authorization` headers per domain.
Note: The `global`, `baseDomain` and `newDomain` keys are all optional.
#### headerFile Example
```sh
# header.json
{
"global": {
"SomeHeaderName": "Value applied to all requests to both domains"
},
"baseDomain": {
"SomeHeaderName": "Value applied to all requests to BaseDomain"
},
"newDomain": {
"SomeHeaderName": "Value applied to all requests to NewDomain"
}
}
```
#### Precedence
If the `headerFile` and a target's `requestHeaders` contain duplicate header keys,
the target's `requestHeaders` value takes precedence.
```
headerFile.global < headerFile.Domain < target.requestHeaders
```
### Output
#### stdout
If the flag `--outputFile` is not passed, the findings are written to
stdout, see [Example output](#example-output)
#### outputFile
Via `--outputFile` a path to a file can be passed. The findings will be written
to this file instead of stdout.
##### outputFile Example
```sh
# findings.json
[
{
"url": "/v1/expected_jsonmissmatch",
"error": "JSON mismatch",
"diff": "@ [\"foo\"]\n- \"baz\"\n+ \"bar\"\n"
}
]
```
## Exit codes
On successful execution `apijc` exits with code `0`.
On any issue the exit code will be `> 0`
## TODOs
- [x] read `requestBody` JSON from files
- [ ] allow to skip the diff comparison for JSON response body fields on a target (e.g. `id`)