{"id":13588534,"url":"https://github.com/radarlabs/api-diff","last_synced_at":"2025-12-24T16:34:37.585Z","repository":{"id":46220487,"uuid":"272524578","full_name":"radarlabs/api-diff","owner":"radarlabs","description":"A command line tool for diffing json rest APIs","archived":false,"fork":false,"pushed_at":"2022-06-13T14:29:29.000Z","size":1332,"stargazers_count":229,"open_issues_count":3,"forks_count":16,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-17T20:42:20.977Z","etag":null,"topics":["api","compare","diff","evaluation","json","quality-metrics","ranking-quality","regression-testing","rest","side-by-side","side-by-sidediff","sxs"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radarlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-15T19:20:46.000Z","updated_at":"2025-03-03T12:14:56.000Z","dependencies_parsed_at":"2022-08-31T02:11:59.190Z","dependency_job_id":null,"html_url":"https://github.com/radarlabs/api-diff","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radarlabs%2Fapi-diff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radarlabs%2Fapi-diff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radarlabs%2Fapi-diff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radarlabs%2Fapi-diff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radarlabs","download_url":"https://codeload.github.com/radarlabs/api-diff/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247792091,"owners_count":20996877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","compare","diff","evaluation","json","quality-metrics","ranking-quality","regression-testing","rest","side-by-side","side-by-sidediff","sxs"],"created_at":"2024-08-01T15:06:46.638Z","updated_at":"2025-12-24T16:34:37.577Z","avatar_url":"https://github.com/radarlabs.png","language":"TypeScript","funding_links":[],"categories":["TypeScript","工具"],"sub_categories":["开发"],"readme":"# api-diff\n\nThis is a tool for comparing one json http server to another. It supports a wide variety of input formats as well as the ability to save a baseline run to compare against future runs.\n\nThis tool is probably most useful in scenarios involving search/ranking problems, where there are alogrithm and data changes over time that require manual evaluation of the wins and losses.\n\nIt also includes a script for talking to json http services with saved configurations.\n\n## Output / Demo\n\n### [text (console) output](https://radarlabs.github.io/api-diff/text-diff-as-html.html)\n\n![start of text diff](https://radarlabs.github.io/api-diff/text-output-2.png)\n\n### [html output](https://radarlabs.github.io/api-diff/diff.html)\n![html diff](http://images.ctfassets.net/f2vbu16fzuly/2izT1q1tXM4UO5VmaLAcnB/f47165fe0bd1c3a01abca54d975b6938/Screen_Shot_2020-11-13_at_8.55.29_AM.png)\n\nNote that this is an interactive evaluation form for figuring out which queries improved and which got worse. Each result is assigned an id based on the md5 hash of the query params + delta, and scores are saved to local storage. in your web browser. This means if you're doing a lot of compares, where many of the diffs are the same between runs, you won't need to re-rank them.\n\n## tl;dr\n\n### Try it out yourself!\n\nThis is a contrived example because our old and new servers are the same, but they return random data, so it is a good way to quickly see diffs.\n\n\nFrom CSV input:\n```\napi-diff \\                                 ✔ \n  --old.host http://names.drycodes.com/ \\\n  --new.host http://names.drycodes.com/ \\\n  --endpoint \"/10\" \\\n  --extra_params separator=\" \" nameOptions=starwarsCharacters \\\n  --input_csv docs/examples/input-names-drycodes.csv\n```\n\nFrom a file of http paths\n\n```\napi-diff \\\n  --old.host http://names.drycodes.com/ \\\n  --new.host http://names.drycodes.com/ \\\n  --input_queries docs/examples/input-names-drycodes.txt\n```\n\n### Compare two servers\n\n```\napi-diff \\\n  `# defaults to http, port 80` \\\n  --new.host localhost:3100 \\\n  --old.host localhost:4100 \\\n  `# csv input, use headings as query parameter keys` \\\n  --input_csv ~/RadarCode/geocode-acceptance-tests/input/addresses.csv \\\n  `# remap csv column \"query\" to query param \"text\"` \\\n  --key_map query=text \\\n  `# run against /v1/search on our hosts` \\\n  --endpoint /v1/search \\\n  `# extra options to append to every query` \\\n  --extra_params size=1 sources=oa,osm \\\n  `# ignore all fields named these things in computing our diff` \\\n  --ignored_fields bbox geometry attribution timestamp \\\n  `# filter down responses to the first entry in the \"addresses\" array \\\n  --response_filter '$.addresses[0]' \\\n  `# output an interactive html diff. other options are text and json` \\\n  --output_mode html \\\n  --output_file diffs.html\n```\n\n### Generate a baseline\n\n```\n api-diff generate-baseline \\\n   `# the host to run against` \\\n   --old.host localhost:3100 \\\n   `# input csv file with headers corresponding to cgi params` \\\n   --input_csv ~/RadarCode/geocode-acceptance-tests/input/addresses.csv \\\n   `# remap the \"query\" header in our csv to the \"text\" cgi param` \\\n   --key_map query=text  \\\n   `# run queries against this endpoint` \\\n   --endpoint /v1/search \\\n   `# add extra query params to every query` \\\n   --extra_params size=1 sources=osm,oa \\\n   \u003e addresses-baseline.json\n```\n\n### Compare against a baseline\n```\n api-diff \\\n  --new.host https://pelias-staging.apicom.com \\\n  --input_json_baseline addresses-baseline.json \\\n  --ignored_fields bbox geometry attribution timestamp \\ \n  --output_mode html \\ \n  \u003e diffs.html\n```\n\n### Use api tool with a [config file](#configuration)\n```\n# use config.hjson which defines staging \u0026 local envs\nAPI_DIFF_CONFIG=config.hjson api-tool \\\n  --prod \\\n  --endpoint /geocode/forward \\\n  near=\"40.74,-74\" \\\n  query=\"30 jay st\"\n```\n\n### Use compare tool with a [config file](#configuration)\n```\n# use config.hjson which defines staging \u0026 local envs\nAPI_DIFF_CONFIG=config.hjson api-diff\n  --old.staging \\\n  --new.local \\\n  --input_csv ~/geocode-acceptance-tests/input/addresses.csv\n```\n\nThis also works with a url because I've defined in my config file how auth works and where to find the keys, and what kinds of keys different hosts need\n\n```\nAPI_DIFF_CONFIG=config.hjson api-tool \\\n  \"http://api.radar.io/v1/geocode/forward?query=30 jay st\"\n```\n\nBecause I've defined a \"prod\" entry in config.hjson, and put keys into .env, this command will execute wth the necessary authentication.\n\n## Installation\n\n```\nnpm install -g @radarlabs/api-diff\n```\n\nTo run any of the examples in this doc from this source tree, simply run `yarn --silent command` instead of `command`, so `yarn --silent api-diff` instead of `api-diff`\n\n## Usage\n\nThis tool might seem like it has a zillion options, but I promise, it's not that bad!\n\nAt it's core, you need to specify two servers (old and new), and an input configuration. There are a bunch of ways to do that.\n\n## Configuration\n\nAll of the tools in this repo can load a configuration file that makes it easy to have saved defaults of servers to compare. A config file is specified via the environment variable API_DIFF_CONFIG, wth the idea that you can have multiple configs (for different services, like an api server and a geocode server) and use shell aliases to wrap the config.\n\nThe config file is specified in [hjson](https://hjson.github.io/) which allows for comments and trailing commas.\n\nAn example config looks like this\n```\n{\n  # this will be uppercased as the prefix to api key env variables\n  name: \"apicom\",\n\n  # \"param\" or \"header\"\n  # param means the key will be sent as a cgi param with \n  #     the key authParam defined below \n  # header means the key will be sent in the http \n  #     Authorization header. If your server expects a header like \"Bearer XXXXXXX\",\n  #     then set the api key to that entire string.\n  authStyle: \"header\",\n\n  # only required if using authStyle=\"param\"\n  # authParam: \"api_key\",\n\n  # optional, if using authStyle=\"header\"\n  # authType: \"Basic\",\n\n  # first keyType listed here will be the default if a\n  # keyType is not specified in the commandline options to # compare\n  keyTypes: ['test', 'live'],\n\n  hosts: {\n    prod: {\n      host: 'https://prod.api.com',\n      aliases: ['production'],\n    }\n    staging: {\n      host: 'https://staging.api.com',\n    },\n    local: {\n      host: 'http://localhost:8000',\n      keyEnv: 'staging',\n    },\n    user: {\n      takesArg: true,\n      host: 'https://USER-staging.api.com'\n      keyEnv: 'staging',\n    },\n  }\n}\n```\n\nThis defines a pretty common server setup. There is a production API at prod.api.com, that uses https (the default is http). It implicitly defines a \"prod\" key env in our config. There is a staging api at staging.api.com with a \"staging\" key env. Additionally, developers run local apis at localhost:8000, which use staging keys and should be communicated with over http (the default). Finally, each user has their own staging env at USER-staging.api.com which again uses staging keys.\n\n### Authorization\n\napi-diff looks in the shell environment for api keys if needed. It loads `~/.api-keys.env` into its env. For this config, that file would look like:\n\n```\n# keys are name + keyEnv + keyType + KEY\n# all uppercase, joined with underscores\nAPICOM_PROD_TEST_KEY=XXXX\nAPICOM_PROD_LIVE_KEY=XXXX\nAPICOM_STAGING_TEST_KEY=XXXX\nAPICOM_STAGING_LIVE_KEY=XXXX\n```\n\nIf your server requires authorization in request params, set authStyle=\"param\" and authParam to the request parameter name the server expects. If your server uses the HTTP Authoization header, then set authStyle=\"header\" and (optionally) authType to what auth type prefix the server expects, such as \"Basic\" or \"Bearer\" - this can be omitted if your server takes bare keys in the Authoization header.\n\nOur config defines two types of keys - test and live. It also defines two key environments. One for \"prod\" and one for \"staging,\" the host configs for \"local\" and \"user\" are both configured to look in the \"staging\" key env\"\n\n### Specifying a server\nFor compare, two servers must be specified, with --new.X and --old.X options. For the api tool, the server is specifed in the same way but without the \"new/old.\" prefix.\n\nServers can be specified in two ways:\n\n- by using host entries from our config file. So `--old.prod`, `--old.staging`, `--old.user blackmad` etc would all work based on our examples (similarly, `--new.staging`, `--new.prod`, etc)\n- a combination of `--old.host` (required), `--old.key` (only if auth is needed). And similarly, `--new.host`, and `--new.key`\n\nThese can be mixed! \n\n\n### Reading queries\n\n- `--input_params` - the input file has one query per line, of the form `param1=A%20B\u0026param2=Z`. requires `--endpoint` argument. `--method` used, defaults to GET.\n- `--input_queries` - the input file has one path + params per line, like `/endpoint?param1=A%20\u0026p2=Z`. `--method` used, defaults to GET.\n- `--input_csv` - for reading csvs. Combined with `--endpoint` and `--method`\n  - Assumes that the first line of the file is column headings, unless `--key_map` is used with *all numeric keys*\n  - Column headings are used as parameter names\n  - `--key_map` remaps column headings. `--key_map query=text` remaps the csv heading \"query\" to the query param \"text\". `--key_map 0=text 1=near` will cause the parser to assume the csv does not have named column headings (because all the keys are numbers), and will use the first column as the query param \"text\", the second as the query param \"near\"\n\n### other options\n- `--concurrency` - how many queries to run at a time per host\n- `--timeout` - per query timeout, defaults to 30s\n- `--unchanged` - whether or not to output unchanged queries, defaults to false\n- `--ignored_fields` - fields to leave out when computing differences from server responses\n\n### output options\n- `--color` - text mode only. whether or not to colorize the output. Defaults to true if sending to stdout, false if redirecting output. Use this option to force colorized output. Suggest always using it.\n- `--output_mode` - json, html or text (console output)\n- `--output_file` - defaults to stdout, where to output to\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradarlabs%2Fapi-diff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradarlabs%2Fapi-diff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradarlabs%2Fapi-diff/lists"}