{"id":20454897,"url":"https://github.com/robb-j/repo-api-service","last_synced_at":"2026-05-12T07:34:03.616Z","repository":{"id":178090786,"uuid":"647006255","full_name":"robb-j/repo-api-service","owner":"robb-j","description":"Wrap a git repo with an HTTP API to read and write files","archived":false,"fork":false,"pushed_at":"2024-11-27T15:54:01.000Z","size":132,"stargazers_count":1,"open_issues_count":11,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-15T23:25:10.677Z","etag":null,"topics":["deno","git"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robb-j.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-29T21:17:42.000Z","updated_at":"2024-11-27T15:54:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"b4171aaf-8078-4a2c-a995-c7e1a5087a14","html_url":"https://github.com/robb-j/repo-api-service","commit_stats":null,"previous_names":["robb-j/repo-api-service"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robb-j%2Frepo-api-service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robb-j%2Frepo-api-service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robb-j%2Frepo-api-service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robb-j%2Frepo-api-service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robb-j","download_url":"https://codeload.github.com/robb-j/repo-api-service/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242005820,"owners_count":20056434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deno","git"],"created_at":"2024-11-15T11:17:20.391Z","updated_at":"2026-05-12T07:34:03.584Z","avatar_url":"https://github.com/robb-j.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# repo-api-service\n\nEver wanted an HTTP API to query what's in a git repo and retrive files? And\nmaybe even write back to the repo and commit changes upstream? Repo Api Service\ndoes that. It is a service that checks out a git repo and keeps it up to date.\nThere is then an API to get and query files in the repository and another\nendpoint to write changes to the repo and commit and push them upstream.\n\nThis is designed to be run as a single container bound to a single git\nrepository. So to support multiple repositories you run multiple containers with\ndifferent configurations.\n\n## Endpoints\n\n\u003e Examples are written with [httpie](https://httpie.io/) with\n\u003e [robb-j/r0b-blog](https://github.com/robb-j/r0b-blog) as the repository being\n\u003e queried.\n\n### `GET /`\n\nCheck the app is online and find out the app version.\n\n### `GET /healthz`\n\nEnsure the app is healthy, useful for\n[Kubernetes checks](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).\n\n### `GET /query`\n\nSearch and retrieve files from the git repository. You can either get a specific\nfile or get multiple files using a\n[glob pattern](https://en.wikipedia.org/wiki/Glob_(programming)).\n\n**get a specific file**\n\nUse the `file` search parameter to get a specific file and it will response will\nbe the body of the request.\n\n```sh\nhttp $URL/query file==content/index.md\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eResponse\u003c/summary\u003e\n\n```http\nHTTP/1.1 200 OK\ndate: Sun, 02 Jul 2023 12:52:25 GMT\ntransfer-encoding: chunked\nvary: Accept-Encoding\n\n---\ntitle: r0b's random ramblings\nlayout: home\n---\n\n# r0b's random ramblings\n\nThis is a place for me to jot down thing's I've experimented with\nand document them for my future self ... or other people too I guess.\n```\n\n\u003c/details\u003e\n\n**query with a glob**\n\nUse the `glob` parameter to retrieve multiple files and it will return them as a\n`multipart/form-data` response where each entry is an item in the form data\nwhere the name is the path of the matched file and the body is the contents of\nthe file.\n\n```sh\nhttp $URL/query \"glob==content/post/*.md\"\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eResponse\u003c/summary\u003e\n\n```http\nHTTP/1.1 200 OK\ncontent-type: multipart/form-data; boundary=----3515918059508545431921584621\ndate: Sun, 02 Jul 2023 12:56:27 GMT\ntransfer-encoding: chunked\nvary: Accept-Encoding\n\n------3515918059508545431921584621\nContent-Disposition: form-data; name=\"content/post/bundle-javascript-with-eleventy-and-esbuild.md\"; filename=\"content/post/bundle-javascript-with-eleventy-and-esbuild.md\"\nContent-Type: application/octet-stream\n\n---\ntitle: Bundle JavaScript with Eleventy and esbuild\ndate: 2021-06-27\ndraft: false\nsummary: \u003e\n  How to add JavaScript and bundle it together for an Eleventy project plus integration with the development server for automatic reloading.\n---\n\n[Static site generators are great...](https://blog.r0b.io/post/compile-sass-with-eleventy/)\nas I have previously mentioned,\nhere is how to bundle JavaScript into your Eleventy site too.\n------3515918059508545431921584621\n...\n```\n\n\u003c/details\u003e\n\n**parsing data**\n\nYou can tell the service to parse the file contents for you for both `file` and\n`glob` queries by using the `format` parameter. These formats are supported:\n`json`, `yaml`,`csv`,`markdown`,`toml`, `ini` and `binary`. If not specified it\ndefaults to `binary`.\n\n```sh\nhttp $URL/query file==content/index.md format==markdown\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eResponse\u003c/summary\u003e\n\n```http\nHTTP/1.1 200 OK\ncontent-encoding: gzip\ncontent-length: 207\ncontent-type: application/json\ndate: Sun, 02 Jul 2023 12:57:58 GMT\nvary: Accept-Encoding\n\n{\n    \"attrs\": {\n        \"layout\": \"home\",\n        \"title\": \"r0b's random ramblings\"\n    },\n    \"body\": \"# r0b's random ramblings\\n\\nThis is a place for me to jot down thing's I've experimented with\\nand document them for my future self ... or other people too I guess.\\n\",\n    \"frontMatter\": \"title: r0b's random ramblings\\nlayout: home\"\n}\n```\n\n\u003c/details\u003e\n\n**CSV columns**\n\nWhen getting CSV files you can also specify `columns` to convert CSV records\ninto objects rather than just arrays in the JSON.\n\n```sh\nhttp $URL/query file==some.csv format==csv columns==date,title,age\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eResponse\u003c/summary\u003e\n\n```http\nHTTP/1.1 200 OK\ncontent-encoding: gzip\ncontent-length: 207\ncontent-type: application/json\ndate: Sun, 02 Jul 2023 12:57:58 GMT\nvary: Accept-Encoding\n\n[\n  {\"date\": \"2023-06-01\", \"title\": \"Something\", \"age\": 42},\n  {\"date\": \"2023-06-01\", \"title\": \"Something\", \"age\": 42},\n  {\"date\": \"2023-06-01\", \"title\": \"Something\", \"age\": 42}\n]\n```\n\n\u003c/details\u003e\n\n### `PUT /file`\n\nUse the file endpoint to write a file to repository, commit it to git and push\nthe change upstream. It will write the file in the request body at the location\nspecified by the `file` parameter.\n\nUse the `message` parameter to set the commit message, this will be prefixed\nwith `repo-api-service` and defaults to `repo-api-service: automated commit` if\nnot set.\n\n```sh\ncat page.md | http put $URL/file file==page.md message==\"Update page\"\n```\n\nIt will return a http/200 if everything went ok. If it failed, it will rollback\nthe git back to the state to before the file was written.\n\n### `GET /expand`\n\nIf you want to see whats in a repository without retrieving the whole files, you\ncan ask the API to expand a glob for you.\n\n```sh\nhttp $URL/expand \"glob==content/post/*.md\"\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eResponse\u003c/summary\u003e\n\n```http\nHTTP/1.1 200 OK\ncontent-encoding: gzip\ncontent-length: 587\ncontent-type: application/json\ndate: Sun, 02 Jul 2023 13:06:06 GMT\nvary: Accept-Encoding\n\n[\n    \"content/post/automating-developer-operations-for-nodejs.md\",\n    \"content/post/bundle-javascript-with-eleventy-and-esbuild.md\",\n    \"content/post/compile-sass-with-eleventy.md\",\n    \"content/post/connecting-an-rpi-to-802.1x.md\",\n    \"content/post/creating-a-nova-extension-with-typescript.md\",\n    \"content/post/creating-custom-javascript-errors.md\",\n    \"content/post/creating-drag-interactions-with-set-pointer-capture-in-java-script.md\",\n    \"content/post/deploying-esp32-with-spiffs-using-github-actions.md\",\n    \"content/post/embed-jsdoc-comments-in-an-eleventy-website-with-ts-morph.md\",\n    \"content/post/esm-nodejs-typescript-with-subpath-exports.md\",\n    \"content/post/getting-started-with-kube-prometheus-stack.md\",\n    \"content/post/host-an-ics-calendar-feed-with-eleventy.md\",\n    \"content/post/minimal-rpi-kiosk.md\",\n    \"content/post/my-first-generator-function.md\",\n    \"content/post/quick-concise-array-to-map-conversion-in-javascript.md\",\n    \"content/post/regrets-of-a-research-software-engineers-tech-stacks.md\",\n    \"content/post/running-node-js-as-a-systemd-service.md\",\n    \"content/post/spoofing-an-rpi-mac-address.md\",\n    \"content/post/tales-from-the-bashrc-bashrc.md\",\n    \"content/post/tales-from-the-bashrc-d1.md\",\n    \"content/post/tales-from-the-bashrc-npr.md\",\n    \"content/post/trying-to-make-a-vanilla-web-app.md\",\n    \"content/post/useful-rpi-wifi-commands.md\",\n    \"content/post/using-jsx-without-react.md\",\n    \"content/post/using-urlpattern-to-add-edit-on-github-support-to-my-blog.md\",\n    \"content/post/yoath-released.md\"\n]\n```\n\n\u003c/details\u003e\n\n### `GET /changed`\n\n\u003e new in **0.2.0**\n\nFind out which files changed at certain times and/or at certain paths or globs.\n\n```sh\nhttp :9000/changed since==\"10 days ago\" paths==\"content/post/*\"\n```\n\nThere are these URL search parameters available, you can specify any combination\nor none of them:\n\n- `since` — specify the lower bound of the date range, only showing files that\n  were changed after this date\n- `until` — specify the higher bound of the date range, to only show files\n  changed before this date\n- `paths` — narrow down the paths you wish to search within. This can be set\n  multiple times to look at different files or use multiple glob patterns. The\n  request will fail if a path is specified that doesn't exist in the repository.\n\n\u003e Git dates — You can see the accepted formats from\n\u003e [this git commit](https://github.com/git/git/commit/34dc6e73b01011fcbe0f314d47fd6120382ae145)\n\n\u003cdetails\u003e\n\u003csummary\u003eResponse\u003c/summary\u003e\n\n```http\nHTTP/1.1 200 OK\ncontent-encoding: gzip\ncontent-length: 136\ncontent-type: application/json\ndate: Wed, 02 Aug 2023 13:49:09 GMT\nvary: Accept-Encoding\n\n[\n    \"content/post/useful-rpi-wifi-commands.md\",\n    \"content/post/host-an-ics-calendar-feed-with-eleventy.md\",\n    \"content/post/host-a-ics-calendar-feed-with-eleventy.md\",\n    \"content/post/embed-jsdoc-comments-in-an-eleventy-website-with-ts-morph.md\"\n]\n```\n\n\u003c/details\u003e\n\n## Configuration\n\nThe service can be configured by a configuration file, CLI arguments or\nenvironment variables.\n\nTo see how to configure the app, run `deno run -A source/config.ts` which shows\nthe default configuration, how it is configured and the table below. The JSON\nconfiguration file should be placed at the root of the project as `config.json`.\n\n| name             | type    | flag   | variable    | fallback                               |\n| ---------------- | ------- | ------ | ----------- | -------------------------------------- |\n| auth.key         | string  | ~      | AUTH_KEY    |                                        |\n| env              | string  | ~      | DENO_ENV    | development                            |\n| git.commit       | boolean | ~      | GIT_COMMIT  | false                                  |\n| git.commitPrefix | string  | ~      | ~           | repo-api-service                       |\n| git.pull         | boolean | ~      | GIT_PULL    | false                                  |\n| git.push         | boolean | ~      | GIT_PUSH    | false                                  |\n| git.remote       | string  | ~      | GIT_REMOTE  | https://github.com/robb-j/r0b-blog.git |\n| git.syncInterval | number  | ~      | ~           | 300000                                 |\n| meta.name        | string  | ~      | APP_NAME    | robb-j/repo-api-service                |\n| meta.version     | string  | ~      | APP_VERSION | 1.2.3                                  |\n| port             | number  | --port | APP_PORT    | 8000                                   |\n\n- `env` — Set the \"mode\" of the service, set to `development` to enable debug\n  logging.\n- `git.remote` — The remote of the git repository for the service.\n- `git.pull` — Whether to pull the git repository while synchronising.\n- `git.push` — Whether to push back to the git remote.\n- `git.syncInterval` — How often to synchronise the git repository, in\n  milliseconds.\n- `auth.key` — Turn on authentication, if set all requests will need to have a\n  `Authorization: Bearer $KEY` header with the same value for them to be\n  accepted. Setting to empty string `\"\"` disables this\n\n### Git permissions\n\nThe user that runs the repo-api-service needs to the correct permissions to\npull/push from the the repository in question. This can be an ssh key for\nexample. If using the container, this is the `deno` user and the credentials\nshould be put into `/home/deno/.ssh` and you need to make sure they have the\ncorrect file permissions and ownership.\n\n## Deplopment\n\nEach version is released as a container on `ghcr.io`, built automatically with\nGitHub Actions. These images are currently build to `amd64` and `arm64`\narchitectures. You could use a **docker-compose.yml** file like this to run the\ncontainer:\n\n```yml\nversion: '2.4'\n\nservices:\n  repo-api:\n    image: ghcr.io/robb-j/repo-api-service:0.1.0\n    environment:\n      NO_PUSH: 'true'\n      NO_PULL: 'true'\n      REMOTE_URL: git@github.com:robb-j/r0b-blog.git\n    volumes:\n      - ./repos/blog:/app/repo\n      - ~/.ssh:/home/deno/.ssh\n    ports:\n      - 8000:8000\n```\n\nThe container runs on port `8000` by default and the repository goes into\n`/app/repo`. You can configure git access to the `deno` user by mounting\ncredentials into `/home/deno/.ssh`.\n\nMake sure any SSH credentials or files in the `.ssh` folder have the correct\nfile permissions and ownership otherwise they will be ignored. It is also useful\nto add in `.ssh/known_hosts` so that the container doesn't need to confirm the\nhosts.\n\nIf you want to use a config file, mount that at `/app/config.json` in the\ncontainer.\n\n## API Client\n\nThere is an API client at `https://esm.r0b.io/repo-api-client@0.3.0/mod.js`, you\ncan see the source code at\n[client/repo-api-client.js](./client/repo-api-client.js).\n\n```js\nimport { RepoApi } from 'https://esm.r0b.io/repo-api-client@0.3.0/mod.js'\n\nconst api = new RepoApi('https://example.com')\n\nconst indexFile = await api.queryFile('content/index.md', {\n  format: 'markdown',\n})\n\nconst csv = await api.queryCsvFile('products.csv', ['id', 'date', 'name'])\n\nconst pages = await api.queryGlob('**/*.md', { format: 'markdown' })\n\nconst usage = await api.queryCsvGlob('**/usage.csv', ['id', 'date', 'amount'])\n\nconst contents = 'hello there'\nawait api.write('hello.txt', contents, 'add hello.txt file')\n\nconst fileNames = await api.expandGlob('**/*.md')\n```\n\n## Development\n\nThere are deno tasks for local development and they are configured to run the\napp on port `9000` and have the repository in `repo`. Git pull \u0026 push are\ndisabled by default so can manually check out the repository in question or just\nwork with static files. You can set environment variables by creating a `.env`\nfile at the root of the project.\n\n```sh\ndeno task dev\n```\n\n**release process**\n\n1. Make sure the `CHANGELOG.md` and [Configuration](#configuration) is up to\n   date\n2. Bump the version in `app.json` and `client/mod.js`\n3. Commit the change as `vX.Y.Z`\n4. Tag the commit as `vX.Y.Z`\n5. Push the commit \u0026 tag and it'll build the container.\n6. Publish the latest version of the client JavaScript by uploading to the S3\n   bucket with public-read and the current version in the comment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobb-j%2Frepo-api-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobb-j%2Frepo-api-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobb-j%2Frepo-api-service/lists"}