{"id":13616840,"url":"https://github.com/scarf-sh/gateway","last_synced_at":"2026-02-01T06:05:55.211Z","repository":{"id":152346183,"uuid":"587801518","full_name":"scarf-sh/gateway","owner":"scarf-sh","description":"Scarf Gateway is a universal redirect layer for any digital artifact or URL, anywhere online. Scarf Gateway acts much like a customizable link shortener that also lets you serve software like Docker containers, Python packages, or anything other kind of software you distribute. ","archived":false,"fork":false,"pushed_at":"2025-02-26T11:47:30.000Z","size":456,"stargazers_count":83,"open_issues_count":3,"forks_count":3,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-14T03:38:14.188Z","etag":null,"topics":["analytics","containers","package-registry","proxy","redirect","registry"],"latest_commit_sha":null,"homepage":"https://scarf.sh","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scarf-sh.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-01-11T16:10:11.000Z","updated_at":"2025-02-28T04:30:16.000Z","dependencies_parsed_at":"2025-01-14T12:39:39.058Z","dependency_job_id":"bf559f95-9af8-41b0-aeb9-15a393fd5235","html_url":"https://github.com/scarf-sh/gateway","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/scarf-sh/gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scarf-sh%2Fgateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scarf-sh%2Fgateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scarf-sh%2Fgateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scarf-sh%2Fgateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scarf-sh","download_url":"https://codeload.github.com/scarf-sh/gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scarf-sh%2Fgateway/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28970194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T05:48:53.985Z","status":"ssl_error","status_checked_at":"2026-02-01T05:47:55.855Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","containers","package-registry","proxy","redirect","registry"],"created_at":"2024-08-01T20:01:33.967Z","updated_at":"2026-02-01T06:05:55.195Z","avatar_url":"https://github.com/scarf-sh.png","language":"Haskell","funding_links":[],"categories":["Haskell"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cimg src=\"/assets/gateway.svg\" width=\"300\" /\u003e\n  \u003cimg referrerpolicy=\"no-referrer-when-downgrade\" src=\"https://static.scarf.sh/a.png?x-pxid=55057c42-7e5c-4f06-b3c5-8745e7e0a06f\" /\u003e\n\u003c/h1\u003e\n\nScarf Gateway is a universal redirect layer for any digital artifact or URL, anywhere online. Scarf Gateway acts much like a customizable link shortener that also lets you serve software like Docker containers, Python packages, or anything other kind of software you distribute. Host your Docker containers, NPM packages, PyPI packages, your binaries, and more, all from the same place. Scarf Gateway also makes it easy to switch between different registries without disrupting your public endpoints, allowing for a more flexible package distribution system that is decoupled from where your artifacts are hosted.\n\nWhen requests are made to Scarf Gateway, the traffic is automatically redirected to the configured location, while ensuring seamless compatibility with popular package managers, container runtimes, and HTTP clients.\n\n```bash\n# Serve container pulls from Docker Hub, GHCR, etc via your domain\ndocker pull yourdomain.com/your-namespace/your-image\n# Redirect Python package downloads PyPI, etc via your domain\npip install --extra-index-url yourdomain.com your-python-package\n# Redirect HTTP requests, file downloads, etc to anywhere through your domain\nwget yourdomain.com/linux/v1.2.0/your-tarball.tgz\n```\n\nScarf Gateway is a core service that [Scarf](https://scarf.sh) runs internally. Scarf hosts a robust, globally distributed Gateway service, and augments it with extra functionality including:\n  - A web dashboard for managing your package configuration\n  - Robust analytics, data visualization, export, etc.\n\nScarf Gateway natively supports Docker/OCI Containers, Python packages, NPM packages, and any other file, URL, or URL template.\n\n## Building the project\n\nMake sure you have configured:\n  - Haskell GHC 9.4.4\n  - Cabal 3.8.1.0\n\nRun:\n\n```\ncabal install\n```\n\nAlternatively, this project uses [Nix Flakes](https://nixos.wiki/wiki/Flakes). The Gateway is set as the default package. So to build:\n\n```\nnix build\n```\n\n## Getting Started\n\nTo use Scarf Gateway, you should first create a manifest with a set of redirection rules. For example:\n\nmy-manifest.json\n```\n{\n  \"rules\": [\n    {\n      \"repository-name\": \"library/hello-world\",\n      \"package-id\": \"aaf2ec15-5244-484b-845a-ffd559e5f802\",\n      \"domain\": \"testorg.docker.scarf.sh\",\n      \"registry\": \"registry-1.docker.io\",\n      \"type\": \"docker-v1\"\n     }\n  ]\n}\n```\n\nThen, just run the gateway using the manifest above:\n\n```\nscarf-gateway --manifest ./my-manifest.json\n```\n\nNow gateway is running at port 8081. You can pull docker images with `docker pull localhost:8081/hello-world`\n\n## Gateway Rules\n\nThere are different manifest rules for docker, file, and python packages. The rule is composed by some basic properties:\n- `type` - It can be `docker-v1` for docker images, `python-v1` for python packages, and `file-v1` for any kind of files or executable.\n- `package-id` - A unique identifier for the package. It can be any string from your system.\n- `domain` - A custom domain through where the package is served. Domains can be different for each file. E.g. `my-domain.com`\n\n### Docker Rules\n\nFor Docker Packages (`docker-v1`) some additional properties are required:\n\n- `repository-name` - The image name used by the docker pull command. E.g. `library/hello-world`, `library/nginx`\n- `backend-registry` - The backend registry domain. The most common are `registry-1.docker.io` for Docker Hub, and `ghcr.io` for GitHub.\n\nExample of Docker rule:\n\n```\n{\n  \"type\": \"docker-v1\",\n  \"package-id\": \"6d3cda25-4ffc-461f-a1b8-72abf9ab3fc1\",\n  \"domain\": \"registry.helloworlders\",\n  \"repository-name\": \"library/hello-world\", \n  \"registry\": \"registry-1.docker.io\"\n}\n```\n\n### File Rules\n\nFor File Packages (`file-v1`) some additional properties are required:\n\n- `incoming-path` - The URL path to download the file. E.g. `/minikube-{platform}-{version}.tar.gz`\n- `outgoing-url` - The URL to redirect the file. E.g. `https://github.com/kubernetes/minikube/releases/downloads/minikube-{platform}-{version}.tar.gz`\n\nExample of File rule:\n\n```\n{\n  \"type\": \"file-v1\",\n  \"package-id\": \"a45901e6-8cad-4a27-8776-2349212c6a0d\",\n  \"domain\": \"avi.gateway.scarf.sh\",\n  \"incoming-path\": \"/{identifer}@{version}\",\n  \"outgoing-url\": \"https://deno.land/x/{identifer}@{version}\"\n}\n```\n    \n### Python Rules\n\nFor Python Packages (`python-v1`) some additional properties are required:\n\n- `file-name` - The full name of the file. See examples below.\n- `package-name` - The package name.\n- `version` - The version of the package. \n- `hash-value` - The hash value. E.g (md5, 123...). \n- `gpg-sig` - A value of either true or false to indicate whether or not there is a GPG signature.\n- `requires-python` - The Python version(s) that the distribution is guaranteed to be compatible with. The value follows the PEP 345 spec.\n- `backend-url` - The backend URL to download the package.\n- `backend-simple-index`- The manifest backend index. If not present, assume https://pypi.org/simple/\n\nExample of Python rule\n\n```\n{\n  \"type\": \"python-v1\",\n  \"package-id\": \"013e1ebd-3bc0-411d-b971-6ff1c20267c3\",\n  \"domain\": \"fabioluz.gateway.scarf53.sh\",\n  \"file-name\": \"numpy-1.6.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl\",\n  \"package-name\": \"numpy\",\n  \"version\": \"1.6.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl\",\n  \"hash-value\": {\n      \"hash\": \"5bd0a2a68903f1b286dd646f42f92f7de7bde6bbf3c4829a3f078400f48fa1e7\",\n      \"type\": \"sha256\"\n  },\n  \"backend-url\": \"https://files.pythonhosted.org/packages/a6/b9/a9e4411c08a568a9558e4d4efc15cd26cf9f2f84e4d7ea800742fedb858c/numpy-1.6.1-cp27-none-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl#sha256=5bd0a2a68903f1b286dd646f42f92f7de7bde6bbf3c4829a3f078400f48fa1e7\"\n  \"backend-simple-index\": \"https://pypi.org/simple/\"\n}\n```\n\nFor further details, please check [Scarf.Gateway.Manifest](/src/Scarf/Gateway/Manifest.hs)\n\n## Caveats\n\n\n### Bandwidth usage: redirect vs proxy\n\nThe gateway redirects whenever possible. Some versions of some container runtimes, however, do not properly authenticate with container registries when the client is redirected , and therefore the gateway is forced to proxy the requests to serve the download successfully. When Scarf Gateway must proxy, be prepared to pay for the bandwidth bill, as the container images themselves will pass through your infrastructure.\n\n## Logging\n\nRequests will be logged to stdout. Request details are in JSON format to be easily processed for any kind of analytics you might want to do. To learn more about getting advanced insights from Scarf Gateway traffic, check out the free and paid features of the entire [Scarf](https://scarf.sh) platform.\n\n## Community\n\n### Code of conduct\n\nThis project is for everyone. We ask that our users and contributors take a few minutes to review our [Code of Conduct](https://github.com/scarf-sh/code-of-conduct).\n\n### Communication\n\n* [Issues](https://github.com/scarf-sh/gateway/issues) can be used for open discussions. If you know someone who should hear about the message, tag them explicitly using the @username GitHub syntax.\n\n* We also have a [slack server](https://join.slack.com/t/scarf-community/shared_invite/zt-1q9vpx13r-H9fy07psWSwM4SGF~vEsJA) that anyone can join where we can discuss about the gateway or all matters open source!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscarf-sh%2Fgateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscarf-sh%2Fgateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscarf-sh%2Fgateway/lists"}