{"id":20237154,"url":"https://github.com/blinkk/fileset","last_synced_at":"2026-01-31T08:08:17.618Z","repository":{"id":42512116,"uuid":"318244338","full_name":"blinkk/fileset","owner":"blinkk","description":"A high-performance TypeScript static server for high-traffic sites. Features include preview branches, redirects, localization-awareness, and atomic deployments.","archived":false,"fork":false,"pushed_at":"2024-01-08T18:42:59.000Z","size":2350,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-17T12:51:48.629Z","etag":null,"topics":["gcs","google-app-engine","google-cloud-storage","static","typescript"],"latest_commit_sha":null,"homepage":"https://blinkk.github.io/fileset/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blinkk.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-03T15:54:14.000Z","updated_at":"2022-01-10T20:43:16.000Z","dependencies_parsed_at":"2024-01-06T01:26:14.185Z","dependency_job_id":"c935c6ff-0b8a-4056-8375-c0754d0eb284","html_url":"https://github.com/blinkk/fileset","commit_stats":null,"previous_names":["blinkkcode/fileset"],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blinkk%2Ffileset","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blinkk%2Ffileset/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blinkk%2Ffileset/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blinkk%2Ffileset/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blinkk","download_url":"https://codeload.github.com/blinkk/fileset/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241681466,"owners_count":20002316,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gcs","google-app-engine","google-cloud-storage","static","typescript"],"created_at":"2024-11-14T08:25:12.098Z","updated_at":"2026-01-31T08:08:17.613Z","avatar_url":"https://github.com/blinkk.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fileset\n\n[![NPM Version][npm-image]][npm-url]\n[![GitHub Actions][github-image]][github-url]\n[![Known Vulnerabilities][snyk-image]][snyk-url]\n[![codecov][codecov-image]][codecov-url]\n[![TypeScript Style Guide][gts-image]][gts-url]\n\nFileset is a light, high-performance TypeScript static web server intended for\nhigh-traffic static sites. Features include:\n\n- Atomic deployments\n- (To be implemented) Scheduled deployments\n- Redirects\n- Localization-aware redirects\n- Isolated per-branch staging environments backed by Google Account authentication\n- Ultra-fast TTFB and payload transfer speeds\n- A simple web UI for inspecting uploaded files\n\nThe server runs on Google App Engine and proxies requests to Google Cloud Storage.\n\n## Concept\n\nMany websites can be built and deployed as fully static content (i.e. just HTML,\nCSS, and JavaScript – with no dynamic backend). Despite being static content,\nwebsites may still have other requirements such as localization, redirects, or\natomic deployments that end up adding slight dynamic functionality to an\notherwise fully static project.\n\nFileset aims to bridge the gap by offering a thin middle layer on top of Google\nCloud Storage: static files are uploaded to Google Cloud Storage via the CLI,\nwith the Fileset server handling request traffic. The server determines whether\nto issue a response sprinkled with one of the above dynamic features, and\notherwise proxies traffic directly to Google Cloud Storage – leveraging Google's\nglobal network for performance.\n\n## Table of contents\n\n- [Concept](#concept)\n- [Usage](#usage)\n  - [Server setup](#server-setup)\n  - [Upload files](#upload-files)\n- [Uploader authentication](#uploader-authentication)\n  - [Local testing (authenticate using user account)](#local-testing-authenticate-using-user-account)\n  - [Continuous deployment (authenticate using service account)](#continuous-deployment-authenticate-using-service-account)\n- [Environments](#environments)\n- [Testing](#testing)\n  - [Response headers](#response-headers)\n  - [Query parameters](#query-parameters)\n- [Tips](#tips)\n  - [Usage within Makefile](#usage-within-makefile)\n  - [Usage with Amagaki](#usage-with-amagaki)\n  - [Usage with Grow.dev](#usage-with-growdev)\n\n## Usage\n\nThere are two main tasks required in order to use Fileset:\n\n1. First, you'll need to deploy the server. This only needs to be done once,\n   when the project is originally set up.\n2. Second, you'll need to upload files to be served. Files must be uploaded each\n   time you want to serve new files, update redirects, etc.\n\n### Server setup\n\n1. Within your project, create a directory to house the server configuration, e.g.\n   `./backend/server`.\n\n2. Copy the files from [./example/server/](./example/server/) into this directory.\n\n3. Modify the settings in `app.yaml` and `secrets.yaml`.\n\n4. Setup and deploy the app using the provided\n   [Makefile](./example/server/Makefile). The app will be deployed to an App\n   Engine service named `fileset`, so it will not conflict with your current\n   deployment.\n\n```bash\nmake project=\u003cAppId\u003e setup\nmake project=\u003cAppId\u003e deploy\n```\n\n### Upload files\n\n1. **Create a `fileset.yaml` configuration file.** The minimum example is below. See\n   the [example fileset.yaml](./example/fileset.yaml) for full configuration\n   options.\n\n```yaml\ngoogle_cloud_project: \u003cAppId\u003e\n```\n\n2. **Generate your files.** Use a static site generator or just manually create a directory containing files\n   to upload. In step (3) below, the files in the directory `./build` are\n   uploaded.\n\n3. **Upload your files.** The uploader will look for `fileset.yaml` within the\n   specified directory first. If it's not found, it will look up in the parent\n   folder. If the config file doesn't exist in either folder, the uploader will\n   abort.\n\n```bash\nnpx fileset upload ./build\n```\n\nThat's it! Files have been uploaded to Google Cloud Storage and the uploaded\ndirectory is now being served by the application server. You can verify by\nvisiting:\n\n```\nhttps://fileset-dot-\u003cAppId\u003e.appspot.com\n```\n\n## Uploader authentication\n\nYou'll need to be authenticated to upload files and deploy the serving manifests.\n\n### Local testing (authenticate using user account)\n\nIf you are testing locally, your user account can be used to authenticate to\nCloud Datastore and Cloud Storage. Simply run the below command to create\ncredentials used for authentication:\n\n```bash\ngcloud auth application-default login\n```\n\n### Continuous deployment (authenticate using service account)\n\nIf you are using a service account for deployment, you'll need to ensure it has\nthe right permissions. When using Fileset with Google Cloud Build, simply run\n`make setup` from the `example/server` directory to configure your project's\nCloud Build Service account with the right permissions.\n\nThe following permissions are needed:\n\n- Cloud Datastore (manifests are stored here): Cloud Datastore Owner\n  (`datastore.owner`)\n- Cloud Storage (files are uploaded here): Storage Object Admin\n  (`storage.objectAdmin`)\n\nIf using the Cloud Build service account (or any other service account), you'll\nneed to add the above two permissions to the account. That can be done via the\nIAM page (`https://console.cloud.google.com/access/iam?project=\u003cAppId\u003e`) or via\nthe `gcloud` CLI.\n\nThe provided [Makefile](./example/server/Makefile) also sets up Google Cloud\nBuild permissions for you.\n\n```bash\nmake project=\u003cAppId\u003e setup\n```\n\n## Environments\n\nFileset uses Git branches to determine whether files should be in production\n(and public) or in staging (and restricted via Google Account authentication).\nThe Git branch is determined by inspecting the local Git environment when the\n`upload` command is invoked.\n\nThe best way to understand how this works is by following the examples below:\n\n```bash\n# main branch\n# ✓ public\n# ✓ production URL\n# ✓ also available from staging URLs (restricted)\n\n(main) $ npx fileset upload build\n...\nPublic URL:            https://appid.appspot.com\nURL (via commit):      https://f3a9abb-dot-fileset-dot-appid.appspot.com\nURL (via branch name): https://master-dot-fileset-dot-appid.appspot.com\n```\n\n```bash\n# staging branch\n# ✓ not public\n# ✓ staging URL only (restricted)\n\n(staging) $ npx fileset upload build\n...\nURL (via commit):      https://4fb48ce-dot-fileset-dot-appid.appspot.com\nURL (via branch name): https://staging-dot-fileset-dot-appid.appspot.com\n```\n\n## Testing\n\n### Response headers\n\nYou can verify Fileset server is working as you expect by looking for the following headers:\n\n| Header           | Description                                                                                                                                        |\n| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `x‑fileset‑site` | The site being served. Usually this will be `default` but for multi-site installations, this will be useful for determining which site is serving. |\n| `x‑fileset‑ref`  | The Git commit sha that corresponds to the serving manifest that is handling your request.                                                         |\n| `x‑fileset‑blob` | The blob directory key corresponding to the file being served. This is the SHA-1 hash of the file's content.                                       |\n| `x‑fileset‑ttl`  | For scheduled deployments, the value of this header will correspond to the timestamp for the timed deployment being served.                        |\n\n### Query parameters\n\nYou can simulate geolocation behavior using query parameters:\n\n| Parameter | Name                | Description                                          |\n| --------- | ------------------- | ---------------------------------------------------- |\n| `hl`      | Language            | Overrides the incoming `accept-language` header.     |\n| `gl`      | Geolocation         | Overrides the incoming `x-appengine-country` header. |\n| `ncr`     | No country redirect | Disables localization-aware redirects.               |\n\n## Tips\n\n### Usage within Makefile\n\nThe absolute path to the `fileset` executable can be specified to invoke the CLI.\n\n```\n./node_modules/.bin/fileset upload build\n```\n\n### Usage with Amagaki\n\nFirst, build the site to the `./build` directory. Then, upload the directory to Fileset.\n\n```shell\nnpx @blinkk/amagaki build\nnpx @blinkk/fileset upload build\n```\n\n### Usage with Grow.dev\n\nFirst, build the site to the `./build` directory. Then, upload the directory to Fileset.\n\n```shell\ngrow build --deployment=prod\nnpx @blinkk/fileset upload build\n```\n\n[github-image]: https://github.com/blinkk/fileset/workflows/Run%20tests/badge.svg\n[github-url]: https://github.com/blinkk/fileset/actions\n[codecov-image]: https://codecov.io/gh/blinkk/fileset/branch/main/graph/badge.svg\n[codecov-url]: https://codecov.io/gh/blinkk/fileset\n[gts-image]: https://img.shields.io/badge/code%20style-google-blueviolet.svg\n[gts-url]: https://github.com/google/gts\n[npm-image]: https://img.shields.io/npm/v/@blinkk/fileset.svg\n[npm-url]: https://npmjs.org/package/@blinkk/fileset\n[snyk-image]: https://snyk.io/test/github/blinkk/fileset/badge.svg\n[snyk-url]: https://snyk.io/test/github/blinkk/fileset\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblinkk%2Ffileset","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblinkk%2Ffileset","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblinkk%2Ffileset/lists"}