{"id":13723295,"url":"https://github.com/google/webpackager","last_synced_at":"2025-05-07T16:32:43.198Z","repository":{"id":38298146,"uuid":"204401927","full_name":"google/webpackager","owner":"google","description":null,"archived":false,"fork":false,"pushed_at":"2024-12-11T22:13:51.000Z","size":5303,"stargazers_count":75,"open_issues_count":10,"forks_count":20,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-19T22:27:45.034Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"docs/supported_media.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-26T05:26:03.000Z","updated_at":"2025-03-08T06:29:54.000Z","dependencies_parsed_at":"2024-06-18T22:38:45.592Z","dependency_job_id":"11564bc4-8684-4e1c-a06a-8f00495aa2e7","html_url":"https://github.com/google/webpackager","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fwebpackager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fwebpackager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fwebpackager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fwebpackager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google","download_url":"https://codeload.github.com/google/webpackager/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252915511,"owners_count":21824582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T01:01:38.797Z","updated_at":"2025-05-07T16:32:38.182Z","avatar_url":"https://github.com/google.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# Web Packager\n\n[![Build Status](https://travis-ci.org/google/webpackager.svg?branch=main)](https://travis-ci.org/google/webpackager)\n[![GoDoc](https://godoc.org/github.com/google/webpackager?status.svg)](https://godoc.org/github.com/google/webpackager)\n\nWeb Packager is a command-line tool to \"package\" websites in accordance\nwith the specifications proposed at [WICG/webpackage][]. It may look like\n[gen-signedexchange][], but is rather *based on* gen-signedexchange and\nfocuses on automating generation of Signed HTTP Exchanges (aka. SXGs) and\noptimizing the page loading.\n\n[WICG/webpackage]: https://github.com/WICG/webpackage/\n[gen-signedexchange]: https://github.com/WICG/webpackage/tree/main/go/signedexchange\n\nWeb Packager HTTP Server is an HTTP server built on top of Web Packager.\nIt functions like a reverse-proxy, receiving signing requests over HTTP.\nFor more detail, see [cmd/webpkgserver/README.md](cmd/webpkgserver/README.md).\nThis README focuses on the command-line tool.\n\nWeb Packager retrieves HTTP responses from servers and turns them into\nsigned exchanges. Those signed exchanges are written into files in a way to\npreserve the URL path structure, so can be deployed easily in some typical\ncases. In addition, Web Packager applies some optimizations to the signed\nexchanges to help the content get rendered quicker.\n\nWeb Packager is purposed primarily for a showcase of how to speed up the\npage loading with [privacy-preserving prefetch][]. Web developers may port\nthe logic from this codebase to their systems or integrate Web Packager into\ntheir systems. The Web Packager's code is designed to allow some injections\nof custom logic; see the GoDoc comments for details. Note, however, that Web\nPackager is currently at an early stage now: see Limitations below.\n\n[privacy-preserving prefetch]: https://wicg.github.io/webpackage/draft-yasskin-webpackage-use-cases.html#private-prefetch\n\nWeb Packager is *not* related to [webpack][].\n\n[webpack]: https://webpack.js.org/\n\n\n## Prerequisite\n\nWeb Packager is written in the Go language thus requires a Go system to run.\nSee [Getting Started on golang.org](https://golang.org/doc/install) for how\nto install Go on your computer.\n\nYou will also need a certificate and private key pair to use for the signing\nthe exchanges. Note the certificate must:\n\n*   use an ECDSA private key (e.g. prime256v1) and\n*   have [CanSignHttpExchanges extension][].\n\n(For example, [DigiCert][] offers the right kind of certificates.)\n\n[CanSignHttpExchanges extension]: https://wicg.github.io/webpackage/draft-yasskin-http-origin-signed-responses.html#cross-origin-cert-req\n[DigiCert]: https://www.digicert.com/account/ietf/http-signed-exchange.php\n\nThen you will need to convert your certificate into\nthe [application/cert-chain+cbor][] format, which you can do using the\ninstructions at:\n\n*   [Creating our first signed exchange](https://github.com/WICG/webpackage/blob/main/go/signedexchange/README.md#creating-our-first-signed-exchange)\n    to generate a self-signed certificate for testing.\n*   [Creating a signed exchange using a trusted certificate](https://github.com/WICG/webpackage/blob/main/go/signedexchange/README.md#creating-a-signed-exchange-using-a-trusted-certificate)\n    to use a CA-issued certificate.\n\n[application/cert-chain+cbor]: https://wicg.github.io/webpackage/draft-yasskin-http-origin-signed-responses.html#cert-chain-format\n\n\n## Limitations\n\nIn this early phase, we may make backward-breaking changes to the commandline\nor API.\n\nWeb Packager aims to automatically meet most but not all [Google SXG Cache\nrequirements](docs/cache_requirements.md). In particular, pages that do not use\nresponsive design should specify a [`supported-media`\nannotation](docs/supported_media.md).\n\nWeb Packager does not handle [request matching][] correctly. It should not\nmatter unless your web server implements content negotiation using the\n`Variants` and `Variant-Key` headers (*not* the `Vary` header). We plan to\nsupport the request matching in future, but there is no ETA (estimated time of\navailability) at this moment.\n\n**Note:** The above limitation is not expected to be a big deal even if your\n    server serves signed exchanges conditionally using content negotiation:\n    if you already have signed exchanges, you should not need Web Packager.\n\n[request matching]: https://wicg.github.io/webpackage/loading.html#request-matching\n\n\n## Install\n\n```shell\ngo get -u github.com/google/webpackager/cmd/...\n```\n\n\n## Usage\n\nThe simplest command looks like:\n\n```shell\nwebpackager \\\n    --cert_cbor=cert.cbor \\\n    --private_key=priv.key \\\n    --cert_url=https://example.com/cert.cbor \\\n    --url=https://example.com/hello.html\n```\n\nIt will retrieve an HTTP response from https://example.com/, generate\na signed exchange with the given pair of certificate (`cert.cbor`) and\nprivate key (`priv.key`), then write it to `./sxg/hello.html.sxg`.\nIf `hello.html` had subresources that could be preloaded together,\n`webpackager` would also retrieve those resources and generate their signed\nexchanges under `./sxg`. Web Packager recognizes `\u003clink rel=\"preload\"\u003e`\nand equivalent `Link` HTTP headers. It also adds the preload links for CSS\n(stylesheets) used in HTML, and may use more heuristics in future. See the\ndefaultproc package to find how exactly the HTTP response is processed.\n\n`--cert_url` specifies where the client will expect to find the CBOR-format\ncertificate chain. `--cert_cbor` is optional when it can be fetched from\n`--cert_url`. Note the reverse is not true: `--cert_url` is always required.\n\nThe `--url` flag can be repeated as many times as you want. For example:\n\n```shell\nwebpackager \\\n    --cert_cbor=cert.cbor \\\n    --private_key=priv.key \\\n    --cert_url=https://example.com/cert.cbor \\\n    --url=https://example.com/foo/ \\\n    --url=https://example.com/bar/ \\\n    --url=https://example.com/baz/\n```\n\nwould generate the following three files:\n\n*   `./sxg/foo/index.html.sxg` for `https://example.com/foo/`\n*   `./sxg/bar/index.html.sxg` for `https://example.com/bar/`\n*   `./sxg/baz/index.html.sxg` for `https://example.com/baz/`\n\n**Note:** `webpackager` expects all target URLs to have the same origin.\n    In particular, the output files collide if you specify more than one URL\n    that has the same path but a different domain.\n\n### Using URL File\n\n`webpackage` also accepts `--url_file=FILE`. `FILE` is a plain text file\nwith one URL on each line. For example, you could create `urls.txt` with:\n\n```\n# This is a comment.\nhttps://example.com/foo/\nhttps://example.com/bar/\nhttps://example.com/baz/\n```\n\nthen run:\n\n```\nwebpackager \\\n    --cert_cbor=cert.cbor \\\n    --private_key=priv.key \\\n    --cert_url=https://example.com/cert.cbor \\\n    --url_file=urls.txt\n```\n\n### Changing Output Directory\n\nYou can change the output directory with the `--sxg_dir` flag:\n\n```shell\nwebpackager \\\n    --cert_cbor=cert.cbor \\\n    --private_key=priv.key \\\n    --cert_url=https://example.com/cert.cbor \\\n    --sxg_dir=/tmp/sxg \\\n    --url=https://example.com/hello.html\n```\n\n### Setting Expiration\n\nThe signed exchanges last one hour by default. You can change the duration\nwith the `--expiry` flag. For example:\n\n```shell\nwebpackager \\\n    --cert_cbor=cert.cbor \\\n    --private_key=priv.key \\\n    --cert_url=https://example.com/cert.cbor \\\n    --expiry=72h \\\n    --url=https://example.com/hello.html\n```\n\nwould make the signed exchanges valid for 72 hours (3 days). The maximum\nis `168h` (7 days), due to the specification.\n\n### Other Flags\n\n`webpackager` provides more flags for advanced usage (e.g. to set request\nheaders). Run the tool with `--help` to see those flags.\n\n\n## Appendix: Deploying SXGs\n\nThe steps below illustrate an example of deploying Signed HTTP Exchanges on\nan Apache server.\n\n1.  Upload `cert.cbor` to your server. Make it available at `--cert_url`.\n\n2.  Upload `*.sxg` files to your server. Put them next to the original files\n    (e.g. `hello.html.sxg` should stay in the same directory as `hello.html`).\n    For example, if you are using the `sftp` command to upload, you can:\n\n    ```\n    sftp\u003e cd public_html\n    sftp\u003e put -r sxg/*\n    ```\n\n    assuming `public_html` to be the document root and `sxg` to be where you\n    generated the `*.sxg` files.\n\n3.  Edit or create `.htaccess` in `public_html` (or the Apache's config file)\n    to add the following settings:\n\n    ```\n    AddType application/signed-exchange;v=b3 .sxg\n\n    \u003cFiles \"cert.cbor\"\u003e\n      AddType application/cert-chain+cbor .cbor\n    \u003c/Files\u003e\n\n    RewriteEngine On\n    RewriteCond %{HTTP:Accept} application/signed-exchange\n    RewriteCond %{REQUEST_FILENAME} !\\.sxg$\n    RewriteCond %{REQUEST_FILENAME}\\.sxg -s\n    RewriteRule .+ %{REQUEST_URI}.sxg [L]\n\n    Header set X-Content-Type-Options: \"nosniff\"\n    ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fwebpackager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle%2Fwebpackager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fwebpackager/lists"}