{"id":19645240,"url":"https://github.com/racket/racket-pkg-website","last_synced_at":"2026-03-08T12:37:36.385Z","repository":{"id":25443140,"uuid":"28873051","full_name":"racket/racket-pkg-website","owner":"racket","description":"A frontend for the Racket Package Catalog.","archived":false,"fork":false,"pushed_at":"2024-10-11T16:05:45.000Z","size":1120,"stargazers_count":11,"open_issues_count":33,"forks_count":16,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-01-09T20:45:28.539Z","etag":null,"topics":["racket"],"latest_commit_sha":null,"homepage":null,"language":"Racket","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/racket.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING_LESSER.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-01-06T16:53:32.000Z","updated_at":"2024-10-11T16:05:49.000Z","dependencies_parsed_at":"2024-09-14T13:16:30.794Z","dependency_job_id":null,"html_url":"https://github.com/racket/racket-pkg-website","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/racket%2Fracket-pkg-website","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/racket%2Fracket-pkg-website/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/racket%2Fracket-pkg-website/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/racket%2Fracket-pkg-website/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/racket","download_url":"https://codeload.github.com/racket/racket-pkg-website/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240952961,"owners_count":19884019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["racket"],"created_at":"2024-11-11T14:32:57.112Z","updated_at":"2026-03-08T12:37:36.373Z","avatar_url":"https://github.com/racket.png","language":"Racket","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Racket Package Catalog Server\n\nThe Racket Package Catalog comprises three pieces of software that work\nin tandem:\n\n - **pkg-index**, a.k.a. the **backend**: responsible for managing the\n   package database and user database, and periodically polls package\n   sources to update checksums. Eventually, all Racket package clients\n   can poll the package server, instead of checking each package\n   individually, but package clients do not access the backend's\n   information directly.\n\n - **website**, a.k.a. the **frontend**: responsible for formatting\n   the package database and rendering it as a web site, and also for\n   publishing the current package database (as determined by the\n   backend) to the site that package clients consult.\n\nBoth frontend and backend run in the same Racket process, but as\nseparate server threads. Each also has additional threads for periodic\nand internal tasks. The frontend sends package-change requests to the\nbackend, and it watches for periodic updates (especially new\nchecksums) from the pkg-index backend. The servers were originally\nimplemented separately, and there is some value to keeping them\nconceptually separate. There's also a backup thread that periodically\nuploads package and user information to an S3 bucket.\n\nAlthough the implementation here is not necessarily tied to the main\nRacket deployment's configuation, various configuration options make\nthe most sense in terms of the main deployment's structure:\n\n - `https://pkgs.racket-lang.org/` is an S3-hosted web site, which\n   makes it as available as possible. The content of this site is\n   uploaded is generated by the frontend server. (Also,\n   `pkg.racket-lang.org` is set up forward to `pkgs.racket-lang.org`,\n   in case someone forgets the \"s\" in \"pkgs\".)\n\n   The frontend server will tend to forward back into this static\n   content, but the \"login\" button in the static view goes to the\n   dynamic frontend server. After a user has logged in, the dynamic\n   server tends to serve information directly, instead of forwarding.\n   So, the server needs to be up and working well for logged-in use\n   or gathering package updates, but not for querying the most\n   recently published package updates.\n\n - `https://pkgd.racket-lang.org/` is the server frontend and backend\n   implemented here. More precisely, it's an Apache instance that\n   sends most URLs to the frontend server, but anything path that\n   starts `api` or `jsonp` is sent to the backend server (so that the\n   backend functionality remains accessible). But to the degree that\n   the frontend server needs to talk to the backend server, it does so\n   more directly.\n\n## Prerequisites\n\nIn addition to Racket v8.17 or later, you will need to install the\nfollowing Racket packages:\n\n    raco pkg install --skip-installed \\\n         'https://github.com/racket/infrastructure-userdb.git#main' \\\n         reloadable \\\n         aws \\\n         s3-sync \\\n         plt-service-monitor\n\n## Configuration\n\nYou can run the server in test mode with `make run` or `./run` or\n`racket src/main.rkt` with no further configuration, except that\n\n - You'll need to have a certificate in place. Use `make keys` to\n   produce a self-signed certificate. The certificate and key are\n   dropped into `compiled/root` so that they're in the right place for\n   a default configuration.\n\n - Probably you'll want to seed the set of registered packages.\n   See \"Adding packages\" below.\n\nBy default, all package state and generated files go into `compiled`\nin the current directory. (The current directory when you run the\nserver doesn't have to be the top of this Git repo checkout.)\n\nAn advantage of using `make run` or `./run` is that it sets\n`PLTSTDERR` to turn on lots of logging. You can run just the frontend\nas `src/website/main.rkt` or just the backend with\n`src/pkg-index/main.rkt`. Running just the frontend requires running\nthe backend at least once to generate its output for the frontend,\nthough, or configuring the frontend to use another source via\n*pkg-index-url* as described below.\n\nWhen you use `make run` or `./run`, it actually runs\n`configs/${CONFIG}.rkt`, so you can set `CONFIG` as an environment\nvariable or makefile variable to pick a configuration there. If\n`CONFIG` is not defined, `testing` is used (which is an empty\nconfiguration). For example, to select `configs/live.rkt`, set\n`CONFIG` to `live`. A good place to do this is in the `run-prelude`\nfile; see the description of `run-prelude` below.\n\nWithin a configuration file, configuration details are to be given as\na hashtable to `main`. Whenusing the testing configuration of `./run`\nor when using `racket src/main.rkt`, you can supply a `--config`\nargument to specify a module that exports a `config` hashtable.\n\nKeys useful for deployment:\n\n - *port*: number; defaults to the value of the `SITE_PORT` environment\n   variable, if defined; otherwise, 7443.\n - *pkg-index-port*: number; defaults to the value of the `SITE_PKG_INDEX_PORT`\n   environment variable, if defined; otherwise, 9004.\n - *ssl?*: boolean; default is `#t`, unless `PKG_SERVER_HTTP` is defined.\n - *reloadable?*: boolean; default is `#t` if the `SITE_RELOADABLE` environment\n   variable is defined; otherwise, `#f`.\n - *recent-seconds*: number, in seconds; default is 172800. Packages\n   modified fewer than this many seconds ago are considered \"recent\",\n   and displayed as such in the UI.\n - *static-output-type*: either `'aws-s3` or `'file`, indicates where the\n   frontend write the static-site data:\n    - When `'file` (the default),\n\t   - *static-content-target-directory*: either `#f` or a string\n\t\t denoting a path to a folder to which the static content of\n\t\t the site will be copied for local serving.\n    - When `'aws-s3`,\n       - *aws-s3-bucket+path*: a string naming an S3 bucket and path.\n         Must end with a forward slash, `.../`. AWS access keys are\n         loaded per the documentation for the `aws` module; usually\n         from a file `~/.aws-keys`.\n - *dynamic-urlprefix*: string; absolute or relative URL, prepended to\n   URLs targetting dynamic content on the site, i.e., for when the\n   frontend wants to serve a link back to itself.\n - *static-urlprefix*: string; absolute or relative URL, prepended to\n   relative URLs referring to static HTML files placed in\n   *static-generated-directory* for when the dynamic server wants\n   to refer to static content.\n - *pkg-index-generated-directory*: a string pointing to where the\n   backend places its redered files; the main rendered file\n   that the frontend cares about is `pkgs-all.json.gz`, although\n   the backend write a whole package catalog and web site there.\n - *user-directory*: directory containing the user database.\n - *email-sender-address*: string; defaults to `pkgs@racket-lang.org`.\n   Used as the \"from\" address when sending authentication emails on\n   behalf of the server.\n - *email-transport*: `'smtp` or `'sendmail`.\n - *stmp-server*, *stmp-port*, *stmp-sending-server*, *smtp-user+password-file*:\n   SMTP relay configuration for authentication emails, defaults to\n   \"smtp-relay.gmail.com\", 465, \"racket-lang.org\", and \"~/.email\\_key\",\n   where \"~/.email\\_key\" normally has the user \"email@plt-scheme.org\"\n   and an app-specific password for that account.\n - *beat-s3-bucket*: string or #f; defaults to #f a bucket name for\n   regsitering heartbeats, or `#f` to disable heartbeats; the\n   region is determined automatically from the bucket name. \n - *beat-publish-task-name*: string; defaults to \"pkgd-publish\"; a task\n   name for heartbeats after publish information for all packages.\n\nKeys useful for development:\n\n - *package-index-url*: string; an alternative source that the frontend\n   uses to get `pkgs-all.json.gz`, such as\n   `http://pkgs.racket-lang.org/pkgs-all.json.gz` to pull from\n   the live database instead of the running backend. The\n   default is based on *pkg-index-generated-directory* unless\n   the `PACKAGE_INDEX_URL` environment variable is defined.\n - *package-fetch-interval*; number, in seconds; default is 300.\n - *session-lifetime*: number, in seconds; default is 604800.\n - *static-generated-directory*: string; names a directory\n   within which generated static HTML files are to be placed.\n   Must be writable by the user running the server.\n - *disable-cache?*: boolean; default is `#f`; a `#t` value causes the\n   frontend to always redirect to itself to serve the package dynamically,\n   instead of redirecting to generated static files.\n - *backend-baseurl*: string; defaults to a `https://localhost:`\n   followed by *pkg-index-port*; must point to the backend package\n   server API root, such that (for example) `/jsonp/authenticate`,\n   when appended to it, resolves to the authentication call.   \n - *pkg-build-baseurl*: string; defaults to\n   `http://pkg-build.racket-lang.org/`. Used to build URLs relative to\n   the package build host, such as for documentation links and build\n   reports.\n - *pkg-index*: `#f` or hash table; use `#f` to disable the backend\n   server entirely, or provide a hash table to configure the backend\n   specifically.\n - *backup*: `#f` or hash table; use `#f` to disable the backup\n   task entirely, or provide a hash table to configure the backup\n   specifically.\n\nBackend keys for a *pkg-index* configuration table within the main\nconfiguration:\n\n - *port*: number; defaults to *pkg-index-port* from the enclosing\n   configuration or `9004`; port on which the backend site will be served.\n - *ssl?* - boolean; defaults to *ssl?* from the enclosing\n   configuration or to `#t`; a true value serves HTTPS and requires\n   *root*/`server-cert.pem` and *root*/`private-key.pem`.\n - *src*: path; defaults to `src/pkg-index` relative to here\n - *static.src-path*: path; defaults to *src*`/static`, the location\n   of of (non-generated) HTML/JS/CSS files to be copied to be\n   `static-path` (see below), although these files are mostly not used\n   anymore\n - *static-path*: path; defaults to *src*`/static-gen`; staging area\n   where all static resources - both generated and non-generated - are\n   written.\n - *notice-path*: path; defaults to *static-path*`/notice.json`;\n   whenever the server has a message for site users, the message will\n   be placed in this file.\n - *root*: path; defaults to *pkg-index-generated-directory* from the\n   outer configurartion; determines several other defaults\n - *users.new-path*: path; defaults to *user-directory* from the\n   outer configuration, which defaults to\n   *pkg-index-generated-directory*`/users.new`;\n   directory in which to hold user records, one file per user\n - *cache-path*: path; defaults to *root*`/cache`; names a\n    directory where files `summary.rktd` and `summary.rktd.etag`\n    will be created.\n - *pkgs-path*: path; defaults to *root*`/pkgs`; names a directory\n   where one file of package information for each package in the\n   catalog will be stored.\n - *github-client_id* (obsolete): string or #f; defaults to the contents of the\n   file at *root*`/client_id`, if it exists; should be a Github client ID string\n   (hex; twenty characters long, i.e. 10 bytes of data, hex-encoded), used only\n   if package downloaing is forced to use the GitHub API by setting the\n   `PLT_USE_GITHUB_API` environment variable.\n - *github-client_secret* (obsolete): string or #f; defaults to the contents of the\n   file at *root*`/client_secret`, if it exists; should be a Github client secret\n   string (hex; forty characters long, i.e. 20 bytes of data, hex-encoded), used\n   only when `github-client_id` is used.\n - *s3-bucket*: string or `#f`; defaults to the contents of the\n    environment variable `S3_BUCKET`, if it is defined, `#f` otherwise;\n    AWS credentials are found by the `s3` package, typically from `~/.aws-keys`;\n    if set to `#f`, S3 synchronization will be disabled.\n - *s3-bucket-region* - string; defaults to the contents of the\n    environment variable `S3_BUCKET_REGION`, if it is defined;\n    otherwise, to `#f`; needs to be non-`#f` if *s3-bucket* is.\n - *beat-s3-bucket*: string or #f; defaults to *beat-s3-bucket* from the\n   enclosing configuration table or #f; a bucket name for\n   regsitering heartbeats, or `#f` to disable heartbeats; the\n   region is determined automatically from the bucket name. \n - *beat-update-task-name*: string; defaults to \"pkgd-update\"; a task\n   name for heartbeats after updating information for all packages.\n - *beat-upload-task-name*: string; defaults to \"pkgd-upload\"; a task\n   name for heartbeats after uploading information for all packages.\n - `beat-update-task-name` - string; defaults to \"pkgd-update\". A task\n   name for heartbeats after updating information for all packages.\n - *redirect-to-static-proc*: function from HTTP request to HTTP\n   response, which should issue a redirect pointing to a static\n   resource; defaults to a function which replaces the scheme with the\n   contents of the configuration variable *redirect-to-static-scheme*,\n   the host with *redirect-to-static-host*, and the port to\n   *redirect-to-static-port*. These, in turn, default to `\"http\"`,\n   `\"pkgs.racket-lang.org\"` and `80`, respectively.\n - *atom-self-link*: string; defaults to\n   `https://pkg.racket-lang.org/rss`; sed as the `rel=self` link in\n   the header of the generated ATOM feed.\n - *atom-link*: string; defaults to `https://pkg.racket-lang.org/`;\n   used as the default site link in the header of the generated ATOM\n   feed.\n - *atom-id*: string; defaults to `https://pkg.racket-lang.org/`;\n   used as the ATOM feed ID.\n - *atom-compute-package-url*: function from package-name symbol to\n   URL string; defaults to a function which calls `format` with the\n   package name and a format template-string from\n   `atom-package-url-format-string`, which in turn defaults to\n   `http://pkg.racket-lang.org/#[~a]`.\n\nBackend keys for a *backup* configuration table within the main\nconfiguration:\n\n - *s3-bucket*: string or #f; defaults to #f a bucket name for\n   uploads, or `#f` to disable backup uploads; the\n   region is determined automatically from the bucket name.\n - *beat-backup-task-name*: string; defaults to \"pkgd-backup\"; a task\n   name for heartbeats after uploading backup data.\n\n## Development Setup\n\n### Adding packages\n\nInstead of manually adding packages to a fresh instance of the package\nweb server, use `raco pkg catalog-copy` to copy an existing catalog into\na directory tree. Then, move the `pkg` directory (no \"s\") in the catalog\ncopy to be *root*`/pkgs` (with \"s\") where *root* is the server's root\ndirectory — so, \"compiled/root/pkgs\" by default.\n\n```\n  $ raco pkg catalog-copy https://pkgs.racket-lang.org compiled/pkgs-copy\n  $ mkdir -p compiled/root/pkgs\n  $ mv compiled/pkgs-copy/pkg/* compiled/root/pkgs/\n```\n\nBeware, however, that the backend will start by updating the checksum\nof every package, so consider using a specific package name in place\nof `*` or a glob that selects a small set of packages.\n\n### Warm up\n\nWhen the server is started, the backend starts by looking for new\nchecksums, while the frontend immediately checks for backend updates.\nSince those run concurrently, you may not immediately see updates via\nthe frontend even when the backend has completed its scan (which you\nmight infer from logging output). At that point, restarting is the\nfastest way to warm up the frontend.\n\n### Adding users\n\nYou can use the website frontend to add a user. Email for a new user\nis sent via sendmail or an SMTP relay, so if you don't have that\nconfigured, just watch the logs to see the token that would have been\nsent.\n\n### Automatic code reloading\n\nIf you would like to enable the automatic code-reloading feature, set\nthe environment variable `SITE_RELOADABLE` to a non-empty string or\nset the `reloadable?` configuration variable to `#t`.\n\nYou must also delete any compiled code `.zo` files. Otherwise, the\nsystem will not be able to correctly replace modules while running.\n\nTherefore, when using automatic code reloading, use just\n\n    make run\n\nand make sure to run `make clean` beforehand, if you've run `make\ncompile` at all previously.\n\n## Deployment\n\n### Static Content\n\nThe site can be set up to run either\n\n 0. entirely dynamically, generating package pages on-the-fly for each\n    request;\n 0. both statically and dynamically, with HTML renderings of package\n    pages stored on and served from disk like other static resources\n    such as Javascript and CSS; or\n 0. both statically and dynamically, as the previous option, but\n    additionally replicating both static and generated content to a\n    local file-system directory and invoking an optional update hook\n    that can be used to further replicate the content to S3 or a\n    remote host.\n\nThe default is mixed static/dynamic, with no additional replication.\n\nFor a fully dynamic site, set configuration variable *disable-cache?*\nto `#t`.\n\nTo enable replication, set configuration variable\n*static-content-target-directory* to a non-`#f` value, and optionally\nset *static-content-update-hook* to a string containing a shell\ncommand to execute every time the static content is updated.\n\n#### S3 Content\n\nTo set up an S3 bucket — let's call it `s3.example` — for use with\nthis site, follow these steps:\n\n 0. Create the bucket (\"`s3.example`\")\n 0. Optionally add a CNAME record to DNS mapping `s3.example` to\n    `s3.example.s3-website-us-east-1.amazonaws.com`. If you do, static\n    resources will be available at `http://s3.example/`; if not, at\n    the longer URL.\n 0. Enable \"Static Website Hosting\" for the bucket. Set the index\n    document to `index.html` and the error document to `not-found`.\n\nThen, under \"Permissions\", click \"Add bucket policy\", and add\nsomething like the following.\n\n    {\n      \"Id\": \"RacketPackageWebsiteS3Policy\",\n      \"Version\": \"2012-10-17\",\n      \"Statement\": [\n        {\n          \"Sid\": \"RacketPackageWebsiteS3PolicyStmt1\",\n          \"Action\": \"s3:*\",\n          \"Effect\": \"Allow\",\n          \"Resource\": [\"arn:aws:s3:::s3.example\",\n                       \"arn:aws:s3:::s3.example/*\"],\n          \"Principal\": {\n            \"AWS\": [\"\u003c\u003c\u003cARN OF THE USER TO WHOM ACCESS SHOULD BE GRANTED\u003e\u003e\u003e\"]\n          }\n        }\n      ]\n    }\n\nThe user will need to be able to read and write objects and set CORS\npolicy. (CORS is configured automatically by code in\n`src/static.rkt`.)\n\n### Supervision\n\nStartable using djb's [daemontools](http://cr.yp.to/daemontools.html);\nsymlink this directory into your services directory and start it as\nusual. The `run` script starts the program, and `log/run` sets up\nlogging of stdout/stderr.\n\nIf the file `run-prelude` exists in the current directory on startup,\nit will be dotted in before racket is invoked. A prelude is useful to\nupdate `PATH` for a locally-built racket `bin` directory or to select\nan appropriate `CONFIG` setting.\n\nOn Debian, daemontools can be installed with `apt-get install\ndaemontools daemontools-run`, and the services directory is\n`/etc/service/`.\n\n### Control signals\n\nYou can send signals to the running service by creating files in\n`/etc/service/webservice/signals/`. For example:\n\n - creating `.pull-required` causes the server to shell out to `git\n   pull` and then exit. Daemontools will restart it.\n\n - creating `.restart-required` causes it to exit, to be restarted by\n   daemontools.\n\n - creating `.reload` causes an explicit code reload. Useful when\n   automatic code reloading is disabled.\n\n - creating `.fetchindex` causes an immediate refetch of the package\n   index from the backend server.\n\n - creating `.rerender` causes an immediate rerendering of all\n   generated static HTML files.\n\nSee `src/signals.rkt` for details of the available signals.\n\nSo long as `sudo chmod 0777 /etc/service/webservice/signals`, these\nare useful for non-root administrators to control the running service.\n\nIn particular, a git `post-receive` hook can be used to create the\n`.pull-required` signal in order to update the service on git push.\n\n## Copyright and License\n\nCopyright \u0026copy; 2014 Tony Garnock-Jones\n\n    This program is free software: you can redistribute it and/or modify\n    it under the terms of the GNU Lesser General Public License as published by\n    the Free Software Foundation, either version 3 of the License, or\n    (at your option) any later version.\n\n    This program is distributed in the hope that it will be useful,\n    but WITHOUT ANY WARRANTY; without even the implied warranty of\n    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n    GNU Lesser General Public License for more details.\n\n    You should have received a copy of the GNU Lesser General Public License\n    along with this program.  If not, see \u003chttp://www.gnu.org/licenses/\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fracket%2Fracket-pkg-website","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fracket%2Fracket-pkg-website","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fracket%2Fracket-pkg-website/lists"}