{"id":20564612,"url":"https://github.com/yjs/y-redis","last_synced_at":"2025-04-06T01:09:10.613Z","repository":{"id":41416633,"uuid":"100140298","full_name":"yjs/y-redis","owner":"yjs","description":"Alternative backend for y-websocket","archived":false,"fork":false,"pushed_at":"2024-03-29T21:51:26.000Z","size":306,"stargazers_count":123,"open_issues_count":0,"forks_count":23,"subscribers_count":15,"default_branch":"master","last_synced_at":"2024-04-14T08:50:29.094Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yjs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":"dmonad","patreon":null,"open_collective":"y-collective","ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2017-08-12T22:13:25.000Z","updated_at":"2024-04-12T22:08:45.000Z","dependencies_parsed_at":"2024-03-17T02:04:18.876Z","dependency_job_id":"7b97a583-4a2f-488a-9bf2-f1d88aaa3329","html_url":"https://github.com/yjs/y-redis","commit_stats":{"total_commits":67,"total_committers":4,"mean_commits":16.75,"dds":0.5074626865671642,"last_synced_commit":"6789d89c2fb53be9e88af230ffc0bccabc118d93"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yjs%2Fy-redis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yjs%2Fy-redis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yjs%2Fy-redis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yjs%2Fy-redis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yjs","download_url":"https://codeload.github.com/yjs/y-redis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419860,"owners_count":20936012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T04:28:09.123Z","updated_at":"2025-04-06T01:09:10.593Z","avatar_url":"https://github.com/yjs.png","language":"JavaScript","readme":"\n# y-redis :tophat: \n\u003e y-websocket compatible backend using Redis for scalability. **This is beta\n\u003e software!**\n\ny-redis is an alternative backend for y-websocket. It only requires a redis\ninstance and a storage provider (S3 or Postgres-compatible). \n\n* **Memory efficient:** The server doesn't maintain a Y.Doc in-memory. It\nstreams updates through redis. The Yjs document is only loaded to memory for the\ninitial sync. \n* **Scalable:** You can start as many y-redis instances as you want to handle\na fluctuating number of clients. No coordination is needed.\n- **Auth:** y-redis works together with your existing infrastructure to\nauthenticate clients and check whether a client has read-only / read-write\naccess to a document.\n- **Database agnostic:** You can persist documents in S3-compatible backends, in\nPostgres, or implement your own storage provider.\n\n### Licensing\n\ny-redis is dual-licensed (either [AGPL](./LICENSE) or proprietary).\n\nPlease contact me to buy a license if you intend to use y-redis in your\ncommercial product: \u003ckevin.jahns at pm.me\u003e\n\nOtherwise, you may use this software under the terms of the AGPL, which requires\nyou to publish your source code under the terms of the AGPL too.\n\n### Components\n\nRedis is used as a \"cache\" and a distribution channel for document updates.\nNormal databases are not fast enough for handling real-time updates of\nfast-changing applications (e.g. collaborative drawing applications that\ngenerate hundreds of operations per second). Hence a redis-cache for temporary\nstorage makes sense to distribute documents as fast as possible to all peers.\n\nA persistent storage (e.g. S3 or Postgres) is used to persist document updates\npermanently. You can configure in which intervals you want to persist data from\nredis to the persistent storage. You can even implement a custom persistent\nstorage technology.\n\nThe y-redis **server component** (`/bin/server.js`) is responsible for accepting\nwebsocket-connections and distributing the updates via redis streams. Each\n\"room\" is represented as a redis stream. The server component assembles updates\nstored redis and in the persistent storage (e.g. S3 or Postgres) for the initial\nsync. After the initial sync, the server doesn't keep any Yjs state in-memory.\nYou can start as many server components as you need. It makes sense to put the\nserver component behind a loadbalancer, which can potentially auto-scale the\nserver component based on CPU or network usage. \n\nThe separate y-redis **worker component** (`/bin/worker.js`) is responsible for\nextracting data from the redis cache to a persistent database like S3 or\nPostgres. Once the data is persisted, the worker component cleans up stale data\nin redis. You can start as many worker components as you need. It is recommended\nto run at least one worker, so that the data is eventually persisted. The worker\ncomponents coordinate which room needs to be persisted using a separate\nworker-queue (see `y:worker` stream in redis).\n\nYou are responsible for providing a REST backend that y-redis will call to check\nwhether a specific client (authenticated via a JWT token) has access to a\nspecific room / document. Example servers can be found in\n`/bin/auth-server-example.js` and `/demos/auth-express/server.js`.\n\n### Missing Features\n\nI'm looking for sponsors that want to sponsor the following work:\n\n- Ability to kick out users when permissions on a document changed\n- Configurable docker containers for y-redis server \u0026 worker\n- Helm chart\n- More exhaustive logging and reporting of possible issues\n- More exhaustive testing\n- Better documentation \u0026 more documentation for specific use-cases\n- Support for Bun and Deno\n- Perform expensive tasks (computing sync messages) in separate threads\n\nIf you are interested in sponsoring some of this work, please send a mail to\n\u003ckevin.jahns at pm.me\u003e.\n\n# Quick Start (docker-compose)\n\nYou can get everything running quickly using\n[docker-compose](https://docs.docker.com/compose/). The compose file runs the\nfollowing components:\n\n- redis\n- minio as a s3 endpoint\n- a single y-redis server\n- a single y-redis worker\n\nThis can be a good starting point for your application. If your cloud provider\nhas a managed s3 service, you should probably use that instead of minio. If you\nwant to use minio, you need to setup proper volumes and backups.\n\nThe full setup gives insight into more specialized configuration options.\n\n```sh\ngit clone https://github.com/yjs/y-redis.git\ncd y-redis\nnpm i\n```\n\n### Setup the environment variables\n\n```sh\ncp .env.docker.template .env\n# generate unique authentication tokens\nnpx 0ecdsa-generate-keypair --name auth \u003e\u003e .env\n```\n\nThe sample configuration configures s3 using minio.\nHave a look at `.env.template` for more configuration options.\n\n### Run demo\n\n```sh\ncd ./demos/auth-express\ndocker compose up\n# open http://localhost:5173 in a browser\n```\n\n# Full setup\n\nComponents are configured via environment variables. It makes sense to start by\ncloning y-redis and getting one of the demos to work.\n\nNote: If you want to use any of the docker commands, feel free to use podman (a\nmore modern alternative) instead.\n\n#### Start a redis instance\n\nSetup redis on your computer. Follow the [official\ndocumentation](https://redis.io/docs/install/install-redis/). This is\nrecommended if you want to debug the redis stream.\n\nAlternatively, simply run redis via docker:\n\n```sh\n# start the official redis docker container on port 6379\ndocker run -p 6379:6379 redis\n# or `npm run redis`\n```\n\n#### Start an S3 instance\n\nSetup an S3-compatible store at your favorite cloud provider.\n\nAlternatively, simply run a *minio* store as a docker container:\n\n```sh\ndocker run -p 9000:9000 -p 9001:9001 quay.io/minio/minio server /data --console-address \\\":9001\\\"\n# or `npm run minio`\n```\n\nThis is just a dev setup. Have a look at the minio documentation if you want to\nrun it in production.\n\n#### Clone demo\n\n```sh\ngit clone https://github.com/yjs/y-redis.git\ncd y-redis\nnpm i\n```\n\nAll features are configurable using environment variables. For local development\nit makes sense to setup a `.env` file, that stores project-specific secrets. Use\n`.env.template` as a template to setup environment variables. Make sure to read\nthe documentation carefully and configure every single variable.\n\n```sh\n# setup environment variables\ncp .env.template .env\nnano .env\n```\n\nThen you can run the different components in separate terminals:\n\n```sh\n# run the server\nnpm run start:server\n# run a single worker in a separate terminal\nnpm run start:worker\n# start the express server in a separater terminal\ncd demos/auth-express\nnpm i\nnpm start\n```\n\nOpen [`http://localhost:5173`](http://localhost:5173) in a browser.\n","funding_links":["https://github.com/sponsors/dmonad","https://opencollective.com/y-collective"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyjs%2Fy-redis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyjs%2Fy-redis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyjs%2Fy-redis/lists"}