{"id":20217992,"url":"https://github.com/matrix-org/sliding-sync","last_synced_at":"2025-10-09T21:10:36.946Z","repository":{"id":82118020,"uuid":"578137794","full_name":"matrix-org/sliding-sync","owner":"matrix-org","description":"Proxy implementation of MSC3575's sync protocol.","archived":false,"fork":false,"pushed_at":"2024-09-13T15:32:57.000Z","size":3827,"stargazers_count":274,"open_issues_count":70,"forks_count":42,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-06-08T18:29:29.737Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/matrix-org/matrix-spec-proposals/pull/3575","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matrix-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-14T10:50:27.000Z","updated_at":"2025-05-29T08:01:50.000Z","dependencies_parsed_at":"2023-10-11T14:17:06.127Z","dependency_job_id":"4c515aa2-7e4b-4908-9e70-c0c88416bf7e","html_url":"https://github.com/matrix-org/sliding-sync","commit_stats":null,"previous_names":["matrix-org/syncv3"],"tags_count":40,"template":false,"template_full_name":null,"purl":"pkg:github/matrix-org/sliding-sync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrix-org%2Fsliding-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrix-org%2Fsliding-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrix-org%2Fsliding-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrix-org%2Fsliding-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matrix-org","download_url":"https://codeload.github.com/matrix-org/sliding-sync/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrix-org%2Fsliding-sync/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002074,"owners_count":26083285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T06:36:45.826Z","updated_at":"2025-10-09T21:10:36.908Z","avatar_url":"https://github.com/matrix-org.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# Sliding Sync\n\nRun a sliding sync proxy. An implementation of [MSC3575](https://github.com/matrix-org/matrix-doc/blob/kegan/sync-v3/proposals/3575-sync.md).\n\n## Proxy version to MSC API specification\n\nThis describes which proxy versions implement which version of the API drafted\nin MSC3575. See https://github.com/matrix-org/sliding-sync/releases for the\nchanges in the proxy itself.\n\nAs of v0.99.12, the proxy implements [this version of the MSC](https://github.com/matrix-org/matrix-spec-proposals/blob/9450ced7fb9cf5ea9077d029b3adf36aebfa8709/proposals/3575-sync.md) with the following exceptions:\n - the `limited` flag is not set in responses.\n - Delta tokens are unsupported.\n\n\n## Usage\n\n*NOTE: The proxy works fine with Dendrite and Synapse, but it doesn't work well with Conduit due to spec violations in the `state` of a room in `/sync`. Running the proxy with Conduit will cause more expired connections (HTTP 400s) when room state changes, and log lines like `WRN Accumulator.filterToNewTimelineEvents: seen the same event ID twice, ignoring`.*\n\n### Setup\nRequires Postgres 13+.\n\nFirst, you must create a Postgres database and secret:\n```bash\n$ createdb syncv3\n$ echo -n \"$(openssl rand -hex 32)\" \u003e .secret # this MUST remain the same throughout the lifetime of the database created above.\n```\n\nThe Sliding Sync proxy requires some environment variables set to function. They are described when the proxy is run with missing variables.\n\nHere is a short description of each, as of writing:\n```\nSYNCV3_SERVER        Required. The destination homeserver to talk to (CS API HTTPS URL) e.g 'https://matrix-client.matrix.org' (Supports unix socket: /path/to/socket)\nSYNCV3_DB            Required. The postgres connection string: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING\nSYNCV3_SECRET        Required. A secret to use to encrypt access tokens. Must remain the same for the lifetime of the database.\nSYNCV3_BINDADDR      Default: 0.0.0.0:8008. The interface and port to listen on. (Supports unix socket: /path/to/socket)\nSYNCV3_TLS_CERT      Default: unset. Path to a certificate file to serve to HTTPS clients. Specifying this enables TLS on the bound address.\nSYNCV3_TLS_KEY       Default: unset. Path to a key file for the certificate. Must be provided along with the certificate file.\nSYNCV3_PPROF         Default: unset. The bind addr for pprof debugging e.g ':6060'. If not set, does not listen.\nSYNCV3_PROM          Default: unset. The bind addr for Prometheus metrics, which will be accessible at /metrics at this address.\nSYNCV3_OTLP_URL      Default: unset. The OTLP HTTP URL to send spans to e.g https://localhost:4318 - if unset does not send OTLP traces.\nSYNCV3_OTLP_USERNAME Default: unset. The OTLP username for Basic auth. If unset, does not send an Authorization header.\nSYNCV3_OTLP_PASSWORD Default: unset. The OTLP password for Basic auth. If unset, does not send an Authorization header.\nSYNCV3_SENTRY_DSN    Default: unset. The Sentry DSN to report events to e.g https://sliding-sync@sentry.example.com/123 - if unset does not send sentry events.\nSYNCV3_LOG_LEVEL     Default: info. The level of verbosity for messages logged. Available values are trace, debug, info, warn, error and fatal\nSYNCV3_MAX_DB_CONN   Default: unset. Max database connections to use when communicating with postgres. Unset or 0 means no limit.\n```\n\nIt is easiest to host the proxy on a separate hostname than the Matrix server, though it is possible to use the same hostname by forwarding the used endpoints.\n\nIn both cases, the path `https://example.com/.well-known/matrix/client` must return a JSON with at least the following contents:\n```json\n{\n    \"m.homeserver\": {\n        \"base_url\": \"https://example.com\"\n    },\n    \"org.matrix.msc3575.proxy\": {\n        \"url\": \"https://syncv3.example.com\"\n    }\n}\n```\n\n#### Same hostname\nThe following nginx configuration can be used to pass the required endpoints to the sync proxy, running on local port 8009 (so as to not conflict with Synapse):\n```nginx\nlocation ~ ^/(client/|_matrix/client/unstable/org.matrix.msc3575/sync) {\n    proxy_pass http://localhost:8009;\n    proxy_set_header X-Forwarded-For $remote_addr;\n    proxy_set_header X-Forwarded-Proto $scheme;\n    proxy_set_header Host $host;\n}\n\nlocation ~ ^(\\/_matrix|\\/_synapse\\/client) {\n    proxy_pass http://localhost:8008;\n    proxy_set_header X-Forwarded-For $remote_addr;\n    proxy_set_header X-Forwarded-Proto $scheme;\n    proxy_set_header Host $host;\n}\n\nlocation /.well-known/matrix/client {\n    add_header Access-Control-Allow-Origin *;\n}\n```\n\n### Running\nThere are three ways to run the proxy:\n- Compiling from source:\n```\n$ CGO_ENABLED=0 go build ./cmd/syncv3\n$ SYNCV3_SECRET=$(cat .secret) SYNCV3_SERVER=\"https://matrix-client.matrix.org\" SYNCV3_DB=\"user=$(whoami) dbname=syncv3 sslmode=disable password='DATABASE_PASSWORD_HERE'\" SYNCV3_BINDADDR=0.0.0.0:8008 ./syncv3\n```\n\n- Using a Docker image:\n```\ndocker run --rm -e \"SYNCV3_SERVER=https://matrix-client.matrix.org\" -e \"SYNCV3_SECRET=$(cat .secret)\" -e \"SYNCV3_BINDADDR=:8008\" -e \"SYNCV3_DB=user=$(whoami) dbname=syncv3 sslmode=disable host=host.docker.internal password='DATABASE_PASSWORD_HERE'\" -p 8008:8008 ghcr.io/matrix-org/sliding-sync:latest\n```\n\n- Precompiled binaries:\n\nDownload the binary for your architcture (eg. `syncv3_linux_amd64` for 64-bit AMD/Intel) from https://github.com/matrix-org/sliding-sync/releases/latest\n```\n$ SYNCV3_SECRET=$(cat .secret) SYNCV3_SERVER=\"https://matrix-client.matrix.org\" SYNCV3_DB=\"user=$(whoami) dbname=syncv3 sslmode=disable password='DATABASE_PASSWORD_HERE'\" SYNCV3_BINDADDR=0.0.0.0:8008 ./syncv3_linux_amd64\n```\n\nOptionally also set `SYNCV3_TLS_CERT=path/to/cert.pem` and `SYNCV3_TLS_KEY=path/to/key.pem` to listen on HTTPS instead of HTTP.\nMake sure to tweak the `SYNCV3_DB` environment variable if the Postgres database isn't running on the host.\n\nRegular users may now log in with their sliding-sync compatible Matrix client. If developing sliding-sync, a simple client is provided (although it is not included in the Docker image).\n\nTo use the stub client, visit http://localhost:8008/client/ (with trailing slash) and paste in the `access_token` for any account on `SYNCV3_SERVER`. Note that this will consume to-device messages for the device associated with that access token.\n\nWhen you hit the Sync button nothing will happen initially, but you should see:\n\n```\nINF Poller: v2 poll loop started ip=::1 since= user_id=@kegan:matrix.org\n```\n\nWait for the first initial v2 sync to be processed (this can take minutes!) and then v3 APIs will be responsive.\n\nNote that some clients might require that your home server advertises support for sliding-sync in the `.well-known/matrix/client` endpoint; details are in [the work-in-progress specification document](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#unstable-prefix).\n\nThere are some notes on the log format in [docs/log_format.md](./docs/log_format.md).\n\n### Prometheus\n\nTo enable metrics, pass `SYNCV3_PROM=:2112` to listen on that port and expose a scraping endpoint `GET /metrics`.\nIf you want to hook this up to a prometheus, you can just define `prometheus.yml`:\n```yaml\nglobal:\n    scrape_interval: 30s\n    scrape_timeout: 10s\nscrape_configs:\n    - job_name: ss\n      static_configs:\n       - targets: [\"host.docker.internal:2112\"]\n```\nthen run Prometheus in a docker container:\n```bash\ndocker run -p 9090:9090 -v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus\n```\nto play with the data, use PromLens and point it at http://localhost:9090:\n```bash\ndocker run -p 8080:8080 prom/promlens\n```\nUseful queries include:\n - `rate(sliding_sync_poller_num_payloads{job=\"ss\"}[5m])` : This shows the payload rate from pollers to API processes,\n   broken down by type. A stacked graph display is especially useful as the height then represents the total payload\n   rate. This can be used to highlight abnormal incoming data, such as excessive payload rates. It can also be used\n   to gauge how costly certain payload types are. In general, receipts and device data tend to be the most frequent\n   background noise. A full list of payload types are defined in the [pubsub directory](https://github.com/matrix-org/sliding-sync/blob/main/pubsub/v2.go).\n - `sliding_sync_poller_num_pollers` : Absolute count of the number of /sync v2 pollers over time. Useful either as a single value,\n   or display over time. The higher this value, the more pressure is put on the upstream Homeserver.\n - `sliding_sync_api_num_active_conns` : Absolute count of the number of active sliding sync connections. Useful either as a single value,\n   or display over time. The higher this value, the more pressure is put on the proxy API processes.\n - `sum(increase(sliding_sync_poller_process_duration_secs_bucket[1m])) by (le)` : Useful heatmap to show how long /sync v2 responses take to process.\n   This can highlight database pressure as processing responses involves database writes and notifications over pubsub.\n - `sum(increase(sliding_sync_api_process_duration_secs_bucket[1m])) by (le)` : Useful heatmap to show how long sliding sync responses take to calculate,\n   which excludes all long-polling requests. This can highlight slow sorting/database performance, as these requests should always be fast.\n\n### Can I delete the proxy database and start over?\n\n*It is not recommended to do this.*\n\nYes, with caveats. Most of the data is just a copy of data from the upstream homeserver. The exceptions to this are:\n - device list changes (table: `syncv3_device_list_updates`)\n - to-device messages (table: `syncv3_to_device_messages` and related sequence `syncv3_to_device_messages_seq`)\n\nBoth of these are critical for E2EE to work correctly as it ensures A) clients know up-to-date devices for users, B) clients can reliably send messages directly to devices.\nIf the device list changes are deleted, Alice may not realise that Bob logged in on a new device and hence not encrypt for that new device. Encryption keys are sent in to-device messages, so deleting them will inevitably cause some messages to become undecryptable. If you don't care about causing undecryptable messages then you can just delete the database and start over. If you do care, preserve those tables/sequences. We cannot guarantee that the database will remain in a consistent state (things may break if you keep some tables) as we do not test this scenario.\n\n### Profiling\n\nTo help debug performance issues, you can make the proxy listen for PPROF requests by passing `SYNCV3_PPROF=:6060` to listen on `:6060`.\nTo debug **why a request is slow**:\n```\nwget -O 'trace.pprof' 'http://localhost:6060/debug/pprof/trace?seconds=20'\n```\nThen perform the slow request within 20 seconds. Send `trace.pprof` to someone who will then run `go tool trace trace.pprof` and look at \"User-defined Tasks\" for slow HTTP requests.\n\nTo debug **why the proxy is consuming lots of memory**, run:\n```\nwget -O 'heap.pprof' 'http://localhost:6060/debug/pprof/heap'\n```\nThen send `heap.pprof` to someone who will then run `go tool pprof heap.pprof` and probably type something like `top10`:\n```\n(pprof) top10\nShowing nodes accounting for 83.13MB, 100% of 83.13MB total\nShowing top 10 nodes out of 82\n      flat  flat%   sum%        cum   cum%\n   43.01MB 51.74% 51.74%    43.01MB 51.74%  github.com/tidwall/gjson.ParseBytes\n   31.85MB 38.31% 90.05%    31.85MB 38.31%  github.com/matrix-org/sliding-sync/sync3.(*JoinedRoomsTracker).Startup\n       4MB  4.82% 94.87%        4MB  4.82%  runtime.allocm\n    1.76MB  2.12% 96.99%     1.76MB  2.12%  compress/flate.NewWriter\n    0.50MB  0.61% 97.59%        1MB  1.21%  github.com/matrix-org/sliding-sync/sync3.(*SortableRooms).Sort\n    0.50MB   0.6% 98.20%     0.50MB   0.6%  runtime.malg\n    0.50MB   0.6% 98.80%     0.50MB   0.6%  github.com/matrix-org/sliding-sync/sync3.(*InternalRequestLists).Room\n    0.50MB   0.6% 99.40%     0.50MB   0.6%  github.com/matrix-org/sliding-sync/sync3.(*Dispatcher).notifyListeners\n    0.50MB   0.6%   100%     0.50MB   0.6%  runtime.acquireSudog\n         0     0%   100%     1.76MB  2.12%  bufio.(*Writer).Flush\n```\n\nTo debug **why the proxy is using 100% CPU**, run:\n```\nwget -O 'profile.pprof' 'http://localhost:6060/debug/pprof/profile?seconds=10'\n```\nThen send `profile.pprof` to someone who will then run `go tool pprof -http :5656 profile.pprof` and typically view the flame graph: View -\u003e Flame Graph.\n\nTo build **with PGO**, run:\n```\n$ wget -O 'default.pgo' 'http://localhost:6060/debug/pprof/profile?seconds=30'\n$ CGO_ENABLED=0 go build -o after_pgo ./cmd/syncv3\n```\n\n\n### Developers' cheat sheet\n\nSanity check everything builds:\n\n```shell\nCGO_ENABLED=0 go build ./cmd/syncv3\ngo list ./... | xargs -n1 go test -c -o /dev/null\n```\n\nRun all unit and integration tests:\n\n```shell\ngo test -p 1 -count 1 $(go list ./... | grep -v tests-e2e) -timeout 120s\n```\n\nRun end-to-end tests:\n\n```shell\n# Will need to `docker login` to ghcr and pull the image.\ndocker run -d --rm -e \"SYNAPSE_COMPLEMENT_DATABASE=sqlite\" -e \"SERVER_NAME=synapse\" -p 8888:8008 ghcr.io/matrix-org/synapse-service:v1.94.0\n\nexport SYNCV3_SECRET=foobar\nexport SYNCV3_SERVER=http://localhost:8888\nexport SYNCV3_DB=\"user=$(whoami) dbname=syncv3_test sslmode=disable\"\n\n(go build ./cmd/syncv3 \u0026\u0026 dropdb syncv3_test \u0026\u0026 createdb syncv3_test \u0026\u0026 cd tests-e2e \u0026\u0026 ./run-tests.sh -count=1 .)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrix-org%2Fsliding-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatrix-org%2Fsliding-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrix-org%2Fsliding-sync/lists"}