{"id":30110573,"url":"https://github.com/monoscope-tech/timefusion","last_synced_at":"2026-03-17T21:36:18.306Z","repository":{"id":287915295,"uuid":"930022267","full_name":"monoscope-tech/timefusion","owner":"monoscope-tech","description":"A timeseries database created for events, logs, traces and metrics. Speaks the postgres dialect, and stores data in s3 via delta lake protocol","archived":false,"fork":false,"pushed_at":"2026-02-16T19:51:33.000Z","size":2333,"stargazers_count":159,"open_issues_count":5,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2026-02-17T01:29:04.011Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/monoscope-tech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-09T23:15:14.000Z","updated_at":"2026-02-03T06:16:24.000Z","dependencies_parsed_at":"2025-04-14T16:49:07.385Z","dependency_job_id":"2ffed90a-4c5e-421a-ae56-60e92a687343","html_url":"https://github.com/monoscope-tech/timefusion","commit_stats":null,"previous_names":["apitoolkit/timefusion","monoscope-tech/timefusion"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/monoscope-tech/timefusion","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monoscope-tech%2Ftimefusion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monoscope-tech%2Ftimefusion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monoscope-tech%2Ftimefusion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monoscope-tech%2Ftimefusion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/monoscope-tech","download_url":"https://codeload.github.com/monoscope-tech/timefusion/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monoscope-tech%2Ftimefusion/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30632057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T17:32:55.572Z","status":"ssl_error","status_checked_at":"2026-03-17T17:32:38.732Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-10T04:47:37.934Z","updated_at":"2026-03-17T21:36:18.290Z","avatar_url":"https://github.com/monoscope-tech.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Timefusion\n\nA very specialized timeseries database created for events, logs, traces and metrics.\n\nIts designed to allow users plug in their own s3 storage and buckets and have their stored to their accounts.\nThis way, timefusion is used as a compute and cache engine, not primary data storage.\n\nTimefusion speaks the postgres dialect, so you can insert and read from it using any postgres client or driver.\n\n## Configuration\n\nTimefusion can be configured using the following environment variables:\n\n| Variable               | Description                                      | Default                     |\n| ---------------------- | ------------------------------------------------ | --------------------------- |\n| `PORT`                 | HTTP server port                                 | `80`                        |\n| `PGWIRE_PORT`          | PostgreSQL wire protocol port                    | `5432`                      |\n| `AWS_S3_BUCKET`        | AWS S3 bucket name                               | Required                    |\n| `AWS_S3_ENDPOINT`      | AWS S3 endpoint URL                              | `https://s3.amazonaws.com`  |\n| `AWS_ACCESS_KEY_ID`    | AWS access key                                   | -                           |\n| `AWS_SECRET_ACCESS_KEY`| AWS secret key                                   | -                           |\n| `AWS_S3_LOCKING_PROVIDER` | Delta Lake locking provider ('dynamodb')      | -                           |\n| `DELTA_DYNAMO_TABLE_NAME` | DynamoDB table name for Delta Lake locking    | -                           |\n| `TIMEFUSION_TABLE_PREFIX` | Prefix for Delta tables                       | `timefusion`                |\n| `BATCH_INTERVAL_MS`    | Interval between batch inserts in milliseconds   | `1000`                      |\n| `MAX_BATCH_SIZE`       | Maximum number of rows in a single batch         | `1000`                      |\n| `ENABLE_BATCH_QUEUE`   | Whether to use batch queue for inserts           | `false` (direct insertion)  |\n| `MAX_PG_CONNECTIONS`   | Maximum number of concurrent PostgreSQL connections | `100`                     |\n\nFor local development, you can set `QUEUE_DB_PATH` to a location in your development environment.\n\n### Delta Lake DynamoDB Locking\n\nFor multi-writer scenarios where multiple instances of TimeFusion may write to the same Delta tables concurrently, it's recommended to enable DynamoDB locking:\n\n1. Create a DynamoDB table with the following configuration:\n   - Table name: Choose any name (e.g., `timefusion-delta-locks`)\n   - Partition key: `key` (String type)\n   - On-demand billing mode is recommended\n\n2. Set the following environment variables:\n   ```\n   AWS_S3_LOCKING_PROVIDER=dynamodb\n   DELTA_DYNAMO_TABLE_NAME=timefusion-delta-locks\n   ```\n\n3. Ensure your AWS credentials have the following DynamoDB permissions:\n   - `dynamodb:GetItem`\n   - `dynamodb:PutItem`\n   - `dynamodb:UpdateItem`\n   - `dynamodb:DeleteItem`\n\nThis configuration ensures safe concurrent writes to Delta tables by using DynamoDB for distributed locking.\n\n**Note for S3-Compatible Storage (e.g., OVH, MinIO)**: When using S3-compatible stores that don't support conditional PUT operations, DynamoDB locking is strongly recommended to prevent data corruption in multi-writer scenarios. See [DELTA_CONFIG.md](DELTA_CONFIG.md) for detailed configuration options and trade-offs.\n\n## Usage\n\nThere currently exists only 1 table. otel_logs_and_spans.\nYou can access it via psql: eg if running locally:\n\n```\n$ psql \"postgresql://postgres:postgres@localhost:12345/postgres\"\n\npsql (16.8 (Homebrew), server 0.28.0)\nWARNING: psql major version 16, server major version 0.28.\n         Some psql features might not work.\nType \"help\" for help.\n\npostgres=\u003e insert into otel_logs_and_spans (name, id, project_id, hashes, timestamp, date) values ('name3', 'id2', 'pid3', ARRAY[], '2025-04-14 02:00:24.898000', '2025-04-14 02:00:24.898000');\nINSERT 0 1\n\npostgres=\u003e select name, id, project_id,timestamp from otel_logs_and_spans limit 10;\n                            name                             |                  id                  |              project_id              |         timestamp\n-------------------------------------------------------------+--------------------------------------+--------------------------------------+----------------------------\n GET api/v1/validations/profundity-interior/(?P\u003cpk\u003e[^/.]+)/$ | 00000000-09ab-47bc-b628-2554626d1261 | 00000000-876e-41fa-be63-52d5bcfc037e | 2025-04-14 20:45:08.713740\n GET api/v1/validations/tire-pressure/(?P\u003cpk\u003e[^/.]+)/$       | 00000000-3d2a-445d-b7bf-3e56125b48d4 | 00000000-876e-41fa-be63-52d5bcfc037e | 2025-04-14 22:01:00.816390\n POST api/v1/validations/warnings-of-wear/$                  | 00000000-4ced-48f4-830d-64d3531eb7f0 | 00000000-876e-41fa-be63-52d5bcfc037e | 2025-04-14 21:18:08.635637\n\n```\n\n```\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmonoscope-tech%2Ftimefusion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmonoscope-tech%2Ftimefusion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmonoscope-tech%2Ftimefusion/lists"}