{"id":17131032,"url":"https://github.com/jac18281828/rustdata","last_synced_at":"2025-03-24T05:12:29.156Z","repository":{"id":254491488,"uuid":"846698857","full_name":"jac18281828/rustdata","owner":"jac18281828","description":"Example data ingestion and processing in rust","archived":false,"fork":false,"pushed_at":"2024-08-27T18:33:45.000Z","size":68,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-29T10:49:49.442Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jac18281828.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-23T19:03:07.000Z","updated_at":"2024-08-27T18:33:48.000Z","dependencies_parsed_at":"2024-12-01T11:37:04.209Z","dependency_job_id":null,"html_url":"https://github.com/jac18281828/rustdata","commit_stats":null,"previous_names":["jac18281828/rustdata"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jac18281828%2Frustdata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jac18281828%2Frustdata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jac18281828%2Frustdata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jac18281828%2Frustdata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jac18281828","download_url":"https://codeload.github.com/jac18281828/rustdata/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245212298,"owners_count":20578443,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T19:13:49.742Z","updated_at":"2025-03-24T05:12:29.138Z","avatar_url":"https://github.com/jac18281828.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rustdata\n\n## Disclaimer\n\nThis is an unscientific comparision of Parquet and Postgress for a simple stat calculation.\n\nThe dataset is 2500 rows of Ethereum Event data.  Eigenlayer IRewardsCoordinator.RewardsClaimed.  The data is stored to the database and then serialized back with the purpose of computing some common statistics, mean, min, max, etc.\n\n\n## Postgres\n\n#### 1. Start Postgres database\n\n```bash\ndocker run --name db-postgres -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=password -e POSTGRES_DB=edb -p 5432:5432 -d postgres\n```\n\n#### 2. Upload Postgres data\n\n```bash\ncargo run --release --bin pg\n```\n\n## Parquet\n\n#### 1. run the client\n\n```bash\ncargo run --release --bin pq\n```\n\n## Comparison Chart\n\n| Storage                                | Write Time (us) | Read Time (us) | Storage Size (kB) |\n| -------------------------------------- | --------------- | -------------- | ----------------- |\n| [Parquet](https://parquet.apache.org/) | 2687            | 339            | 583               |\n| [Postgres](https://www.postgresql.org) | 1037531         | 2539           | 1048              |\n| [DuckDB](https://duckdb.org/)          | 277615          | 798            | 1300              |\n| SQLite                                 | 1040026         | 320            | 792               |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjac18281828%2Frustdata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjac18281828%2Frustdata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjac18281828%2Frustdata/lists"}