{"id":15566190,"url":"https://github.com/ericfreese/data-comparison","last_synced_at":"2025-08-12T13:35:12.986Z","repository":{"id":254479077,"uuid":"846654050","full_name":"ericfreese/data-comparison","owner":"ericfreese","description":"A tool for comparing similar rectangular data sets","archived":false,"fork":false,"pushed_at":"2024-08-30T19:04:10.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-23T03:23:17.791Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ericfreese.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-23T17:02:22.000Z","updated_at":"2024-08-30T21:28:54.000Z","dependencies_parsed_at":"2024-08-23T19:15:00.088Z","dependency_job_id":"9853a399-e7f4-4ec9-9182-a487e6161102","html_url":"https://github.com/ericfreese/data-comparison","commit_stats":{"total_commits":3,"total_committers":1,"mean_commits":3.0,"dds":0.0,"last_synced_commit":"e2bedbb5261eb51b00719dccd09306c8ed3e78b3"},"previous_names":["ericfreese/data-comparison"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericfreese%2Fdata-comparison","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericfreese%2Fdata-comparison/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericfreese%2Fdata-comparison/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ericfreese%2Fdata-comparison/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ericfreese","download_url":"https://codeload.github.com/ericfreese/data-comparison/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240702650,"owners_count":19843957,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T17:01:17.350Z","updated_at":"2025-02-25T16:25:23.703Z","avatar_url":"https://github.com/ericfreese.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Comparison Tool\n\nTakes two CSV files with overlapping column names and generates a comparison report (printed to stdout in CSV format) by grouping the data in each file by specified qualitative columns, aggregating (summing) specified quantitative columns, joining the aggregated data together on the grouped columns, and reporting before/after values of the quantitative aggregates.\n\n## Example\n\n([`xsv`](https://github.com/BurntSushi/xsv) used for illustration purposes)\n\nAssume you have two files `before.csv` and `after.csv`:\n\n```\n$ xsv table before.csv\nname  color  amount\nfoo   red    3\nfoo   green  2\nfoo   blue   4\nbar   red    5\nbar   green  3\n\n$ xsv table after.csv\nname  color  amount\nfoo   red    5\nfoo   green  2\nbar   red    5\nbar   green  2\nbar   blue   4\n```\n\nYou can compare them in the following ways:\n\n```\n$ data-compare before.csv after.csv -m amount -g color | xsv table\ncolor  b_amount  a_amount  d_amount\nblue   4.000     4.000     0.000\ngreen  5.000     4.000     -1.000\nred    8.000     10.000    2.000\n\n$ data-compare before.csv after.csv -m amount -g name | xsv table\nname  b_amount  a_amount  d_amount\nbar   8.000     11.000    3.000\nfoo   9.000     7.000     -2.000\n\n$ data-compare before.csv after.csv -m amount -g color name | xsv table\ncolor  name  b_amount  a_amount  d_amount\nblue   bar             4.000     4.000\nblue   foo   4.000               -4.000\ngreen  bar   3.000     2.000     -1.000\ngreen  foo   2.000     2.000     0.000\nred    bar   5.000     5.000     0.000\nred    foo   3.000     5.000     2.000\n\n$ data-compare /tmp/before.csv /tmp/after.csv -m amount -g color=red name | xsv table\ncolor  name  b_amount  a_amount  d_amount\nred    bar   5.000     5.000     0.000\nred    foo   3.000     5.000     2.000\n```\n\n## Usage\n\n```\nusage: data-compare [-h] -m col [col ...] -g col [col ...] before after\n\npositional arguments:\n  before                file to start with\n  after                 file to compare against\n\noptions:\n  -h, --help            show this help message and exit\n  -m col [col ...], --measures col [col ...]\n                        columns to sum and compare\n  -g col [col ...], --group-cols col [col ...]\n                        columns to group and join by\n```\n\nFilters can be applied to a grouped column by suffixing it with `=\u003cvalue\u003e` so using `-g foo=bar` would group by the `foo` column and filter rows to only those where `foo` is equal to `bar`\n\n## Initializing virtual env for dev\n\n```sh\npipenv sync --dev\n```\n\n## Run in dev environment\n\n```sh\npipenv run python data-compare.py\n```\n\n## Build standalone executable\n\n```sh\npipenv run pyinstaller data-compare.spec\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fericfreese%2Fdata-comparison","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fericfreese%2Fdata-comparison","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fericfreese%2Fdata-comparison/lists"}