{"id":13557505,"url":"https://github.com/entrepreneur-interet-general/Merge-Machine","last_synced_at":"2025-04-03T11:32:18.358Z","repository":{"id":57440927,"uuid":"110987707","full_name":"entrepreneur-interet-general/Merge-Machine","owner":"entrepreneur-interet-general","description":"Merge Dirty Data with Clean Reference Tables","archived":false,"fork":false,"pushed_at":"2021-08-03T18:25:07.000Z","size":1331,"stargazers_count":35,"open_issues_count":35,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-28T15:45:51.995Z","etag":null,"topics":["elasticsearch","entity-resolution","entrepreneur-interet-general","python","record-linkage"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/entrepreneur-interet-general.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-16T15:22:37.000Z","updated_at":"2022-12-19T03:32:43.000Z","dependencies_parsed_at":"2022-09-19T14:41:08.603Z","dependency_job_id":null,"html_url":"https://github.com/entrepreneur-interet-general/Merge-Machine","commit_stats":null,"previous_names":["eig-2017/merge-machine"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2FMerge-Machine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2FMerge-Machine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2FMerge-Machine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2FMerge-Machine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/entrepreneur-interet-general","download_url":"https://codeload.github.com/entrepreneur-interet-general/Merge-Machine/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246993188,"owners_count":20865957,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","entity-resolution","entrepreneur-interet-general","python","record-linkage"],"created_at":"2024-08-01T12:04:23.442Z","updated_at":"2025-04-03T11:32:13.349Z","avatar_url":"https://github.com/entrepreneur-interet-general.png","language":"Python","funding_links":[],"categories":["Python","python"],"sub_categories":[],"readme":"# The Magical CSV Merge Machine\n\n## What does it do ?\n\nA python3 library to link a dirty CSV file with a clean reference table. It is meant to as generic as possible and includes a labeller to learn optimal parameters for each matching scenario.\n\n![Alt Text](documentation/labeller_peek.gif)\n\n## How to install ?\n\n### Manual install\n\n#### Non-Python Requirements\n\nThis library relies on Elasticsearch. We used version 5.6.7 for developpment.. We recommend Elasticsearch 5.X. Instructions [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html).\n\n#### PIP3 install\n\n```\npip3 install merge-machine\n```\n\nFrom source (recommended, for the meantime...):\n```\ngit clone https://github.com/entrepreneur-interet-general/Merge-Machine.git\ncd Merge-Machine\npip3 install -e .\n```\n\n## How to use ?\n\n### General use example (install the package first...)\nSee [examples/example.py](https://github.com/eig-2017/Merge-Machine/blob/master/examples/example.py).\n\n### Resource creation example (to use advanced analyzers)\nSee [examples/gen\\_resource\\_example.py](https://github.com/eig-2017/Merge-Machine/blob/master/examples/gen_resource_example.py).\n\n### Guidelines\nSee [HOW\\_TO.md](https://github.com/eig-2017/Merge-Machine/blob/master/HOW_TO.md).\n\n## How it works ?\n\nThe reference is indexed in Elasticsearch with multiple analyzers (languages specific, integers, n\\_grams...). The labeller then proposes training samples from the source which it tries to match to rows of the reference file. Upon user confirmation (match / not match) it updates its belief on which Elasticsearch queries are most performant to use for matching. When labelling is over, the \"best query\" (a weighted combination of multiple ES queries with different analyzers on different fields) is used for each row of the source to try to find a match in the ES-indexed referential.\n\n## How to contribute ?\n\nFeel free to report bugs via issues and make pull requests...\n\n## Credits\n\nThis library was developped by Léo Bouloc during 10 months in 2017 at the French [Ministry of Research and Higher Education](http://www.enseignementsup-recherche.gouv.fr/) in the context of the [\"Entrepreneur d'Intérêt Général\" program](https://www.etalab.gouv.fr/decouvrez-la-1e-promotion-des-entrepreneurs-dinteret-general) funded by the French Government.\n\n## See also\n\nThis library was developped as a component of larger matching service:\n* ONLINE SERVICE COMING SOON !!!\n* [code](https://github.com/eig-2017/the-magical-csv-merge-machine)\n\nCheck out:\n* [The Merge Machine](https://themergemachine.com), A simple web interface to link your data\n\nOther similar libraries include:\n* [match_id](https://github.com/matchID-project) (Identity record linking)\n* [dedupe](https://github.com/dedupeio/dedupe) (Record linking and deduping)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fentrepreneur-interet-general%2FMerge-Machine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fentrepreneur-interet-general%2FMerge-Machine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fentrepreneur-interet-general%2FMerge-Machine/lists"}