{"id":20274889,"url":"https://github.com/t2solve/recordlinkagenet","last_synced_at":"2025-10-16T14:52:40.377Z","repository":{"id":58725471,"uuid":"533280924","full_name":"t2solve/recordlinkagenet","owner":"t2solve","description":"library for dataset comparison","archived":false,"fork":false,"pushed_at":"2023-09-22T18:14:04.000Z","size":286,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-17T17:48:20.216Z","etag":null,"topics":["csharp","csharp-library","distance-measures","record-linkage","string-matching"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/t2solve.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-09-06T10:50:49.000Z","updated_at":"2024-11-08T07:13:51.000Z","dependencies_parsed_at":"2023-01-20T21:30:46.825Z","dependency_job_id":null,"html_url":"https://github.com/t2solve/recordlinkagenet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t2solve%2Frecordlinkagenet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t2solve%2Frecordlinkagenet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t2solve%2Frecordlinkagenet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t2solve%2Frecordlinkagenet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/t2solve","download_url":"https://codeload.github.com/t2solve/recordlinkagenet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241766485,"owners_count":20016766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csharp","csharp-library","distance-measures","record-linkage","string-matching"],"created_at":"2024-11-14T13:06:51.310Z","updated_at":"2025-10-16T14:52:35.341Z","avatar_url":"https://github.com/t2solve.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"![dotnet 6.0 build](https://github.com/t2solve/recordlinkagenet/actions/workflows/dotnet.yml/badge.svg)\n![code quality check](https://github.com/t2solve/recordlinkagenet/actions/workflows/codeql.yml/badge.svg)\n![code coverage](https://github.com/t2solve/recordlinkagenet/actions/workflows/codecoverage.yml/badge.svg)\n![Code Coverage](https://img.shields.io/badge/Code%20Coverage-60%25-yellow?style=flat)\n\n# Overview\n\n**aim:** opensource library which offers help to compare datasets (csv, database tables,classes) in a memory-limited environment  \n\n**license** BSD 2-Clause\n\nThis project is a pure c# port of the super useful python package [recordlinkage](https://recordlinkage.readthedocs.io/en/latest/about.html).\nBesides it tries to use the effective parts of the c# language (e.g. linq, dataflow).\n\n## features\n- string comparision with multiple string metrics\n- uses scoring method to calculate overall similarity \n- uses own datatable struture to reduce memory footprint (in comparsison to system.data.datatable)\n- uses dataflow to reduce memory footprint\n- uses parallelism to reduce runtime\n- limits: right now every datacell is string\n\n## plattforms:\nall plattform which supports [.NET 6.0](https://dotnet.microsoft.com/en-us/download/dotnet/6.0)\nso:\n\n- Linux\n- MacOs\n- Windows\n\n## minimal examples\nThis project should look and feel like using the pyhton equivalent:\n```c#       \n//we create some testdata //see UnitTest.TestDataPerson\nList\u003cTestDataPerson\u003e testDataPeopleA = new List\u003cTestDataPerson\u003e\n{\n    new TestDataPerson(\"Thomas\", \"Mueller\", \"Lindetrasse\", \"Testhausen\", \"12345\"),\n    new TestDataPerson(\"Thomas\", \"Mueller\", \"Lindenstrasse\", \"Testcity\", \"012345\"),\n    new TestDataPerson(\"Thomas\", \"Müller\", \"Lindenstrasse\", \"Testcity\", \"012345\"),\n    new TestDataPerson(\"Tomas\", \"Müller\", \"Lindenstroad\", \"Testhausen\", \"012342\"),\n    new TestDataPerson(\"Tomas\", \"Müller\", \"Lindenstroad\", \"Dorf\", \"012342\")\n};\nDataTableFeather tabA = TableConverter.CreateTableFeatherFromDataObjectList(testDataPeopleA);\n\n//we load some data from sqlite file\nDataTableFeather tabB = RecordLinkageNet.Util.SqliteReader.ReadTableFromSqliteFile(\"filenameof.sqlite\",\"testtablename\");\n\nConditionList conList = new ConditionList();\nCondition.StringMethod testMethod = Condition.StringMethod.JaroWinklerSimilarity;\nconList.String(\"NameFirst\", \"NameFirst\", testMethod);\nconList.String(\"Street\", \"Street\", testMethod);\nconList.String(\"PostalCode\", \"PostalCode\", Condition.StringMethod.Exact);\nconList.String(\"NameLast\", \"NameLast\", testMethod);\n\n//configure comparison\nConfiguration config = Configuration.Instance;\nconfig.AddIndex(new IndexFeather().Create(tabB, tabA));\nconfig.AddConditionList(conList);\nconfig.SetStrategy(Configuration.CalculationStrategy.WeightedConditionSum);\nconfig.SetNumberTransposeModus(NumberTransposeHelper.TransposeModus.LOG10); ;\n\n//we init a worker\nWorkScheduler workScheduler = new WorkScheduler();\nvar pipeLineCancellation = new CancellationTokenSource();//for optional cancellation\nvar resultTask = workScheduler.Compare(pipeLineCancellation.Token);\n\nawait resultTask;\n\nint amount = resultTask.Result.Count();\n```\n\nMore Details could be found at [Examples Repository](https://github.com/t2solve/RecordLinkageNetExamples)\n\nThe project implements mutliple metrics for string comparision as extensions:\n\n- HammingDistance\n- DamerauLevenshteinDistance\n- JaroDistance\n- JaroWinklerSimilarity\n- ShannonEntropyDistance\n\n```c# \nusing RecordLinkageNet.Core.Distance;\n \nvar result1 = \"foo\".HammingDistance(\"bar\");//3\nvar result2 = \"foo\".DamerauLevenshteinDistance(\"bar\");//3\nvar result3 = \"foo\".JaroWinklerSimilarity(\"bar\");//0\n```\nThe distances metrics are well tested with results from python lib [jellyfish](https://github.com/jamesturk/jellyfish).\n\n## structure:\n\n| folder | description |\n| ----------- | ----------- |\n| RecordLinkageNet | c# library code  |\n| UnitTest | test for the lib  |\n\n## thanks to\n- [jamesturk](https://github.com/jamesturk) for [jellyfish](https://github.com/jamesturk/jellyfish) and his c implementation of string metrics\n- [jeff-atwood](https://codereview.stackexchange.com/users/136/jeff-atwood) for [Shannon Entropy](https://codereview.stackexchange.com/a/909)\n- [wickedshimmy](https://gist.github.com/wickedshimmy) and [joannaksk](https://gist.github.com/joannaksk) for [basic Damerau Levenshtein Distance](https://gist.github.com/joannaksk/da110f9b05ff38d3f4ea4d149a0eb55e)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ft2solve%2Frecordlinkagenet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ft2solve%2Frecordlinkagenet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ft2solve%2Frecordlinkagenet/lists"}