{"id":19811385,"url":"https://github.com/diging/author-disambiguation","last_synced_at":"2025-05-01T08:32:56.240Z","repository":{"id":47139274,"uuid":"73090067","full_name":"diging/author-disambiguation","owner":"diging","description":null,"archived":false,"fork":false,"pushed_at":"2021-09-12T11:12:35.000Z","size":1821,"stargazers_count":12,"open_issues_count":1,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-06T11:51:50.980Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/diging.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-11-07T15:06:28.000Z","updated_at":"2024-11-07T11:53:05.000Z","dependencies_parsed_at":"2022-09-07T16:40:38.283Z","dependency_job_id":null,"html_url":"https://github.com/diging/author-disambiguation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diging%2Fauthor-disambiguation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diging%2Fauthor-disambiguation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diging%2Fauthor-disambiguation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diging%2Fauthor-disambiguation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/diging","download_url":"https://codeload.github.com/diging/author-disambiguation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251847841,"owners_count":21653583,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T09:26:10.351Z","updated_at":"2025-05-01T08:32:55.688Z","avatar_url":"https://github.com/diging.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Author-Disambiguation\nThis is a research project in which we address the problem of disambiguation in a co authorship network. Say, we are looking at all the papers by the same author. The same author may have slight variations in his first name or last name or both.\n\nFor example, the author David Albertini has the following variations in his name :\n* D.F. Albertini\n* David. F. Albertini\n* D. Albertini\n* AlbertniDF\n\nNow, how do we say that 2 papers belong to the same author career? This is technically the problem of disambiguation. In literature some also call it as Record Linkage.\n\nWe took a machine learning approach to solve this problem. It turns out to be binary classification problem where 2 Author-Paper instances either match or don't match. (1 or 0).\n\n### Clone this Repo\nClone this repository into whatever directory you'd like to work on it from:\n\n```bash\ngit clone https://github.com/diging/author-disambiguation.git\n```\n\n### Install the following\n*   [Python 2.7](https://www.python.org/download/releases/2.7/)\n*   [Tethne](http://pythonhosted.org/tethne/)\n    *   `pip install tethne`\n*   [pandas](http://pandas.pydata.org/)\n    *   `pip install pandas`\n*   [scikit-learn](http://scikit-learn.org/stable/)\n    *   `pip install -U scikit-learn`\n\n### Files \n\n* `PaperParser.py`\nThis class uses [Tethne](http://pythonhosted.org/tethne/) to parse WOS tagged-file data and write the output to a CSV file. \nThen we can use the class `DataAnalysisTool.py` on the output csv to perform data analysis\n\n    * There are 2 possible use-cases\n        \n        ```python\n        parser = PaperParser('/Users/aosingh/TethneDataAnalysis/MBL History Data/1971/Albertini_David.txt',\n                             '/Users/aosingh/AuthorDisambiguation/Dataset',)\n        parser.parseFile()\n        ```\n        \n        ```python\n        parser = PaperParser('/Users/aosingh/TethneDataAnalysis/MBL History Data/',\n                             '/Users/aosingh/AuthorDisambiguation/Dataset', \n                             output_filename='records.csv')\n        parser.parseDirectory()\n        ```\n\n\n* `DataAnalysisTool.py`\nThis class has methods and tools to analyse a bunch of (World of Science)WOS papers objects. DataSet is read from a CSV. \nA CSV file of expected format can be easily created using the class `PaperParser.py` as explained above.\n\n    * An example usage of this analysis tool can be to get all papers based on different variations of the first names and last names.\n    \n        ```python\n          fileName = '/Users/aosingh/AuthorDisambiguation/Dataset/Albertini_David.csv' #this CSV is generated using the class PaperParser.py\n          analyzer = DataAnalysisTool(fileName) # Please check the class DataAnalysisTool.py for more details\n          ALBERTINI_FIRSTNAME = ['DAVID', 'DF', 'DAVID F', 'D F', 'D']\n          ALBERITNI_LASTNAME = ['ALBERTINI', 'ALBERTIN', 'ALBERTINDF']\n          papers = analyzer.getPapersForAuthor(ALBERITNI_LASTNAME, ALBERTINI_FIRSTNAME)\n        ```\n     \n      \n\n* `DistanceMetric.py`\nWe can define various similarity metrics in this class. \n\n    * As of now, I have defined a method to calculate cosine similarity. \n    \n        ```python\n        input1 = \"CARNEGIE INST WASHINGTON,DEPT EMBRYOL\"\n        input2 = \"CARNEGIE INST WASHINGTON,DEPT\"\n        vector1 = sentence_to_vector(input1) #Counter({'EMBRYOL': 1, 'WASHINGTON': 1, 'INST': 1, 'CARNEGIE': 1, 'DEPT': 1})\n        vector2 = sentence_to_vector(input2) #Counter({'WASHINGTON': 1, 'INST': 1, 'CARNEGIE': 1, 'DEPT': 1})\n        cosine_similarity(vector1, vector2)  #0.894427191\n        ```\n\n\n* `TrainingDataGenerator.py`\nThis class is responsible for generating Training records. By training records, we mean the following 2 things.\n\n    * Generate records in the form of : AUTHOR_INSTANCE_1, AUTHOR_INSTANCE_2, MATCH(0,1). The corresponding output file is called `train.csv`\n    \n        To be precise, following are the column names in the file `train.csv`\n        ```python \n                   ['FIRST_NAME1', \n                   'FIRST_NAME2', \n                   'LAST_NAME1', \n                   'LAST_NAME2',\n                   'EMAILADDRESS1', \n                   'EMAILADDRESS2', \n                   'INSTITUTE1', \n                   'INSTITUTE2',\n                   'AUTHOR_KW1', \n                   'AUTHOR_KW2', \n                   'COAUTHORS1', \n                   'COAUTHORS2', \n                   'MATCH']\n        ```\n    * Scores in between the 2 AUTHOR INSTANCES. Each column is a score in between 0 and 1. We will train our classifiers on these records. The corresponding output file is called `scores.csv`\n    \n        To be precise, following are the column names in the file `scores.csv`\n        ```python \n                    ['INSTIT_SCORE',\n                    'BOTH_NAME_SCORE',\n                    'FNAME_SCORE',\n                    'FNAME_PARTIAL_SCORE',\n                    'LNAME_SCORE',\n                    'LNAME_PARTIAL_SCORE',\n                    'EMAIL_ADDR_SCORE',\n                    'AUTH_KW_SCORE',\n                    'COAUTHOR_SCORE',\n                    'MATCH']\n        ```\n    * Example of usage of this class\n        \n        ```python\n         fileName = '/Users/aosingh/AuthorDisambiguation/Dataset/Albertini_David.csv' #this CSV is generated using the class PaperParser.py\n         analyzer = DataAnalysisTool(fileName) # Please check the class DataAnalysisTool.py for more details\n         ALBERTINI_FIRSTNAME = ['DAVID', 'DF', 'DAVID F', 'D F', 'D']\n         ALBERITNI_LASTNAME = ['ALBERTINI', 'ALBERTIN', 'ALBERTINDF']\n         papers = analyzer.getPapersForAuthor(ALBERITNI_LASTNAME, ALBERTINI_FIRSTNAME)\n         training_data_generator = TrainingDataGenerator(papers, random=False)\n         training_data_generator.generate_records() # generate train.csv.\n         training_data_generator.calculate_scores() # Generate scores.csv\n        ```\n    \n\n        \n   \n\n\n\n\n    \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiging%2Fauthor-disambiguation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiging%2Fauthor-disambiguation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiging%2Fauthor-disambiguation/lists"}