{"id":15014160,"url":"https://github.com/davidberenstein1957/crosslingual-coreference","last_synced_at":"2026-04-02T18:45:25.290Z","repository":{"id":42655082,"uuid":"474043389","full_name":"davidberenstein1957/crosslingual-coreference","owner":"davidberenstein1957","description":"A multi-lingual approach to AllenNLP CoReference Resolution along with a wrapper for spaCy.","archived":false,"fork":false,"pushed_at":"2024-04-16T13:20:41.000Z","size":561,"stargazers_count":109,"open_issues_count":10,"forks_count":19,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-01-07T06:34:53.741Z","etag":null,"topics":["coreference","coreference-resolution","hacktoberfest","natural-language-processing","nlp","python","spacy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidberenstein1957.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-25T14:31:51.000Z","updated_at":"2025-12-16T08:23:49.000Z","dependencies_parsed_at":"2023-07-18T07:14:23.006Z","dependency_job_id":"bd726c7e-e54a-485f-97ba-751f910aa755","html_url":"https://github.com/davidberenstein1957/crosslingual-coreference","commit_stats":{"total_commits":48,"total_committers":5,"mean_commits":9.6,"dds":0.5,"last_synced_commit":"195bd6ba351c1d3aea714a85c7312365c36d12e5"},"previous_names":["pandora-intelligence/crosslingual-coreference"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/davidberenstein1957/crosslingual-coreference","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidberenstein1957%2Fcrosslingual-coreference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidberenstein1957%2Fcrosslingual-coreference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidberenstein1957%2Fcrosslingual-coreference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidberenstein1957%2Fcrosslingual-coreference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidberenstein1957","download_url":"https://codeload.github.com/davidberenstein1957/crosslingual-coreference/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidberenstein1957%2Fcrosslingual-coreference/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31313300,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coreference","coreference-resolution","hacktoberfest","natural-language-processing","nlp","python","spacy"],"created_at":"2024-09-24T19:45:16.544Z","updated_at":"2026-04-02T18:45:25.268Z","avatar_url":"https://github.com/davidberenstein1957.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Crosslingual Coreference\nCoreference is amazing but the data required for training a model is very scarce. In our case, the available training for non-English languages also proved to be poorly annotated. Crosslingual Coreference, therefore, uses the assumption a trained model with English data and cross-lingual embeddings should work for languages with similar sentence structures.\n\n[![Current Release Version](https://img.shields.io/github/release/pandora-intelligence/crosslingual-coreference.svg?style=flat-square\u0026logo=github)](https://github.com/pandora-intelligence/crosslingual-coreference/releases)\n[![pypi Version](https://img.shields.io/pypi/v/crosslingual-coreference.svg?style=flat-square\u0026logo=pypi\u0026logoColor=white)](https://pypi.org/project/crosslingual-coreference/)\n[![PyPi downloads](https://static.pepy.tech/personalized-badge/crosslingual-coreference?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=orange\u0026left_text=pip%20downloads)](https://pypi.org/project/crosslingual-coreference/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square)](https://github.com/ambv/black)\n\n# Install\n\n```\npip install crosslingual-coreference\n```\n# Quickstart\n```python\nfrom crosslingual_coreference import Predictor\n\ntext = (\n    \"Do not forget about Momofuku Ando! He created instant noodles in Osaka. At\"\n    \" that location, Nissin was founded. Many students survived by eating these\"\n    \" noodles, but they don't even know him.\"\n)\n\n# choose minilm for speed/memory and info_xlm for accuracy\npredictor = Predictor(\n    language=\"en_core_web_sm\", device=-1, model_name=\"minilm\"\n)\n\nprint(predictor.predict(text)[\"resolved_text\"])\nprint(predictor.pipe([text])[0][\"resolved_text\"])\n# Note you can also get 'cluster_heads' and 'clusters'\n# Output\n#\n# Do not forget about Momofuku Ando!\n# Momofuku Ando created instant noodles in Osaka.\n# At Osaka, Nissin was founded.\n# Many students survived by eating instant noodles,\n# but Many students don't even know Momofuku Ando.\n```\n![](https://raw.githubusercontent.com/Pandora-Intelligence/crosslingual-coreference/master/img/example_en.png)\n\n## Models\nAs of now, there are two models available \"spanbert\", \"info_xlm\", \"xlm_roberta\", \"minilm\", which scored 83, 77, 74 and 74 on OntoNotes Release 5.0 English data, respectively.\n- The \"minilm\" model is the best quality speed trade-off for both mult-lingual and english texts.\n- The \"info_xlm\" model produces the best quality for multi-lingual texts.\n- The AllenNLP \"spanbert\" model produces the best quality for english texts.\n\n## Chunking/batching to resolve memory OOM errors\n\n```python\nfrom crosslingual_coreference import Predictor\n\npredictor = Predictor(\n    language=\"en_core_web_sm\",\n    device=0,\n    model_name=\"minilm\",\n    chunk_size=2500,\n    chunk_overlap=2,\n)\n```\n\n## Use spaCy pipeline\n```python\nimport spacy\n\ntext = (\n    \"Do not forget about Momofuku Ando! He created instant noodles in Osaka. At\"\n    \" that location, Nissin was founded. Many students survived by eating these\"\n    \" noodles, but they don't even know him.\"\n)\n\n\nnlp = spacy.load(\"en_core_web_sm\")\nnlp.add_pipe(\n    \"xx_coref\", config={\"chunk_size\": 2500, \"chunk_overlap\": 2, \"device\": 0}\n)\n\ndoc = nlp(text)\nprint(doc._.coref_clusters)\n# Output\n#\n# [[[4, 5], [7, 7], [27, 27], [36, 36]],\n# [[12, 12], [15, 16]],\n# [[9, 10], [27, 28]],\n# [[22, 23], [31, 31]]]\nprint(doc._.resolved_text)\n# Output\n#\n# Do not forget about Momofuku Ando!\n# Momofuku Ando created instant noodles in Osaka.\n# At Osaka, Nissin was founded.\n# Many students survived by eating instant noodles,\n# but Many students don't even know Momofuku Ando.\nprint(doc._.cluster_heads)\n# Output\n#\n# {Momofuku Ando: [5, 6],\n# instant noodles: [11, 12],\n# Osaka: [14, 14],\n# Nissin: [21, 21],\n# Many students: [26, 27]}\n```\n### Visualize spacy pipeline\nThis only works with spacy \u003e= 3.3.\n```python\nimport spacy\nfrom spacy.tokens import Span\nfrom spacy import displacy\n\ntext = (\n    \"Do not forget about Momofuku Ando! He created instant noodles in Osaka. At\"\n    \" that location, Nissin was founded. Many students survived by eating these\"\n    \" noodles, but they don't even know him.\"\n)\n\nnlp = spacy.load(\"nl_core_news_sm\")\nnlp.add_pipe(\"xx_coref\", config={\"model_name\": \"minilm\"})\ndoc = nlp(text)\nspans = []\nfor idx, cluster in enumerate(doc._.coref_clusters):\n    for span in cluster:\n        spans.append(\n            Span(doc, span[0], span[1]+1, str(idx).upper())\n        )\n\ndoc.spans[\"custom\"] = spans\n\ndisplacy.render(doc, style=\"span\", options={\"spans_key\": \"custom\"})\n```\n\n## More Examples\n![](https://raw.githubusercontent.com/Pandora-Intelligence/crosslingual-coreference/master/img/example_total.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidberenstein1957%2Fcrosslingual-coreference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidberenstein1957%2Fcrosslingual-coreference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidberenstein1957%2Fcrosslingual-coreference/lists"}