{"id":15014054,"url":"https://github.com/explosion/spacy-alignments","last_synced_at":"2025-10-19T14:31:54.269Z","repository":{"id":42452919,"uuid":"319569270","full_name":"explosion/spacy-alignments","owner":"explosion","description":"💫 A spaCy package for Yohei Tamura's Rust tokenizations library ","archived":false,"fork":false,"pushed_at":"2023-11-03T15:15:32.000Z","size":22,"stargazers_count":27,"open_issues_count":0,"forks_count":4,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-01-29T18:38:16.786Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/explosion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-08T08:07:25.000Z","updated_at":"2024-09-08T16:33:00.000Z","dependencies_parsed_at":"2023-01-29T21:46:32.506Z","dependency_job_id":"89b6032b-9bef-4667-880e-51c4d55ac23c","html_url":"https://github.com/explosion/spacy-alignments","commit_stats":{"total_commits":17,"total_committers":3,"mean_commits":5.666666666666667,"dds":"0.17647058823529416","last_synced_commit":"a4845b2e5222fbb7c2ad0e9d6e7ba8257a6c86af"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fspacy-alignments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fspacy-alignments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fspacy-alignments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/explosion%2Fspacy-alignments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/explosion","download_url":"https://codeload.github.com/explosion/spacy-alignments/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237152769,"owners_count":19263780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T19:45:07.791Z","updated_at":"2025-10-19T14:31:53.890Z","avatar_url":"https://github.com/explosion.png","language":"Python","readme":"\u003ca href=\"https://explosion.ai\"\u003e\u003cimg src=\"https://explosion.ai/assets/img/logo.svg\" width=\"125\" height=\"125\" align=\"right\" /\u003e\u003c/a\u003e\n\n# spacy-alignments: Align tokenizations for spaCy + transformers\n\nA spaCy package for Yohei Tamura's Rust\n[tokenizations](https://github.com/tamuhey/tokenizations/) library with Python\nbindings.\n\n## Installation\n\n```\npip install -U pip setuptools wheel\npip install spacy-alignments\n```\n\nIf no binary wheel is available for your platform, you will need to [install\nRust](https://www.rust-lang.org/tools/install) in order to build\n`spacy-alignments` from source.\n\n## spacy-alignments vs. pytokenizations\n\nThe `spacy_alignments` module is a drop-in replacement for `tokenizations`:\n\n```python\nimport spacy_alignments as tokenizations\na2b, b2a = tokenizations.get_alignments([\"å\", \"BC\"], [\"abc\"])\nassert a2b == [[0], [0]]\nassert b2a == [[0, 1]]\n```\n\nThe only difference between this package and the original\n[`pytokenizations`](https://pypi.org/project/pytokenizations/) is that it\nswitches the build system to `setuptools-rust` to make it easier for us at\nExplosion to build source and binary packages for a wider range of platforms.\n\n## Bug reports and other issues\n\nPlease use [spaCy's issue tracker](https://github.com/explosion/spaCy/issues) to report a bug, or open a new thread on the\n[discussion board](https://github.com/explosion/spaCy/discussions)\nfor any other issue.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexplosion%2Fspacy-alignments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexplosion%2Fspacy-alignments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexplosion%2Fspacy-alignments/lists"}