{"id":14997617,"url":"https://github.com/kennethenevoldsen/spacy-wrap","last_synced_at":"2025-06-13T17:07:56.949Z","repository":{"id":37978362,"uuid":"453770672","full_name":"KennethEnevoldsen/spacy-wrap","owner":"KennethEnevoldsen","description":"spaCy-wrap is a wrapper library for spaCy for including fine-tuned transformers from Huggingface in your spaCy pipeline allowing you to include existing fine-tuned models within your SpaCy workflow.","archived":false,"fork":false,"pushed_at":"2024-04-15T16:37:36.000Z","size":2311,"stargazers_count":46,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-09T12:20:07.869Z","etag":null,"topics":["deep-learning","huggingface","huggingface-transformers","language-model","machine-learning","natural-language-processing","nlp","pytorch","spacy","spacy-extension","spacy-extensions","spacy-models","spacy-nlp","spacy-pipeline","spacy-transformers","text-classification","transformers"],"latest_commit_sha":null,"homepage":"https://KennethEnevoldsen.github.io/spacy-wrap/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KennethEnevoldsen.png","metadata":{"files":{"readme":"readme.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"citation.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-01-30T18:49:52.000Z","updated_at":"2024-07-01T00:43:08.000Z","dependencies_parsed_at":"2023-02-19T10:16:25.274Z","dependency_job_id":"cb6cefa3-5bb3-482f-98bb-35730048ae45","html_url":"https://github.com/KennethEnevoldsen/spacy-wrap","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/KennethEnevoldsen/spacy-wrap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KennethEnevoldsen%2Fspacy-wrap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KennethEnevoldsen%2Fspacy-wrap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KennethEnevoldsen%2Fspacy-wrap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KennethEnevoldsen%2Fspacy-wrap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KennethEnevoldsen","download_url":"https://codeload.github.com/KennethEnevoldsen/spacy-wrap/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KennethEnevoldsen%2Fspacy-wrap/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259686130,"owners_count":22896054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","huggingface","huggingface-transformers","language-model","machine-learning","natural-language-processing","nlp","pytorch","spacy","spacy-extension","spacy-extensions","spacy-models","spacy-nlp","spacy-pipeline","spacy-transformers","text-classification","transformers"],"created_at":"2024-09-24T17:05:02.721Z","updated_at":"2025-06-13T17:07:56.926Z","avatar_url":"https://github.com/KennethEnevoldsen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://github.com/kennethenevoldsen/spacy-wrap\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/KennethEnevoldsen/spacy-wrap/main/docs/_static/icon.png\" width=\"300\" align=\"right\" /\u003e\u003c/a\u003e\n# spaCy-wrap: For Wrapping fine-tuned transformers in spaCy pipelines\n\n[![PyPI version](https://badge.fury.io/py/spacy-wrap.svg)](https://pypi.org/project/spacy-wrap/)\n[![python version](https://img.shields.io/badge/Python-%3E=3.8-blue)](https://github.com/kennethenevoldsen/spacy-wrap)\n[![Code style: black](https://img.shields.io/badge/Code%20Style-Black-black)](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html)\n[![github actions pytest](https://github.com/kennethenevoldsen/spacy-wrap/actions/workflows/tests.yml/badge.svg)](https://github.com/kennethenevoldsen/spacy-wrap/actions)\n[![github actions docs](https://github.com/kennethenevoldsen/spacy-wrap/actions/workflows/documentation.yml/badge.svg)](https://kennethenevoldsen.github.io/spacy-wrap/)\n![github coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/KennethEnevoldsen/33fb85a2c440013df494c1fce884633c/raw/3813a0369fdd61b39a806b7b91839ff405ef809a/badge-spacy-wrap-coverage.json)\n\n\nspaCy-wrap is a minimal library intended for wrapping fine-tuned transformers from the [Huggingface model hub](https://huggingface.co/models?pipeline_tag=text-classification\u0026sort=downloads) in your spaCy pipeline allowing the inclusion of existing models within [SpaCy](https://spacy.io) workflows. \n\nAs far as possible it follows a similar API as [spacy-transformers](https://github.com/explosion/spacy-transformers).\n\n**NOTE**: Since the release of spaCy-wrap, Explosion released the [spacy-huggingface-pipelines](https://github.com/explosion/spacy-huggingface-pipelines) it takes the approach of wrapping the Huggingface pipeline as opposed to the transformer. That means token aggregation and conversion into spans happens at\nthe Huggingface pipeline, while in spaCy-wrap it happens at the logits of the model which can sometimes lead to unfortunate differences in results.\nI generally recommend using the spacy-huggingface-pipelines for most use cases, but if you need to use the transformer output more directly \nspaCy-wrap can have its uses.\n\n## Installation\n\nInstalling spacy-wrap is simple using pip:\n\n```\npip install spacy_wrap\n```\n\n## Examples\nThe following shows a simple example of how you can quickly add a fine-tuned transformer model from the Huggingface model hub for either [text classification](https://huggingface.co/models?pipeline_tag=text-classification\u0026sort=downloads), [named entity](https://huggingface.co/models?pipeline_tag=token-classification\u0026sort=downloads) or [token classification](https://huggingface.co/models?pipeline_tag=token-classification\u0026sort=downloads). \n\n### Sequence Classification\nIn this example, we will use a [model](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english) fine-tuned for sentiment classification on SST2. This model classifies whether a text is positive or negative. We will add this model to a blank English pipeline:\n\n\n```python\nimport spacy\nimport spacy_wrap\n\nnlp = spacy.blank(\"en\")\n\nconfig = {\n    \"doc_extension_trf_data\": \"clf_trf_data\",  # document extention for the forward pass\n    \"doc_extension_prediction\": \"sentiment\",  # document extention for the prediction\n    \"model\": {\n        # the model name or path of huggingface model\n        \"name\": \"distilbert-base-uncased-finetuned-sst-2-english\",  \n    },\n}\n\ntransformer = nlp.add_pipe(\"sequence_classification_transformer\", config=config)\n\ndoc = nlp(\"spaCy is a wonderful tool\")\n\nprint(doc.cats)\n# {'NEGATIVE': 0.001, 'POSITIVE': 0.999}\nprint(doc._.sentiment)\n# 'POSITIVE'\nprint(doc._.clf_trf_data)\n# TransformerData(wordpieces=...\n```\nThese pipelines can also easily be applied to multiple documents using the `nlp.pipe` as one would expect from a spaCy component:\n\n```python\ndocs = nlp.pipe(\n    [\n        \"I hate wrapping my own models\",\n        \"Isn't there a tool for this?!\",\n        \"spacy-wrap is great for wrapping models\",\n    ]\n)\n\nfor doc in docs:\n    print(doc._.sentiment)\n# 'NEGATIVE'\n# 'NEGATIVE'\n# 'POSITIVE'\n```\n\n\n \u003cbr /\u003e \n\n\u003cdetails\u003e\n  \u003csummary\u003e More Examples \u003c/summary\u003e\n\nIt is always nice to have more than one example. Here is another one where we add the Hate speech model for Danish to a blank Danish pipeline:\n\n```python\nimport spacy\nimport spacy_wrap\n\nnlp = spacy.blank(\"da\")\n\nconfig = {\n    \"doc_extension_trf_data\": \"clf_trf_data\",  # document extention for the forward pass\n    \"doc_extension_prediction\": \"hate_speech\",  # document extention for the prediction\n    # choose custom labels\n    \"labels\": [\"Not hate Speech\", \"Hate speech\"],\n    \"model\": {\n        \"name\": \"DaNLP/da-bert-hatespeech-detection\",  # the model name or path of huggingface model\n    },\n}\n\ntransformer = nlp.add_pipe(\"classification_transformer\", config=config)\n\ndoc = nlp(\"Senile gamle idiot\") # old senile idiot\n\ndoc._.clf_trf_data\n# TransformerData(wordpieces=...\ndoc._.hate_speech\n# \"Hate speech\"\ndoc._.hate_speech_prob\n# {'prob': array([0.013, 0.987], dtype=float32), 'labels': ['Not hate Speech', 'Hate speech']}\n```\n\n\u003c/details\u003e\n\n\u003cbr /\u003e \n\n\n### Token Classification\nWe can also use the model for token classification: \n\n```python\nimport spacy\nimport spacy_wrap\nnlp = spacy.blank(\"en\")\n\nconfig = {\"model\": {\"name\": \"vblagoje/bert-english-uncased-finetuned-pos\"}, \n          # \"predictions_to\": [\"pos\"]  # optional, can be \"pos\", \"tag\" or \"ents\"\n}\n\nsnlp.add_pipe(\"token_classification_transformer\", config=config)\n\ntext = \"My name is Wolfgang and I live in Berlin\"\n\ndoc = nlp(text)\nprint(doc._.tok_clf_predictions)\n# ['PRON', 'NOUN', 'AUX', 'PROPN', 'CCONJ', 'PRON', 'VERB', 'ADP', 'PROPN']\n```\n\nBy default, spacy-wrap will automatically detect it the labels follow the universal POS tags as well. If so it will also assign it to the `token.pos`, similar regular spacy pipelines:\n\n```python\nprint(doc[0].pos_)\n# 'PRON'\n```\n\n### Named Entity Recognition\nIn this example, we use a model fine-tuned for named entity recognition. spacy-wrap will in this case infer from the IOB tags that the model is intended for named entity recognition and assign it to `doc.ents`.\n\n```python\nimport spacy\nimport spacy_wrap\nnlp = spacy.blank(\"en\")\n\n# specify model from the hub\nconfig = {\"model\": {\"name\": \"dslim/bert-base-NER\"}, \n          \"predictions_to\": [\"ents\"]} # forced to be named entity recognition, if left out it will be estimated from the labels\n\n# add it to the pipe\nnlp.add_pipe(\"token_classification_transformer\", config=config)\n\ndoc = nlp(\"My name is Wolfgang and I live in Berlin.\")\n\nprint(doc.ents)\n# (Wolfgang, Berlin)\n```\n\n# 📖 Documentation\n\n| Documentation              |                                             |\n| -------------------------- | ------------------------------------------- |\n| 🔧 **[Installation]**       | Installation instructions for spacy-wrap.   |\n| 📰 **[News and changelog]** | New additions, changes and version history. |\n| 🎛 **[Documentation]**      | The reference for spacy-wrap's API.         |\n\n[Documentation]: https://kennethenevoldsen.github.io/spacy-wrap/index.html\n[Installation]: https://kennethenevoldsen.github.io/spacy-wrap/installation.html\n[News and changelog]: https://kennethenevoldsen.github.io/spacy-wrap/news.html\n\n# 💬 Where to ask questions\n\n| Type                           |                        |\n| ------------------------------ | ---------------------- |\n| 🚨 **FAQ**                      | [FAQ]                  |\n| 🚨 **Bug Reports**              | [GitHub Issue Tracker] |\n| 🎁 **Feature Requests \u0026 Ideas** | [GitHub Issue Tracker] |\n| 👩‍💻 **Usage Questions**          | [GitHub Discussions]   |\n| 🗯 **General Discussion**       | [GitHub Discussions]   |\n\n\n[FAQ]: https://kennethenevoldsen.github.io/spacy-wrap/faq.html\n[github issue tracker]: https://github.com/kennethenevoldsen/spacy-wrap/issues\n[github discussions]: https://github.com/kennethenevoldsen/spacy-wrap/discussions\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkennethenevoldsen%2Fspacy-wrap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkennethenevoldsen%2Fspacy-wrap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkennethenevoldsen%2Fspacy-wrap/lists"}