{"id":17508151,"url":"https://github.com/nickcrews/spacy-address","last_synced_at":"2025-04-23T12:31:57.854Z","repository":{"id":258387262,"uuid":"868615646","full_name":"NickCrews/spacy-address","owner":"NickCrews","description":"Parse oneline US addresses using a spaCy NER model trained on OSM data","archived":false,"fork":false,"pushed_at":"2024-12-04T10:32:18.000Z","size":500,"stargazers_count":7,"open_issues_count":5,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-29T23:41:14.591Z","etag":null,"topics":["address","address-parsing","osm","osm-data","spacy","spacy-nlp","usaddress"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NickCrews.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-06T20:34:39.000Z","updated_at":"2025-02-12T21:16:51.000Z","dependencies_parsed_at":"2024-10-18T19:16:06.024Z","dependency_job_id":"cf87728b-b360-4b3a-8dd0-c0b8363ce873","html_url":"https://github.com/NickCrews/spacy-address","commit_stats":null,"previous_names":["nickcrews/spacy-address"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NickCrews%2Fspacy-address","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NickCrews%2Fspacy-address/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NickCrews%2Fspacy-address/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NickCrews%2Fspacy-address/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NickCrews","download_url":"https://codeload.github.com/NickCrews/spacy-address/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250434951,"owners_count":21430189,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["address","address-parsing","osm","osm-data","spacy","spacy-nlp","usaddress"],"created_at":"2024-10-20T04:12:32.442Z","updated_at":"2025-04-23T12:31:57.802Z","avatar_url":"https://github.com/NickCrews.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spacy-address\n\nUse [spaCy](https://spacy.io/)'s NER pipeline to parse oneline US addresses\n\nUses the the labeled data from [usaddress](https://github.com/datamade/usaddress)\nwith spaCy's very easy [training infrastructure](https://spacy.io/usage/training)\n\nInspired by the code and blog from https://github.com/swapnil-saxena/address-parser.\n\n## Usage\n\nThere are currently two models, `en-us-address-ner-sm` and `en-us-address-ner-lg`,\nfollowing the naming conventions for small and large that spaCy uses.\n\n### en-us-address-ner-sm\n\nYou probably want this one. Much better efficiency for not much worse accuracy.\n\nAs of 2024-10-06:\n\n- F1 score for NER of .978\n- model size of ~5MB\n- on my 2021 M1 MacBook Pro, tags 1000 addresses in ~.2 sec\n\n### en-us-address-ner-lg\n\nMuch larger and slower, a little more accurate.\n\nAs of 2024-10-06:\n\n- F1 score for NER of .982\n- model size of ~420MB\n- on my 2021 M1 MacBook Pro, tags 1000 addresses in ~2 sec\n\nYou can find the released models in various [github releases](https://github.com/NickCrews/spacy-address/releases).\nThere, you can see the most up to date model size and F1 score.\nThe speed isn't reported anywhere easily, unfortunately.\n\nYou can install from a release directly with pip:\n\n```bash\npython -m pip install \"en-us-address-ner-sm @ https://github.com/NickCrews/spacy-address/releases/download/20241029-205717-sm/en_us_address_ner_sm-0.0.0-py3-none-any.whl\"\n```\n\nNow, this is accessible from python:\n\n```python\nimport spacy\n\nnlp = spacy.load(\"en-us-address-ner-sm\")\ndoc = nlp(\"CO John SMITH, 123 E St elias stree S,   Oklahoma City, OK 99507-1234\")\nfor ent in doc.ents:\n    print(f\"{ent.text} ({ent.label_})\")\n# CO John SMITH (Recipient)\n# 123 (AddressNumber)\n# E (StreetNamePreDirectional)\n# St elias (StreetName)            # St isn't confused as an abbreviation for street!\n# stree (StreetNamePostType)       # Typos are tagged correctly!\n# S (StreetNamePostDirectional)\n# Oklahoma City (PlaceName)        # Oklahoma isn't confused as a state!\n# OK (StateName)\n# 99507-1234 (ZipCode)\n\n# For convenience I include the taggings for autcomplete, IDE support, etc\nfrom en_us_address_ner_sm import labels\n[ent.text for ent in doc.ents if ent.label_ == labels.StateName]\n# ['OK']\n```\n\nThis uses the tags from the\n\"United States Thoroughfare, Landmark, and Postal Address Data Standard (Publication 28)\".\nSee [labels.py](./labels.py) for details\n\n## Goals\n\nI have tried using various probabilstic address parsers/taggers. None of them\nquite suited my needs. Here is what I was aiming for\n\n- Speed: I am doing bulk processing, I want to parse 10s of millions of addresses in \u003c10 minutes (we're not there yet, see timings above.)\n- Support for PO Boxes.\n- Support for finegrained tagging, eg split \"Aspen Avenue\" into (\"Aspen\", StreetName), (\"Avenue\", StreetPostType).\n  I need this for performing entity resolution, I only really care about the street name.\n- Python API\n- An installation story that isn't a total pain.\n\n## Comparison vs Peers\nHere is an incomplete list of how this project compares with some other projects\nI've tried:\n\n### [Libpostal](https://github.com/openvenues/libpostal)\n\n- appears fairly unmaintained. It still works, but mostly all development\nappears to keep the lights on.\n- pain in the butt to deploy. You have to build from source using git, gcc, no `pip`.\n- Requires a few GB of data (this amount might not be totally correct, but anyways a lot)\n- it's not thread safe, and hard to integrate with other projects.\nSee https://github.com/Maxxen/duckdb-postal/issues/1\n- Uses OpenStreetMap, with addresses from around the world\n- Uses [a slightly different set of tags](https://github.com/OpenCageData/address-formatting)\nfrom this project. They are less USPS-specific, more applicable for worldwide addresses,\nbut aren't as granular, ie we tag \"NW\" as a StreetPreDirectional, they don't\ngo to that level of detail. We tag post office boxes and rural routes, they don't.\nMight be more differences than what I have here. I'm open to expanding to a different\ntagging scheme, but I NEED to support USPS PO box tagging, and of separating out the\nstreet name (\"Aspen\") vs street type (\"Avenue\").\n- According to [their very in depth blog post](https://medium.com/@albarrentine/statistical-nlp-on-openstreetmap-b9d573e6cc86),\nit's quite more technically specialized than the other solutions here.\nDoes a lot of hand-written, address-domain specific things like normalizing st -\u003e street,\nhand-parsing numbers with custom grammar rules, etc.\n- uses something called a \"averaged perceptron\" which they claim is much faster than\n conditional random fields (as usaddress uses). IDK what spacy is doing...\n- I would be very interested in a speed comparison vs us, building a c app\nthat does bulk processing.\n- They claim a ~99.5% accuracy, which is similar to what we achieve.\nI would love to test this though.\n\n### [PyPostal](https://github.com/openvenues/pypostal)\nThe python bindings to libpostal.\n- has same installation problems as the above.\n- only provides a one-by-one inference API, no batch API\n- I would be very interested in a speed comparison vs us.\n\n### [USAddress](https://github.com/datamade/usaddress)\n- Python only. Installable with `pip`!\n- Not actively maintained, but basic maintenance happens.\n- Older architecture (conditional random fields, but that doesn't mean anything to me...)\n- Uses same set of taggings as us.\n- Also uses OpenStreetMap data.\n- Tokenizes in python using hardcoded rules. in spacy, the tokenizer is another trained model.\n  IDK really the consequences of this.\n- I would be very interested in a speed comparison vs us.\n- IDK what their accuracy numbers are.\n\n## Licence\n\nReleased under the MIT license.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickcrews%2Fspacy-address","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnickcrews%2Fspacy-address","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickcrews%2Fspacy-address/lists"}