https://github.com/diyclassics/la_core_web_md
spaCy-compatible md core model for Latin
https://github.com/diyclassics/la_core_web_md
Last synced: over 1 year ago
JSON representation
spaCy-compatible md core model for Latin
- Host: GitHub
- URL: https://github.com/diyclassics/la_core_web_md
- Owner: diyclassics
- License: mit
- Created: 2023-04-07T14:10:53.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-05-04T20:46:17.000Z (about 3 years ago)
- Last Synced: 2025-03-24T11:56:52.505Z (over 1 year ago)
- Language: Jupyter Notebook
- Homepage:
- Size: 1.87 MB
- Stars: 3
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 🪐 spaCy Project: la_core_web_md
Code required to train spaCy-compatible md core model for Latin, i.e pipeline with POS tagger, morphologizer, lemmatizer, dependency parser, and NER trained on all available Latin UD treebanks, i.e. Perseus, PROIEL, ITTB, UDante, and LLCT (see below). The model contains floret vectors (50,000) trained on Wikipedia, Oscar, and UD. NER is based on custom tagged data based on tagger output and manual annotation, supplemented by data from the Herodotos Project; this data is included in `assets/ner/`.
## 📋 project.yml
The [`project.yml`](project.yml) defines the data assets required by the
project, as well as the available commands and workflows. For details, see the
[spaCy projects documentation](https://spacy.io/usage/projects).
### ⏯ Commands
The following commands are defined by the project. They
can be executed using [`spacy project run [name]`](https://spacy.io/api/cli#project-run).
Commands are only re-run if their inputs have changed.
| Command | Description |
| --- | --- |
| `assets` | Download assets |
| `preprocess` | Convert different UD treebanks so that they use shared formatting, feature defs, etc. |
| `convert` | Convert the data to spaCy's format |
| `norm-corpus` | Convert norm attribute to u-v and i-j norm |
| `init-labels` | Initialize labels for components |
| `train` | Train tagger/parser/etc. on Latin UD treebanks |
| `evaluate` | Evaluate model on the test data and save the metrics |
| `convert-ner` | Convert the NER data to spaCy's binary format |
| `train-ner` | Train the NER model for the model |
| `assemble` | Assemble core model, i.e. add NER component to dep model |
| `assemble-meta` | Assemble meta.json files so that all metrics are included |
| `package` | Package the trained core model |
| `document` | Document core_web_md |
| `clean` | Remove intermediate files |
### ⏭ Workflows
The following workflows are defined by the project. They
can be executed using [`spacy project run [name]`](https://spacy.io/api/cli#project-run)
and will run the specified commands in order. Commands are only re-run if their
inputs have changed.
| Workflow | Steps |
| --- | --- |
| `all` | `assets` → `preprocess` → `convert` → `norm-corpus` → `init-labels` → `train` → `evaluate` → `convert-ner` → `train-ner` → `assemble` → `assemble-meta` → `package` → `document` |
### 🗂 Assets
The following assets are defined by the project. They can
be fetched by running [`spacy project assets`](https://spacy.io/api/cli#project-assets)
in the project directory.
| File | Source | Description |
| --- | --- | --- |
| `assets/original/UD_Latin-Perseus` | Git | |
| `assets/original/UD_Latin-PROIEL` | Git | |
| `assets/original/UD_Latin-ITTB` | Git | |
| `assets/original/UD_Latin-LLCT` | Git | |
| `assets/original/UD_Latin-UDante` | Git | |
## Install
- To install the current version...
- `pip install https://huggingface.co/diyclassics/la_core_web_md/resolve/main/la_core_web_md-any-py3-none-any.whl`
## Model repository
- The model itself can be found here:
- https://huggingface.co/diyclassics/la_core_web_md
## Current version
| Feature | Description |
| --- | --- |
| **Name** | `la_core_web_md` |
| **Version** | `3.5.2` |
| **spaCy** | `>=3.5.2,<3.6.0` |
| **Default Pipeline** | `normer`, `tok2vec`, `tagger`, `morphologizer`, `trainable_lemmatizer`, `parser`, `lemma_fixer`, `ner` |
| **Components** | `senter`, `normer`, `tok2vec`, `tagger`, `morphologizer`, `trainable_lemmatizer`, `parser`, `lemma_fixer`, `ner` |
| **Vectors** | -1 keys, 50000 unique vectors (300 dimensions) |
| **Sources** | UD_Latin-Perseus
UD_Latin-PROIEL
UD_Latin-ITTB
UD_Latin-LLCT
UD_Latin-UDante |
| **License** | `MIT` |
| **Author** | [Patrick J. Burns; with Nora Bernhardt [ner], Tim Geelhaar [tagger, morphologizer, parser, ner], Vincent Koch [ner]](https://diyclassics.github.io/) |
### Accuracy
| Type | Score |
| --- | --- |
| `ENTS_F` | 85.62 |
| `ENTS_P` | 82.37 |
| `ENTS_R` | 89.12 |
| `NER_LOSS` | 4022.72 |
| `NER_TOK2VEC_LOSS` | 519.13 |
| `SENTS_F` | 93.30 |
| `SENTS_P` | 92.70 |
| `SENTS_R` | 93.90 |
| `TAG_ACC` | 93.57 |
| `POS_ACC` | 96.87 |
| `MORPH_ACC` | 91.75 |
| `LEMMA_ACC` | 94.17 |
| `DEP_UAS` | 81.94 |
| `DEP_LAS` | 75.86 |
| `TOK2VEC_LOSS` | 8302715.33 |
| `TAGGER_LOSS` | 927840.47 |
| `MORPHOLOGIZER_LOSS` | 1996080.55 |
| `TRAINABLE_LEMMATIZER_LOSS` | 769005.09 |
| `PARSER_LOSS` | 6609683.39 |
NB: For full details on tags etc., see the README.md in the model package.
### Bibliography
- Cecchini, F.M., Passarotti, M., Marongiu, P., and Zeman, D. 2018. “Challenges in Converting the Index Thomisticus Treebank into Universal Dependencies.” In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018). 27–36.
- Cecchini, F.M., Sprugnoli, R., Moretti, G., and Passarotti, M. 2020. “UDante: First Steps Towards the Universal Dependencies Treebank of Dante’s Latin Works.” In Dell’Orletta, F., Monti, J., and Tamburini, F. eds. Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-It 2020. Accademia University Press. 99–105. doi:10.4000/books.aaccademia.8653. http://books.openedition.org/aaccademia/8653.
- Celano, G.G.A., Crane, G., Almas, B., and et al. 2014. “The Ancient Greek and Latin Dependency Treebank v.2.1.” https://perseusdl.github.io/treebank_data/.
- Erdmann, A., Wrisley, D.J., Allen, B., Brown, C., Bodénès, S.C., Elsner, M., Feng, Y., Joseph, B., Joyeaux-Prunel, B., and Marneffe, M.-C. 2019. “Practical, Efficient, and Customizable Active Learning for Named Entity Recognition in the Digital Humanities.” In Proceedings of North American Association of Computational Linguistics (NAACL 2019). Minneapolis, Minnesota.
- Haug, D.T., and Jøhndal, M. 2008. “Creating a Parallel Treebank of the Old Indo-European Bible Translations.” In Proceedings of the Second Workshop on Language Technology for Cultural Heritage Data (LaTeCH 2008). 27–34.
- Honnibal, M., and Montani, I. 2023. “SpaCy: Industrial-Strength Natural Language Processing in Python” (version v. 3.5.0). https://spacy.io/.
- Korkiakangas, T. 2021. “Late Latin Charter Treebank: Contents and Annotation.” Corpora 16 (2) (August 1): 191–203. doi:10.3366/cor.2021.0217.
- Passarotti, M., and Dell’Orletta, F. 2010. “Improvements in Parsing the Index Thomisticus Treebank. Revision, Combination and a Feature Model for Medieval Latin.” In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10). Valletta, Malta: European Language Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2010/pdf/178_Paper.pdf.