{"id":20385571,"url":"https://github.com/emrys-hong/fastai_sequence_tagging","last_synced_at":"2025-04-12T09:41:36.758Z","repository":{"id":56496255,"uuid":"145080215","full_name":"Emrys-Hong/fastai_sequence_tagging","owner":"Emrys-Hong","description":"sequence tagging for NER for ULMFiT","archived":false,"fork":false,"pushed_at":"2020-11-04T02:36:07.000Z","size":1927,"stargazers_count":20,"open_issues_count":1,"forks_count":6,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-26T04:41:20.335Z","etag":null,"topics":["crf","fastai","named-entity-recognition","pytorch","sequence-tagging","ulmfit"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Emrys-Hong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-17T06:19:48.000Z","updated_at":"2025-01-17T09:39:22.000Z","dependencies_parsed_at":"2022-08-15T19:50:33.476Z","dependency_job_id":null,"html_url":"https://github.com/Emrys-Hong/fastai_sequence_tagging","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Emrys-Hong%2Ffastai_sequence_tagging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Emrys-Hong%2Ffastai_sequence_tagging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Emrys-Hong%2Ffastai_sequence_tagging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Emrys-Hong%2Ffastai_sequence_tagging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Emrys-Hong","download_url":"https://codeload.github.com/Emrys-Hong/fastai_sequence_tagging/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248549281,"owners_count":21122865,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crf","fastai","named-entity-recognition","pytorch","sequence-tagging","ulmfit"],"created_at":"2024-11-15T02:34:48.518Z","updated_at":"2025-04-12T09:41:36.715Z","avatar_url":"https://github.com/Emrys-Hong.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fastai_sequence_tagging\nsequence tagging for NER for ULMFiT\n\n## data\nto replicate result:\nyou can download the ```data/``` folder from [here](https://www.dropbox.com/sh/z308tqyqrvakl66/AADsiYKx3vfNZ3LQGInz0Q-qa?dl=0), and put it in root directory.\n\n## run training\nI am currently doing experiments in jupyter notebook ```coNLL_three_layer.ipynb```\n\n## files modified from lesson10.ipynb\n1. concat both forward and backward outputs from language model ```W_LM = [W_forward, W_backward]```\n\n2. feeding word vectors from GloVe to a BiLSTM and get output ```W_glove```\n\n3. concatenating these outputs ```W = [W_glove, W_LM]```\n\n4. feeding ```W``` to another BiLSTM to get final result.\n\n## results\nF1 score of 76. \n\n(need to improve by fine tuning parameters, see how the toks are preprocessed, [adding char embedding](http://alanakbik.github.io/papers/coling2018.pdf), [adding CRF layer](https://arxiv.org/abs/1603.01360).\n\n## questions\n1. which layer of lanuage model should be used for Sequence tagging problem\n\n2. how to build a better language model for sequence tagging\n\n## relevant papers\n[Regularizing and Optimizing LSTM Language Models](https://arxiv.org/pdf/1708.02182.pdf)\n\n[deep contextualized word representations](https://arxiv.org/abs/1802.05365)\n\n[End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](http://www.aclweb.org/anthology/P16-1101)\n\n[Semi-supervised sequence tagging with bidirectional language models](https://arxiv.org/abs/1705.00108)\n\n[Contextual String Embeddings for Sequence Labeling](http://alanakbik.github.io/papers/coling2018.pdf)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femrys-hong%2Ffastai_sequence_tagging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femrys-hong%2Ffastai_sequence_tagging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femrys-hong%2Ffastai_sequence_tagging/lists"}