{"id":19373640,"url":"https://github.com/4ai/ls-llama","last_synced_at":"2026-02-28T17:31:07.610Z","repository":{"id":197937796,"uuid":"698473159","full_name":"4AI/LS-LLaMA","owner":"4AI","description":"A Simple but Powerful SOTA NER Model | Official Code For Label Supervised LLaMA Finetuning","archived":false,"fork":false,"pushed_at":"2024-03-17T06:21:30.000Z","size":3711,"stargazers_count":154,"open_issues_count":9,"forks_count":25,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-11T01:03:25.776Z","etag":null,"topics":["conll2003","llama","llama2","llms","named-entity-recognition","ontonotes","sequence-classification","token-classification"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2310.01208","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/4AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-30T03:09:04.000Z","updated_at":"2025-05-08T15:46:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"c0fb3ff0-ad86-483b-8030-74cdd358ecaf","html_url":"https://github.com/4AI/LS-LLaMA","commit_stats":{"total_commits":12,"total_committers":2,"mean_commits":6.0,"dds":0.08333333333333337,"last_synced_commit":"2f87df12d030388b8901187845beec75e46d6744"},"previous_names":["4ai/ls-llama"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FLS-LLaMA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FLS-LLaMA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FLS-LLaMA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4AI%2FLS-LLaMA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/4AI","download_url":"https://codeload.github.com/4AI/LS-LLaMA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253668096,"owners_count":21944982,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conll2003","llama","llama2","llms","named-entity-recognition","ontonotes","sequence-classification","token-classification"],"created_at":"2024-11-10T08:30:36.960Z","updated_at":"2026-02-28T17:31:07.571Z","avatar_url":"https://github.com/4AI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LS-LLaMA: Label Supervised LLaMA Finetuning\n\n\u003ch2\u003e📢: For convenience, we build a bi-directional LLMs toolkit \u003ca href='https://github.com/WhereIsAI/BiLLM'\u003eBiLLM\u003c/a\u003e for language understanding. Welcome to use it.\u003c/h2\u003e\n\n\u003cp align=\"center\"\u003e\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/label-supervised-llama-finetuning/named-entity-recognition-on-conll03-4)](https://paperswithcode.com/sota/named-entity-recognition-on-conll03-4?p=label-supervised-llama-finetuning)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/label-supervised-llama-finetuning/named-entity-recognition-on-ontonotes-5-0-1)](https://paperswithcode.com/sota/named-entity-recognition-on-ontonotes-5-0-1?p=label-supervised-llama-finetuning)\n\u003c/p\u003e\n\n\n\u003cp align='center'\u003e\n\u003cimg src='./docs/lsllama.png'/\u003e\n\u003c/p\u003e\n\n## Usage\n\nOur implementation currently supports the following sequence classification benchmarks: \n1. SST2 (2 classes) / SST5 (5 classes)\n2. AGNews (4 classes)\n3. Twitter Financial News Sentiment (twitterfin, 3 classes)\n\nand token classification benchmarks for named entity recognition (NER): CoNLL2003 and OntonotesV5.\n\nCommands for training LS-LLaMA and LS-unLLaMA on different tasks can follow the templates below:\n```console\nfoo@bar:~$ CUDA_VISIBLE_DEVICES=0 python file_name.py dataset_name model_size\n```\n\n`file_name.py` can be one of `unllama_seq_clf.py`, `unllama_token_clf.py`, `llama_seq_clf.py`, and `llama_token_clf.py`, for training LS-LLaMA and LS-unLLaMA on sequence- and token-level classification.\n\n`dataset_name` can be one of `sst2`, `sst5`, `agnews`, `twitterfin`, `conll03`, and `ontonotesv5`.\n\n`model_size` can be `7b` or `13b`, corresponding to LLaMA-2-7B and LLaMA-2-13B.\n\nFor example, the following command will train LS-unLLaMA based on LLaMA-2-7B on AGNews for sequence classification:\n```console\nfoo@bar:~$ CUDA_VISIBLE_DEVICES=0 python unllama_seq_clf.py agnews 7b\n```\n\n## Implementations\n\nLoad Pretrained Models\n\n```python\nfrom transformers import AutoTokenizer\nfrom modeling_llama import (\n    LlamaForSequenceClassification, LlamaForTokenClassification,\n    UnmaskingLlamaForSequenceClassification, UnmaskingLlamaForTokenClassification,\n)\n\n\nmodel_id = 'meta-llama/Llama-2-7b'\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = LlamaForSequenceClassification.from_pretrained(model_id).bfloat16()\nmodel = LlamaForTokenClassification.from_pretrained(model_id).bfloat16()\nmodel = UnmaskingLlamaForSequenceClassification.from_pretrained(model_id).bfloat16()\nmodel = UnmaskingLlamaForTokenClassification.from_pretrained(model_id).bfloat16()\n```\n\nFor more usage, please refer to `unllama_seq_clf.py`, `unllama_token_clf.py`, `llama_seq_clf.py`, `llama_token_clf.py`.\n\n# Citation\n\n```\n@article{li2023label,\n  title={Label supervised llama finetuning},\n  author={Li, Zongxi and Li, Xianming and Liu, Yuzhang and Xie, Haoran and Li, Jing and Wang, Fu-lee and Li, Qing and Zhong, Xiaoqin},\n  journal={arXiv preprint arXiv:2310.01208},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4ai%2Fls-llama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F4ai%2Fls-llama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4ai%2Fls-llama/lists"}