{"id":21493570,"url":"https://github.com/alienkevin/tagllm","last_synced_at":"2026-07-18T00:02:20.322Z","repository":{"id":227511095,"uuid":"767834551","full_name":"AlienKevin/tagllm","owner":"AlienKevin","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-28T23:17:16.000Z","size":60839,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-23T21:24:52.383Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlienKevin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-06T01:18:18.000Z","updated_at":"2024-04-28T23:17:20.000Z","dependencies_parsed_at":"2024-04-26T15:38:49.113Z","dependency_job_id":null,"html_url":"https://github.com/AlienKevin/tagllm","commit_stats":null,"previous_names":["alienkevin/tagllm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlienKevin%2Ftagllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlienKevin%2Ftagllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlienKevin%2Ftagllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlienKevin%2Ftagllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlienKevin","download_url":"https://codeload.github.com/AlienKevin/tagllm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244029505,"owners_count":20386420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T15:43:27.889Z","updated_at":"2025-10-23T16:56:20.187Z","avatar_url":"https://github.com/AlienKevin.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Setup Virtual Environment\n```\nconda create -n tagllm python=3.10 -y\nconda activate tagllm\npython -m pip install opencc-python-reimplemented datasets evaluate sacrebleu scikit-learn\n```\n\n# Dataset Generation\nThe original datasets for translation are stored in `datasets/`. Run the following script to generate a new folder called `translations/` that collates all datasets together into a HuggingFace dataset:\n```\npython gen_translation_dataset.py\n```\n\n# Training\nWe adapt the [training script](https://huggingface.co/blog/mlabonne/orpo-llama-3) written by Maxime Labonne to fine-tune Llama 3 with QLoRA to our custom tasks and datasets. To speed up training and reduce memory requirement for the learnable tags configuration, we adapt Unsloth AI's hand-optimized Llama 3 [QLoRA fine-tuning script](https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing) to our tasks. See the `training_scripts/` folder for all the scripts used for each configuration and task in our paper:\n\n| Configuration | Task | Script |\n| -------- | ------- | -------- |\n| No Tags  | Language Domain Adaptation | `qlora-lang-no-tag.ipynb` |\n| No Tags  | POS Tagging | `qlora-pos-no-tag.ipynb` |\n| No Tags  | Translation | `qlora-translation-no-tag.ipynb` |\n| Natural Tags 1 | POS Tagging | `qlora-pos-no-lang.ipynb` |\n| Natural Tags 1 | Translation | `qlora-translation-no-lang.ipynb` |\n| Natural Tags 2 | Language Domain Adaptation | `qlora-lang.ipynb` |\n| Natural Tags 2 | POS Tagging | `qlora-pos.ipynb` |\n| Natural Tags 2 | Translation | `qlora-translation.ipynb` |\n| Learnable Tags | Language Domain Adaptation | `tagllm-lang-unsloth.ipynb` |\n| Learnable Tags | POS Tagging | `tagllm-pos-unsloth.ipynb` |\n| Learnable Tags | Translation | `tagllm-translation-unsloth.ipynb` |\n\n# Testing\nWe wrote our own test scripts for each configuration and task. Each test script should generate a JSONL output under the `experiment_results/` folder. See the `test_scripts/` folder for details:\n\n| Configuration | Task | Script |\n| -------- | ------- | -------- |\n| Plain Llama 3 | POS Tagging | `test_pos.ipynb` |\n| Plain Llama 3 | Translation | `test_translation.ipynb` |\n| No Tags  | POS Tagging | `test_pos_qlora_no_tag.ipynb` |\n| No Tags  | Translation | `test_translation_qlora_no_tag.ipynb` |\n| Natural Tags 1 | POS Tagging | `test_pos_qlora_no_lang.ipynb` |\n| Natural Tags 1 | Translation | `test_translation_qlora_no_lang.ipynb` |\n| Natural Tags 2 | POS Tagging | `test_pos.ipynb` |\n| Natural Tags 2 | Translation | `test_translation_qlora.ipynb` |\n| Learnable Tags | POS Tagging | `test_pos_tagllm.ipynb` |\n| Learnable Tags | Translation | `test_translation_tagllm.ipynb` |\n\n# Evaluation\nRun `python eval_pos.py` to generate `pos_evaluation_results.csv` under `experiment_results/`. We use HuggingFace's poseval module to reliably calculate the accuracy and F1 measures for the POS tagging task.\n\nSimilarly, run `python eval_translation.py` to generate `translation_evaluation_results.csv` under `experiment_results/`. We use the standard SacreBLEU package to reliably calculate the BLEU and chrF++ scores for the translation task.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falienkevin%2Ftagllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falienkevin%2Ftagllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falienkevin%2Ftagllm/lists"}