{"id":18918076,"url":"https://github.com/dengbocong/prompt-tuning","last_synced_at":"2025-04-15T09:32:20.016Z","repository":{"id":38950403,"uuid":"230193111","full_name":"DengBoCong/prompt-tuning","owner":"DengBoCong","description":"A pipeline for Prompt-tuning","archived":false,"fork":false,"pushed_at":"2022-06-07T15:12:35.000Z","size":70,"stargazers_count":41,"open_issues_count":2,"forks_count":14,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-13T12:07:17.021Z","etag":null,"topics":["classification","deep-learning","few-shot-learning","fine-tuning","nlp","pretrained-models","prompt","prompt-tuning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DengBoCong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-12-26T04:12:16.000Z","updated_at":"2024-09-11T06:44:50.000Z","dependencies_parsed_at":"2022-09-18T20:32:59.288Z","dependency_job_id":null,"html_url":"https://github.com/DengBoCong/prompt-tuning","commit_stats":null,"previous_names":["dengbocong/state-op"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DengBoCong%2Fprompt-tuning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DengBoCong%2Fprompt-tuning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DengBoCong%2Fprompt-tuning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DengBoCong%2Fprompt-tuning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DengBoCong","download_url":"https://codeload.github.com/DengBoCong/prompt-tuning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249043007,"owners_count":21203398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","deep-learning","few-shot-learning","fine-tuning","nlp","pretrained-models","prompt","prompt-tuning"],"created_at":"2024-11-08T10:29:32.979Z","updated_at":"2025-04-15T09:32:19.674Z","avatar_url":"https://github.com/DengBoCong.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003ePrompt-Tuning\u003c/h1\u003e\n\n+ A pipeline for Prompt-tuning\n+ 集成主流的Prompt-tuning相关方法，以及search template策略\n+ 提供Prompt-tuning完整的执行pipeline\n\n# Requirements\n本项目相关的依赖包参考requirements.txt，也可直接使用如下指令安装：\n```shell\npip install -r requirements.txt\n```\n\n# Usage\n+ core下放置相关prompt-tuning模型\n+ core/gen_template下是相关template生成方法，执行入口参考run_gen_template.py，执行示例如下：\n```python\npython3 run_gen_template.py \\\n    --task_name CoLA \\\n    --k 16 \\\n    --dev_rate 1 \\\n    --data_loader glue \\\n    --template_generator lm_bff \\\n    --data_dir data/original/CoLA \\\n    --output_dir data/output \\\n    --generator_config_path data/config/lm_bff.json\n```\n+ 模型实现放在core目录下，执行入口参考run_prompt_tuning.py，执行示例如下：\n```python\npython3 run_prompt_tuning.py \\\n    --data_dir data/CoLA/ \\\n    --do_train \\\n    --do_eval \\\n    --do_predict \\\n    --model_name_or_path bert \\\n    --num_k 16 \\\n    --max_steps 1000 \\\n    --eval_steps 100 \\\n    --learning_rate 1e-5 \\\n    --output_dir result/ \\\n    --seed 16\n    --template \"*cls**sent_0*_It_was*mask*.*sep+*\" \\\n    --mapping \"{'0':'terrible','1':'great'}\" \\\n    --num_sample 16 \\\n```\n+ data放置相关config及datasets，由于数据集比较庞大，可使用scripts下的下载脚本自行下载，如下：\n```shell\ncd data\nsh download_clue_dataset.sh\nsh download_glue_dataset.sh\n```\n+ tools放置相关工具方法及数据集处理方法等\n\n# Paper\n更详细的论文解读和阅读笔记 ☞  [点这里](https://github.com/DengBoCong/nlp-paper)\n\n+ [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://arxiv.org/pdf/2001.07676.pdf)\n+ [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://arxiv.org/pdf/2010.15980.pdf)\n+ [Making Pre-trained Language Models Better Few-shot Learners](https://arxiv.org/pdf/2012.15723.pdf)\n+ [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://arxiv.org/pdf/2101.00190.pdf)\n+ [GPT Understands, Too](https://arxiv.org/pdf/2103.10385.pdf)\n+ [The Power of Scale for Parameter-Efficient Prompt Tuning](https://arxiv.org/pdf/2104.08691.pdf)\n+ [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://arxiv.org/pdf/2108.04106.pdf)\n+ [PPT: Pre-trained Prompt Tuning for Few-shot Learning](https://arxiv.org/pdf/2109.04332.pdf)\n+ [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://arxiv.org/pdf/2110.07904.pdf)\n\n# Reference\n+ https://github.com/princeton-nlp/LM-BFF\n+ https://github.com/shmsw25/Channel-LM-Prompting\n\n# Dataset\n+ GLUE：https://nlp.cs.princeton.edu/projects/lm-bff/datasets.tar\n```shell\nsh scripts/download_glue_dataset.sh\n```\n+ CLUE：https://github.com/CLUEbenchmark/CLUE\n```shell\nsh scripts/download_clue_dataset.sh\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdengbocong%2Fprompt-tuning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdengbocong%2Fprompt-tuning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdengbocong%2Fprompt-tuning/lists"}