{"id":21741216,"url":"https://github.com/imkett/pcae","last_synced_at":"2025-04-13T03:42:14.340Z","repository":{"id":65651927,"uuid":"527451533","full_name":"ImKeTT/PCAE","owner":"ImKeTT","description":"[KBS] PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation PyTorch Implementation","archived":false,"fork":false,"pushed_at":"2023-04-10T11:26:16.000Z","size":532,"stargazers_count":25,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-26T20:51:30.721Z","etag":null,"topics":["bart","controllable-generation","deep-learning","nlp","plug-and-play-language-models","text-generation","variational-autoencoder"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ImKeTT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-22T07:10:06.000Z","updated_at":"2024-12-05T09:47:41.000Z","dependencies_parsed_at":"2023-02-17T18:45:35.947Z","dependency_job_id":null,"html_url":"https://github.com/ImKeTT/PCAE","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImKeTT%2FPCAE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImKeTT%2FPCAE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImKeTT%2FPCAE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImKeTT%2FPCAE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ImKeTT","download_url":"https://codeload.github.com/ImKeTT/PCAE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248661055,"owners_count":21141387,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bart","controllable-generation","deep-learning","nlp","plug-and-play-language-models","text-generation","variational-autoencoder"],"created_at":"2024-11-26T06:17:24.963Z","updated_at":"2025-04-13T03:42:14.316Z","avatar_url":"https://github.com/ImKeTT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation\n\nOfficial PyTorch implementation of *[PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation](https://www.sciencedirect.com/science/article/pii/S0950705122008942)*, published in *Knowledge-Based Systems*. We provide PCAE as well as all implemented baselines (PPVAE and Optimus) under pre-trained BART.\n\n![pcae_struct](pics/pcae_struct.jpg)\n\n\n\n## News\n\n- [2023-02-02] We are sharing fine-tuned BART VAE [here](https://drive.google.com/file/d/1hp_vm1rQIxWgCSkgm7cKtasGWxCG1kL8/view?usp=sharing) now!\n- [2022-10-10] Our paper is available on [arXiv](https://arxiv.org/abs/2210.03496) now.\n- [2022-09-27] Our paper is now in this [paper list](https://github.com/ImKeTT/CTG-latentAEs), which aims at collecting all kinds of latent variational auto-encoders that controllably generate texts. Feel free to check it out and contribute!\n- [2022-09-27] We release our PCAE and baseline codes under the setup of pre-trained BART.\n- [2022-09-06] Our work PCAE is [available](https://www.sciencedirect.com/science/article/pii/S0950705122008942) online.\n- [2022-08-21] Our paper PCAE is accepted to *Knowledge-Based Systems*.\n\n## Setup\n\nMake sure you have installed\n\n```bash\ntransformers\ntqdm\ntorch\nnumpy\n```\n\n## Dataset\n\nWe conduct five tasks span from three datasets: *Yelp review*, *Titles* and *Yahoo Question*. \n\nWe provide our full processed datasets in:\n\n-  [BaiduPan](https://pan.baidu.com/s/11vEqD_liL_U8brCEC6Nohg?pwd=bx81) (password `bx81`) \n- [GoogleDrive](https://drive.google.com/file/d/1XDHN3rbXhl-dc_cqIFsQCd01pr6BiQjn/view?usp=sharing)\n\nPlease download `data.zip` and unzip it to the current folder.\n\nYou can also try your own data, follow the split in `data` folder. Note that, for PPVAE, you have to manually split negative samples for every control signal.\n\n## Training\n\n### Stage 1 BART VAE Finetuning\n\nYou can download full `checkpoints` folder from [here](https://drive.google.com/file/d/1hp_vm1rQIxWgCSkgm7cKtasGWxCG1kL8/view?usp=sharing), unzip the `checkpoints.zip` to the current folder.\n\nOr you can train the BART VAE from the scratch: \n\nFinetuning on three datasets. (choose DATA from `yelp`, `yahoo`, `titles`, and EPOCH from 8, 10, 10):\n\n```shell\nDATA=yelp\nEPOCH=8\npython train.py --run_mode vae_ft --dataset $DATA --zmanner hidden\\\n--gpu 0 1 --dim_z 128 --per_gpu_train_batch_size 64\\\n--train_epochs $EPOCH --fb_mode 1 --lr 1e-4 --first_token_pooling --eval_every 500\n```\n\n\n\n![acc_results](pics/acc_results.jpg)\n\n### Stage 2.1 PCAE Plug-in Training\n\nPlug-in training of PCAE. Choose arguments below:\n\n+ TASK: [sentiment,  tense,  topics, quess_s, quess] \n\n  (topics, quess_s, quess corresponds to $topics_S,topics_M, topics_L$ in the paper respectively)\n\n+ SAMPLE_N: [100, 300, 500, 800, 1000]\n\n+ NUM_LAYER: int number from 8 to 15 is fine\n\n+ EPOCH: 10 to 20 is fine, less SAMPLE_N means less EPOCH required\n\n```shell\nTASK=sentiment\nEPOCH=10\nSAMPLE_N=100\nNUM_LAYER=10\n\npython train.py --run_mode pcae --task $TASK --zmanner hidden\\\n--gpu 0 --dim_z 128 --per_gpu_train_batch_size 5\\\n--plugin_train_epochs $EPOCH --fb_mode 1 --sample_n $SAMPLE_N\\\n--layer_num $NUM_LAYER --lr 1e-4 --use_mean\n```\n\n### Stage 2.2 PPVAE Plug-in Training\n\nPlug-in training of PPVAE BART. Choose arguments below:\n\n+ TASK: [sentiment,  tense,  topics, quess_s, quess]\n+ SAMPLE_N: [100, 300, 500, 800, 1000]\n+ TASK_LABEL: [pos, neg] for sentiment task; [present, past] for tense task; [0, 1, 2, 3, 4] for topics task; [0, 1, 2, 3, ..., 9] for quess task\n+ EPOCH: 10 to 20 is fine, less SAMPLE_N means less EPOCH required\n\nFor example, if you want to train PPVAE to generate **positive** sentences in sentiment task with 100 training samples per class, run:\n\n```shell\nTASK=sentiment\nEPOCH=10\nSAMPLE_N=100\nTASK_LABEL=pos\n\npython train.py --run_mode ppvae --task $TASK --zmanner hidden\\\n--gpu 0 --dim_z 128 --per_gpu_train_batch_size 5\\\n--plugin_train_epochs $EPOCH --fb_mode 1 --sample_n $SAMPLE_N\\\n--task_label $TASK_LABEL --lr 1e-4 --ppvae_dim_bottle 25\n```\n\n### Stage 2.3 Optimus_{bart} Plug-in Finetuning\n\nPlug-in finetuning of Optimus under BART setup. Choose arguments below:\n\n+ TASK: [sentiment,  tense,  topics, quess_s, quess] \n+ SAMPLE_N: [100, 300, 500, 800, 1000]\n+ EPOCH: 10 to 20 is fine, less SAMPLE_N means less EPOCH required\n\n```shell\nTASK=sentiment\nEPOCH=10\nSAMPLE_N=100\n\npython train.py --run_mode optimus --task $TASK --zmanner hidden\\\n--gpu 0 --dim_z 128 --per_gpu_train_batch_size 5\\\n--plugin_train_epochs $EPOCH --fb_mode 1 --sample_n $SAMPLE_N\\\n--lr 1e-4\n```\n\n## Others\n\nPlease [email](tuisaac163@gmail.com) me or open an issue if you have further questions.\n\nif you find our work useful, please cite the paper and star the repo~ :)\n\n```bibtex\n@article{tu2022pcae,\n  title={PCAE: A framework of plug-in conditional auto-encoder for controllable text generation},\n  author={Tu, Haoqin and Yang, Zhongliang and Yang, Jinshuai and Zhang, Siyu and Huang, Yongfeng},\n  journal={Knowledge-Based Systems},\n  volume={256},\n  pages={109766},\n  year={2022},\n  publisher={Elsevier}\n}\n```\n\nWe thank open sourced codes related to VAEs and plug-and-play models, which inspired our work!!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimkett%2Fpcae","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimkett%2Fpcae","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimkett%2Fpcae/lists"}