{"id":28306822,"url":"https://github.com/applied-machine-learning-lab/uni-ctr","last_synced_at":"2026-03-03T12:32:07.199Z","repository":{"id":259835386,"uuid":"866882921","full_name":"Applied-Machine-Learning-Lab/Uni-CTR","owner":"Applied-Machine-Learning-Lab","description":"Code implementation of TOIS paper \"A Unified Framework for Multi-Domain CTR Prediction via Large Language Models\"","archived":false,"fork":false,"pushed_at":"2025-09-06T02:50:13.000Z","size":451,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-06T04:22:42.097Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Applied-Machine-Learning-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-03T03:47:14.000Z","updated_at":"2025-09-06T02:50:15.000Z","dependencies_parsed_at":"2024-10-28T08:48:49.551Z","dependency_job_id":"fee9aa90-ff22-4040-8eb2-b13f4d691a50","html_url":"https://github.com/Applied-Machine-Learning-Lab/Uni-CTR","commit_stats":null,"previous_names":["applied-machine-learning-lab/uni-ctr"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Applied-Machine-Learning-Lab/Uni-CTR","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Applied-Machine-Learning-Lab%2FUni-CTR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Applied-Machine-Learning-Lab%2FUni-CTR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Applied-Machine-Learning-Lab%2FUni-CTR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Applied-Machine-Learning-Lab%2FUni-CTR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Applied-Machine-Learning-Lab","download_url":"https://codeload.github.com/Applied-Machine-Learning-Lab/Uni-CTR/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Applied-Machine-Learning-Lab%2FUni-CTR/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273983110,"owners_count":25202095,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-24T04:12:46.426Z","updated_at":"2026-03-03T12:32:07.136Z","avatar_url":"https://github.com/Applied-Machine-Learning-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"2024 08 25 We are pleased to announce that our paper has been accepted for publication in **TOIS**(CCF-A) (ACM Transactions on Information Systems) 🎉🎉!\n\n# Contents\n\n- [Uni-CTR Description](#uni-ctr-description)\n- [Dataset](#dataset)\n- [Environment Requirements](#environment-requirements)\n- [Quick Start](#quick-start)\n- [Script Description](#script-description)\n   * [Script and Sample Code](#script-and-sample-code)\n   * [Script Parameters](#script-parameters)\n      + [Uni-CTR](#uni-ctr)\n      + [Multi-Domain Models](#multi-domain-models)\n      + [Single-Domain Models](#single-domain-models)\n   * [Training Process](#training-process)\n      + [Training](#training)\n- [Model Description](#model-description)\n   * [Performance](#performance)\n      + [Training Performance](#training-performance)\n      + [Inference Performance](#inference-performance)\n- [Description of Random Situation](#description-of-random-situation)\n\n# [Uni-CTR Description](#contents)\n\nThe proposed framework for Uni-CTR comprises three parts,. Initially, the input text undergoes processing via the selected **LLM Backbone** to extract the commonalities and distinctions of the data across domains. Subsequently, Subsequently, LLM provides the representations obtained from different layers to the **domain-specific networks** to learn domain-specific characteristics. Additionally, a **general network** is incorporated to learn the representations of all known domains, which enables zero-shot prediction of newly unseen domains.\n\n[A Unified Framework for Multi-Domain CTR Prediction via Large Language Models](https://arxiv.org/abs/2312.10743)\n![avatar](./figure/model.PNG)\n![avatar](./figure/performance.PNG)\n\n\n\n# [Dataset](#contents)\n\n- [Amazon Review Data (2018)](https://cseweb.ucsd.edu/~jmcauley/datasets/amazon_v2/)\n\n# [Environment Requirements](#contents)\n\n- Hardware（GPU）\n\t- Prepare hardware environment with GPU  processor.\n- Framework\n\t- Pytorch\n- Requirements\n\t- accelerate  \n\t- huggingface-hub  \n\t- numpy  \n\t- peft  \n\t- scipy  \n\t- sympy  \n\t- tensorboard  \n\t- tokenizers  \n\t- torch-summary  \n\t- torchvision  \n\t- tqdm  \n\t- transformers  \n\t- scikit-learn  \n\t- pandas  \n\t- tensorflow  \n\t- matplotlib\n\n# [Quick Start](#contents)\n\nAfter configuring the environment, you can start training and evaluation as follows:\n\n- running on GPU\n\n  ```python\n  # run training and evaluation example\n  python training/main.py\n  ```\n\n# [Script Description](#contents)\n\n## [Script and Sample Code](#contents)\n\n```bash\n.\n├── configs                                          # configurations for different paradigm models\n│   ├── __init__.py                                  # relative package import\n│   ├── config.py                                    # configuration for Uni-CTR\n│   ├── config_multi_domain.py                       # configuration for multi-domain baselines\n│   └── config_single_domain.py                      # configuration for single-domain baselines\n├── layers                                           # network layers in models (mostly from package DeepCTR-torch)\n│   ├── __init__.py                                  # relative package import\n│   ├── activation.py                                # activation networks\n│   ├── core.py                                      # core networks including ladders\n│   ├── interaction.py                               # modules for single-domain models\n│   ├── sequence.py                                  # sequence processing networks\n│   └── utils.py                                     # other data processing methods and additional networks\n├── miscellaneous\n├── models                                           # all baseline models\n│   ├── autoint.py\n│   ├── basemodel.py\n│   ├── dcn.py\n│   ├── deepfm.py\n│   ├── fibinet.py\n│   ├── mmoe.py\n│   ├── ple.py\n│   ├── pnn.py\n│   ├── sharedbottom.py\n│   ├── star.py\n│   └── xdeepfm.py\n├── preprocessing                                    # data preprocessing\n│   ├── amazon_review_data                           # preprocessing methods for Amazon Review Data (2018)\n│   │   ├── data_analysis.ipynb                      # analyse the distributions of the domains\n│   │   ├── multi_domain_raw_data_processing.py      # data preprocessing for baseline models\n│   │   ├── multi_domain_text_processing.py          # prompt generation\n│   │   └── one_for_all.py                           # whole dataset preprocessing pipeline for Uni-CTR\n│   └── utils.py                                     # data preprocessing methods\n├── training                                         # training files\n│   ├── main.py                                      # train file for Uni-CTR\n│   ├── main_multi_domain.py                         # train file for multi-domain models\n│   └── main_single_domain.py                        # train file for single-domain models\n├── requirements.txt                                 # package requirements\n├── callbacks.py                                     # Early Stopping for single-domain models\n├── inputs.py                                        # data transformation\n└── utils.py                                         # general functions for Uni-CTR \n\n```\n\n## [Script Parameters](#contents)\n\n### [Uni-CTR](#contents)\n\nParameters for Uni-CTR can be set in `configs/config.py`\n\n- Parameters for Amazon Review Data (2018)\n\n```python\ntext_encoder_models = [  \n    # Name, num_hidden_layers, text_embedding_dim, max_length\n    [\"Llama-2-7b-hf\", 24, 2048, 4096],  \n]\ntext_encoder_model_name, layer_num, text_embedding_dim, max_length = text_encoder_models[0]\nladder_frequency = 4\n\nladder_block = [\"wo_block\", \"w_lora\", \"w_self_attention\", \"w_transformer_block\"]  \nladder_block = ladder_block[3]  \nr = 4  \nnum_heads = 2  \nnarrowed_ratio = 0.25  \nuse_peft = True\nmixed_precision = True\ndropout = 0.2\nepochs = 10\nbatch_size = 3 * len(device_ids)\nseed = 2012\nlr = 8e-5  \nmax_lr = 5e-4\nweight_decay = 0.001\n```\n\n### [Multi-Domain Models](#contents)\n\nParameters for multi-domain can be set in `configs/config_multi_domain.py`\n\n- Parameters for Amazon Review Data (2018)\n\n```python\nmultiplier = 6\nembed_dim = 32\ndropout = 0.2  \nepochs = 10  \nbatch_size = 2048  \nseed = 2012  \nlr = 1e-7  \nmax_lr = 1e-3\nweight_decay = 0.002\n```\n\n\n### [Single-Domain Models](#contents)\n\nParameters for multi-domain can be set in `configs/config_single_domain.py`\n\n- Parameters for Amazon Review Data (2018)\n\n```python\nembed_dim = 32\nepoch = 10\nbatch_size = 2048\nseed = 2012\nlr = 1e-7  \nmax_lr = 1e-3\nweight_decay = 0.002\n```\n\n## [Training Process](#contents)\n\n### Training\n\n- running on GPUs with `DistributedDataParallel`\n\n  - Start training:\n    ```shell\n    tmux new -s my_session\t# (Optional)\n    cd multi-domain\n    CUDA_VISIBLE_DEVICES=0,1 nohup torchrun --nproc_per_node=2 training/main.py \u003e output.log 2\u003e\u00261 \u0026\n    ```\n    \n  - Press the following keys to detach from session `my_session`:\n    \n    \u003ckbd\u003eCtrl\u003c/kbd\u003e + \u003ckbd\u003eB\u003c/kbd\u003e + \u003ckbd\u003eD\u003c/kbd\u003e\n  \t\n  - Use the following code to attach session `my_session`:\n    ```shell\n    tmux attach-session -t my_session\n    ```\n\n\n- The python command above will run in the background, you can view the results through the file `ms_log/output.log`.\n\n  ```txt\n   13%|█▎        | 31/14524 [06:23\u003c36:26:32,  1.36it/s, train_auc=0.713, train_loss=0.47034054]\n   ...\n  ```\n\n- The model checkpoint will be saved in the current directory.\n\n# [Model Description](#contents)\n\n## [Performance](#contents)\n\n### Training Performance\n\n| Parameters          | GPU                                                    |\n| ------------------- | ------------------------------------------------------ |\n| Model Version       | Uni-CTR                                                |\n| Resource            | GPU 8$\\times$NVIDIA V100 32G                           |\n| Uploaded Date       | 12/09/2023 (month/day/year)                            |\n| Pytorch Version     | 2.0.1                                                  |\n| Dataset             | [1]                                                    |\n| Domains             | [0,2,3]                                                |\n| Training Parameters | epoch=10, batch_size=3$\\times$len(device_ids), lr=1e-4 |\n| Optimizer           | AdamW                                                  |\n| Loss Function       | Sigmoid Cross Entropy With Logits                      |\n| outputs             | AUC                                                    |\n| Loss                |                                                        |\n| Per Step Time       | ms                                                     |\n\n### Inference Performance\n\n| Parameters      | GPU                           |\n| --------------- | ----------------------------- |\n| Model Version   | Uni-CTR                       |\n| Resource        | GPU 8$\\times$NVIDIA V100 32G |\n| Uploaded Date   | 12/09/2023 (month/day/year)   |\n| Pytorch Version | 2.0.1                         |\n| Dataset         | [1]                           |\n| Domains         | [0,2,3]                       |\n| batch_size      | 150$\\times$len(device_ids)                           |\n| outputs         | AUC                           |\n| AUC             | [0.7523, 0.7569, 0.7246]       |\n\n\n# [Description of Random Situation](#contents)\n\nWe set the random seed before training in model_config.py.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapplied-machine-learning-lab%2Funi-ctr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapplied-machine-learning-lab%2Funi-ctr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapplied-machine-learning-lab%2Funi-ctr/lists"}