{"id":13958326,"url":"https://github.com/guoday/Tencent2020_Rank1st","last_synced_at":"2025-07-20T23:31:03.074Z","repository":{"id":40596762,"uuid":"281659186","full_name":"guoday/Tencent2020_Rank1st","owner":"guoday","description":"The code for 2020 Tencent College Algorithm Contest, and the online result ranks 1st.","archived":false,"fork":false,"pushed_at":"2023-03-16T04:19:00.000Z","size":1015,"stargazers_count":1025,"open_issues_count":1,"forks_count":318,"subscribers_count":19,"default_branch":"master","last_synced_at":"2024-11-19T18:16:30.155Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/guoday.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-07-22T11:29:34.000Z","updated_at":"2024-11-19T03:32:52.000Z","dependencies_parsed_at":"2022-07-11T07:00:22.316Z","dependency_job_id":"f0c68041-0072-43ad-955f-a4a970b4be56","html_url":"https://github.com/guoday/Tencent2020_Rank1st","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoday%2FTencent2020_Rank1st","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoday%2FTencent2020_Rank1st/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoday%2FTencent2020_Rank1st/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoday%2FTencent2020_Rank1st/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/guoday","download_url":"https://codeload.github.com/guoday/Tencent2020_Rank1st/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226844968,"owners_count":17691136,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-08T13:01:29.604Z","updated_at":"2024-11-28T01:31:39.040Z","avatar_url":"https://github.com/guoday.png","language":"Python","funding_links":[],"categories":["其他_推荐系统"],"sub_categories":["网络服务_其他"],"readme":"## 赛题介绍-广告受众基础属性预估\n\n比赛将为参赛者提供一组用户在长度为 91 天(3 个月)的时间窗口内的广告点击历史记录作为训练数据集。每条记录中包含了日期 (从 1 到 91)、用户信息 (年龄，性别)，被点击的广告的信息(素材 id、广告 id、产品 id、产品类目 id、广告主 id、广告主行业 id 等)，以及该用户当天点击该广告的次数。测试数据集将会是另一组用户 的广告点击历史记录。提供给参赛者的测试数据集中不会包含这些用户的年龄和性别信息。 本赛题要求参赛者预测测试数据集中出现的用户的年龄和性别。\n\n### 1. 环境配置\n\n- Pytorch\n- Linux Ubuntu 16.04, 256G内存，4*p100\n- pip install transformers==2.8.0 pandas gensim scikit-learn filelock gdown\n\n### 2. 模型介绍\n\n![avatar](picture/model.png)\n![avatar](picture/mlm.png)\n![avatar](picture/fusion-layer.png)\n![avatar](picture/output.png)\n\n### 3. 低配置资源建议\n\n\n1)内存不足或者只是想简单跑下完整代码，请只使用初赛数据:\n\n去掉src/prepocess.py的8, 15, 22行\n\n2)如果显存不足，请下载10中的bert-small模型，并调整batch size\n\n### 4. 运行完整过程\n\n可运行以下脚本，运行整个过程并生成结果。或按照3-7节的说明依次运行。\n\n```shell\nbash run.sh\n```\n\n### 5. 数据下载\n\n通过该[网站](https://drive.google.com/file/d/15onAobxlim_uRUNWSMQuK6VxDsmGTtp4/view?usp=sharing)下载数据集到data目录，或运行下面的命令进行下载\n\n```shell\ngdown https://drive.google.com/uc?id=15onAobxlim_uRUNWSMQuK6VxDsmGTtp4\nunzip data.zip \nrm data.zip\n```\n\n### 6. 数据预处理\n\n合并所有文件，并分为点击记录文件(click.pkl)，用户文件(train_user.pkl/test_user.pkl)\n\n```\npython src/preprocess.py\n```\n\n### 7. 特征提取\n\n```shell\npython src/extract_features.py\n```\n\n### 8. 预训练 Word2Vector 与 BERT\n\n这里提供两种方式获得预训练权重: 重新预训练或下载预训练好的权重 \n\n注: Word2Vector和BERT权重必须一致，即要么全部重新预训练，要么全部下载\n\n#### 1) 预训练Word2Vector\n\n预训练word2vector\n\n```shell\npython src/w2v.py\n```\n\n或下载预训练好的[W2V](https://drive.google.com/file/d/1SUpukAeXR5Ymyf3wH3SRNdQ3Hl2HazQa/view?usp=sharing)\n\n```shell\ngdown https://drive.google.com/uc?id=1SUpukAeXR5Ymyf3wH3SRNdQ3Hl2HazQa\nunzip w2v.zip \ncp w2v/* data/\nrm -r w2v*\n```\n\n#### 2) 预训练BERT\n\n预训练BERT (如果GPU是v100，可以安装apex并在参数上加--fp16进行加速)\n\n```shell\ncd BERT\nmkdir saved_models\npython run.py \\\n    --output_dir saved_models \\\n    --model_type roberta \\\n    --config_name roberta-base \\\n    --mlm \\\n    --block_size 128 \\\n    --per_gpu_train_batch_size 64 \\\n    --per_gpu_eval_batch_size 64 \\\n    --gradient_accumulation_steps 1 \\\n    --learning_rate 5e-5 \\\n    --weight_decay 0.01 \\\n    --adam_epsilon 1e-6 \\\n    --max_grad_norm 1.0 \\\n    --max_steps 100000 \\\n    --mlm_probability 0.2 \\\n    --warmup_steps 10000 \\\n    --logging_steps 50 \\\n    --save_steps 10000 \\\n    --evaluate_during_training \\\n    --save_total_limit 500 \\\n    --seed 123456 \\\n    --tensorboard_dir saved_models/tensorboard_logs    \nrm -r saved_models/bert-base    \ncp -r saved_models/checkpoint-last saved_models/bert-base\nrm saved_models/bert-base/optimizer.pt\ncp saved_models/vocab.pkl saved_models/bert-base/vocab.pkl\ncd ..\n```\n\n或下载预训练好的[BERT-base](https://drive.google.com/file/d/1ToAJwl_oRAeRNyYF_FK0B2APVXlPFTlq/view?usp=sharing)\n\n```shell\ngdown https://drive.google.com/uc?id=1ToAJwl_oRAeRNyYF_FK0B2APVXlPFTlq\nunzip bert-base.zip\nmv bert-base BERT/\nrm bert-base.zip\n```\n\n### 9. 训练模型\n\n```shell\nmkdir saved_models\nmkdir saved_models/log\nfor((i=0;i\u003c5;i++));  \ndo  \n  python run.py \\\n      --kfold=5 \\\n      --index=$i \\\n      --train_batch_size=256 \\\n      --eval_steps=5000 \\\n      --max_len_text=128 \\\n      --epoch=5 \\\n      --lr=1e-4 \\\n      --output_path=saved_models \\\n      --pretrained_model_path=BERT/bert-base \\\n      --eval_batch_size=512 2\u003e\u00261 | tee saved_models/log/$i.txt\ndone  \n```\n\n合并结果，结果为submission.csv\n\n```shell\npython src/merge_submission.py\n```\n\n### 10. 不同规模的预训练模型\n\n由于此次比赛融合了不同规模大小的预训练模型，在此也提供不同规模的预训练模型: \n\n[BERT-small](https://drive.google.com/file/d/1bDneO-YhBs5dx-9qC-WrBf3jUc_QCIYn/view?usp=sharing), [BERT-base](https://drive.google.com/file/d/1ToAJwl_oRAeRNyYF_FK0B2APVXlPFTlq/view?usp=sharing), [BERT-large](https://drive.google.com/file/d/1yQeh3O6E_98srPqTVwAnVbr1v-X0A7R-/view?usp=sharing), [BERT-xl](https://drive.google.com/file/d/1jViHtyljOJxxeOBmxn9tOZg_hmWOj0L2/view?usp=sharing)\n\n其中bert-base效果最好\n\n```shell\n#bert-small\ngdown https://drive.google.com/uc?id=1bDneO-YhBs5dx-9qC-WrBf3jUc_QCIYn\n#bert-base\ngdown https://drive.google.com/uc?id=1ToAJwl_oRAeRNyYF_FK0B2APVXlPFTlq\n#bert-large\ngdown https://drive.google.com/uc?id=1yQeh3O6E_98srPqTVwAnVbr1v-X0A7R-\n#bert-xl\ngdown https://drive.google.com/uc?id=1jViHtyljOJxxeOBmxn9tOZg_hmWOj0L2\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguoday%2FTencent2020_Rank1st","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguoday%2FTencent2020_Rank1st","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguoday%2FTencent2020_Rank1st/lists"}