{"id":19125336,"url":"https://github.com/ryangawei/house-property-qa","last_synced_at":"2026-06-18T08:31:12.860Z","repository":{"id":68816783,"uuid":"304855376","full_name":"ryangawei/House-Property-QA","owner":"ryangawei","description":"CCF BDCI 2020 房产行业聊天问答匹配大赛, A 榜 95/2985","archived":false,"fork":false,"pushed_at":"2021-03-23T01:47:12.000Z","size":4208,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-01T15:43:58.163Z","etag":null,"topics":["deep-learning","nlp","question-answering","text-matching"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryangawei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-17T10:45:39.000Z","updated_at":"2021-04-09T18:54:21.000Z","dependencies_parsed_at":"2023-08-20T02:46:18.842Z","dependency_job_id":null,"html_url":"https://github.com/ryangawei/House-Property-QA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ryangawei/House-Property-QA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryangawei%2FHouse-Property-QA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryangawei%2FHouse-Property-QA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryangawei%2FHouse-Property-QA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryangawei%2FHouse-Property-QA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryangawei","download_url":"https://codeload.github.com/ryangawei/House-Property-QA/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryangawei%2FHouse-Property-QA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34483274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","nlp","question-answering","text-matching"],"created_at":"2024-11-09T05:35:28.236Z","updated_at":"2026-06-18T08:31:12.831Z","avatar_url":"https://github.com/ryangawei.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CCF BDCI 2020 房产行业聊天问答匹配大赛\n\n**A 榜 95/2985**\n\n运行src/run_model.py 即可开始训练（注：参数 data_type 目前只能使用 qa_data3, 其它的还有BUG）\n\n## TODO\n\n* 使用其他损失函数（pair-wise, list-wise, triplet loss）\n* 先通过 CMRC 微调，再通过 HouseQA 二次微调，控制 CMRC 的正负样本比例（接近 HouseQA 的正负比例）\n* LCQMC\n* 增加交叉验证的 k 值\n* 爬取房产论坛数据，扩大数据集\n* 使用两阶段微调，point-wise (CrossEntropyLoss) + pair-wise (HingeLoss)\n\n## 训练 Regression 模型需要修改的位置\n\n```python\nall_pred = np.zeros(shape=(len(df_train)))\n\ntrain_loader = DataLoader(train_set,\n                        batch_size=kwargs['batch_size'],\n                        # shuffle=True  # 如果使用分类训练，建议True --\u003e\n                        )\n\n# criterion = torch.nn.MarginRankingLoss(margin=1.0)\n# criterion = torch.nn.MSELoss()\ncriterion = torch.nn.CrossEntropyLoss()\n\n# Train\n# CrossEntropy\n# loss = criterion(model_outputs, y)\n# MSE\n# loss = criterion(model_outputs, y.float().unsqueeze(-1))\n\n# 使用 HingeLoss\ntrain_qa_id_sub = sample[2].numpy()\nloss = get_hinge_loss(model_outputs, train_qa_id_sub, criterion)\n\n# Valid\n# MSELoss\n# loss = criterion(model_outputs, y_true.float().unsqueeze(-1)).cpu().detach().item()\n# HingeLoss\n# valid_qa_id_sub = sample[2].numpy()\n# loss = get_hinge_loss(model_outputs, valid_qa_id_sub, criterion)\n# y_pred = model_outputs.cpu().detach().squeeze(-1).numpy()\n# CrossEntropy\nloss = criterion(model_outputs, y_true).cpu().detach().item()\ny_pred = F.softmax(model_outputs.cpu().detach(), dim=1).numpy()\n\n# 如果使用回归模型\n# valid_f1, thr = search_f1(valid_true, valid_pred)\n# logger.info(\"Epoch {}, valid loss {:.5f}, valid f1 {:.4f}\".format(epoch, valid_loss, valid_f1)))\n\ntorch.save(\n            {\n                \"model_name\": \"BertForHouseQA\",\n                \"epoch\": epoch,\n                \"valid_loss\": valid_loss,\n                \"valid_f1\": valid_f1,\n                \"model_state_dict\": model.state_dict(),\n                # \"optimizer_state_dict\": optimizer.state_dict(),\n                # \"thr\": thr\n                # 'scheduler_state_dict': scheduler.state_dict()\n            },\n\n# MSELoss\n# all_f1, all_thr = search_f1(all_true, all_pred)\n# logger.info(\"All f1 {:.4f}, all thr {:.4f}\".format(all_f1, all_thr))\n\n\nall_f1, checkpoint_path = train_pytorch(batch_size=128, valid_batch_size=512, epoch=15, lr=2e-5, weight_decay=1e-3, \n                                        n_splits=10, patience=8, device=1, inputs=inputs, \n                                        outputs=outputs, test_inputs=test_inputs)\n```\n\n## 切换早停指标需要修改的代码\n\n```python\nstopper = EarlyStopping(patience=kwargs['patience'], mode='max')    # 'max' for F1, 'min' for loss\n\nstop_flag, best_flag = stopper.step(valid_f1)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryangawei%2Fhouse-property-qa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryangawei%2Fhouse-property-qa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryangawei%2Fhouse-property-qa/lists"}