{"id":13535263,"url":"https://github.com/zhpmatrix/BERTem","last_synced_at":"2025-04-02T00:33:01.450Z","repository":{"id":117937423,"uuid":"195616106","full_name":"zhpmatrix/BERTem","owner":"zhpmatrix","description":"论文实现(ACL2019)：《Matching the Blanks: Distributional Similarity for Relation Learning》","archived":false,"fork":false,"pushed_at":"2022-12-08T00:21:05.000Z","size":325,"stargazers_count":154,"open_issues_count":9,"forks_count":24,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-26T17:11:20.732Z","etag":null,"topics":["acl2019","bert-pytorch","fewrel","matching-the-blanks","nlp","relation-extraction"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhpmatrix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-07-07T06:00:24.000Z","updated_at":"2024-12-16T08:50:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"48a21ffa-9e0f-4a1b-b7fe-df329c0d5198","html_url":"https://github.com/zhpmatrix/BERTem","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhpmatrix%2FBERTem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhpmatrix%2FBERTem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhpmatrix%2FBERTem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhpmatrix%2FBERTem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhpmatrix","download_url":"https://codeload.github.com/zhpmatrix/BERTem/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246735358,"owners_count":20825222,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["acl2019","bert-pytorch","fewrel","matching-the-blanks","nlp","relation-extraction"],"created_at":"2024-08-01T08:00:52.273Z","updated_at":"2025-04-02T00:33:01.084Z","avatar_url":"https://github.com/zhpmatrix.png","language":"Jupyter Notebook","readme":"### 实现说明\n\n主要实现文章前半部分的工作，PyTorch实现，基于[huggingface](https://github.com/huggingface/pytorch-pretrained-BERT)的工作，PyTorch才是世界上最屌的框架，逃。\n\n### 实现参考\n\n![img1](http://wx2.sinaimg.cn/mw690/aba7d18bgy1g47p0g5ln3j210n0drtas.jpg)\n\n\n### 代码说明\n\n（1）主要修改：[modeling.py](https://github.com/zhpmatrix/BERTem/blob/master/pytorch_pretrained_bert/modeling.py)\n\noutput representation: **BertForSequenceClassification**\n\ninput representation:  **BertEmbeddings**\n\ninput和output都实现了多种策略，可以结合具体的任务，找到最佳的组合。\n\n\n（2）非主要实现：examples下的关于classification的文件\n\n（3）服务部署：基于Flask，可以在本地开启一个服务。具体实现在[tacred\\_run\\_infer.py](https://github.com/zhpmatrix/BERTem/blob/master/examples/tacred_run_infer.py)中。\n\n（4）代码仅供参考，不提供数据集，不提供预训练模型，不提供训练后的模型（希望理解吧）。\n\n（5）相关工作可以参考[我的博客-神经关系抽取](https://zhpmatrix.github.io/2019/06/30/neural-relation-extraction/)，可能比这个代码更有价值一些吧。\n\n\n### 实现结果：\n\n 数据集TACRED上的结果：\n\n|模型序号|输入类型|输出类型|指标类型|P|R|F1|备注|\n|------|------|------|------|------|------|------|------|\n|0|entity marker|sum(entity start)|micro|**0.68**|**0.63**|**0.65**|**base-model**,lr=3e-5,epoch=3|\n||||macro|**0.60**|**0.54**|**0.55**|\n|1|entity marker|sum(entity start)|micro|**0.70**|**0.62**|**0.65**|**large-model**,lr=3e-5,epoch=1|\n||||macro|**0.63**|**0.52**|**0.55**|\n|-1|None|None|micro|**0.69**|**0.66**|**0.67**|手误之后，再也找不到了，尴尬|||\n||||macro|**0.58**|**0.50**|**0.53**||||\n\n\n数据集SemEval2010 Task 8上的结果：\n\n|模型序号|输入类型|输出类型|指标类型|P|R|F1|备注|\n|------|------|------|------|------|------|------|------|\n|0|entity marker|maxpool(entity emb)+relu|micro|**0.86**|**0.86**|**0.86**|bert-large|\n||||macro|**0.82**|**0.83**|**0.82**||||\n\n\n### 混合精度加速结果\n\n在具体任务上，延续之前的setting，将train和dev合并共同作为新的train集，test集不变。在fp32\n和fp16的两种setting下，比较相同batch\\_size下，一个epoch的用时或者每个迭代的用时。\n\n|比较方面|fp32|fp16|备注|\n|------|------|------|------|\n|训练阶段|1.04it/s|4.41it/s|12.76it/s（独占显卡）|\n|推断阶段|4.14it/s|8.63it/s||\n|测试集指标|0.65/0.55|0.64/0.53|格式：micro/macro|\n|模型大小|421M|212M||\n","funding_links":[],"categories":["BERT  Knowledge Graph Task :"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhpmatrix%2FBERTem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhpmatrix%2FBERTem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhpmatrix%2FBERTem/lists"}