{"id":13595422,"url":"https://github.com/littlepan0413/DuEE_baseline","last_synced_at":"2025-04-09T13:31:56.399Z","repository":{"id":40970129,"uuid":"241523338","full_name":"littlepan0413/DuEE_baseline","owner":"littlepan0413","description":"事件抽取基线模型","archived":false,"fork":false,"pushed_at":"2023-02-16T01:31:50.000Z","size":1432,"stargazers_count":11,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-06T18:43:58.019Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/littlepan0413.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-02-19T03:25:40.000Z","updated_at":"2021-10-26T08:07:47.000Z","dependencies_parsed_at":"2024-01-16T22:19:53.718Z","dependency_job_id":"f5d4cb17-ca2f-4f8f-9dd5-4c2f11085dab","html_url":"https://github.com/littlepan0413/DuEE_baseline","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littlepan0413%2FDuEE_baseline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littlepan0413%2FDuEE_baseline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littlepan0413%2FDuEE_baseline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littlepan0413%2FDuEE_baseline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/littlepan0413","download_url":"https://codeload.github.com/littlepan0413/DuEE_baseline/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248049338,"owners_count":21039204,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T16:01:49.712Z","updated_at":"2025-04-09T13:31:56.066Z","avatar_url":"https://github.com/littlepan0413.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"English | [简体中文](./README.zh.md)\n\n# Event Extraction Baseline model (EE-Baseline)\n\nEE-Baseline is a event extraction baseline model for the event extraction dataset DuEE 1.0. This model splits event extraction into two sub-tasks: trigger extraction and argument extraction, solved as two sequence labeling problems in a pipelined fashion.\n\n#### Trigger extraction model based on sequence labeling (Tri-SeqL)\n\nTrigger extraction aims to predict whether a token triggers an event. We formulates trigger extraction as a sequence labeling problem with labels indicate the event types. This model is built upon a pre-trained language model ERNIE combined with a CRF layer.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"pictures/trigger_model.png\" width=\"550\" height=\"400\" alt=\"Trigger extraction model based on sequence labeling\" align=center /\u003e\n\u003c/div\u003e\n\n\u003e For above example, the model recognizes the trigger \"求婚\" and assigns it to the label \"B-结婚\" \"I-结婚\", finally, we get event type is \"结婚\"。\n\n#### Argument extraction model based on Sequence labeling (Arg-SeqL)\n\nArgument extraction aims to extract arguments and corresponding argument roles they play. We formulates argument extraction as a sequence labeling problem with labels indicate the argument roles. This model is also built upon a pre-trained language model ERNIE combined with a CRF layer.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"pictures/role_model.png\" width=\"550\" height=\"400\" alt=\"Argument extraction model based on Sequence labeling\" align=center /\u003e\n\u003c/div\u003e\n\n\u003e For above example,  the model recognizes the arguments: 1) \"李荣浩\", assigns it labels \"B-求婚者\"、\"I-求婚者\" \"I-求婚者\"; 2) \"杨丞琳\", assigns it labels \"B-求婚对象\" \"I-求婚对象\" \"I-求婚对象\". Finally, we get argument roles and argument pairs is \u003c求婚者, 李荣浩\u003e \u003c求婚对象, 杨丞琳\u003e.\n\n\n#### other versions\n\n- [DuEE-PaddleHub](./DuEE-PaddleHub): Code simplified version base on PaddleHub\n\n\n## Getting Started\n\n### Environment Requirements\n\n- python 2.7.x\n- paddlepaddle-gpu \u003e= 1.5.0 (see more details about PaddlePaddle [PaddlePaddle Homepage](https://www.paddlepaddle.org.cn/install/quick))\n\nThe code is tested on a single GPU Tesla K40m, with CUDA version=10.1, GPU Driver Version=418.39.\n\n###### Install dependencies\n\n```\npip install -r ./requirements.txt\n```\n\n### Integration steps\n\n##### Step 1: Data preparation\n\nIncluding steps Download pre-trained ERNIE model、Examples process and Schmea process\n\n```shell\nsh bin/script/data_preparation.sh\n```\n\nRe-executing this step requires deleting files `./model/ERNIE_1.0_max-len-512.tar.gz`、`./data/train.json`、`./data/dev.json`、`./data/test.json`、`./dict/vocab_trigger_label_map.txt`、`./dict/vocab_roles_label_map.txt`\n\n##### Step 2: Train and results process\n\nIncluding steps train and predict results of Tri-SeqL、train and predict results of Arg-SeqL、Prediction results process.\n\n```shell\nsh bin/script/train_and_eval.sh\n```\n\nRe-executing this step requires deleting files `./save_model/trigger`、`./save_model/role`、 `./save_model/trigger/pred_trigger.json` and `./save_model/role/pred_role.json`.\n\n### Detailed steps\n\n##### Step 1: Download pre-trained ERNIE model\n\n```shell\ncd ./model\nwget https://ernie.bj.bcebos.com/ERNIE_1.0_max-len-512.tar.gz --no-check-certificate\nmkdir ERNIE_1.0_max-len-512\ntar -zxvf ERNIE_1.0_max-len-512.tar.gz -C ERNIE_1.0_max-len-512\n```\nDownload ERNIE1.0 Base（max-len-512）model and extract it into `./model/ERNIE_1.0_max-len-512/`\n\n\n##### Step 2: Examples process\n\n\nProcess examples into `./data/`, create files `train.json`、`dev.json` and `test.json`\n\n```python\npython bin/data_process.py origin_events_process ./data/eet_events.json ./data/\n```\n\n##### Step 3: Schmea process\n\n- Trigger label process for Tri-SeqL and save file into `./dict/vocab_trigger_label_map.txt`\n\n```python\npython bin/data_process.py schema_event_type_process ./dict/event_schema.json ./dict/vocab_trigger_label_map.txt\n```\n\n- Argument Role Label for Arg-SeqL and save file into `./dict/vocab_roles_label_map.txt`\n\n```python\npython bin/data_process.py schema_role_process ./dict/event_schema.json ./dict/vocab_roles_label_map.txt\n```\n\n##### Step 4: Train Tri-SeqL\n\n```shell\ncd ./bin\nHERE=$(readlink -f \"$(dirname \"$0\")\")\ncd ${HERE}/..\nDATA_DIR=${HERE}/../../data\nPRETRAIN_MODEL=${HERE}/../../model/ERNIE_1.0_max-len-512\nSAVE_MODEL=${HERE}/../../save_model\nDICT=${HERE}/../../dict\nGPUID=0\nTRIGGER_SAVE_MODEL=${SAVE_MODEL}/trigger\n\nsh script/train_event_trigger.sh ${GPUID} ${DATA_DIR} ${TRIGGER_SAVE_MODEL} ${PRETRAIN_MODEL} ${DICT}\n\n```\n\n##### Step 5: Prediction Tri-SeqL\n\n```shell\ncd ./bin\nHERE=$(readlink -f \"$(dirname \"$0\")\")\ncd ${HERE}/..\nDATA_DIR=${HERE}/../../data\nPRETRAIN_MODEL=${HERE}/../../model/ERNIE_1.0_max-len-512\nSAVE_MODEL=${HERE}/../../save_model\nDICT=${HERE}/../../dict\nGPUID=0\nTRIGGER_SAVE_MODEL=${SAVE_MODEL}/trigger\n\nsh script/predict_event_trigger.sh ${GPUID} ${DATA_DIR} ${PRETRAIN_MODEL} ${TRIGGER_SAVE_MODEL}/final_model ${DICT}\n```\n\n##### Step 6: Train Arg-SeqL\n\n```shell\ncd ./bin\nHERE=$(readlink -f \"$(dirname \"$0\")\")\ncd ${HERE}/..\nDATA_DIR=${HERE}/../../data\nPRETRAIN_MODEL=${HERE}/../../model/ERNIE_1.0_max-len-512\nSAVE_MODEL=${HERE}/../../save_model\nDICT=${HERE}/../../dict\nGPUID=0\nROLE_SAVE_MODEL=${SAVE_MODEL}/role\n\nsh script/train_event_role.sh ${GPUID} ${DATA_DIR} ${ROLE_SAVE_MODEL} ${PRETRAIN_MODEL} ${DICT}\n```\n\n##### Step 7: Prediction Arg-SeqL\n\n```shell\ncd ./bin\nHERE=$(readlink -f \"$(dirname \"$0\")\")\ncd ${HERE}/..\nDATA_DIR=${HERE}/../../data\nPRETRAIN_MODEL=${HERE}/../../model/ERNIE_1.0_max-len-512\nSAVE_MODEL=${HERE}/../../save_model\nDICT=${HERE}/../../dict\nGPUID=0\nROLE_SAVE_MODEL=${SAVE_MODEL}/role\n\nsh script/predict_event_role.sh ${GPUID} ${DATA_DIR} ${PRETRAIN_MODEL} ${ROLE_SAVE_MODEL}/final_model ${DICT}\n```\n\n##### step 8: Prediction results process\n\n- Transform test set（`./data/test.json`）to evaluation format file `./result/gold.json`\n\n```python\npython bin/predict_eval_process.py test_data_2_eval ./data/test.json ./result/gold.json\n```\n\n- Integrate and transform prediction results into evaluation format\n\nIntegrate predict results file of Tri-SeqL（`./save_model/trigger/pred_trigger.json`）、predict results file of Arg-SeqL（`./save_model/role/pred_role.json`）、events schema file（`./dict/event_schema.json`）into  evaluation format file `./result/pred.json`\n\n```python\npython bin/predict_eval_process.py predict_data_2_eval ./save_model/trigger/pred_trigger.json ./save_model/role/pred_role.json ./dict/event_schema.json ./result/pred.json\n```\n\n# Evaluation\n\nZip your prediction json (`./result/pred.json`) file and submit it to official website\n\n## Discussion\n\nIf you have any question, you can submit an issue in github and we will respond periodically.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flittlepan0413%2FDuEE_baseline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flittlepan0413%2FDuEE_baseline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flittlepan0413%2FDuEE_baseline/lists"}