{"id":19287251,"url":"https://github.com/magicgh/self-map","last_synced_at":"2025-04-22T04:31:57.845Z","repository":{"id":223878959,"uuid":"761794566","full_name":"magicgh/Self-MAP","owner":"magicgh","description":"[ACL 2024] On the Multi-turn Instruction Following for Conversational Web Agents","archived":false,"fork":false,"pushed_at":"2024-07-12T07:13:07.000Z","size":141,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-07-12T08:35:59.426Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/magicgh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-22T14:03:13.000Z","updated_at":"2024-07-12T07:13:11.000Z","dependencies_parsed_at":"2024-02-22T15:28:49.901Z","dependency_job_id":"988fc528-7be2-4b5e-8876-f5e65075a0c8","html_url":"https://github.com/magicgh/Self-MAP","commit_stats":null,"previous_names":["magicgh/self-map"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicgh%2FSelf-MAP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicgh%2FSelf-MAP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicgh%2FSelf-MAP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicgh%2FSelf-MAP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/magicgh","download_url":"https://codeload.github.com/magicgh/Self-MAP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223888467,"owners_count":17220083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T22:05:38.916Z","updated_at":"2024-11-09T22:05:44.471Z","avatar_url":"https://github.com/magicgh.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":["Datasets","Dataset"],"readme":"# Self-MAP\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./static/method.png\" alt=\"Self-MAP-Overview\" width=\"400\"/\u003e \u003c!-- Adjust the width as needed --\u003e\n  \u003cbr\u003e\n  Official Implementation of \u003cem\u003eOn the Multi-turn Instruction Following for Conversational Web Agents\u003c/em\u003e (\u003ca href=\"https://aclanthology.org/2024.acl-long.477.pdf\"\u003ePaper\u003c/a\u003e, \u003ca href=\"https://huggingface.co/datasets/magicgh/MT-Mind2Web\"\u003eDatasets\u003c/a\u003e)\n\u003c/p\u003e\n\n\n## Setup\nUse Python version \u003c= 3.11, and install the required packages using the following command:\n```bash\npip install -r requirements.txt\n```\n\n## Usage\n1. Download the MT-Mind2Web dataset and place it in the `data/` directory.\n2. Fill in the DATA_PATH and LOG_PATH variables as within the [`src/candidate_generation/conf/config.yaml`](./src/candidate_generation/conf/config.yaml) and [`src/action_prediction/conf/config.yaml`](./src/action_prediction/conf/config.yaml) files.  \n\n    ```yaml\n    data:\n        data_path: $(DATA_PATH)\n        train_split_file: data/train/*.json\n        test_split_files:\n            test_task: data/test_task/*.json\n            test_website: data/test_website/*.json\n            test_subdomain: data/test_subdomain/*.json\n    ```\n    ```yaml\n    hydra:\n        run:\n            dir: $(LOG_PATH)\n    ```\n3. Set the environment variable `OPENAI_API_KEY` to your OpenAI API key.\n\n## Candidate Generation\n### Fine-tuning\n```bash\npython src/candidate_generation/train.py model=deberta-v3-base\n```\n### Evaluation\nGenerate the ranks and scores for element candidates within the test-X set, where X can represent *task*, *website*, or *subdomain*.\n```bash\npython src/candidate_generation/evaluate.py\\\n    --model_path ${MODEL_PATH}\\\n    --data_path ${DATA_PATH}\\\n    --split_file data/test_${X}/*.json\\\n    --output_dir ${OUTPUT_PATH}\\\n```\n### Generate Pickle Files\nGenerate the pickle files for conversational action planning.\n```python\nimport pickle\n\ndef load_pickle(file_path):\n    with open(file_path, 'rb') as file:\n        data = pickle.load(file)\n    return data\n\ndef write_pickle(data, file_path):\n    with open(file_path, 'wb') as file:\n        pickle.dump(data, file)\n\nall_dict = {\"scores\": {}, \"ranks\": {}}\nfor file_path in [test_task_path, test_website_path, test_subdomain_path]:\n    data = load_pickle(file_path)\n    for key in [\"scores\", \"ranks\"]:\n        for annotation_id, element in data[key].items():\n            all_dict[key].setdefault(annotation_id, {}).update(element)\n\nwrite_pickle(all_dict, output_path)\n```\n## Conversational Action Planning\n### Fine-tuning\n```bash\ntorchrun --nproc-per-node 4 --master_port=10086 \\\n    /src/action_prediction/train.py \\\n    model=flan-t5-base \\\n    train.per_device_train_batch_size=8 \\\n    train.gradient_accumulation_steps=1 \\\n    train.fsdp=False \\\n    train.num_gpus=4 \\\n    train.epoch=5 \\\n    run_id=\"full\" \\\n    ++self_map.generation=False \\\n    ++self_map.memory_simplification=False \\\n    ++self_map.memory_refinement=False \\\n    ++self_map.multifaceted_matching=False \\\n```\n### Evaluation\n```bash\npython src/action_prediction/evaluate.py\\\n  +model_path=${MODEL_PATH}\\\n  model=flan-t5-base\\\n  +output_path=${OUTPUT_PATH}\\\n  +top_k=50\\\n  ++self_map.generation=False \\\n  ++self_map.memory_simplification=False \\\n  ++self_map.memory_refinement=False \\\n  ++self_map.multifaceted_matching=False \\\n```\nSet *self_map.${generation, memory_simplification, memory_refinement, multifaceted_matching}* to True to enable the corresponding module.\n\n## LICENSE\nOur code is derived from the [Mind2Web](https://github.com/OSU-NLP-Group/Mind2Web/) project under the MIT License.  \nOur MT-Mind2Web dataset is made available under the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/) license.\n\n## Citation\n```\n@inproceedings{self-map,\n    author = {Deng, Yang and Zhang, Xuan and Zhang, Wenxuan and Yuan, Yifei and Ng, See-Kiong and Chua, Tat-Seng},\n    booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},\n    doi = {10.18653/v1/2024.acl-long.477},\n    editor = {Ku, Lun-Wei  and  Martins, Andre  and  Srikumar, Vivek},\n    pages = {8795--8812},\n    publisher = {Association for Computational Linguistics},\n    title = {On the Multi-turn Instruction Following for Conversational Web Agents},\n    url = {https://aclanthology.org/2024.acl-long.477},\n    year = {2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagicgh%2Fself-map","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmagicgh%2Fself-map","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagicgh%2Fself-map/lists"}