{"id":28795760,"url":"https://github.com/ukplab/structadapt","last_synced_at":"2025-10-27T05:09:55.327Z","repository":{"id":49784637,"uuid":"402366953","full_name":"UKPLab/StructAdapt","owner":"UKPLab","description":"Structural Adapters in Pretrained Language Models for AMR-to-Text Generation (EMNLP 2021)","archived":false,"fork":false,"pushed_at":"2023-03-30T00:38:15.000Z","size":1211,"stargazers_count":23,"open_issues_count":3,"forks_count":8,"subscribers_count":16,"default_branch":"master","last_synced_at":"2023-04-09T21:51:27.689Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UKPLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-09-02T09:42:46.000Z","updated_at":"2023-03-28T07:20:09.000Z","dependencies_parsed_at":"2022-09-02T20:22:48.206Z","dependency_job_id":null,"html_url":"https://github.com/UKPLab/StructAdapt","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/UKPLab/StructAdapt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UKPLab%2FStructAdapt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UKPLab%2FStructAdapt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UKPLab%2FStructAdapt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UKPLab%2FStructAdapt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UKPLab","download_url":"https://codeload.github.com/UKPLab/StructAdapt/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UKPLab%2FStructAdapt/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260477932,"owners_count":23015066,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-18T03:10:15.303Z","updated_at":"2025-10-27T05:09:50.287Z","avatar_url":"https://github.com/UKPLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Structural Adapters in Pretrained Language Models for AMR-to-Text Generation\n\nThis repository contains the code for the [EMNLP 2021 paper](https://arxiv.org/pdf/2103.09120.pdf) \"Structural Adapters in Pretrained Language Models for AMR-to-Text Generation\". Please cite the paper if you find it useful. \n\n**StructAdapt** is an adapter-based method that encodes graph structures into pretrained language models. Contrary to prior work, **StructAdapt** effectively models interactions among the nodes based on the graph connectivity, only training graph\nstructure-aware adapter parameters.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"img/figure1-adapter-paper.png\" width=\"500\"\u003e\n\u003c/p\u003e\n\nFigure 1(a) shows a standard adapter, whereas Figure 1(b) illustrates **StructAdapt**.\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"img/adapter.png\" width=\"500\"\u003e\n\u003c/p\u003e\n\n## Datasets\n\nIn our experiments, we use the following datasets: [AMR2017](https://catalog.ldc.upenn.edu/LDC2017T10), [AMR2020](https://catalog.ldc.upenn.edu/LDC2020T02).\n\n## Environment\n\nThe easiest way to proceed is to create a conda environment:\n```\nconda create -n structadapt python=3.6\n```\n\nFurther, install PyTorch and PyTorch Geometric:\n\n```\nconda install -c pytorch pytorch=1.6.0\npip install torch-scatter==2.0.5 -f https://data.pyg.org/whl/torch-1.6.0+${CUDA}.html\npip install torch-sparse==0.6.8 -f https://data.pyg.org/whl/torch-1.6.0+${CUDA}.html\npip install torch-geometric==1.6.1\n```\nwhere `{CUDA}` is your CUDA version. Please, refer to [PyTorch Geometric installation page](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html) for more details.\n\nFinally, install the packages required:\n\n```\npip install -r requirements.txt\n```\n\n## Preprocess\n\nConvert the dataset into the format required for the model:\n\n```\n./preprocess/preprocess_AMR.sh \u003cdataset_folder\u003e\n```\n\n\n## Finetuning\n\nFor training **StructAdapt** using the AMR dataset, execute:\n```\n./train.sh \u003cmodel\u003e \u003cdata_dir\u003e \u003cgpu_id\u003e \n```\n \nOptions for `\u003cmodel\u003e` are `t5-small`, `t5-base`, `t5-large`. \n\nExample:\n```\n./train.sh t5-large data/processed_amr 0\n```\n\n\n## Decoding\n\nFor decoding, run:\n```\n./test.sh \u003cmodel\u003e \u003ccheckpoint_folder\u003e \u003cdata_dir\u003e \u003cgpu_id\u003e\n```\n\nExample:\n```\n./test.sh t5-base ckpt-folder data/processed_amr 0\n```\n\n\n## Trained Model\n\nA checkpoint of **StrutAdapt** (T5-large) trained on LDC2020T02 dataset can be found [here](https://public.ukp.informatik.tu-darmstadt.de/ribeiro/structadapt/ckpt_amr2020.tar.gz). This model achieves a _BLEU score_ of _48.21_. The outputs can be downloaded [here](https://raw.githubusercontent.com/UKPLab/StructAdapt/master/outputs/test_generations.txt).\n\n## More\nFor more details regarding hyperparameters, please refer to [HuggingFace](https://huggingface.co/).\n\n\n**Contact person:** Leonardo Ribeiro, ribeiro@aiphes.tu-darmstadt.de\n\n## Citation\n\n```\n@inproceedings{ribeiro-etal-2021-structural,\n    title = \"Structural Adapters in Pretrained Language Models for {AMR}-to-{T}ext Generation\",\n    author = \"Ribeiro, Leonardo F. R.  and\n      Zhang, Yue  and\n      Gurevych, Iryna\",\n    booktitle = \"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing\",\n    month = nov,\n    year = \"2021\",\n    address = \"Online and Punta Cana, Dominican Republic\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2021.emnlp-main.351\",\n    pages = \"4269--4282\",\n    abstract = \"Pretrained language models (PLM) have recently advanced graph-to-text generation, where the input graph is linearized into a sequence and fed into the PLM to obtain its representation. However, efficiently encoding the graph structure in PLMs is challenging because such models were pretrained on natural language, and modeling structured data may lead to catastrophic forgetting of distributional knowledge. In this paper, we propose StructAdapt, an adapter method to encode graph structure into PLMs. Contrary to prior work, StructAdapt effectively models interactions among the nodes based on the graph connectivity, only training graph structure-aware adapter parameters. In this way, we incorporate task-specific knowledge while maintaining the topological structure of the graph. We empirically show the benefits of explicitly encoding graph structure into PLMs using StructAdapt, outperforming the state of the art on two AMR-to-text datasets, training only 5.1{\\%} of the PLM parameters.\",\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fukplab%2Fstructadapt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fukplab%2Fstructadapt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fukplab%2Fstructadapt/lists"}