{"id":19184215,"url":"https://github.com/princeton-vl/attach-juxtapose-parser","last_synced_at":"2025-07-27T13:13:47.632Z","repository":{"id":55582917,"uuid":"308182103","full_name":"princeton-vl/attach-juxtapose-parser","owner":"princeton-vl","description":"Code for the paper \"Strongly Incremental Constituency Parsing with Graph Neural Networks\"","archived":false,"fork":false,"pushed_at":"2023-06-30T20:16:12.000Z","size":5993,"stargazers_count":36,"open_issues_count":0,"forks_count":6,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-08T00:02:05.681Z","etag":null,"topics":["machine-learning","neurips-2020","nlp","parsing"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2010.14568","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/princeton-vl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-10-29T01:13:44.000Z","updated_at":"2025-02-28T17:11:56.000Z","dependencies_parsed_at":"2025-04-20T05:30:49.784Z","dependency_job_id":"abb0f568-c91f-4451-8d8f-f903c40fdb20","html_url":"https://github.com/princeton-vl/attach-juxtapose-parser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-vl%2Fattach-juxtapose-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-vl%2Fattach-juxtapose-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-vl%2Fattach-juxtapose-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/princeton-vl%2Fattach-juxtapose-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/princeton-vl","download_url":"https://codeload.github.com/princeton-vl/attach-juxtapose-parser/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252973678,"owners_count":21834107,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","neurips-2020","nlp","parsing"],"created_at":"2024-11-09T11:06:45.593Z","updated_at":"2025-05-08T00:04:05.686Z","avatar_url":"https://github.com/princeton-vl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Strongly Incremental Constituency Parsing with Graph Neural Networks\n\n\n![Example actions](images/actions.jpg)\n\nCode for the paper:  \n\n[Strongly Incremental Constituency Parsing with Graph Neural Networks](https://arxiv.org/abs/2010.14568)      \n[Kaiyu Yang](https://yangky11.github.io/) and [Jia Deng](https://www.cs.princeton.edu/~jiadeng/)     \nNeural Information Processing Systems (NeurIPS) 2020   \n\n```bibtex\n@inproceedings{yang2020attachjuxtapose,\n  title={Strongly Incremental Constituency Parsing with Graph Neural Networks},\n  author={Yang, Kaiyu and Deng, Jia},\n  booktitle={Neural Information Processing Systems (NeurIPS)},\n  year={2020}\n}\n```\n\n## Requirements\n\n1. Make sure your gcc version is at least 5 (`gcc --version`). I encountered segmentation faults with gcc 4.8.5. But if it works for you, it's probably fine. \n1. Download and install [Miniconda Python 3](https://docs.conda.io/en/latest/miniconda.html) (Anaconda should also work).\n1. `cd` into the root of this repo.  \n1. Edit [parser.yaml](./parser.yaml) according to your system. For example, remove [- cudatoolkit=10.2](./parser.yaml#L11) if you don't have a GPU. Change the version of cudatoolkit if necessary.  \n1. Install Python dependencies using conda: `conda env create -f parser.yaml \u0026\u0026 conda activate parser`. If you have troubles with the aforementioned two steps, you may manually install the packages in [parser.yaml](./parser.yaml) in whatever way that works for you. \n1. Compile the [Evalb](https://nlp.cs.nyu.edu/evalb/) program used for evaluation: `cd EVALB \u0026\u0026 make \u0026\u0026 cd ..`\n\n\n## Data\n\nWe include the preprocessed PTB and CTB data in the [data](./data) directory. No additional data needs to be downloaded. For PTB, we use exactly the same data files as [self-attentive-parser](https://github.com/nikitakit/self-attentive-parser). For CTB, the data files are obtained following [distance-parser](https://github.com/hantek/distance-parser), which is also adopted by [HPSG-Neural-Parser](https://github.com/DoodleJZ/HPSG-Neural-Parser). It basically selects a subset of CTB 8.0 that corresponds to CTB 5.1.\n\n\n## Training\n\nUse [train.py](./train.py) for training models. By default, `python train.py` trains the parser on PTB using XLNet encoder and graph decoder. It saves training logs and model checkpoints to `./runs/default`. We use [hydra](https://hydra.cc/) to manage command-line arguments. Please refer to [conf/train.yaml](./conf/train.yaml) for the complete list of them. Below are some examples:\n\n* To save results to `./runs/EXPID`, where `EXPID` is an arbitrary experiment identifier:\n```bash\npython train.py exp_id=EXPID\n```\n\n* To use BERT instead of XLNet\n```bash\npython train.py model=ptb_bert_graph\n```\n\n* To train on CTB using Chinese BERT as the encoder:\n```bash\npython train.py dataset=ctb model=ctb_bert_graph\n```\n\n\n\n## Results and Pre-trained Models\n\nWe provide hyperparameters, training logs and pre-trained models for reproducing our main results (Table 1 and Table 2 in the paper). In the paper, we ran each experiment 5 times with beam search and reported the mean and its standard errors (SEM). Whereas the numbers below are results of 1 run without beam search.\n\n\n#### Constituency parsing on PTB\n\n| Model         | EM       | F1      | LP      | LR       | Hyperparameters | Training log  | Pre-trained model | \n| ------------- | -------- | ------- | ------- | -------- | --------------- | ------------- | ----------------- |\n| Ours (BERT)   | 57.41    | 95.80   | 96.01   | 95.59    | [ptb_bert_graph.yaml](./conf/model/ptb_bert_graph.yaml) | [ptb_bert_graph.txt](https://huggingface.co/kaiyuy/attach-juxtapose-parser-ptb-bert/blob/main/ptb_bert_graph.txt)   | [ptb_bert_graph.pth](https://huggingface.co/kaiyuy/attach-juxtapose-parser-ptb-bert) |\n| Ours (XLNet)  | 59.48    | 96.44   | 96.64   | 96.24    | [ptb_xlnet_graph.yaml](./conf/model/ptb_xlnet_graph.yaml) | [ptb_xlnet_graph.txt](https://huggingface.co/kaiyuy/attach-juxtapose-parser-ptb-xlnet/blob/main/ptb_xlnet_graph.txt)   | [ptb_xlnet_graph.pth](https://huggingface.co/kaiyuy/attach-juxtapose-parser-ptb-xlnet) |\n\n\n#### Constituency parsing on CTB\n\n\n\n| Model         | EM               | F1              | LP              | LR               | Hyperparameters | Training log  | Pre-trained model | \n| ------------- | ---------------- | --------------- | --------------- | ---------------- | --------------- | ------------- | ----------------- |\n| Ours (BERT)   | 49.43            | 93.52           | 93.66           | 93.38            | [ctb_bert_graph.yaml](./conf/model/ctb_bert_graph.yaml) | [ctb_bert_graph.txt](https://huggingface.co/kaiyuy/attach-juxtapose-parser-ctb-bert/blob/main/ctb_bert_graph.txt)   | [ctb_bert_graph.pth](https://huggingface.co/kaiyuy/attach-juxtapose-parser-ctb-bert)  |\n\n\n## Evaluation\n\n\nTo evaluate a model checkpoint on PTB:\n```bash\npython test.py model_path=PATH_TO_MODEL dataset=ptb\n```\n\n`PATH_TO_MODEL` is the path to the `*.pth` file generated by the training script or downloaded from our pre-trained models.\n\n\nTo evaluate on CTB:\n```bash\npython test.py model_path=PATH_TO_MODEL dataset=ctb\n```\n\n\nTo evaluate with beam search:\n```bash\npython test.py model_path=PATH_TO_MODEL dataset=ptb/ctb beam_size=10\n```\n\nPlease refer to [conf/test.yaml](./conf/test.yaml) for the complete list of command-line arguments.\n\n\n#### Automatic Mixed Precision (AMP) Support\nThe evaluation script has [amp](https://pytorch.org/docs/stable/amp.html) enabled by default. In our experiments, amp speeds up the evaluation when using GTX 2080 Ti or Quadro RTX 6000, but it makes no difference when using GTX 1080 Ti. You may have to disable it when comparing speed with prior works without amp support.\n```bash\npython test.py model_path=PATH_TO_MODEL amp=false\n```\n\n\n#### GPU memory\n\nWe use a batch size of 150 during evaluation to fit our 11 GB GPU memory. Feel free to change it according to your hardware.\n```bash\npython test.py model_path=PATH_TO_MODEL eval_batch_size=XXX\n```\n\n\n## Parsing User-Provided Texts\n\nYou can use the attach-juxtapose parser to parse your own sentences. \nFirst, download the [spaCy](https://spacy.io/) models used for tokenization and POS tagging:\n```bash\npython -m spacy download en_core_web_sm\npython -m spacy download zh_core_web_sm\n```\n\nThen, store the sentences in a text file, one sentence per line. See [input_examples.txt](./input_examples.txt) and [input_examples_chinese.txt](./input_examples_chinese.txt) for examples.  \nFinally, run the parser from a model checkpoint `PATH_TO_MODEL`, saving the parse trees to a output file, e.g., `output.txt` or `output_chinese.txt`:\n```bash\npython parse.py model_path=PATH_TO_MODEL input=input_examples.txt output=output.txt\npython parse.py language=chinese model_path=PATH_TO_MODEL input=input_examples_chinese.txt output=output_chinese.txt\n```\n\n\n\n\n## Static Type Checking\n\nThe codebase uses [Python 3 type hints](https://docs.python.org/3.6/library/typing.html) extensively. We use [mypy](http://mypy-lang.org/) for static type checking. Run `mypy` to typecheck the entire codebase. [mypy.ini](./mypy.ini) is the configuration file for mypy.\n\n\n## Credits\n\n* The code for the self-attention layers ([models/utils.py](./models/utils.py)) is based on [self-attentive-parser](https://github.com/nikitakit/self-attentive-parser). \n* We include the code of the [Evalb](https://nlp.cs.nyu.edu/evalb/) tool for calculating evaluation metrics. \n* The code is formatted using [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black).\n\n\n## Third-Party Implementations\n\n* [SuPar](https://github.com/yzhangcs/parser)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprinceton-vl%2Fattach-juxtapose-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprinceton-vl%2Fattach-juxtapose-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprinceton-vl%2Fattach-juxtapose-parser/lists"}