{"id":15009218,"url":"https://github.com/swabhs/open-sesame","last_synced_at":"2025-04-09T14:10:27.691Z","repository":{"id":22345287,"uuid":"94537817","full_name":"swabhs/open-sesame","owner":"swabhs","description":"A frame-semantic parsing system based on a softmax-margin SegRNN.","archived":false,"fork":false,"pushed_at":"2022-05-07T19:07:54.000Z","size":502,"stargazers_count":231,"open_issues_count":29,"forks_count":66,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-02T12:11:22.380Z","etag":null,"topics":["crf","deep-learning","dynet","frame-semantic-parsing","natural-language-processing","nlp","python27"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/swabhs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-16T11:31:33.000Z","updated_at":"2025-03-20T15:36:18.000Z","dependencies_parsed_at":"2022-08-07T10:15:31.469Z","dependency_job_id":null,"html_url":"https://github.com/swabhs/open-sesame","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swabhs%2Fopen-sesame","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swabhs%2Fopen-sesame/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swabhs%2Fopen-sesame/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swabhs%2Fopen-sesame/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/swabhs","download_url":"https://codeload.github.com/swabhs/open-sesame/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248054195,"owners_count":21039952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crf","deep-learning","dynet","frame-semantic-parsing","natural-language-processing","nlp","python27"],"created_at":"2024-09-24T19:23:43.973Z","updated_at":"2025-04-09T14:10:27.659Z","avatar_url":"https://github.com/swabhs.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Open-SESAME\n\nA frame-semantic parser for automatically detecting [FrameNet](https://framenet.icsi.berkeley.edu/fndrupal/) frames and their frame-elements from sentences. The model is based on  softmax-margin segmental recurrent neural nets, described in our paper [Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold](https://arxiv.org/abs/1706.09528). An example of a frame-semantic parse is shown below\n\n![Frame-semantics example](fig/fsp-example.png)\n\n## Installation\n\nThis project is built on python==3.7.9 and the [DyNet](http://dynet.readthedocs.io/en/latest/python.html) library. Additionally, it uses some packages from [NLTK](https://www.nltk.org/). \n\n```sh\n$ pip install dynet==2.0.3\n$ pip install nltk==3.5\n$ python -m nltk.downloader averaged_perceptron_tagger wordnet\n```\n\n## Data Preprocessing\n\nThis codebase only handles data in the XML format specified under FrameNet. The default version is FrameNet 1.7, but the codebase is backward compatible with versions 1.6 and 1.5.\n\nAs a first step the data is preprocessed for ease of readability.\n\n0. Clone the repository.\n```sh\n$ git clone https://github.com/swabhs/open-sesame.git\n$ cd open-sesame/\n ```\n\n1. Create a directory for the data, `$DATA`, containing the (extracted) [FrameNet version 1.7](https://drive.google.com/open?id=1s4SDt_yDhT8qFs1MZJbeFf-XeiNPNnx7) data. This should be under `$DATA/fndata-1.7/`.\n\n2. Second, this project uses pretrained [GloVe word embeddings](https://nlp.stanford.edu/projects/glove/) of 100 dimensions, trained on 6B tokens. [Download](http://nlp.stanford.edu/data/glove.6B.100d.zip) and extract under `$DATA/embeddings_glove/`.\n\n3. Optionally, make alterations to the configurations in `configurations/global_config.json`, if you have decided to either use a different version of FrameNet, or different pretrained embeddings, and so on.\n\n4. In this repository, data is formatted in a [format similar to CoNLL 2009](https://ufal.mff.cuni.cz/conll2009-st/task-description.html), but with BIO tags, for ease of reading, compared to the original XML format. See sample CoNLL formatting [here](https://github.com/swabhs/open-sesame/blob/master/sample.fn1.7.train.conll). Preprocess the data by executing:\n```sh\n$ python -m sesame.preprocess\n```\nThe above script writes the train, dev and test files in the required format into the `data/neural/fn1.7/` directory. A large fraction of the annotations are either incomplete, or inconsistent. Such annotations are discarded, but logged under `preprocess-fn1.7.log`, along with the respective error messages. To include exemplars, use the option ```--exemplar``` with the above command.\n\n\n## Training\n\nFrame-semantic parsing involves target identification, frame identification and argument identification --- each step is trained independently of the others. Details can be found in our [paper](https://arxiv.org/abs/1706.09528), and also below.\n\nTo train a model, execute:\n\n```sh\n$ python -m sesame.$MODEL --mode train --model_name $MODEL_NAME\n```\n\nThe $MODELs are specified below. Training saves the best model on validation data in the directory `logs/$MODEL_NAME/best-$MODEL-1.7-model`. The same directory will also save a `configurations.json` containing current model configuration.\n\nIf training gets interrupted, it can be restarted from the last saved checkpoint by specifying `--mode refresh`.\n\n## Pre-trained Models\n\nThe downloads need to be placed under the base-directory. On extraction, these will create a `logs/` directory containing pre-trained models for target identification, frame identification using gold targets, and argument identification using gold targets and frames.\n\n*Note* There is a [known open issue](https://github.com/swabhs/open-sesame/issues/15) about pretrained models not being able to replicate the reported performance on a different machine. It is recommended to train and test from scratch - performance can be replicated (within a small margin of error) to the performance reported below.\n\n|           |  FN 1.5 Dev | FN 1.5 Test | FN 1.5 Models                                                                             |  FN 1.7 Dev | FN 1.7 Test | FN 1.7 Models                                                                             |\n|-----------|------------:|------------:|------------------------------------------------------------------------------------------------------|------------:|------------:|------------------------------------------------------------------------------------------------------|\n| Target ID |       79.85 |       73.23 | [Download](https://drive.google.com/open?id=1xrBKFOwdqLDim2X3kGysV6B8k1ThXDZh) | 80.26 | 73.25 | [Download](https://drive.google.com/open?id=1sS0OPw1uYxeOUK0drkvfZsFkRNgnVUAC) |\n| Frame ID  |       89.27 |       86.40 | [Download](https://drive.google.com/open?id=1Ma1h1x8QDsJFQW047_9KvNLblkxlDdGb) | 89.74 | 86.55 | [Download](https://drive.google.com/open?id=1me1V0CrZF5HVWiDBqZ4LHZVSpsWfW3-8)  |\n| Arg ID    |       60.60 |       59.48 | [Download](https://drive.google.com/open?id=1TopWIEvmzQ8MBypo-xrCAokW4bZg1oGz) | 61.21 | 61.36 | [Download](https://drive.google.com/open?id=1ys-DIGhJSHgt8VjstMtlkPnYqtlzMSHe)  |\n\n## Test\n\nThe different models for target identification, frame identification and argument identification, *need to be executed in that order*.\nTo test under a given model, execute:\n\n```sh\n$ python -m sesame.$MODEL --mode test --model_name $MODEL_NAME\n```\n\nThe output, in a CoNLL 2009-like format will be written to `logs/$MODEL_NAME/predicted-1.7-$MODEL-test.conll` and in the [frame-elements file format](https://github.com/Noahs-ARK/semafor/tree/master/training/data) to `logs/$MODEL_NAME/predicted-1.7-$MODEL-test.fes` for frame and argument identification.\n\n### 1. Target Identification\n\n`$MODEL = targetid`\n\nA bidirectional LSTM model takes into account the lexical unit index in FrameNet to identify targets. This model has *not* been described in the [paper](https://arxiv.org/abs/1706.09528). Moreover, FN 1.7 exemplars cannot be used for target identification.\n\n### 2. Frame Identification\n\n`$MODEL = frameid`\n\nFrame identification is based on a bidirectional LSTM model. Targets and their respective lexical units need to be identified before this step. At test time, example-wise analysis is logged in the model directory. Exemplars can be used for frame identification using the ```--exemplar``` flag during training, but do not help (in fact reduce performance to 88.17).\n\n### 3. Argument (Frame-Element) Identification\n\n`$MODEL = argid`\n\nArgument identification is based on a segmental recurrent neural net, used as the *baseline* in the [paper](https://arxiv.org/abs/1706.09528). Targets and their respective lexical units need to be identified, and frames corresponding to the LUs predicted before this step. At test time, example-wise analysis is logged in the model directory. Exemplars can be used for argument identification using the ```--exemplar``` flag during training.\n\n## Prediction on unannotated data\n\nFor predicting targets, frames and arguments on unannotated data, pretrained models are needed. Input needs to be specified in a file containing one sentence per line. The following steps result in the full frame-semantic parsing of the sentences:\n\n```sh\n$ python -m sesame.targetid --mode predict \\\n                            --model_name fn1.7-pretrained-targetid \\\n                            --raw_input sentences.txt\n$ python -m sesame.frameid --mode predict \\\n                           --model_name fn1.7-pretrained-frameid \\\n                           --raw_input logs/fn1.7-pretrained-targetid/predicted-targets.conll\n$ python -m sesame.argid --mode predict \\\n                         --model_name fn1.7-pretrained-argid \\\n                         --raw_input logs/fn1.7-pretrained-frameid/predicted-frames.conll\n```\n\nThe resulting frame-semantic parses will be written to `logs/fn1.7-pretrained-argid/predicted-args.conll` in the same CoNLL 2009-like format.\n\n## Contact and Reference\n\nFor questions and usage issues, please contact `sswayamd@alumni.cmu.edu`. If you use open-sesame for research, please cite [our paper](https://arxiv.org/pdf/1706.09528.pdf) as follows:\n\n```\n@article{swayamdipta:17,\n  title={{Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold}},\n  author={Swabha Swayamdipta and Sam Thomson and Chris Dyer and Noah A. Smith},\n  journal={arXiv preprint arXiv:1706.09528},\n  year={2017}\n}\n```\nCopyright [2018] [Swabha Swayamdipta]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswabhs%2Fopen-sesame","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fswabhs%2Fopen-sesame","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswabhs%2Fopen-sesame/lists"}