{"id":16536310,"url":"https://github.com/timbmg/structured-self-attentive-sentence-embedding","last_synced_at":"2025-03-16T19:31:51.680Z","repository":{"id":40986246,"uuid":"141027063","full_name":"timbmg/Structured-Self-Attentive-Sentence-Embedding","owner":"timbmg","description":"Re-Implementation of \"A Structured Self-Attentive Sentence Embedding\" by Lin et al., 2017","archived":false,"fork":false,"pushed_at":"2024-08-21T08:36:06.000Z","size":707,"stargazers_count":24,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-27T12:37:18.850Z","etag":null,"topics":["attention","deep-learning","machine-learning","neural-networks","nlp","pytorch","recurrent-neural-networks","self-attention","self-attentive-rnn","sentiment-analysis","text-classification","vizualisation","yelp-dataset"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/1703.03130.pdf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timbmg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-07-15T12:51:18.000Z","updated_at":"2024-08-21T08:35:04.000Z","dependencies_parsed_at":"2024-08-21T10:06:38.093Z","dependency_job_id":null,"html_url":"https://github.com/timbmg/Structured-Self-Attentive-Sentence-Embedding","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timbmg%2FStructured-Self-Attentive-Sentence-Embedding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timbmg%2FStructured-Self-Attentive-Sentence-Embedding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timbmg%2FStructured-Self-Attentive-Sentence-Embedding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timbmg%2FStructured-Self-Attentive-Sentence-Embedding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timbmg","download_url":"https://codeload.github.com/timbmg/Structured-Self-Attentive-Sentence-Embedding/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243826791,"owners_count":20354220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","deep-learning","machine-learning","neural-networks","nlp","pytorch","recurrent-neural-networks","self-attention","self-attentive-rnn","sentiment-analysis","text-classification","vizualisation","yelp-dataset"],"created_at":"2024-10-11T18:30:33.413Z","updated_at":"2025-03-16T19:31:51.170Z","avatar_url":"https://github.com/timbmg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A Structured Self-Attentive Sentence Embedding\nRe-Implementation of [_A Structured Self-Attentive Sentence Embedding_](https://arxiv.org/pdf/1703.03130.pdf) by Lin et al., 2017.\n\n## Results\n\n| Set        | Loss     | Accuracy |\n|:-----------|---------:|---------:|\n| Training   | 1.136    | 77.26%   |\n| Validation | 1.587    | 60.91%   |\n\nAbove results were obtained after training for 5 epochs. The training set contained 20000 examples and validation set 1000 examples. The model with the best validation loss was choosen. Note that the training set size in the paper is much bigger.\n\n## Data\nThe Yelp dataset can be download [here](http://pan.webis.de/clef16/pan16-web/author-profiling.html). \nAfter downloading, the file only has to be unzipped.\n\n## Training\nYou can run the training procedure with the default settings with the following command:  \n`python3 train.py --data-dir \u003cdir of unzipped yelp data\u003e`\n\nFor more information about training settings run:  \n`python3 train.py --help`\n\n## Analysis \u0026 Visualization\nOnce the model is trained, the attention pattern can be visualized as done in the paper.\nThe following python script will create an HTML file with the reviews and respective attention pattern. \nAlso the confusion matrix for the classification will be created.  \n`python3 viz.py --html --cm --data-dir \u003cdir of unzipped yelp data\u003e --validation-set \u003cpath to saved validation split\u003e`\n\n### Attention Pattern \n![](imgs/viz2.png)\n\n### Confusion Matrix\n![](imgs/cm.png)\n\n## Differences with the paper\n- Adam instead of SGD\n- No gradient clipping\n- No dropout\n- No GLOVE word embedding initialization\n\n## Requirements\n- Implemented and tested with python 3.6.5\n- Python library versions can be found in [requirements.txt](requirements.txt)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimbmg%2Fstructured-self-attentive-sentence-embedding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimbmg%2Fstructured-self-attentive-sentence-embedding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimbmg%2Fstructured-self-attentive-sentence-embedding/lists"}