{"id":13862089,"url":"https://github.com/kaushalshetty/Structured-Self-Attention","last_synced_at":"2025-07-14T11:32:20.775Z","repository":{"id":92597112,"uuid":"119800854","full_name":"kaushalshetty/Structured-Self-Attention","owner":"kaushalshetty","description":"A Structured Self-attentive Sentence Embedding","archived":false,"fork":false,"pushed_at":"2019-09-22T21:00:24.000Z","size":504,"stargazers_count":495,"open_issues_count":2,"forks_count":106,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-11-22T22:34:05.704Z","etag":null,"topics":["attention","attention-mechanism","attention-model","attention-weights","classification","deep-learning","python3","pytorch","self-attention","self-attentive-rnn","sentence-embeddings","visualization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kaushalshetty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-01T07:38:55.000Z","updated_at":"2024-11-21T07:35:19.000Z","dependencies_parsed_at":"2023-03-02T10:15:38.503Z","dependency_job_id":null,"html_url":"https://github.com/kaushalshetty/Structured-Self-Attention","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kaushalshetty/Structured-Self-Attention","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaushalshetty%2FStructured-Self-Attention","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaushalshetty%2FStructured-Self-Attention/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaushalshetty%2FStructured-Self-Attention/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaushalshetty%2FStructured-Self-Attention/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kaushalshetty","download_url":"https://codeload.github.com/kaushalshetty/Structured-Self-Attention/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaushalshetty%2FStructured-Self-Attention/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265285644,"owners_count":23740568,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","attention-mechanism","attention-model","attention-weights","classification","deep-learning","python3","pytorch","self-attention","self-attentive-rnn","sentence-embeddings","visualization"],"created_at":"2024-08-05T06:01:36.939Z","updated_at":"2025-07-14T11:32:20.461Z","avatar_url":"https://github.com/kaushalshetty.png","language":"Python","funding_links":[],"categories":["Python","Paper implementations｜论文实现","Paper implementations"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"readme":"# Structured Self-attentive sentence embeddings \nImplementation for the paper A Structured Self-Attentive Sentence Embedding, which was published in ICLR 2017: https://arxiv.org/abs/1703.03130 .\n#### USAGE:\nFor binary sentiment classification on imdb dataset run :\n`python classification.py \"binary\"`\n\nFor multiclass classification on reuters dataset run :\n`python classification.py \"multiclass\"`\n\nYou can change the model parameters in the `model_params.json file`\nOther tranining parameters like number of attention hops etc can be configured in the `config.json` file.\n\nIf you want to use pretrained glove embeddings , set the `use_embeddings` parameter to `\"True\"` ,default is set to False. Do not forget to download the `glove.6B.50d.txt` and place it in the glove folder.\n\n\n\n#### Implemented:\n* Classification using self attention\n* Regularization using Frobenius norm\n* Gradient clipping\n* Visualizing the attention weights\n\nInstead of pruning ,used averaging over the sentence embeddings.\n\n#### Visualization:\nAfter training, the model is tested on 100 test points. Attention weights for the 100 test data are retrieved and used to visualize over the text using heatmaps. A file visualization.html gets saved in the visualization/ folder after successful training. The visualization code was provided by Zhouhan Lin (@hantek). Many thanks.\n\n\nBelow is a shot of the visualization on few datapoints.\n![alt text](https://github.com/kaushalshetty/Structured-Self-Attention/blob/master/visualization/attention.png \"Attention Visualization\")\n\n\n\nTraining accuracy 93.4%\nTested on 1000 points with 90.2% accuracy\n\n---\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaushalshetty%2FStructured-Self-Attention","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaushalshetty%2FStructured-Self-Attention","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaushalshetty%2FStructured-Self-Attention/lists"}