{"id":29315251,"url":"https://github.com/shreyas-bhat/snli","last_synced_at":"2025-08-18T19:05:31.933Z","repository":{"id":116136635,"uuid":"443039326","full_name":"Shreyas-Bhat/SNLI","owner":"Shreyas-Bhat","description":"Natural language inference task on SNLI and carried out POS tag probing","archived":false,"fork":false,"pushed_at":"2022-01-04T18:43:26.000Z","size":497,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-07T11:41:48.905Z","etag":null,"topics":["nli-tasks","nlp","probing-task","snli"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Shreyas-Bhat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-12-30T10:17:33.000Z","updated_at":"2022-01-05T07:03:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"e3acce87-f645-42c2-b53e-bddea9b20bec","html_url":"https://github.com/Shreyas-Bhat/SNLI","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Shreyas-Bhat/SNLI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shreyas-Bhat%2FSNLI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shreyas-Bhat%2FSNLI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shreyas-Bhat%2FSNLI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shreyas-Bhat%2FSNLI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Shreyas-Bhat","download_url":"https://codeload.github.com/Shreyas-Bhat/SNLI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shreyas-Bhat%2FSNLI/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271043524,"owners_count":24689772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nli-tasks","nlp","probing-task","snli"],"created_at":"2025-07-07T11:31:17.568Z","updated_at":"2025-08-18T19:05:31.919Z","avatar_url":"https://github.com/Shreyas-Bhat.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SNLI\n\n* dataset - https://nlp.stanford.edu/projects/snli/\n* Extracted training, validation and testing data from SNLI dataset to carry out inference task\n* Used BiLSTM architecture \n\u003cp float=\"left\"\u003e\n\u003cimg src=\"images/1.png\" width=\"400\"/\u003e\n\u003cimg src=\"images/2.png\" width=\"400\"/\u003e\n\u003c/p\u003e\n* Got training accuracy of 76.3 and validation accuracy of 75.57 \n\n* Next I carried out POS tag probing \n* We probe on a model representation from the previous tasks i.e a specific frozen\nlayer of the model and we start building the model from this layer. We take the 4th\nlayer from last and add it to a keras.Sequential function.\n* We then add an RNN layer with 2 cells with returns sequences to the next layer\n* We see a significant increase in the validation accuracy due to the fact that the\nBi-LSTM model is able to interpret the POS tags. To address the probe\nconfounder problem, it may seem that due to the complexity of RNN as compared\nto MLP, the model may be memorizing the outputs from the probed layer with\nsome supervision. But, in fact, the learned parameters of the probe is low of ~800\nparameters only. In literature shallow MLPs are preferred which will be a part of\nfuture work. Literature also uses ‘selectivity’ as a metric for the performance of\nthe probe, selectivity = linguistic accuracy - control accuracy\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/3.png\" width=\"400\"/\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshreyas-bhat%2Fsnli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshreyas-bhat%2Fsnli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshreyas-bhat%2Fsnli/lists"}