{"id":17871342,"url":"https://github.com/da03/bert_results","last_synced_at":"2026-02-27T04:04:00.046Z","repository":{"id":85314945,"uuid":"333642530","full_name":"da03/bert_results","owner":"da03","description":null,"archived":false,"fork":false,"pushed_at":"2021-01-29T20:48:32.000Z","size":51701,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-17T19:11:39.658Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/da03.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-28T04:13:29.000Z","updated_at":"2021-01-29T20:48:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"ab198fad-d308-47d0-8afc-2e0011c50cd3","html_url":"https://github.com/da03/bert_results","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da03%2Fbert_results","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da03%2Fbert_results/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da03%2Fbert_results/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/da03%2Fbert_results/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/da03","download_url":"https://codeload.github.com/da03/bert_results/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246906983,"owners_count":20853008,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-28T10:30:33.937Z","updated_at":"2025-10-26T10:04:35.652Z","avatar_url":"https://github.com/da03.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"| Task                      |       Metric         | Receptive Field Density |   Result    |    Log                              |  Json Logs                     |\n|---------------------------|----------------------|-------------------------|-------------|-------------------------------------|--------------------------------|\n| cola                      | matthews correlation | 1.0                     |  56.53      | Huggingface                         |                                | \n|                           |                      | 1.0                     |  54.95      |                                     |                                |\n|                           |                      | 0.1                     |  51.27      | [0.1.txt](cola/log_density_0.1.txt) | [cola/json/0.1](cola/json/0.1) |\n|                           |                      | 0.2                     |  51.80      | [0.2.txt](cola/log_density_0.2.txt) | [cola/json/0.2](cola/json/0.2) |\n|                           |                      | 0.3                     |  49.23      | [0.3.txt](cola/log_density_0.3.txt) | [cola/json/0.3](cola/json/0.3) |\n| sst2                      | accuracy             | 1.0                     |  92.32      | Huggingface                         |                                |\n|                           |                      | 1.0                     |  92.09      |                                     |                                |\n|                           |                      | 0.1                     |  91.06      | [0.1.txt](sst2/log_density_0.1.txt) | [sst2/json/0.1](sst2/json/0.1) |\n|                           |                      | 0.2                     |  92.09      | [0.2.txt](sst2/log_density_0.2.txt) | [sst2/json/0.2](sst2/json/0.2) |\n|                           |                      | 0.3                     |  91.97      | [0.3.txt](sst2/log_density_0.3.txt) | [sst2/json/0.3](sst2/json/0.3) |\n| mrpc                      | accuracy/f1          | 1.0                     | 84.07/88.85 | Huggingface                         |                                |\n|                           |                      | 1.0                     | 83.82/88.58 |                                     |                                |\n|                           |                      | 0.1                     | 83.58/88.31 | [0.1.txt](mrpc/log_density_0.1.txt) | [mrpc/json/0.1](mrpc/json/0.1) |\n|                           |                      | 0.2                     | 83.58/88.39 | [0.2.txt](mrpc/log_density_0.2.txt) | [mrpc/json/0.2](mrpc/json/0.2) |\n|                           |                      | 0.3                     | 83.58/88.39 | [0.3.txt](mrpc/log_density_0.3.txt) | [mrpc/json/0.3](mrpc/json/0.3) |\n| stsb                      | pearson/spearmanr    | 1.0                     | 88.64/88.48 | Huggingface                         |                                |\n|                           |                      | 1.0                     | 88.83/88.68 |                                     |                                |\n|                           |                      | 0.1                     | 64.05/67.01 | [0.1.txt](stsb/log_density_0.1.txt) | [stsb/json/0.1](stsb/json/0.1) |\n|                           |                      | 0.2                     | 80.94/79.87 | [0.2.txt](stsb/log_density_0.2.txt) | [stsb/json/0.2](stsb/json/0.2) |\n|                           |                      | 0.3                     | 84.80/83.84 | [0.3.txt](stsb/log_density_0.3.txt) | [stsb/json/0.3](stsb/json/0.3) |\n| qqp                       | accuracy/f1          | 1.0                     | 90.71/87.49 | Huggingface                         |                                |\n|                           |                      | 1.0                     | 90.71/87.49 |                                     |                                |\n|                           |                      | 0.1                     |             | [0.1.txt](qqp/log_density_0.1.txt)  | [qqp/json/0.1](qqp/json/0.1)   |\n|                           |                      | 0.2                     |             | [0.2.txt](qqp/log_density_0.2.txt)  | [qqp/json/0.2](qqp/json/0.2)   |\n|                           |                      | 0.3                     |             | [0.3.txt](qqp/log_density_0.3.txt)  | [qqp/json/0.3](qqp/json/0.3)   |\n| mnli                      | accuracy             | 1.0                     | 83.91/84.10 | Huggingface                         |                                |\n|                           |                      | 1.0                     | 84.48       |                                     |                                |\n|                           |                      | 0.1                     | 82.64       | [0.1.txt](mnli/log_density_0.1.txt) | [mnli/json/0.1](mnli/json/0.1) |\n|                           |                      | 0.2                     | 84.10       | [0.2.txt](mnli/log_density_0.2.txt) | [mnli/json/0.2](mnli/json/0.2) |\n|                           |                      | 0.3                     | 84.34       | [0.3.txt](mnli/log_density_0.3.txt) | [mnli/json/0.3](mnli/json/0.3) |\n| qnli                      | accuracy             | 1.0                     | 90.66       | Huggingface                         |                                |\n|                           |                      | 1.0                     | 90.68       |                                     |                                |\n|                           |                      | 0.1                     | 90.37       | [0.1.txt](qnli/log_density_0.1.txt) | [qnli/json/0.1](qnli/json/0.1) |\n|                           |                      | 0.2                     | 90.72       | [0.2.txt](qnli/log_density_0.2.txt) | [qnli/json/0.2](qnli/json/0.2) |\n|                           |                      | 0.3                     | 90.66       | [0.3.txt](qnli/log_density_0.3.txt) | [qnli/json/0.3](qnli/json/0.3) |\n| rte                       | accuracy             | 1.0                     | 65.70       | Huggingface                         |                                |\n|                           |                      | 1.0                     | 66.79       |                                     |                                |\n|                           |                      | 0.1                     | 66.06       | [0.1.txt](rte/log_density_0.1.txt)  | [rte/json/0.1](rte/json/0.1)   |\n|                           |                      | 0.2                     | 66.06       | [0.2.txt](rte/log_density_0.2.txt)  | [rte/json/0.2](rte/json/0.2)   |\n|                           |                      | 0.3                     | 63.54       | [0.3.txt](rte/log_density_0.3.txt)  | [rte/json/0.3](rte/json/0.3)   |\n| wnli                      | accuracy             | 1.0                     | 56.34       | Huggingface                         |                                |\n|                           |                      | 1.0                     | 54.93       |                                     |                                |\n|                           |                      | 0.1                     | 56.34       | [0.1.txt](wnli/log_density_0.1.txt) | [wnli/json/0.1](wnli/json/0.1) |\n|                           |                      | 0.2                     | 54.93       | [0.2.txt](wnli/log_density_0.2.txt) | [wnli/json/0.2](wnli/json/0.2) |\n|                           |                      | 0.3                     | 54.93       | [0.3.txt](wnli/log_density_0.3.txt) | [wnli/json/0.3](wnli/json/0.3) |\n\n\n## Reading JSON files\n\n```\nimport json\n\n\nwith open('sst2/json/0.1/val_0.json') as f:\n    distillation = json.load(f)\n\n\nstep = 0 # classification task\nprint (f'When generating {distillation[\"true_target_label\"]}')\nattended_source = [f'{idx}:' + \"'\" + distillation['source_tokens_text'][idx] + \"'\" for idx in distillation['all_unmasked_source_tokens'][step]]\nsource = [f'{idx}:' + \"'\" + distillation['source_tokens_text'][idx] + \"'\" for idx in range(len(distillation['source_tokens_text']))]\nprint(f'  Source: {\", \".join(source)}')\nprint(\"  Attended to source: {}\".format(\", \".join(attended_source)))\n\nlog = distillation['distillation'][0]['log']\n\ntop_labels = log[0]['model_top_predictions']\ntop_ps = log[0]['model_top_probs']\ngold = [f'{label}: {p}' for label, p in zip(top_labels, top_ps)]\nprint (f' ** full model prediction: {\", \".join(gold)}')\nfor l in log:\n    top_labels = l['distilled_top_predictions']\n    top_ps = l['distilled_top_probs']\n    top_preds = [f'{label}: {p}' for label, p in zip(top_labels, top_ps)]\n    print (f'  **added token: {l[\"added_token_text\"]}, KL becomes {l[\"kl\"]}, distillation top predictions: {\", \".join(top_preds)}')\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fda03%2Fbert_results","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fda03%2Fbert_results","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fda03%2Fbert_results/lists"}