{"id":13535015,"url":"https://github.com/yaserkl/BERTvsULMFIT","last_synced_at":"2025-04-02T00:31:19.597Z","repository":{"id":80921516,"uuid":"170186563","full_name":"yaserkl/BERTvsULMFIT","owner":"yaserkl","description":"Comparing Text Classification results using BERT embedding and ULMFIT embedding","archived":false,"fork":false,"pushed_at":"2019-02-13T23:34:54.000Z","size":29216,"stargazers_count":65,"open_issues_count":2,"forks_count":11,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-02T22:32:55.451Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yaserkl.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-11T19:16:17.000Z","updated_at":"2024-03-09T02:47:17.000Z","dependencies_parsed_at":"2023-03-02T08:45:14.215Z","dependency_job_id":null,"html_url":"https://github.com/yaserkl/BERTvsULMFIT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaserkl%2FBERTvsULMFIT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaserkl%2FBERTvsULMFIT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaserkl%2FBERTvsULMFIT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaserkl%2FBERTvsULMFIT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yaserkl","download_url":"https://codeload.github.com/yaserkl/BERTvsULMFIT/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246735034,"owners_count":20825212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T08:00:48.538Z","updated_at":"2025-04-02T00:31:19.569Z","avatar_url":"https://github.com/yaserkl.png","language":"Jupyter Notebook","funding_links":[],"categories":["other resources for BERT:"],"sub_categories":[],"readme":"## Instructions\n\n\\[THIS REPOSITORY IS UNDER DEVELOPMENT AND MOER DATASETS AND MODELS WILL BE ADDED\\]\n\n\\[FEEL FREE TO MAKE PULL REQUEST FOR A NEW DATASET OR NEW MODEL\\]\n\n## 1. Requirements\n\n* CUDA 9.0\n* Python 3.6\n* `bash setup.sh`\n\n\nRun [setup.sh](setup.sh) to download the datasets and install all the required packages.\n\nRun [prepare_datasets.py](prepare_datasets.py) notebook to prepare the datasets.\n\nFor instruction regarding running each model go the respective model directory.\n\nThe [models](models) directory holds the result of these experiments.\n\n\nBert 20NG Confusion Matrix |  Bert 20NG Sankey Plot\n:-------------------------:|:-------------------------:\n\u003cimg src=\"https://github.com/yaserkl/BERTvsULMFIT/raw/master/models/bert/20ng/cm_20ng.png\" alt=\"20 Newsgroup Confusion Matrix\"\u003e  |  \u003cimg src=\"https://github.com/yaserkl/BERTvsULMFIT/raw/master/models/bert/20ng/sankey_20ng.png\" alt=\"20 Newsgroup Sankey Plot\"\u003e\n\n\n\n\n\n\n## 2. Results\n\n### 2.1 BERT\n\nBert (MXNet)               | F1-score | Precision | Recall  | Accuracy | Error Rate\n-------------------------- | :------: | :-------: | :----:  | :------: | :--------:\n20ng                       |   91.24  |   91.46   |  91.13  |   91.04  |    8.96\nIMDB                       |   88.59  |   88.61   |  88.62  |   88.6   |    11.4\nReuters 21578 (R8)         |   94.38  |   93.62   |  95.64  |   98.12  |    1.88\nReuters 21578 (R52)        | **73.80**|   73.48   |**76.01**|   96.35  |    3.65\nOhsumed (all docs)         |   70.45  |   73.97   |  68.84  |   79.30  |    20.70\nOhsumed (first 20k docs)   | **56.52**| **61.49** |**56.04**| **71.04**|  **28.96**\n\n\n### 2.2 ULMFit\n\nULMFit                     | F1-score | Precision | Recall  | Accuracy | Error Rate\n-------------------------- | :------: | :-------: | :----:  | :------: | :--------:\n20ng                       | **92.87**| **93.02** |**92.82**| **92.82**|  **7.18**\nIMDB                       |**91.92** | **91.96** |**91.96**| **91.92**|  **8.08**\nReuters 21578 (R8)         | **94.79**| **94.07** |**96.12**| **98.18**|  **1.82**\nReuters 21578 (R52)        |   73.77  | **75.47** |  75.96  | **96.43**|  **3.57**\nOhsumed (all docs)         |**74.82** | **75.01** |**75.47**| **81.96**|  **18.04**\nOhsumed (first 20k docs)   |   43.76  |   44.46   |  45.49  |   62.5   |    37.5\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyaserkl%2FBERTvsULMFIT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyaserkl%2FBERTvsULMFIT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyaserkl%2FBERTvsULMFIT/lists"}