{"id":21426994,"url":"https://github.com/rensvandeschoot/software-overview-machine-learning-for-screening-text","last_synced_at":"2026-01-03T20:03:28.195Z","repository":{"id":121631912,"uuid":"460562513","full_name":"Rensvandeschoot/software-overview-machine-learning-for-screening-text","owner":"Rensvandeschoot","description":"The repository aims to create an overview and comparison of software used for systematically screening large amounts of textual data using machine learning.","archived":false,"fork":false,"pushed_at":"2024-12-19T08:33:44.000Z","size":57,"stargazers_count":13,"open_issues_count":9,"forks_count":6,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-01-23T07:44:35.313Z","etag":null,"topics":["active-learning","machine-learning","software-development","systematic-reviews"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Rensvandeschoot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-17T18:40:05.000Z","updated_at":"2025-01-05T08:58:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"dcd1622f-638f-4ae4-bfb0-5e8550de52dd","html_url":"https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Rensvandeschoot","download_url":"https://codeload.github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243933425,"owners_count":20370989,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["active-learning","machine-learning","software-development","systematic-reviews"],"created_at":"2024-11-22T21:43:33.680Z","updated_at":"2026-01-03T20:03:28.189Z","avatar_url":"https://github.com/Rensvandeschoot.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Comprehensive Guide to Machine Learning Software for Text Screening\n\nThis project aims to provide a comparison of\ndifferent software tools for machine learning-assisted text screening. The\ncomparison is designed to help researchers and practitioners make informed\ndecisions when selecting a suitable tool for their needs. We compare various\naspects, such as software functionality, data handling capabilities, and\nmachine learning properties.\n\n# Table of Contents\n- [Inclusion Criteria](#inclusion-criteria)\n- [Quick Overview](@overview)\n- [Installation](#installation)\n- [Data Handling](#data-handling)\n- [Machine Learning Properties](#machine-learning-properties)\n- [Excluded Software](#excluded-software)\n- [Software Description](#software)\n- [Contributing](#contributing)\n- [License](#license)\n- [Contact](#contact)\n\n\n# Inclusion Criteria\n\nThe initial selection process for\nselecting the software tools is documented on the [Open Science\nFramework](https://osf.io/g3nkz/) and meet the following\ninclusion criteria:\n\n- Implements a Researcher-in-the-Loop [(RITL)-based active learning cycle](https://www.nature.com/articles/s42256-020-00287-7) for systematically screening large volumes of textual data.\n- Achieves a Technology Readiness Level of at least [TRL7](https://en.wikipedia.org/wiki/Technology_readiness_level).\n- Offers user-friendly software that is accessible to a broad audience.\n- Provides a generic application that is not limited to specific content, fields, or types of interventions.\n- Is still maintained (last update \u003c3 years).\n\n\n# Overview\nThe table below offers a concise overview of various software tools designed\nfor systematically screening large volumes of textual data using machine\nlearning techniques. Each software is evaluated based on the following\nproperties:\n\n-   Is there a website?\n-\tIs the software [open-source](https://opensource.org/osd) (provide a :link: to the source code)?\n-\tIs the software peer-reviewed in a scientific article?\n-\tIs documentation or a manual available (provide a :link:)?\n-\tIs the full version of the software free of charge?\n\n\n|            Software             |                           Website                            |                         Open-Source                         |                               Published                                                                                        |                                    Documentation                                    |            Free             |\n|:-------------------------------:|:------------------------------------------------------------:|:-----------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------:|:---------------------------:|\n|     [Abstrackr](#abstrackr)     |          [:link:](http://abstrackr.cebm.brown.edu)           |                             :x:                             | [![DOI](https://img.shields.io/badge/DOI-10.1145/2110363.2110464-green.svg)](https://doi.org/10.1145/2110363.2110464)          |                                         :x:                                         |     :white_check_mark:      |\n|      [ASReview](#asreview)      |                [:link:](https://asreview.nl/)                |  :white_check_mark:[:link:](https://github.com/asreview/)   | [![DOI](https://img.shields.io/badge/DOI-10.1038/s42256--020--00287--7-green.svg)](https://doi.org/10.1038/s42256-020-00287-7) |            :white_check_mark:[:link:](https://asreview.readthedocs.io/)             |     :white_check_mark:      |\n|       [Colandr](#colandr)       |      [:link:](https://hslib.jabsom.hawaii.edu/colandr)       |                             :x:                             | [![DOI](https://img.shields.io/badge/DOI-10.1111/cobi.13117-green.svg)](https://doi.org/10.1111/cobi.13117)                    | :white_check_mark:[:link:](https://hslib.jabsom.hawaii.edu/colandr/getting_started) |     :white_check_mark:      |\n|  [DistillerSR](#distillersr)    | [:link:](https://www.evidencepartners.com/) | :x: | [![DOI](https://img.shields.io/badge/DOI-10.1016/j.vhri.2020.07.479-green.svg)](https://doi.org/10.1016/j.vhri.2020.07.479)| :white_check_mark:[:link:](https://www.evidencepartners.com/resources) | :x:|\n| [EPPI-Reviewer](#eppi-reviewer) | [:link:](https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914) |                             :x:                             |                                  :x:                                                                                           |   :white_check_mark:[:link:](https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3822)    |             :x:             |\n|        [Rayyan](#rayyan)        |               [:link:](https://www.rayyan.ai/)               |                             :x:                             | [![DOI](https://img.shields.io/badge/DOI-10.1186/s13643--016--0384--4-green.svg)](https://doi.org/10.1186/s13643-016-0384-4)   |             :white_check_mark:[:link:](https://help.rayyan.ai/hc/en-us)             |             :x:             |\n| [SWIFT-Active Screener](#swift-activescreener) |    [:link:](https://www.sciome.com/swift-activescreener/)    |                             :x:                             | [![DOI](https://img.shields.io/badge/DOI-10.1016/j.envint.2020.105623-green.svg)](https://doi.org/10.1016/j.envint.2020.105623) | :white_check_mark:[:link:](https://www.sciome.com/swift-activescreener/knowledgebase/) |             :x:             |\n|     [Covidence](#covidence)     |          [:link:](https://www.covidence.org/)               |                             :x:                              | :x:                                                                                                                            |                  :white_check_mark:[:link:](https://support.covidence.org/help)       |     :x:                     |\n\n\n:white_check_mark: Yes/Implemented;\n:x: No/Not implemented;\n:grey_question: Unknown (requires an issue).\n\n\u003csup\u003e1\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/29\n\n# Installation\n\nThis table summarizes the various installation options available for each\nsoftware tool, highlighting whether:\n\n- The software can be installed locally, ensuring that data and labeling decisions are only stored on the user's device (yes/no)?\n- The software can be installed on a server (yes/no)?\n- The software is available as an online service (Software as a Service - SAAS; yes/no; provide a link to the registration page)?\n\n|                    Software                    |       Local        |       Server       |                                             Online Service                                              |\n|:----------------------------------------------:|:------------------:|:------------------:|:-------------------------------------------------------------------------------------------------------:|\n|            [Abstrackr](#abstrackr)             |        :x:         |        :x:         |                       :white_check_mark:[:link:](http://abstrackr.cebm.brown.edu)                       |\n|             [ASReview](#asreview)              | :white_check_mark: | :white_check_mark: |                                                   :x:                                                   |\n|              [Colandr](#colandr)               |        :x:         |        :x:         |                         :white_check_mark:[:link:](https://www.colandrapp.com/)                         |\n|          [DistillerSR](#distillersr)           |        :x:         |        :x:         | :white_check_mark:[:link:](https://www.distillersr.com/products/distillersr-systematic-review-software) |\n|        [EPPI-Reviewer](#eppi-reviewer)         |        :x:         |        :x:         |                           :white_check_mark:[:link:](https://eppi.ioe.ac.uk/)                           |\n|               [Rayyan](#rayyan)                |        :x:         |        :x:         |                           :white_check_mark:[:link:](https://www.rayyan.ai/)                            |\n|         [RobotAnalyst](#robotanalyst)          |        :x:         |        :x:         |              :white_check_mark:[:link:](http://www.nactem.ac.uk/robotanalyst/)\u003csup\u003e1\u003c/sup\u003e              |\n| [SWIFT-Active Screener](#swift-activescreener) |        :x:         |        :x:         |                          :x:[:link:](https://swift.sciome.com/activescreener/)                          |  \n| [Covidence](#Covidence)                        |        :x:         |        :x:         |                          :white_check_mark:[:link:](https://www.covidence.org/)                         | \n\n:white_check_mark: Yes;\n:x: No;\n:grey_question: Unknown (requires an issue).\n\n# Data Handling\n\nThis table provides an overview of the data input/output capabilities of each\nsoftware, including:\n\n- Supported import data formats.\n- Whether partially labeled data can be imported (yes/no; if yes, as **S**(ingle) or **M**(ultiple) files)?\n- Supported export data formats.\n- If the export file includes the labeling decisions.\n- Whether the export file can be re-imported into the same software, retaining the labeling decisions (Re-Import-1: yes/no)?\n- Whether the export file can be re-imported into reference manager software, retaining the labeling decision (Re-Import-2: yes/no)?\n\n|                    Software                    |             Input data format             |                Partly labeled                 |     Output data format      |     Labeling decisions      |         Re-Import-1         |         Re-Import-2         |\n|:----------------------------------------------:|:-----------------------------------------:|:---------------------------------------------:|:---------------------------:|:---------------------------:|:---------------------------:|:---------------------------:|\n|            [Abstrackr](#abstrackr)             |         RIS, TAB, TXT\u003csup\u003e1\u003c/sup\u003e         |                      :x:                      |        CSV, XML, RIS        |     :white_check_mark:      |             :x:             |     :white_check_mark:      |\n|             [ASReview](#asreview)              | RIS, TSV, CSV, XLSX, TAB, `+`\u003csup\u003e2\u003c/sup\u003e |     :white_check_mark:(S)`+`\u003csup\u003e2\u003c/sup\u003e      |  RIS, TSV, CSV, XLSX, TAB   |     :white_check_mark:      |     :white_check_mark:      |     :white_check_mark:      |\n|              [Colandr](#colandr)               |               RIS, BIB, TXT               |             :white_check_mark:(M)             |             CSV             |     :white_check_mark:      |             :x:             |             :x:             |\n|          [DistillerSR](#distillersr)           |            ENLX, RIS, CSV, ZIP            |             :white_check_mark:(M)             |    RIS, CSV, XLSX, Word     | :grey_question:\u003csup\u003e4\u003c/sup\u003e | :grey_question:\u003csup\u003e4\u003c/sup\u003e | :grey_question:\u003csup\u003e4\u003c/sup\u003e |\n|        [EPPI-Reviewer](#eppi-reviewer)         |         RIS, TXT, `+`\u003csup\u003e3\u003c/sup\u003e         |             :white_check_mark:(M)             |          RIS, XLSX          | :grey_question:\u003csup\u003e5\u003c/sup\u003e | :grey_question:\u003csup\u003e5\u003c/sup\u003e | :grey_question:\u003csup\u003e5\u003c/sup\u003e |\n|               [Rayyan](#rayyan)                |    RIS, ENW, BIB, CSV, XML, CIW, NBIB     |             :white_check_mark:(M)             |     RIS, BIB, ENW, CSV      |     :white_check_mark:      |             :x:             |     :white_check_mark:      |\n| [SWIFT-Active Screener](#swift-activescreener) |           TXT, RIS, XML, BibTex           |             :white_check_mark:(M)             |          CSV, RIS           |     :white_check_mark:      | :grey_question:\u003csup\u003e7\u003c/sup\u003e |     :white_check_mark:      |\n| [Covidence](#covidence)                        |           TXT, RIS, XML                   |             :x:                               |          CSV, RIS           |     :white_check_mark:      |              :x:            |     :white_check_mark:      |\n\n:white_check_mark: Yes/Implemented;\n:x: No/Not implemented;\n:zap: Only for some extensions (add a footnote for more explanation);\n:grey_question: Unknown (requires an issue).\n\n\u003csup\u003e1\u003c/sup\u003e List of PubMed IDs\n\n\u003csup\u003e2\u003c/sup\u003e ASReview provides several open-source tools to convert file formats (e.g., CSV-\u003eRIS or RIS-\u003eXLSX), combine datasets (labeled, partly labeled, or unlabeled), and deduplicate records based on title/abstract/DOI.\n\n\u003csup\u003e3\u003c/sup\u003e EPPI-Reviewer provides a closed-source [online file converter](https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2934) to convert several file formats to RIS.\n\n\u003csup\u003e4\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/54\n\n\u003csup\u003e5\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/21\n\n\u003csup\u003e7\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/40\n\n\n\n\n# Machine Learning Properties\n\nThe tables below provide an overview of the machine learning properties of each software.\n\n\n## Active Learning\n\n### Training Data\n\n- Can the user select training data (prior knowledge) to train the first iteration of the model (yes/no)?\n- What is the minimum training data size (provide a number for **R**elevant and **I**rrelevant records)?\n\n\n\n|                    Software                    |        Tr.Data by user         |                 Minimum Tr.data                  |\n|:----------------------------------------------:|:------------------------------:|:------------------------------------------------:|\n|            [Abstrackr](#abstrackr)             |              :x:               |           :grey_question:\u003csup\u003e1\u003c/sup\u003e            |\n|             [ASReview](#asreview)              |       :white_check_mark:       |                     ≥1R+≥1I                      |\n|              [Colandr](#colandr)               |       :white_check_mark:       |                        10                        |\n|          [DistillerSR](#distillersr)           |       :white_check_mark:       | 25 or 2%\u003csup\u003e2\u003c/sup\u003e                             |\n|        [EPPI-Reviewer](#eppi-reviewer)         |       :white_check_mark:       |                       ≥5R                        |\n|               [Rayyan](#rayyan)                |       :white_check_mark:       |                   ≥50 with ≥5R                   |\n| [SWIFT-Active Screener](#swift-activescreener) | :white_check_mark:\u003csup\u003e4\u003c/sup\u003e |                 ≥1R\u003csup\u003e5\u003c/sup\u003e                  |\n|               [Covidence](#covidence)          |       :white_check_mark:       |                   ≥25 with ≥2R + ≥2I             |\n\n:white_check_mark: Yes/Implemented;\n:x: No/Not implemented;\n:zap: With some effort (add a footnote for more explanation);\n:grey_question: Unknown (requires an issue).\n\n\u003csup\u003e1\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/34\n\n\u003csup\u003e2\u003c/sup\u003e Training takes place after screening 25 records or after screening 2% of the dataset, whichever is greater.\n\n\u003csup\u003e4\u003c/sup\u003e Only relevant records can be provided as training data prior to screening.\n\n\u003csup\u003e5\u003c/sup\u003e If no relevant records are uploaded prior to screening, training will be initiated after screening ≥30 records with atleast ≥1R and ≥1I.\n\n### Model Selection\n\nThe table below provides an overview of the model selection properties for each software.\n\n- Can the user select the active learning model (yes/no)?\n- Can a user upload their own model (yes/no)?\n- Can the feature extraction results be stored (yes/no)?\n- Does (re-)training proceed **A**utomatically or is it triggered **M**anually?\n- Can the user continue labeling during training (yes/no)?\n- Can the user select batch size (yes/no; provide the default)?\n- Is it possible to switch to a different model during screening (yes/no)?\n\n|                    Software                    |    Select model    |     User model     | Store Feat.matrix  | Training |          Continue           | Batch size |      Switch       |\n|:----------------------------------------------:|:------------------:|:------------------:|:------------------:|:--------:|:---------------------------:|:----------:|:-----------------:|\n|            [Abstrackr](#abstrackr)             |        :x:         |        :x:         |        :x:         |    A     |     :white_check_mark:      |    :x:     |        :x:        |\n|             [ASReview](#asreview)              | :white_check_mark: | :white_check_mark: | :white_check_mark: |    A     |     :white_check_mark:      |  :x: (1)   | :zap:\u003csup\u003e1\u003c/sup\u003e |\n|              [Colandr](#colandr)               |        :x:         |        :x:         |        :x:         |    A     |     :white_check_mark:      |  :x: (10)  |        :x:        |\n|          [DistillerSR](#distillersr)           |        :x:         |        :x:         |        :x:         |   A, M   |     :white_check_mark:      |    :x:     |        :x:        |\n|        [EPPI-Reviewer](#eppi-reviewer)         |        :x:         |        :x:         |        :x:         |    M     |     :white_check_mark:      |    :x:     |        :x:        |\n|               [Rayyan](#rayyan)                |        :x:         |        :x:         |        :x:         |    M     |     :white_check_mark:      |    :x:     |        :x:        |\n| [SWIFT-Active Screener](#swift-activescreener) |        :x:         |        :x:         |        :x:         |    A     | :grey_question:\u003csup\u003e3\u003c/sup\u003e |  :x: (30)  |        :x:        |\n|               [Covidence](#covidence)          |        :x:         |        :x:         |        :x:         |    M     |     :white_check_mark:      |    :x:     |        :x:        |\n\n:white_check_mark: Yes/Implemented;\n:x: No/Not implemented;\n:zap: With some effort (add a footnote with more explanation);\n\n\u003csup\u003e1\u003c/sup\u003e Switching to a different model in ASReview is available by exporting the data of the first model and importing the data back into ASReview.\nThe software will recognize all previous labeling decisions, and a new model can be trained.\n\n\u003csup\u003e2\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/29\n\n\u003csup\u003e3\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/40\n\n\n### Overview of Available Models\n\n- Which feature extraction methods are available?\n**BOW** = bag of words;\n**TF–IDF** = term frequency–inverse document frequency;\n\n- Which classifiers are available?\n**NN** =  neural network;\n**LDA** = latent Dirichlet allocation;\n**LL** = log linear;\n**LR**= logistic regression;\n**NB** = naive Bayes;\n**RF** =random forests;\n**SGD** = stochastic gradient descent;\n**SVM** = support vector machine;\n\n\n- Which balancing strategies are available?\n**S / Simple** = no balancing balance strategy;\n**D / Double** = Double balance strategy;\n**T / Triple** = Triple balance strategy;\n**U / Under** = Undersampling balance strategy;\n**A / Aggressive** = Aggressive undersampling balance strategy (after classifier is stable);\n**W / Weighting** = Weighting for data balancing (before and after classifier is stable);\n**M / Mixing** = Mixing: weighting is applied before the classifier is stable and aggressive undersampling is applied after the classifier is stable;\n\n\n- Which query strategies are available?\n**R / Random** = Records are selected randomly;\n**C / Certain** = Certainty based;\n**U / Uncertain** = Uncertainty based;\n**M / Mixed** = A combination of query strategies, for example 90% Certainty based and 10% Random;\n**Cl / Clustering** = Clustering query strategy;\n\n\n\n|                    Software                    |            Feature Extr.                                                                                                                                   |           Classifiers                                                                                                                 |          Balancing           |         Query Stra.         |\n|:----------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------:|:----------------------------:|:---------------------------:|\n|            [Abstrackr](#abstrackr)             |  TF-IDF :grey_question:\u003csup\u003e1\u003c/sup\u003e                                                                                                                        | SVM                                                                                                                                   | :grey_question:\u003csup\u003e1\u003c/sup\u003e  |           R, C, U           |\n|             [ASReview](#asreview)              |  default: one-hot, TF–IDF (bi-gram); via [Dory-extension](https://github.com/asreview/asreview-dory): Multilingual E5, sBert (MXBAI, MPNet), GTR T5, LaBSE | default: LR, NB, RF, SVM; via [Dory-extension](https://github.com/asreview/asreview-dory): XGBoost, NN (2-layer, Dynamic, Warm Start) |          S, D, U, T          |       R, C, U, M, CL        |\n|              [Colandr](#colandr)               | Word2Vec :grey_question:\u003csup\u003e2\u003c/sup\u003e                                                                                                                       | SGD :grey_question: \u003csup\u003e2\u003c/sup\u003e                                                                                                      | :grey_question:\u003csup\u003e2\u003c/sup\u003e  |              C              |\n|          [DistillerSR](#distillersr)           |     :grey_question:\u003csup\u003e3\u003c/sup\u003e                                                                                                                            |   SVM                                                                                                                                 | :grey_question:\u003csup\u003e3\u003c/sup\u003e  | R, C                        |\n|        [EPPI-Reviewer](#eppi-reviewer)         |                TF-IDF                                                                                                                                      |               SVM                                                                                                                     | :grey_question:\u003csup\u003e4\u003c/sup\u003e  |          R, C, Cl           |\n|               [Rayyan](#rayyan)                |     :grey_question:\u003csup\u003e5\u003c/sup\u003e                                                                                                                            |               SVM                                                                                                                     | :grey_question:\u003csup\u003e5\u003c/sup\u003e  |            C, U             |\n| [SWIFT-Active Screener](#swift-activescreener) |                TF-IDF                                                                                                                                      |                LL                                                                                                                     | :grey_question:\u003csup\u003e7\u003c/sup\u003e  |              C              |\n|               [Covidence](#covidence)          |     :grey_question:\u003csup\u003e8\u003c/sup\u003e                                                                                                                            |  :grey_question:\u003csup\u003e8\u003c/sup\u003e                                                                                                          | :grey_question:\u003csup\u003e8\u003c/sup\u003e  |:grey_question:\u003csup\u003e8\u003c/sup\u003e  |  \n\n\n:white_check_mark: Yes/Implemented;\n:x: No/Not implemented;\n:grey_question: Unknown (requires an issue).\n\n\u003csup\u003e1\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/34\n\n\u003csup\u003e2\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/16\n\n\u003csup\u003e3\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/54\n\n\u003csup\u003e4\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/21\n\n\u003csup\u003e5\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/19\n\n\u003csup\u003e7\u003c/sup\u003e See issues https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/40\n\n\u003csup\u003e8\u003c/sup\u003e See issues https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/63\n\n\n## Supervised Learning\n\n|                  Software                   | Feature Extr. |          Classifiers           |          Balancing          | Query Stra. |\n|:-------------------------------------------:|:-------------:|:------------------------------:|:---------------------------:|:-----------:|\n| [EPPI-Reviewer](#eppi-reviewer)\u003csup\u003e1\u003c/sup\u003e |    TF-IDF     | SVM:grey_question:\u003csup\u003e2\u003c/sup\u003e | :grey_question:\u003csup\u003e2\u003c/sup\u003e |  R, C, Cl   |\n\n\u003csup\u003e1\u003c/sup\u003e EPPI-Reviewer offers the option to choose from, or use custom, pre-trained models to find a specific type of literature, e.g., for RCTs.\n\n\u003csup\u003e2\u003c/sup\u003e See issue https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/21\n\n## Unsupervised Learning\n\n| Software |     Q1      |\n|:--------:|:-----------:|\n\n# Excluded Software\n\nThis section contains a list of software that did not fullfill the [inclusion criteria](#inclusion-criteria), but that are still largely used in the scientific community.\n\n## [RobotAnalyst](http://www.nactem.ac.uk/robotanalyst/)\n\nRobotAnalyst was developed as part of the Supporting Evidence-based Public Health Interventions using Text Mining project to support the literature screening phase of systematic reviews. The current version of RobotAnalyst is mounted on a University of Manchester server and is a prototype demo system for research purposes at Manchester University and partners (see [response to issue #29](https://github.com/Rensvandeschoot/software-overview-machine-learning-for-screening-text/issues/29#issuecomment-2517211394)).\n\n# Software\n\nThis section briefly describes the software in alphabetical order.\n\n## [Abstrackr](https://github.com/bwallace/abstrackr-web)\n\nAbstrackr is a collaborative (i.e., multiple reviewers can simultaneously\nscreen citations for a review), web-based annotation tool for the citation\nscreening task.\n\n## [ASReview](www.asreview.nl)\n\nASReview, developed at Utrecht University, helps scholars and practitioners\nto get an overview of the most relevant records for their work as efficiently\nas possible while being transparent in the process. It allows multiple\nmachine learning models, and ships with exploration and simulation modes,\nwhich are especially useful for comparing and designing algorithms.\nFurthermore, it is intended to be easily extensible, allowing third parties\nto add modules that enhance the pipeline with new models, data, and other\nextensions.\n\n## [Colandr](https://hslib.jabsom.hawaii.edu/colandr)\n\nColandr is a free, web-based, open-access tool for conducting evidence\nsynthesis projects.\n\n## [Covidence](https://www.covidence.org/)\n\nCovidence is a collaborative, web-based software platform that streamlines the production of systematic reviews. It is designed to support the entire review lifecycle, including citation screening, full-text screening, risk of bias assessment, and data extraction.\n\n## [DistillerSR](https://www.evidencepartners.com/products/distillersr-systematic-review-software)\n\nDistillerSR automates the management of literature collection, screening, and assessment using AI and intelligent workflows. From a systematic literature review to a rapid review to a living review, DistillerSR makes any project simpler to manage and configure to produce transparent, audit-ready, and compliant results.\n\n\n## [EPPI-Reviewer](https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914)\n\nEPPI-Reviewer is a web-based software program for managing and analysing data\nin literature reviews. It has been developed for all types of systematic\nreview (meta-analysis, framework synthesis, thematic synthesis etc) but also\nhas features that would be useful in any literature review. It manages\nreferences, stores PDF files and facilitates qualitative and quantitative\nanalyses such as meta-analysis and thematic synthesis. It also contains some\nnew ‘text mining’ technology which is promising to make systematic reviewing\nmore efficient.\n\n## [Rayyan](https://www.rayyan.ai/)\n\nRayyan is a free web and mobile app, that helps expedite the initial screening\nof abstracts and titles using a process of semi-automation while incorporating\na high level of usability.\n\n## [SWIFT-Active Screener](https://www.sciome.com/swift-activescreener/)\n\nSWIFT-Active Screener (SWIFT is an acronym for “Sciome Workbench for Interactive\ncomputer-Facilitated Text-mining”) is a freely available interactive workbench\nwhich provides numerous tools to assist with problem formulation and\nliterature prioritization.\n\n# Contributing\n\nIf you know of other software that meets the inclusion criteria, please make a\nPull Request and add it to the overview. If you find any missing, incorrect,\nor incomplete information, please open an issue to discuss it.\n\nBy collaborating on this repository, we can create a valuable resource for\nresearchers, practitioners, and other stakeholders interested in leveraging\nmachine learning for text screening purposes.\n\n# License\n\nThis project is licensed under CC-BY 4.0.\n\n# Contact\n\nFor suggestions, questions, or comments, please file an issue in the issue\ntracker.\n\nThis comparison is maintained by Rens van de Schoot. The goal is to provide a\nfair and unbiased comparison. If you have any concerns regarding the\ncomparison, please open an issue in the issue tracker so that it can be\ndiscussed openly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frensvandeschoot%2Fsoftware-overview-machine-learning-for-screening-text/lists"}