https://github.com/mathworks/covid-19_textanalytics_matlab
https://github.com/mathworks/covid-19_textanalytics_matlab
covid-19 latent-dirichlet-allocation matlab natural-language-processing text-summarization textanalytics topic-modeling
Last synced: about 11 hours ago
JSON representation
- Host: GitHub
- URL: https://github.com/mathworks/covid-19_textanalytics_matlab
- Owner: mathworks
- License: other
- Created: 2020-04-09T20:41:13.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2023-03-23T18:49:52.000Z (about 2 years ago)
- Last Synced: 2025-05-06T02:04:35.318Z (about 11 hours ago)
- Topics: covid-19, latent-dirichlet-allocation, matlab, natural-language-processing, text-summarization, textanalytics, topic-modeling
- Size: 1.07 MB
- Stars: 6
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: license.txt
- Security: SECURITY.md
Awesome Lists containing this project
README
# **Understanding COVID-19 from Research Articles using Text Analytics in MATLAB**
[COVID-19 Open Research Dataset (CORD-19)](https://allenai.org/data/cord-19), offered by Allen Institute for AI and other leading research groups, is a collection of thousands of articles related to COVID-19 and related coronaviruses. Here, we are using text analytics techniques in MATLAB to explore the articles and use topic modeling and document summarization to answer some of the relevant questions.
## **Overview**
***Goal***: Explore relevant articles to understand “what do we know about transmission?”
***Data used***: comm_use_subset from the dataset hosted at [COVID-19 Open Research Dataset (CORD-19)](https://allenai.org/data/cord-19)
***Techniques used***: Topic modeling and Document Summarization
***MATLAB Live Script***: TopicModel_Transmission_comm_use.mlx
## **Steps**
**Step 1**:
First, we use a latent Dirichlet allocation (LDA) method to perform topic modeling to discover underlying topics in the articles. We test [four different solvers](https://www.mathworks.com/help/textanalytics/ref/fitlda.html#d120e1505):
* ‘cgs’: collapsed Gibbs sampling
* ‘avb’: approximate variational Bayes
* ‘cvb0’: variational Bayes, zeroth order
* ‘savb’: stochastic approximate variational Bayes
**Step 2**: After choosing a solver, we then choose the optimum number of topics by comparing validation perplexities for different numbers of topics.
**Step 3**: Build the final model using the chosen solver and optimum number of topics.
**Step 4**: In order to answer the question “what do we know about transmission?”, we choose the most relevant article by identifying the topic with the word, “transmission”, having highest probability and then identifying the document in that topic that has the highest probability.
**Step 5**: An alternate approach is to summarize the top abstracts.
**Next Steps**
There are many ways to dig deeper into this single question. Some of the possible approaches are:
* Use of ngrams (2 or more words) in topic modeling ([Analyze Text Data with Multiword Phrases](https://www.mathworks.com/help/textanalytics/ug/analyze-text-data-using-multi-word-phrases.html)),
* Use of TFIDF for topic modeling ([TFIDF](https://www.mathworks.com/help/textanalytics/ref/bagofwords.tfidf.html)),
* Extract summary with a query (such as transmission) using Maximal Marginal Relevance (MMR) algorithm ([mmrScores](https://www.mathworks.com/help/textanalytics/ref/mmrscores.html)), and
* Perform correlation analysis to find words most commonly associated to transmission ([Co-occurrence Analysis and Visualization](https://www.mathworks.com/matlabcentral/fileexchange/74776-co-occurrenceanalysis-and-visualization)).
## **Summary**
The aim of this example is to show
* how to use text analytics techniques to explore text data and build predictive models, and
* provide a starting point for researchers to build on it and dive deeper into unanswered questions regarding this pandemic.
## **For more information on Text Analytics using MATLAB:**
* [Download a free trial](https://www.mathworks.com/campaigns/products/trials.html?prodcode=TA)
* Visit [Text Analytics Toolbox](https://www.mathworks.com/products/text-analytics.html)
### **Requires**
- [MATLAB R2020a](https://www.mathworks.com/products/matlab.html)
- [Text Analytics Toolbox](https://www.mathworks.com/products/text-analytics.html)
- [Statistics and Machine Learning Toolbox](https://www.mathworks.com/products/statistics.html)
[](https://www.mathworks.com/matlabcentral/fileexchange/74942-covid-19_textanalytics_matlab)
Copyright 2020 - 2020 The MathWorks, Inc.