https://github.com/harmonydata/harmony_original
The Harmony project
https://github.com/harmonydata/harmony_original
ai data-science data-visualization harmonisation harmonization machine-learning mentalhealth multilingual multilingual-nlp natural-language-processing natural-language-understanding naturallanguageprocessing nlp psychology python scikit-learn
Last synced: 3 months ago
JSON representation
The Harmony project
- Host: GitHub
- URL: https://github.com/harmonydata/harmony_original
- Owner: harmonydata
- License: mit
- Created: 2022-10-13T15:26:57.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-06-09T20:52:24.000Z (almost 2 years ago)
- Last Synced: 2024-12-28T03:29:11.901Z (5 months ago)
- Topics: ai, data-science, data-visualization, harmonisation, harmonization, machine-learning, mentalhealth, multilingual, multilingual-nlp, natural-language-processing, natural-language-understanding, naturallanguageprocessing, nlp, psychology, python, scikit-learn
- Language: Jupyter Notebook
- Homepage: https://harmonydata.org/
- Size: 2.77 MB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Harmony version 0.1.0
*A second version of Harmony is in development as an API at https://github.com/harmonydata/harmony*
Harmony is a data harmonisation project that uses Natural Language Processing to help researchers make better use of existing data from different studies by supporting them with the harmonisation of various measures and items used in different studies. Harmony is a collaboration project between the University of Ulster, University College London, the Universidade Federal de Santa Maria in Brazil, and Fast Data Science Ltd.
You can read more at https://harmonydata.org.
There is a live demo at: https://app.harmonydata.org/
This front end is based on the Dash Food Footprint demo: https://dash.gallery/dash-food-footprint/
Runs on Dash interactive Python framework developed by [Plotly](https://plot.ly/).
Developed by Thomas Wood / Fast Data Science
[email protected]
This tool is written in Python using the Dash front end library and the Java library Tika for reading PDFs, and runs on Linux, Mac, and Windows, and can be deployed as a web app using Docker.
# How does Harmony work in layman's terms?
Harmony compares questions from different instruments by converting them to a vector representation and calculating their similarity. You can read more at https://harmonydata.org/how-does-harmony-work/
# FAIR data schema
We have defined a data schema in accordance with the [FAIR principles](https://harmonydata.org/fair-data/).
Questionnaires are represented within Harmony in a tabular format.
The file name is the unique identifier of a questionnaire, e.g. `GAD-7 English.csv`.
Files are tab-separated with the following columns:
* Question No: Alphanumeric, the question ID from the original questionnaire.
* Question: The text of the question
* Options: Any options or Likert scale such as "very often", "more than usual", etc
# Very quick guide to running the tool on your computer
1. Install [Docker](https://docs.docker.com/get-docker/).
2. Open a command line or Terminal window. Change folder to where you downloaded and unzipped the repository, and go to the folder `front_end`. Run the following command:
```
docker build -t harmony
docker run harmony
```
5. Open your browser at `https://localhost:80`. You will see the web app running.
# Deploying the tool to Azure using the Azure Command Line Interface via Azure Container Registry
In command line, if you have installed Azure CLI, log into both the Azure Portal and Azure Container Registry:
```
az login
az acr login --name regprotocolsfds
```
If the admin user is not yet enabled, you can use the command:
```
az acr update -n regprotocolsfds --admin-enabled true
```
Run this script:
```
./build_deploy.sh
```
## Developer's guide: Running the tool on your computer in Python and without using Docker
### Architecture
### Downloading PDF data
cd into data/raw_pdf and run `download_raw_pdfs.sh`.
### Installing requirements
Download and install Java if you don't have it already. Download and install Apache Tika and run it on your computer https://tika.apache.org/download.html
```
java -jar tika-server-standard-2.3.0.jar
```
(the version number of your Jar file name may differ.)
Install everything in `requirements.txt`:
```
pip install -r requirements.txt
```
### Running the front end app locally
Go into `front_end` and run
```
python application.py
```
You can then open your browser at `localhost:8050` and you will see the tool.
## Built With
- [Dash](https://dash.plot.ly/) - Main server and interactive components
- [Plotly Python](https://plot.ly/python/) - Used to create the interactive plots
- [Docker](https://docs.docker.com/) - Used for deployment to the web
- [Apache Tika](https://tika.apache.org/) - Used for parsing PDFs to text
- [spaCy](https://spacy.io/) - Used for NLP analysis
- [NLTK](https://www.nltk.org/) - Used for NLP analysis
- [Scikit-Learn](https://scikit-learn.org/) - Used for machine learning
## Licences of Third Party Software
- Apache Tika: [Apache 2.0 License](https://tika.apache.org/license.html)
- spaCy: [MIT License](https://github.com/explosion/spaCy/blob/master/LICENSE)
- NLTK: [Apache 2.0 License](https://github.com/nltk/nltk/blob/develop/LICENSE.txt)
- Scikit-Learn: [BSD 3-Clause](https://github.com/scikit-learn/scikit-learn/blob/main/COPYING)
## References
* Deploying a Dash webapp via Docker to Azure: https://medium.com/swlh/deploy-a-dash-application-in-azure-using-docker-ed46c4b9d2b2