{"id":15647359,"url":"https://github.com/dermatologist/nlp-qrmine","last_synced_at":"2025-04-14T09:42:10.098Z","repository":{"id":35030706,"uuid":"139090908","full_name":"dermatologist/nlp-qrmine","owner":"dermatologist","description":"Qualitative Research support tools in Python","archived":false,"fork":false,"pushed_at":"2023-12-11T16:49:55.000Z","size":3360,"stargazers_count":45,"open_issues_count":15,"forks_count":14,"subscribers_count":7,"default_branch":"develop","last_synced_at":"2024-04-25T22:01:37.613Z","etag":null,"topics":["hacktoberfest","interview-data","machine-learning","nlp","nlp-machine-learning","python3","qualitative-data-analysis","qualitative-research","research-tool"],"latest_commit_sha":null,"homepage":"http://nuchange.ca/2017/09/grounded-theory-qualitative-research-python.html","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dermatologist.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-29T02:26:30.000Z","updated_at":"2024-04-19T12:23:17.000Z","dependencies_parsed_at":"2023-02-10T04:45:48.605Z","dependency_job_id":"b24a68fa-c9ad-41dc-a59f-caa3f51aed98","html_url":"https://github.com/dermatologist/nlp-qrmine","commit_stats":{"total_commits":293,"total_committers":6,"mean_commits":"48.833333333333336","dds":0.378839590443686,"last_synced_commit":"218c65f64f381f77a68bc143257b2244e0df9a0c"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fnlp-qrmine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fnlp-qrmine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fnlp-qrmine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fnlp-qrmine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dermatologist","download_url":"https://codeload.github.com/dermatologist/nlp-qrmine/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248855961,"owners_count":21172673,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","interview-data","machine-learning","nlp","nlp-machine-learning","python3","qualitative-data-analysis","qualitative-research","research-tool"],"created_at":"2024-10-03T12:18:53.820Z","updated_at":"2025-04-14T09:42:10.074Z","avatar_url":"https://github.com/dermatologist.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# :flashlight: QRMine\n*/ˈkärmīn/*\n\n[![forthebadge made-with-python](http://ForTheBadge.com/images/badges/made-with-python.svg)](https://www.python.org/)[![PyPI download total](https://img.shields.io/pypi/dm/qrmine.svg)](https://pypi.python.org/pypi/qrmine/)\n![Libraries.io SourceRank](https://img.shields.io/librariesio/sourcerank/pypi/qrmine)\n![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/dermatologist/nlp-qrmine)\n[![Documentation](https://badgen.net/badge/icon/documentation?icon=libraries\u0026label)](https://dermatologist.github.io/nlp-qrmine/)\n\nQRMine is a suite of qualitative research (QR) data mining tools in Python using Natural Language Processing (NLP) and Machine Learning (ML). QRMine is work in progress. [Read More..](https://nuchange.ca/2017/09/grounded-theory-qualitative-research-python.html)\n\n## What it does\n\n### NLP\n* Lists common categories for open coding.\n* Create a coding dictionary with categories, properties and dimensions.\n* Topic modelling.\n* Arrange docs according to topics.\n* Compare two documents/interviews.\n* Select documents/interviews by sentiment, category or title for further analysis.\n* Sentiment analysis\n\n\n### ML\n* Accuracy of a neural network model trained using the data\n* Confusion matrix from an support vector machine classifier\n* K nearest neighbours of a given record\n* K-Means clustering\n* Principal Component Analysis (PCA)\n* Association rules\n\n## How to install\n\n* Requires Python 3.11 and a CPU that support AVX instructions\n```text\npip install uv\nuv pip install qrmine\npython -m spacy download en_core_web_sm\n\n```\n\n### Mac users\n* Mac users, please install *libomp* for XGBoost\n```\nbrew install libomp\n```\n\n## How to Use\n\n* input files are transcripts as txt files and a single csv file with numeric data. The output txt file can be specified.\n\n* The coding dictionary, topics and topic assignments can be created from the  entire corpus (all documents) using the respective command line options.\n\n* Categories (concepts), summary and sentiment can be viewed for entire corpus or specific titles (documents) specified using the --titles switch. Sentence level sentiment output is possible with the --sentence flag.\n\n* You can filter documents based on sentiment, titles or categories and do further analysis, using --filters or -f\n\n* Many of the ML functions like neural network takes a second argument (-n) . In nnet -n signifies the number of epochs, number of clusters in kmeans, number of factors in pca, and number of neighbours in KNN. KNN also takes the --rec or -r argument to specify the record.\n\n* Variables from csv can be selected using --titles (defaults to all). The first variable will be ignored (index) and the last will be the DV (dependant variable).\n\n\n### Command-line options\n\n```text\nqrmine --help\n\n```\n\n| Command | Alternate | Description |\n| --- | --- | --- |\n| --inp | -i | Input file in the text format with \u003cbreak\u003e Topic \u003c/break\u003e |\n| --out | -o | Output file name |\n| --csv |   | csv file name |\n| --num | -n  | N (clusters/epochs etc depending on context) |\n| --rec | -r  | Record (based on context) |\n| --titles | -t | Document(s) title(s) to analyze/compare |\n| --codedict |   | Generate coding dictionary |\n| --topics |   | Generate topic model |\n| --assign |   | Assign documents to topics |\n| --cat |   | List categories of entire corpus or individual docs |\n| --summary |   | Generate summary for entire corpus or individual docs |\n| --sentiment |   | Generate sentiment score for entire corpus or individual docs |\n| --nlp |   | Generate all NLP reports |\n| --sentence |   | Generate sentence level scores when applicable |\n| --nnet |   | Display accuracy of a neural network model -n epochs(3)|\n| --svm |   | Display confusion matrix from an svm classifier |\n| --knn |   | Display nearest neighbours -n neighbours (3)|\n| --kmeans |   | Display KMeans clusters -n clusters (3)|\n| --cart |   | Display Association Rules |\n| --pca |   | Display PCA -n factors (3)|\n\n\n## Use it in your code\n```python\nfrom qrmine import Content\nfrom qrmine import Network\nfrom qrmine import Qrmine\nfrom qrmine import ReadData\nfrom qrmine import Sentiment\nfrom qrmine import MLQRMine\n\n```\n* *More instructions and a jupyter notebook available [here.](https://nuchange.ca/2017/09/grounded-theory-qualitative-research-python.html)*\n\n## Input file format\n\n### NLP\nIndividual documents or interview transcripts in a single text file separated by \u003cbreak\u003eTopic\u003c/break\u003e. Example below\n\n```\nTranscript of the first interview with John.\nAny number of lines\n\u003cbreak\u003eFirst_Interview_John\u003c/break\u003e\n\nText of the second interview with Jane.\nMore text.\n\u003cbreak\u003eSecond_Interview_Jane\u003c/break\u003e\n\n....\n```\n\nMultiple files are suported, each having only one break tag at the bottom with the topic.\n(The tag may be renamed in the future)\n\n### ML\n\nA single csv file with the following generic structure.\n\n* Column 1 with identifier. If it is related to a text document as above, include the title.\n* Last column has the dependent variable (DV). (NLP algorithms like the topic asignments may provide the DV)\n* All independent variables (numerical) in between.\n\n```\nindex, obesity, bmi, exercise, income, bp, fbs, has_diabetes\n1, 0, 29, 1, 12, 120, 89, 1\n2, 1, 32, 0, 9, 140, 92, 0\n......\n\n```\n\n## Author\n\n* [Bell Eapen](https://nuchange.ca) (McMaster U) |  [Contact](https://nuchange.ca/contact) | [![Twitter Follow](https://img.shields.io/twitter/follow/beapen?style=social)](https://twitter.com/beapen)\n\n* This software is developed and tested using [Compute Canada](http://www.computecanada.ca) resources.\n* See also:  [:fire: The FHIRForm framework for managing healthcare eForms](https://github.com/E-Health/fhirform)\n* See also: [:eyes: Drishti | An mHealth sense-plan-act framework!](https://github.com/E-Health/drishti)\n\n## Citation\n\nPlease cite QRMine in your publications if it helped your research. Here\nis an example BibTeX entry [(Read paper on arXiv)](https://arxiv.org/abs/2003.13519):\n\n```\n\n@article{eapenbr2019qrmine,\n  title={QRMine: A python package for triangulation in Grounded Theory},\n  author={Eapen, Bell Raj and Archer, Norm and Sartpi, Kamran},\n  journal={arXiv preprint arXiv:2003.13519 },\n  year={2020}\n}\n\n```\n\nQRMine is inspired by [this work](https://github.com/lknelson/computational-grounded-theory) and the associated [paper](https://journals.sagepub.com/doi/abs/10.1177/0049124117729703).\n\n## Give us a star ⭐️\nIf you find this project useful, give us a star. It helps others discover the project.\n\n## Demo\n\n[![QRMine](https://github.com/dermatologist/nlp-qrmine/blob/develop/notes/qrmine.gif)](https://github.com/dermatologist/nlp-qrmine/blob/develop/notes/qrmine.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdermatologist%2Fnlp-qrmine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdermatologist%2Fnlp-qrmine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdermatologist%2Fnlp-qrmine/lists"}