{"id":43142195,"url":"https://github.com/psaegert/pmtrendviz","last_synced_at":"2026-01-31T22:49:25.402Z","repository":{"id":186330246,"uuid":"612135474","full_name":"psaegert/pmtrendviz","owner":"psaegert","description":"Unsupervised Discovery Of Trends In Biomedical Research Based On The PubMed Baseline Repository","archived":false,"fork":false,"pushed_at":"2023-03-10T10:06:23.000Z","size":512,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-01-26T08:46:01.349Z","etag":null,"topics":["biomedical-text-mining","clustering","data-science","document-representation","pubmed","pubmed-abstracts","text-analysis","trends"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/psaegert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-03-10T09:21:29.000Z","updated_at":"2024-01-06T14:07:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"91dba119-7a46-4190-bc31-0753924211af","html_url":"https://github.com/psaegert/pmtrendviz","commit_stats":null,"previous_names":["psaegert/pmtrendviz"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/psaegert/pmtrendviz","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psaegert%2Fpmtrendviz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psaegert%2Fpmtrendviz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psaegert%2Fpmtrendviz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psaegert%2Fpmtrendviz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/psaegert","download_url":"https://codeload.github.com/psaegert/pmtrendviz/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psaegert%2Fpmtrendviz/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28958455,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T22:20:19.638Z","status":"ssl_error","status_checked_at":"2026-01-31T22:18:07.061Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["biomedical-text-mining","clustering","data-science","document-representation","pubmed","pubmed-abstracts","text-analysis","trends"],"created_at":"2026-01-31T22:49:23.058Z","updated_at":"2026-01-31T22:49:25.389Z","avatar_url":"https://github.com/psaegert.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\" style=\"margin-top: 0px;\"\u003eUnsupervised Discovery Of Trends In Biomedical Research Based On The PubMed Baseline Repository\u003c/h1\u003e\n\u003ch2 align=\"center\" style=\"margin-top: 0px;\"\u003eData Science for Text Analytics: Student Project (Public Version)\u003c/h2\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![pytest](https://github.com/psaegert/pmtrendviz/actions/workflows/pytest.yml/badge.svg)](https://github.com/psaegert/pmtrendviz/actions/workflows/pytest.yml)\n[![quality checks](https://github.com/psaegert/pmtrendviz/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/psaegert/pmtrendviz/actions/workflows/pre-commit.yml)\n\n\u003c/div\u003e\n\n![Visual Abstract](images/visual_abstract.png)\n\n# Table of Contents\n1. [Introduction](#introduction)\n1. [Requirements](#requirements)\n1. [Getting Started](#getting-started)\n1. [Usage](#usage)\n2. [Development](#development)\n\n\n# Introduction\n[PubMed](https://pubmed.ncbi.nlm.nih.gov/) is an online database of biomedical literature from MEDLINE, life science journals, and online books. It contains over 35 million citations, covering various areas of research related to biomedicine and health since 1965.\n\nOur work aims to offer easy to access insights into hot research areas, establish structure and organize the vast amount of information available in PubMed.\nWe developed `pmtrendviz`, a python based text-analytics tool that makes use of document embedding and clustering methods to identify research areas without supervision and derive trends on a per-cluster basis for a number of clusters most similar to a given query.\n\n# Requirements\n## Software\n- Python 3.10\n- Docker 20.10.20\n- Node 19.4.0 (see the [instructions](docs/install_node.md))\n- [Git LFS](https://git-lfs.com/) (optional, for installing pre-trained `pmtrendviz` models)\n  - On WSL2, you may need to install `git-lfs` manually, see [this thread](https://stackoverflow.com/questions/68867420/how-to-install-git-lfs-on-wsl2)\n\n## Hardware\n### Minimum\n- 8GB RAM\n- 10GB free disk space\n\n### Recommended\n- 32GB RAM\n- 70GB free disk space\n- GPU (optional)\n\n# Getting Started\n\n## Clone The Repository\n\n```bash\ngit clone https://github.com/psaegert/pmtrendviz.git\ncd pmtrendviz\n```\n\n## Create A Virtual Environment (optional):\n\n### With conda\n\n```bash\nconda create -n pmtrendviz python=3.10\nconda activate pmtrendviz\n```\n\n### With venv and pyenv\n\n```bash\npyenv install 3.10\npyenv local 3.10\npython -m venv .venv\n```\n\n## Install\n\nOption 1 (recommended):\u003cbr\u003e\nInstall the entire package with pip:\n```bash\npip install -e .\n```\n\nOption 2:\u003cbr\u003e\nIf you do not wish to install the package and run the `main.py` script directly, use the following command to install the dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n## Set up Elasticsearch\n```bash\ndocker compose up -d es01 [elasticvue]\n```\n\nNote: The `es01` service is required for all steps of the pipeline.\n\n# Usage\nThe `pmtrendviz` pipeline consists of four distinct steps: Data collection, training, prediction, and visualization, which can be run in the following ways:\n## Crunch the data\nOption 1: CLI (recommended)\u003cbr\u003e\nCheck out the [CLI Documentation](docs/cli.md) or the [minimal CLI example](docs/minimal_cli_example.md)\n\nOption 2: Use `pmtrendviz` in your own python code\u003cbr\u003e\nCheck out the [minimal python example](docs/minimal_python_example.md)\n\n## Visualize the results\nTo start the visualization, run the `start_backend.sh` and `start_frontend.sh` scripts in two separate terminals.\nAfterwards, open `http://localhost:5173/` in your browser, and start typing in the search bar (be patient, it may take a while for the models to load into memory).\n\n# Development\n\n## Setup\nTo set up the development environment, run the following command:\n```bash\npip install -r requirements_dev.txt\n```\n\n## Tools\nWe use\n- [flake8](https://pypi.org/project/flake8/) to enforce linting\n- [mypy](https://pypi.org/project/mypy/) to enforce static typing\n- [isort](https://pypi.org/project/isort/) to enforce import sorting\n- [pytest](https://pypi.org/project/pytest/) to run tests against our code (see `tests/`)\n\n## Pre-Commit Hooks\nTo set up linting, static typing, whitespace trailing, ordering of `requirements.txt` and imports when committing, run the following command:\n```bash\npre-commit install\n```\n\nTo run the pre-commit hooks manually, run the following command:\n```bash\npre-commit run --all-files\n```\n\nTests can be run with the following command:\n```bash\npytest\n```\n\n# Citation\n\nIf you use this code for your own research, please cite our work:\n\n```bibtex\n@misc{pmtrendviz,\n  author = {Paul Saegert and Philipp Steichen},\n  title = {Unsupervised Discovery Of Trends In Biomedical Research Based On The PubMed Baseline Repository},\n  year = {2023},\n  publisher = {GitHub},\n  journal = {GitHub Repository},\n  howpublished = {\\url{https://github.com/psaegert/pmtrendviz}},\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsaegert%2Fpmtrendviz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpsaegert%2Fpmtrendviz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsaegert%2Fpmtrendviz/lists"}