{"id":22312184,"url":"https://github.com/ddofer/trends","last_synced_at":"2025-10-24T08:18:46.790Z","repository":{"id":160614239,"uuid":"635472008","full_name":"ddofer/Trends","owner":"ddofer","description":"Code \u0026 datasets for \"What’s next? Forecasting scientific research trends\"","archived":false,"fork":false,"pushed_at":"2023-10-09T13:57:41.000Z","size":32440,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T10:33:25.773Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ddofer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-02T19:11:17.000Z","updated_at":"2025-04-02T21:10:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"32a515e0-4d54-468e-a7b4-d55657579d73","html_url":"https://github.com/ddofer/Trends","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ddofer/Trends","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddofer%2FTrends","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddofer%2FTrends/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddofer%2FTrends/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddofer%2FTrends/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ddofer","download_url":"https://codeload.github.com/ddofer/Trends/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddofer%2FTrends/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262749223,"owners_count":23358358,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-03T21:35:25.454Z","updated_at":"2025-10-24T08:18:41.732Z","avatar_url":"https://github.com/ddofer.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Trends\nCode \u0026amp; datasets for the paper \"What’s next? Forecasting scientific research trends\"\n\nAbstract\nScientific research trends and interests evolve over time. The ability to identify and forecast these trends is vital. We predict future trends in scientific publications using heterogeneous public sources, including historical publications from PubMed, research and review articles, and patents. We demonstrate that scientific trends can be predicted five years in advance, with preceding publications and future patents serving as leading indicators for emerging scientific topics. We found that the ratio of reviews to original research articles is an informative feature for identifying increasing or declining topics, with declining topics having an excess of reviews. We find that language models provide improved insights and predictions into topic temporal dynamics. Our findings suggest that similar dynamics apply to molecular, technological, and conceptual topics across biomedical research.\n\nArxiv preprint:\nhttps://arxiv.org/abs/2305.04133\n\n```\n@misc{ofer2023whats,\n      title={Whats next? Forecasting scientific research trends}, \n      author={Dan Ofer and Michal Linial},\n      year={2023},\n      eprint={2305.04133},\n      archivePrefix={arXiv},\n      primaryClass={cs.DL}\n}\n```\n\nA website to access the predictions is in progress and will also be made available. \n\nReplicating the pipeline:\n* [OPTIONAL: For adding your own data]: Download datasets of terms results from PubmedByYear manually, using URLs constructed in `PrepData_Trends.ipynb` notebook (click links) - can also add additional topics here. This will output the training data `trends_v6.csv.gz` and long term historical context (`trends_context_v6.csv.gz`) (used for additional features) files.\n      * Training data and context is already provided in this repo.\n* Modelling results and evaluation: `plot trend pred + CV-V2023.ipynb`\n* Analysis of Patents leading future publications, with CRISPR as an use-case: `crispr_patent_paper_Corr.ipynb`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fddofer%2Ftrends","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fddofer%2Ftrends","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fddofer%2Ftrends/lists"}