{"id":20938159,"url":"https://github.com/gibsramen/evident_","last_synced_at":"2025-05-13T22:31:35.261Z","repository":{"id":41848462,"uuid":"454481917","full_name":"gibsramen/evident_","owner":"gibsramen","description":"Effect size calculations for microbiome data","archived":false,"fork":false,"pushed_at":"2022-04-26T17:19:06.000Z","size":1449,"stargazers_count":5,"open_issues_count":6,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-02T09:01:35.520Z","etag":null,"topics":["data-visualization","microbiome","python","qiime2","statistics","visualization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gibsramen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-02-01T17:23:51.000Z","updated_at":"2024-11-30T12:47:18.000Z","dependencies_parsed_at":"2022-08-11T19:20:28.431Z","dependency_job_id":null,"html_url":"https://github.com/gibsramen/evident_","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gibsramen%2Fevident_","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gibsramen%2Fevident_/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gibsramen%2Fevident_/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gibsramen%2Fevident_/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gibsramen","download_url":"https://codeload.github.com/gibsramen/evident_/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254036813,"owners_count":22003655,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-visualization","microbiome","python","qiime2","statistics","visualization"],"created_at":"2024-11-18T22:49:33.030Z","updated_at":"2025-05-13T22:31:34.106Z","avatar_url":"https://github.com/gibsramen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Main CI](https://github.com/gibsramen/evident/actions/workflows/main.yml/badge.svg)](https://github.com/gibsramen/evident/actions/workflows/main.yml)\n[![QIIME 2 CI](https://github.com/gibsramen/evident/actions/workflows/q2.yml/badge.svg)](https://github.com/gibsramen/evident/actions/workflows/q2.yml)\n[![PyPI](https://img.shields.io/pypi/v/evident.svg)](https://pypi.org/project/evident)\n\n# evident\n\nEvident is a tool for performing effect size and power calculations on microbiome diversity data.\n\n## Installation\n\nYou can install the most up-to-date version of evident from PyPi using the following command:\n\n```bash\npip install evident\n```\n\n## QIIME 2\n\nevident is also available as a [QIIME 2](https://qiime2.org/) plugin.\nMake sure you have activated a QIIME 2 environment and run the same installation command as above.\n\nTo check that evident installed correctly, run the following from the command line:\n\n```bash\nqiime evident --help\n```\n\nYou should see something like this if evident installed correctly:\n\n```bash\nUsage: qiime evident [OPTIONS] COMMAND [ARGS]...\n\n  Description: Perform power analysis on microbiome diversity data. Supports\n  calculation of effect size given metadata covariates and supporting\n  visualizations.\n\n  Plugin website: https://github.com/gibsramen/evident\n\n  Getting user support: Please post to the QIIME 2 forum for help with this\n  plugin: https://forum.qiime2.org\n\nOptions:\n  --version            Show the version and exit.\n  --example-data PATH  Write example data and exit.\n  --citations          Show citations and exit.\n  --help               Show this message and exit.\n\nCommands:\n  alpha-effect-size-by-category  Alpha diversity effect size by category.\n  alpha-power-analysis           Alpha diversity power analysis.\n  beta-effect-size-by-category   Beta diversity effect size by category.\n  beta-power-analysis            Beta diversity power analysis.\n  plot-power-curve               Plot power curve.\n  visualize-results              Tabulate evident results.\n```\n\n## Standalone Usage\n\nevident requires two input files:\n\n1. Either an alpha or beta diversity file\n2. Sample metadata\n\nFirst, open Python and import evident\n\n```python\nimport evident\n```\n\nNext, load your diversity file and sample metadata.\nFor alpha diversity, this should be a pandas Series.\nFor beta diversity, this should be an scikit-bio DistanceMatrix.\nSample metadata should be a pandas DataFrame.\nWe'll be using an alpha diversity vector for this tutorial but the commands are nearly the same for beta diversity distance matrices.\n\n```python\nimport pandas as pd\n\nmetadata = pd.read_table(\"data/metadata.tsv\", sep=\"\\t\", index_col=0)\nfaith_pd = metadata[\"faith_pd\"]\n```\n\nThe main data structure in evident is the 'DiversityHandler'.\nThis is the way that evident stores the diversity data and metadata for power calculations.\nFor our alpha diversity example, we'll load the `AlphaDiversityHandler` class from evident.\n`AlphaDiversityHandler` takes as input the pandas Series with the diversity values and the pandas DataFrame containing the sample metadata.\nBy default, evident will only consider metadata columns with, at max, 5 levels.\nTo modify this behavior, provide a value for the `max_levels_per_category` argument.\nAdditionally, evident will not consider any category levels represented by fewer than 3 samples.\nTo modify this behavior, use the `min_count_per_level` argument.\n\n```python\nadh = evident.AlphaDiversityHandler(faith_pd, metadata)\n```\n\nNext, let's say we want to get the effect size of the diversity differences between two groups of samples.\nWe have in our example a column in the metadata \"classification\" comparing two groups of patients with Crohn's disease.\nFirst, we'll look at the mean of Faith's PD between these two groups.\n\n```python\nmetadata.groupby(\"classification\").agg([\"count\", \"mean\", \"std\"])[\"faith_pd\"]\n```\n\nwhich results in\n\n```\n                count       mean       std\nclassification\nB1                 99  13.566110  3.455625\nNon-B1            121   9.758946  3.874911\n```\n\nLooks like there's a pretty large difference between these two groups.\nWhat we would like to do now is calculate the effect size of this difference.\nBecause we are comparing only two groups, we can use Cohen's d.\nevident automatically chooses the correct effect size to calculate - either Cohen's d if there are only two categories or Cohen's f if there are more than 2.\n\n```python\nadh.calculate_effect_size(column=\"classification\")\n```\n\nThis tells us that our effect size is 1.03.\n\nNow let's say we want to see how many samples we need to be able to detect this difference with a power of 0.8.\nevident allows you to easily specify arguments for alpha, power, or total observations for power analysis.\nWe can then plot these results as a power curve to summarize the data.\n\n```python\nfrom evident.plotting import plot_power_curve\nimport numpy as np\n\nalpha_vals = [0.01, 0.05, 0.1]\nobs_vals = np.arange(10, 101, step=10)\nresults = adh.power_analysis(\n    \"classification\",\n    alpha=alpha_vals,\n    total_observations=obs_vals\n)\nplot_power_curve(results, target_power=0.8, style=\"alpha\", markers=True)\n```\n\nWhen we inspect this plot, we can see how many samples we would need to collect to observe the same effect size at different levels of significance and power.\n\n![Power Curve](https://raw.githubusercontent.com/gibsramen/evident/main/imgs/power_curve.png)\n\n## Interactive power curve with Bokeh\n\nevident allows users to *interactively* perform effect size and power calculations using [Bokeh](https://docs.bokeh.org/en/latest/).\nTo create a Bokeh app, use the following command:\n\n```python\nfrom evident.interactive import create_bokeh_app\n\ncreate_bokeh_app(adh, \"app\")\n```\n\nThis will save the necessary files into a new directory `app/`.\nNavigate to the directory containing `app/` (**not** `app/` itself) and execute this command from your terminal:\n\n```bash\nbokeh serve --show app\n```\n\nThis should open up a browser window where you can modify the chosen column, significance, level, and observations.\nWe also provide a command line script to generate an interactive app using some test data.\nYou can access this script at `evident/tests/make_interactive.py`.\n\n![Bokeh App](https://raw.githubusercontent.com/gibsramen/evident/main/imgs/bokeh_screenshot.png)\n\nNote that because evident uses Python to perform the power calculations, it is at the moment not possible to embded this interactive app into a standalone webpage.\n\n## QIIME 2 Usage\n\nevident provides support for the popular QIIME 2 framework of microbiome data analysis.\nWe assume in this tutorial that you are familiar with using QIIME 2 on the command line.\nIf not, we recommend you read the excellent [documentation](https://docs.qiime2.org/) before you get started with evident.\nNote that we have only tested evident on QIIME 2 version 2021.11.\nIf you are using a different version and encounter an error please let us know via an issue.\n\nAs with the standalone version, evident requires a diversity file and a sample metadata file.\nThese inputs are expected to conform to QIIME 2 standards.\n\nTo calculate power, we can run the following command:\n\n```bash\nqiime evident alpha-power-analysis \\\n    --i-alpha-diversity faith_pd.qza \\\n    --m-sample-metadata-file metadata.qza \\\n    --m-sample-metadata-column classification \\\n    --p-alpha 0.01 0.05 0.1 \\\n    --p-total-observations $(seq 10 10 100) \\\n    --o-power-analysis-results results.qza\n```\n\nNotice how we used `$(seq 10 10 100)` to provide input into the `--p-total-observations` argument.\n`seq` is a command on UNIX-like systems that generates a sequence of numbers.\nIn our example, we used `seq` to generate the values from 10 to 100 in intervals of 10 (10, 20, ..., 100).\n\nWith this results artifact, we can visualize the power curve to get a sense of how power varies with number of observations and significance level.\nRun the following command:\n\n```bash\nqiime evident plot-power-curve \\\n    --i-power-analysis-results results.qza \\\n    --p-target-power 0.8 \\\n    --p-style alpha \\\n    --o-visualization power_curve.qzv\n```\n\nYou can view this visualization at [view.qiime2.org](https://view.qiime2.org/) directly in your browser.\n\n## Parallelization\n\nevident provides support for parallelizing effect size calculations through [joblib](https://joblib.readthedocs.io/en/latest/parallel.html).\nParallelization is performed across different columns when using `effect_size_by_category` and `pairwise_effect_size_by_category`.\nConsider parallelization if you have a lot of samples and/or a lot of different metadata categories of interest.\nBy default, no parallelization is performed.\n\nWith Python:\n\n```python\nfrom evident.effect_size import effect_size_by_category\n\neffect_size_by_category(\n    adh,\n    [\"classification\", \"cd_resection\", \"cd_behavior\"],\n    n_jobs=2\n)\n```\n\nWith QIIME 2:\n\n```bash\nqiime evident alpha-effect-size-by-category \\\n    --i-alpha-diversity faith_pd.qza \\\n    --m-sample-metadata-file metadata.qza \\\n    --p-columns classification sex cd_behavior \\\n    --p-n-jobs 2 \\\n    --o-effect-size-results alpha_effect_sizes.qza\n```\n\n## Help with evident\n\nIf you encounter a bug in evident, please post a GitHub issue and we will get to it as soon as we can.\nWe welcome any ideas or documentation updates/fixes so please submit an issue and/or a pull request if you have thoughts on making evident better.\n\nIf your question is regarding the QIIME 2 version of evident, consider posting to the [QIIME 2 forum](https://forum.qiime2.org/).\nYou can open an issue on the [Community Plugin Support](https://forum.qiime2.org/c/community-plugin-support/24) board and tag [@gibsramen](https://forum.qiime2.org/u/gibsramen) if required.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgibsramen%2Fevident_","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgibsramen%2Fevident_","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgibsramen%2Fevident_/lists"}