{"id":13487010,"url":"https://github.com/maximtrp/scikit-posthocs","last_synced_at":"2025-10-21T20:53:22.430Z","repository":{"id":41142701,"uuid":"95150484","full_name":"maximtrp/scikit-posthocs","owner":"maximtrp","description":"Multiple Pairwise Comparisons (Post Hoc) Tests in Python","archived":false,"fork":false,"pushed_at":"2025-09-11T11:20:36.000Z","size":5647,"stargazers_count":376,"open_issues_count":8,"forks_count":41,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-10-21T20:53:17.236Z","etag":null,"topics":["posthoc-comparisons","posthoc-tests","python","scikit","statistical-analysis","statistics","stats"],"latest_commit_sha":null,"homepage":"https://scikit-posthocs.rtfd.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maximtrp.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-06-22T19:41:37.000Z","updated_at":"2025-10-17T18:08:41.000Z","dependencies_parsed_at":"2023-02-18T00:16:03.092Z","dependency_job_id":"5e1555f2-74d6-411c-8a4a-38404ad790f0","html_url":"https://github.com/maximtrp/scikit-posthocs","commit_stats":{"total_commits":462,"total_committers":13,"mean_commits":35.53846153846154,"dds":"0.10606060606060608","last_synced_commit":"9210f9b4d24bb302e742ac29a139e5689c6b6508"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/maximtrp/scikit-posthocs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maximtrp%2Fscikit-posthocs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maximtrp%2Fscikit-posthocs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maximtrp%2Fscikit-posthocs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maximtrp%2Fscikit-posthocs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maximtrp","download_url":"https://codeload.github.com/maximtrp/scikit-posthocs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maximtrp%2Fscikit-posthocs/sbom","scorecard":{"id":629455,"data":{"date":"2025-08-11","repo":{"name":"github.com/maximtrp/scikit-posthocs","commit":"32c7af7961bb73194cb0a76c85e23f818aa2e35e"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4,"checks":[{"name":"Code-Review","score":1,"reason":"Found 4/26 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":1,"reason":"2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: topLevel 'contents' permission set to 'read': .github/workflows/package-publish.yml:8","Warn: no topLevel permission defined: .github/workflows/package-pull.yml:1","Warn: no topLevel permission defined: .github/workflows/package-test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/package-publish.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-publish.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/package-publish.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-publish.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/package-publish.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-publish.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/package-pull.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-pull.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/package-pull.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-pull.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/package-test.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-test.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/package-test.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-test.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/package-test.yml:31: update your workflow using https://app.stepsecurity.io/secureworkflow/maximtrp/scikit-posthocs/package-test.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/package-publish.yml:21","Warn: pipCommand not pinned by hash: .github/workflows/package-publish.yml:22","Warn: pipCommand not pinned by hash: .github/workflows/package-pull.yml:23","Warn: pipCommand not pinned by hash: .github/workflows/package-pull.yml:24","Warn: pipCommand not pinned by hash: .github/workflows/package-pull.yml:25","Warn: pipCommand not pinned by hash: .github/workflows/package-test.yml:23","Warn: pipCommand not pinned by hash: .github/workflows/package-test.yml:24","Warn: pipCommand not pinned by hash: .github/workflows/package-test.yml:25","Info:   0 out of   6 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 third-party GitHubAction dependencies pinned","Info:   0 out of   8 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/package-publish.yml:11"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 8 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-21T07:26:06.295Z","repository_id":41142701,"created_at":"2025-08-21T07:26:06.295Z","updated_at":"2025-08-21T07:26:06.295Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280333495,"owners_count":26312845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["posthoc-comparisons","posthoc-tests","python","scikit","statistical-analysis","statistics","stats"],"created_at":"2024-07-31T18:00:54.298Z","updated_at":"2025-10-21T20:53:22.412Z","avatar_url":"https://github.com/maximtrp.png","language":"Python","funding_links":[],"categories":["The Data Science Toolbox","Statistics","Statistical Analysis","其他_机器学习与深度学习","Python","Uncategorized","概率统计","🎲 Statistics \u0026 Probability","Statistics \u0026 Probability"],"sub_categories":["General Machine Learning Packages","NLP","Specialized Machine Learning Libraries","Uncategorized","Automated Machine Learning","Tools"],"readme":".. image:: images/logo.png\n\n===============\n\n.. image:: http://joss.theoj.org/papers/10.21105/joss.01169/status.svg\n   :target: https://doi.org/10.21105/joss.01169\n.. image:: https://img.shields.io/github/actions/workflow/status/maximtrp/scikit-posthocs/package-test.yml?label=build\n   :target: https://github.com/maximtrp/scikit-posthocs/actions/workflows/package-test.yml\n.. image:: https://img.shields.io/readthedocs/scikit-posthocs.svg\n   :target: https://scikit-posthocs.readthedocs.io\n.. image:: https://img.shields.io/codacy/coverage/50d2a82a6dd84b51b515cebf931067d7/master\n   :target: https://app.codacy.com/gh/maximtrp/scikit-posthocs/dashboard\n.. image:: https://img.shields.io/codacy/grade/50d2a82a6dd84b51b515cebf931067d7\n   :target: https://www.codacy.com/gh/maximtrp/scikit-posthocs/dashboard\n.. image:: https://static.pepy.tech/badge/scikit-posthocs\n   :target: https://pepy.tech/project/scikit-posthocs\n.. image:: https://img.shields.io/github/issues/maximtrp/scikit-posthocs.svg\n   :target: https://github.com/maximtrp/scikit-posthocs/issues\n.. image:: https://img.shields.io/pypi/v/scikit-posthocs.svg\n   :target: https://pypi.python.org/pypi/scikit-posthocs/\n.. image:: https://img.shields.io/conda/vn/conda-forge/scikit-posthocs.svg\n   :target: https://anaconda.org/conda-forge/scikit-posthocs\n\n===============\n\n**scikit-posthocs** is a Python package that provides post hoc tests for\npairwise multiple comparisons that are usually performed in statistical\ndata analysis to assess the differences between group levels if a statistically\nsignificant result of ANOVA test has been obtained.\n\n**scikit-posthocs** is tightly integrated with Pandas DataFrames and NumPy\narrays to ensure fast computations and convenient data import and storage.\n\nThis package will be useful for statisticians, data analysts, and\nresearchers who use Python in their work.\n\n\nBackground\n----------\n\nPython statistical ecosystem comprises multiple packages. However, it\nstill has numerous gaps and is surpassed by R packages and capabilities.\n\n`SciPy \u003chttps://www.scipy.org/\u003e`_ (version 1.2.0) offers *Student*, *Wilcoxon*,\nand *Mann-Whitney* tests that are not adapted to multiple pairwise\ncomparisons. `Statsmodels \u003chttp://statsmodels.sourceforge.net/\u003e`_ (version 0.9.0)\nfeatures *TukeyHSD* test that needs some extra actions to be fluently\nintegrated into a data analysis pipeline.\n`Statsmodels \u003chttp://statsmodels.sourceforge.net/\u003e`_ also has good helper\nmethods: ``allpairtest`` (adapts an external function such as\n``scipy.stats.ttest_ind`` to multiple pairwise comparisons) and\n``multipletests`` (adjusts *p* values to minimize type I and II errors).\n`PMCMRplus \u003chttps://rdrr.io/cran/PMCMRplus/\u003e`_ is a very good R package that\nhas no rivals in Python as it offers more than 40 various tests (including\npost hoc tests) for factorial and block design data. PMCMRplus was an\ninspiration and a reference for *scikit-posthocs*.\n\n**scikit-posthocs** attempts to improve Python statistical capabilities by\noffering a lot of parametric and nonparametric post hoc tests along with\noutliers detection and basic plotting methods.\n\n\nFeatures\n--------\n\n.. image:: images/flowchart.png\n  :alt: Tests Flowchart\n\n- *Omnibus* tests:\n\n  - Durbin test (for balanced incomplete block design).\n  - Mack-Wolfe test.\n  - Hayter (OSRT) test.\n\n- *Parametric* pairwise multiple comparisons tests:\n\n  - Scheffe test.\n  - Student T test.\n  - Tamhane T2 test.\n  - TukeyHSD test.\n\n- *Non-parametric* tests for factorial design:\n\n  - Conover test.\n  - Dunn test.\n  - Dwass, Steel, Critchlow, and Fligner test.\n  - Mann-Whitney test.\n  - Nashimoto and Wright (NPM) test.\n  - Nemenyi test.\n  - van Waerden test.\n  - Wilcoxon test.\n\n- *Non-parametric* tests for block design:\n\n  - Conover test.\n  - Durbin and Conover test.\n  - Miller test.\n  - Nemenyi test.\n  - Quade test.\n  - Siegel test.\n\n- Outliers detection tests:\n\n  - Simple test based on interquartile range (IQR).\n  - Grubbs test.\n  - Tietjen-Moore test.\n  - Generalized Extreme Studentized Deviate test (ESD test).\n\n- Other tests:\n\n  - Anderson-Darling test.\n\n- Global null hypothesis tests:\n  \n  - Fisher's combination test.\n  - Simes test.\n\n- Plotting functionality (e.g. significance plots).\n\nAll post hoc tests are capable of p adjustments for multiple\npairwise comparisons.\n\nDependencies\n------------\n\n- `NumPy and SciPy packages \u003chttps://www.scipy.org/\u003e`_\n- `Statsmodels \u003chttp://statsmodels.sourceforge.net/\u003e`_\n- `Pandas \u003chttp://pandas.pydata.org/\u003e`_\n- `Matplotlib \u003chttps://matplotlib.org/\u003e`_\n- `Seaborn \u003chttps://seaborn.pydata.org/\u003e`_\n\nCompatibility\n-------------\n\nPackage is only compatible with Python 3.\n\nInstall\n-------\n\nYou can install the package using ``pip`` (from PyPi):\n\n.. code:: bash\n\n  pip install scikit-posthocs\n\nOr using ``conda`` (from conda-forge repo):\n\n.. code:: bash\n\n  conda install -c conda-forge scikit-posthocs\n\nThe latest version from GitHub can be installed using:\n\n.. code:: bash\n\n  pip install git+https://github.com/maximtrp/scikit-posthocs.git\n\nExamples\n--------\n\nParametric ANOVA with post hoc tests\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nHere is a simple example of the one-way analysis of variance (ANOVA)\nwith post hoc tests used to compare *sepal width* means of three\ngroups (three iris species) in *iris* dataset.\n\nTo begin, we will import the dataset using statsmodels\n``get_rdataset()`` method.\n\n.. code:: python\n\n  \u003e\u003e\u003e import statsmodels.api as sa\n  \u003e\u003e\u003e import statsmodels.formula.api as sfa\n  \u003e\u003e\u003e import scikit_posthocs as sp\n  \u003e\u003e\u003e df = sa.datasets.get_rdataset('iris').data\n  \u003e\u003e\u003e df.columns = df.columns.str.replace('.', '')\n  \u003e\u003e\u003e df.head()\n      SepalLength   SepalWidth   PetalLength   PetalWidth Species\n  0           5.1          3.5           1.4          0.2  setosa\n  1           4.9          3.0           1.4          0.2  setosa\n  2           4.7          3.2           1.3          0.2  setosa\n  3           4.6          3.1           1.5          0.2  setosa\n  4           5.0          3.6           1.4          0.2  setosa\n\nNow, we will build a model and run ANOVA using statsmodels ``ols()``\nand ``anova_lm()`` methods. Columns ``Species`` and ``SepalWidth``\ncontain independent (predictor) and dependent (response) variable\nvalues, correspondingly.\n\n.. code:: python\n\n  \u003e\u003e\u003e lm = sfa.ols('SepalWidth ~ C(Species)', data=df).fit()\n  \u003e\u003e\u003e anova = sa.stats.anova_lm(lm)\n  \u003e\u003e\u003e print(anova)\n                 df     sum_sq   mean_sq         F        PR(\u003eF)\n  C(Species)    2.0  11.344933  5.672467  49.16004  4.492017e-17\n  Residual    147.0  16.962000  0.115388       NaN           NaN\n\nThe results tell us that there is a significant difference between\ngroups means (p = 4.49e-17), but does not tell us the exact group pairs which\nare different in means. To obtain pairwise group differences, we will carry\nout a posteriori (post hoc) analysis using ``scikits-posthocs`` package.\nStudent T test applied pairwisely gives us the following p values:\n\n.. code:: python\n\n  \u003e\u003e\u003e sp.posthoc_ttest(df, val_col='SepalWidth', group_col='Species', p_adjust='holm')\n                    setosa    versicolor     virginica\n  setosa     -1.000000e+00  5.535780e-15  8.492711e-09\n  versicolor  5.535780e-15 -1.000000e+00  1.819100e-03\n  virginica   8.492711e-09  1.819100e-03 -1.000000e+00\n\nRemember to use a `FWER controlling procedure \u003chttps://en.wikipedia.org/wiki/Family-wise_error_rate#Controlling_procedures\u003e`_,\nsuch as Holm procedure, when making multiple comparisons. As seen from this\ntable, significant differences in group means are obtained for all group pairs.\n\nNon-parametric ANOVA with post hoc tests\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nIf normality and other `assumptions \u003chttps://en.wikipedia.org/wiki/One-way_analysis_of_variance\u003e`_\nare violated, one can use a non-parametric Kruskal-Wallis H test (one-way\nnon-parametric ANOVA) to test if samples came from the same distribution.\n\nLet's use the same dataset just to demonstrate the procedure. Kruskal-Wallis\ntest is implemented in SciPy package. ``scipy.stats.kruskal`` method\naccepts array-like structures, but not DataFrames.\n\n.. code:: python\n\n  \u003e\u003e\u003e import scipy.stats as ss\n  \u003e\u003e\u003e import statsmodels.api as sa\n  \u003e\u003e\u003e import scikit_posthocs as sp\n  \u003e\u003e\u003e df = sa.datasets.get_rdataset('iris').data\n  \u003e\u003e\u003e df.columns = df.columns.str.replace('.', '')\n  \u003e\u003e\u003e data = [df.loc[ids, 'SepalWidth'].values for ids in df.groupby('Species').groups.values()]\n\n``data`` is a list of 1D arrays containing *sepal width* values, one array per\neach species. Now we can run Kruskal-Wallis analysis of variance.\n\n.. code:: python\n\n  \u003e\u003e\u003e H, p = ss.kruskal(*data)\n  \u003e\u003e\u003e p\n  1.5692820940316782e-14\n\nP value tells us we may reject the null hypothesis that the population medians\nof all of the groups are equal. To learn what groups (species) differ in their\nmedians we need to run post hoc tests. ``scikit-posthocs`` provides a lot of\nnon-parametric tests mentioned above. Let's choose Conover's test.\n\n.. code:: python\n\n  \u003e\u003e\u003e sp.posthoc_conover(df, val_col='SepalWidth', group_col='Species', p_adjust = 'holm')\n                    setosa    versicolor     virginica\n  setosa     -1.000000e+00  2.278515e-18  1.293888e-10\n  versicolor  2.278515e-18 -1.000000e+00  1.881294e-03\n  virginica   1.293888e-10  1.881294e-03 -1.000000e+00\n\nPairwise comparisons show that we may reject the null hypothesis (p \u003c 0.01) for\neach pair of species and conclude that all groups (species) differ in their\nsepal widths.\n\nBlock design\n~~~~~~~~~~~~\n\nIn block design case, we have a primary factor (e.g. treatment) and a blocking\nfactor (e.g. age or gender). A blocking factor is also called a *nuisance*\nfactor, and it is usually a source of variability that needs to be accounted\nfor.\n\nAn example scenario is testing the effect of four fertilizers on crop yield in\nfour cornfields. We can represent the results with a matrix in which rows\ncorrespond to the blocking factor (field) and columns correspond to the\nprimary factor (yield).\n\nThe following dataset is artificial and created just for demonstration\nof the procedure:\n\n.. code:: python\n\n  \u003e\u003e\u003e data = np.array([[ 8.82, 11.8 , 10.37, 12.08],\n                       [ 8.92,  9.58, 10.59, 11.89],\n                       [ 8.27, 11.46, 10.24, 11.6 ],\n                       [ 8.83, 13.25,  8.33, 11.51]])\n\nFirst, we need to perform an omnibus test — Friedman rank sum test. It is\nimplemented in ``scipy.stats`` subpackage:\n\n.. code:: python\n\n  \u003e\u003e\u003e import scipy.stats as ss\n  \u003e\u003e\u003e ss.friedmanchisquare(*data.T)\n  FriedmanchisquareResult(statistic=8.700000000000003, pvalue=0.03355726870553798)\n\nWe can reject the null hypothesis that our treatments have the same\ndistribution, because p value is less than 0.05. A number of post hoc tests are\navailable in ``scikit-posthocs`` package for unreplicated block design data.\nIn the following example, Nemenyi's test is used:\n\n.. code:: python\n\n  \u003e\u003e\u003e import scikit_posthocs as sp\n  \u003e\u003e\u003e sp.posthoc_nemenyi_friedman(data)\n            0         1         2         3\n  0 -1.000000  0.220908  0.823993  0.031375\n  1  0.220908 -1.000000  0.670273  0.823993\n  2  0.823993  0.670273 -1.000000  0.220908\n  3  0.031375  0.823993  0.220908 -1.000000\n\nThis function returns a DataFrame with p values obtained in pairwise\ncomparisons between all treatments.\nOne can also pass a DataFrame and specify the names of columns containing\ndependent variable values, blocking and primary factor values.\nThe following code creates a DataFrame with the same data:\n\n.. code:: python\n\n  \u003e\u003e\u003e data = pd.DataFrame.from_dict({'blocks': {0: 0, 1: 1, 2: 2, 3: 3, 4: 0, 5: 1, 6:\n  2, 7: 3, 8: 0, 9: 1, 10: 2, 11: 3, 12: 0, 13: 1, 14: 2, 15: 3}, 'groups': {0:\n  0, 1: 0, 2: 0, 3: 0, 4: 1, 5: 1, 6: 1, 7: 1, 8: 2, 9: 2, 10: 2, 11: 2, 12: 3,\n  13: 3, 14: 3, 15: 3}, 'y': {0: 8.82, 1: 8.92, 2: 8.27, 3: 8.83, 4: 11.8, 5:\n  9.58, 6: 11.46, 7: 13.25, 8: 10.37, 9: 10.59, 10: 10.24, 11: 8.33, 12: 12.08,\n  13: 11.89, 14: 11.6, 15: 11.51}})\n  \u003e\u003e\u003e data\n      blocks  groups      y\n  0        0       0   8.82\n  1        1       0   8.92\n  2        2       0   8.27\n  3        3       0   8.83\n  4        0       1  11.80\n  5        1       1   9.58\n  6        2       1  11.46\n  7        3       1  13.25\n  8        0       2  10.37\n  9        1       2  10.59\n  10       2       2  10.24\n  11       3       2   8.33\n  12       0       3  12.08\n  13       1       3  11.89\n  14       2       3  11.60\n  15       3       3  11.51\n\nThis is a *melted* and ready-to-use DataFrame. Do not forget to pass ``melted``\nargument:\n\n.. code:: python\n\n  \u003e\u003e\u003e sp.posthoc_nemenyi_friedman(data, y_col='y', block_col='blocks', group_col='groups', melted=True)\n            0         1         2         3\n  0 -1.000000  0.220908  0.823993  0.031375\n  1  0.220908 -1.000000  0.670273  0.823993\n  2  0.823993  0.670273 -1.000000  0.220908\n  3  0.031375  0.823993  0.220908 -1.000000\n\n\nData types\n~~~~~~~~~~\n\nInternally, ``scikit-posthocs`` uses NumPy ndarrays and pandas DataFrames to\nstore and process data. Python lists, NumPy ndarrays, and pandas DataFrames\nare supported as *input* data types. Below are usage examples of various\ninput data structures.\n\nLists and arrays\n^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n  \u003e\u003e\u003e x = [[1,2,1,3,1,4], [12,3,11,9,3,8,1], [10,22,12,9,8,3]]\n  \u003e\u003e\u003e # or\n  \u003e\u003e\u003e x = np.array([[1,2,1,3,1,4], [12,3,11,9,3,8,1], [10,22,12,9,8,3]])\n  \u003e\u003e\u003e sp.posthoc_conover(x, p_adjust='holm')\n            1         2         3\n  1 -1.000000  0.057606  0.007888\n  2  0.057606 -1.000000  0.215761\n  3  0.007888  0.215761 -1.000000\n\nYou can check how it is processed with a hidden function ``__convert_to_df()``:\n\n.. code:: python\n\n  \u003e\u003e\u003e sp.__convert_to_df(x)\n  (    vals  groups\n   0      1       1\n   1      2       1\n   2      1       1\n   3      3       1\n   4      1       1\n   5      4       1\n   6     12       2\n   7      3       2\n   8     11       2\n   9      9       2\n   10     3       2\n   11     8       2\n   12     1       2\n   13    10       3\n   14    22       3\n   15    12       3\n   16     9       3\n   17     8       3\n   18     3       3, 'vals', 'groups')\n\nIt returns a tuple of a DataFrame representation and names of the columns\ncontaining dependent (``vals``) and independent (``groups``) variable values.\n\n*Block design* matrix passed as a NumPy ndarray is processed with a hidden\n``__convert_to_block_df()`` function:\n\n.. code:: python\n\n  \u003e\u003e\u003e data = np.array([[ 8.82, 11.8 , 10.37, 12.08],\n                       [ 8.92,  9.58, 10.59, 11.89],\n                       [ 8.27, 11.46, 10.24, 11.6 ],\n                       [ 8.83, 13.25,  8.33, 11.51]])\n  \u003e\u003e\u003e sp.__convert_to_block_df(data)\n  (    blocks groups      y\n   0        0      0   8.82\n   1        1      0   8.92\n   2        2      0   8.27\n   3        3      0   8.83\n   4        0      1  11.80\n   5        1      1   9.58\n   6        2      1  11.46\n   7        3      1  13.25\n   8        0      2  10.37\n   9        1      2  10.59\n   10       2      2  10.24\n   11       3      2   8.33\n   12       0      3  12.08\n   13       1      3  11.89\n   14       2      3  11.60\n   15       3      3  11.51, 'y', 'groups', 'blocks')\n\nDataFrames\n^^^^^^^^^^\n\nIf you are using DataFrames, you need to pass column names containing variable\nvalues to a post hoc function:\n\n.. code:: python\n\n  \u003e\u003e\u003e import statsmodels.api as sa\n  \u003e\u003e\u003e import scikit_posthocs as sp\n  \u003e\u003e\u003e df = sa.datasets.get_rdataset('iris').data\n  \u003e\u003e\u003e df.columns = df.columns.str.replace('.', '')\n  \u003e\u003e\u003e sp.posthoc_conover(df, val_col='SepalWidth', group_col='Species', p_adjust = 'holm')\n\n``val_col`` and ``group_col`` arguments specify the names of the columns\ncontaining dependent (response) and independent (grouping) variable values.\n\n\nSignificance plots\n------------------\n\nP values can be plotted using a heatmap:\n\n.. code:: python\n\n  \u003e\u003e\u003e pc = sp.posthoc_conover(x, val_col='values', group_col='groups')\n  \u003e\u003e\u003e heatmap_args = {'linewidths': 0.25, 'linecolor': '0.5', 'clip_on': False, 'square': True, 'cbar_ax_bbox': [0.80, 0.35, 0.04, 0.3]}\n  \u003e\u003e\u003e sp.sign_plot(pc, **heatmap_args)\n\n.. image:: images/plot-conover.png\n\nCustom colormap applied to a plot:\n\n.. code:: python\n\n  \u003e\u003e\u003e pc = sp.posthoc_conover(x, val_col='values', group_col='groups')\n  \u003e\u003e\u003e # Format: diagonal, non-significant, p\u003c0.001, p\u003c0.01, p\u003c0.05\n  \u003e\u003e\u003e cmap = ['1', '#fb6a4a',  '#08306b',  '#4292c6', '#c6dbef']\n  \u003e\u003e\u003e heatmap_args = {'cmap': cmap, 'linewidths': 0.25, 'linecolor': '0.5', 'clip_on': False, 'square': True, 'cbar_ax_bbox': [0.80, 0.35, 0.04, 0.3]}\n  \u003e\u003e\u003e sp.sign_plot(pc, **heatmap_args)\n\n.. image:: images/plot-conover-custom-cmap.png\n\nCiting\n------\n\nIf you want to cite *scikit-posthocs*, please refer to the publication in\nthe `Journal of Open Source Software \u003chttp://joss.theoj.org\u003e`_:\n\nTerpilowski, M. (2019). scikit-posthocs: Pairwise multiple comparison tests in\nPython. Journal of Open Source Software, 4(36), 1169, https://doi.org/10.21105/joss.01169\n\n.. code::\n\n  @ARTICLE{Terpilowski2019,\n    title    = {scikit-posthocs: Pairwise multiple comparison tests in Python},\n    author   = {Terpilowski, Maksim},\n    journal  = {The Journal of Open Source Software},\n    volume   = {4},\n    number   = {36},\n    pages    = {1169},\n    year     = {2019},\n    doi      = {10.21105/joss.01169}\n  }\n\nAcknowledgement\n---------------\n\nThorsten Pohlert, PMCMR author and maintainer\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaximtrp%2Fscikit-posthocs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaximtrp%2Fscikit-posthocs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaximtrp%2Fscikit-posthocs/lists"}