{"id":44600805,"url":"https://github.com/embodied-computation-group/metadpy","last_synced_at":"2026-02-14T09:39:43.965Z","repository":{"id":41146136,"uuid":"267078371","full_name":"embodied-computation-group/metadpy","owner":"embodied-computation-group","description":"Metacognitive efficiency modelling in Python.","archived":false,"fork":false,"pushed_at":"2023-11-08T17:31:25.000Z","size":67839,"stargazers_count":16,"open_issues_count":3,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-18T12:47:30.332Z","etag":null,"topics":["bayesian","behavior","cognitive-modeling","metacognition","signal-detection-theory"],"latest_commit_sha":null,"homepage":"https://embodied-computation-group.github.io/metadpy/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/embodied-computation-group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-26T15:17:50.000Z","updated_at":"2025-02-17T21:51:31.000Z","dependencies_parsed_at":"2023-01-21T05:16:35.167Z","dependency_job_id":"d685b50e-53aa-44d8-88f1-10fc308c2176","html_url":"https://github.com/embodied-computation-group/metadpy","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/embodied-computation-group/metadpy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embodied-computation-group%2Fmetadpy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embodied-computation-group%2Fmetadpy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embodied-computation-group%2Fmetadpy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embodied-computation-group%2Fmetadpy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/embodied-computation-group","download_url":"https://codeload.github.com/embodied-computation-group/metadpy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embodied-computation-group%2Fmetadpy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29442128,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T07:24:13.446Z","status":"ssl_error","status_checked_at":"2026-02-14T07:23:58.969Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian","behavior","cognitive-modeling","metacognition","signal-detection-theory"],"created_at":"2026-02-14T09:39:43.377Z","updated_at":"2026-02-14T09:39:43.958Z","avatar_url":"https://github.com/embodied-computation-group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://github.com/pre-commit/pre-commit) [![license](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](https://github.com/embodied-computation-group/metadpy/blob/master/LICENSE) [![codecov](https://codecov.io/gh/embodied-computation-group/metadpy/branch/master/graph/badge.svg)](https://codecov.io/gh/embodied-computation-group/metadpy) [![black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/) [![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat\u0026labelColor=ef8336)](https://pycqa.github.io/isort/) [![pip](https://badge.fury.io/py/metadpy.svg)](https://badge.fury.io/py/metadpy)\n\n***\n\n\u003cimg src=\"https://github.com/embodied-computation-group/metadpy/raw/master/docs/source/images/logo.png\" align=\"left\" alt=\"metadpy\" height=\"250\" HSPACE=30\u003e\n\n**metadpy** is a Python implementation of standard Bayesian models of behavioural metacognition. It is aimed to provide simple yet powerful functions to compute standard indexes and metrics of signal detection theory (SDT) and metacognitive efficiency (meta-d’ and hierarchical meta-d’). The only input required is a data frame encoding task performances and confidence ratings at the trial level.\n\n**metadpy** is written in Python 3. It uses [Numpy](https://numpy.org/), [Scipy](https://www.scipy.org/) and [Pandas](https://pandas.pydata.org/\u003e) for most of its operation, comprizing meta-d’ estimation using maximum likelihood estimation (MLE). The (Hierarchical) Bayesian modelling is implemented in [Aesara](https://github.com/aesara-devs/aesara) (now renamed [PyTensor](https://github.com/pymc-devs/pytensor) for versions of [pymc](https://docs.pymc.io/\u003e) \u003e=5.0).\n\n* 📖 [Documentation](https://embodied-computation-group.github.io/metadpy/)  \n* ✏️ [Tutorials](https://embodied-computation-group.github.io/metadpy/tutorials.html)  \n\n# Installation\n\nThe package can be installed using pip:\n\n```shell\npip install metadpy\n```\n\nFor most of the operations, the following packages are required:\n\n* [Numpy](https://numpy.org/) (\u003e=1.15)\n* [Scipy](https://www.scipy.org/) (\u003e=1.3.0)\n* [Pandas](https://pandas.pydata.org/\u003e) (\u003e=0.24)\n* [Matplotlib](https://matplotlib.org/) (\u003e=3.0.2)\n* [Seaborn](https://seaborn.pydata.org/) (\u003e=0.9.0)\n\nBayesian models will require:\n\n* [PyTensor](https://github.com/pymc-devs/pytensor)\n* [pymc](https://docs.pymc.io/\u003e) (\u003e=5.0)\n\n\n# Why metadpy?\n\nmetadpy stands for meta-d' (meta-d prime) in Python. meta-d' is a behavioural metric commonly used in consciousness and metacognition research. It is modelled to reflect metacognitive efficiency (i.e the relationship between subjective reports about performances and objective behaviour).\n\nmetadpy first aims to be the Python equivalent of the [hMeta-d toolbox](https://github.com/metacoglab/HMeta-d) (Matlab and R). It tries to make these models available to a broader open-source ecosystem and to ease their use via cloud computing interfaces. One notable difference is that While the [hMeta-d toolbox](https://github.com/metacoglab/HMeta-d) relies on JAGS for the Bayesian modelling of confidence data (see [**4**]) to analyse task performance and confidence ratings, metadpy is built on the top of [pymc](https://docs.pymc.io/\u003e), and uses Hamiltonina Monte Carlo methods (NUTS).\n\nFor an extensive introduction to metadpy, you can navigate the following notebooks that are Python adaptations of the introduction to the [hMeta-d toolbox](https://github.com/metacoglab/HMeta-d) written in Matlab by Olivia Faul for the [Zurich Computational Psychiatry course](https://github.com/metacoglab/HMeta-d/tree/master/CPC_metacog_tutorial).\n\n# Tutorials\n\n| Notebook | Colab |\n| --- | ---|\n| What metacognition looks like? | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embodied-computation-group/metadpy/blob/master/docs/source/examples/1-What%20metacognition%20looks%20like.ipynb)\n| Fitting the model (MLE) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embodied-computation-group/metadpy/blob/master/docs/source/examples/2-Fitting%20the%20model-MLE.ipynb)\n| Comparing with the hmetad toolbox | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embodied-computation-group/metadpy/blob/master/docs/source/examples/3-Comparison%20with%20the%20hmeta-d%20toolbox.ipynb)\n\n# Examples\n\n| Notebook | Colab |\n| --- | ---|\n| Subject and group level (MLE) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embodied-computation-group/metadpy/blob/master/docs/source/examples/Example%201%20-%20Fitting%20MLE%20-%20Subject%20and%20group%20level.ipynb)\n| Subject and group level (Bayesian) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/embodied-computation-group/metadpy/blob/master/docs/source/examples/Example%202%20-%20Fitting%20Bayesian%20-%20Subject%20level%20(pymc).ipynb)\n\nOr just follow the quick tour below.\n\n# Importing data\n\nClassical metacognition experiments contain two phases: task performance and confidence ratings. The task performance could for example be the ability to distinguish the presence of a dot on the screen. By relating trials where stimuli are present or absent and the response provided by the participant (Can you see the dot: yes/no), it is possible to obtain the accuracy. The confidence rating is proposed to the participant when the response is made and should reflect how certain the participant is about his/her judgement.\n\nAn ideal observer would always associate very high confidence ratings with correct task-I responses, and very low confidence ratings with an incorrect task-1 response, while a participant with a low metacognitive efficiency will have a more mixed response pattern.\n\nA minimal metacognition dataset will therefore consist in a data frame populated with 5 columns:\n* `Stimuli`: Which of the two stimuli was presented [0 or 1].\n* `Response`: The response made by the participant [0 or 1].\n* `Accuracy`: Was the participant correct? [0 or 1].\n* `Confidence`: The confidence level [can be continuous or discrete rating].\n* `ntrial`: The trial number.\n\nDue to the logical dependence between the `Stimuli`, `Responses` and `Accuracy` columns, in practice only two of those columns are necessary, the third being deduced from the others. Most of the functions in `metadpy` will accept DataFrames containing only two of these columns, and will automatically infer the missing information. Similarly, as the metacognition models described here does not incorporate the temporal dimension, the trial number is optional. \n\n`metadpy` includes a simulation function that will let you create one such data frame for one or many participants and condition, controlling for a variety of parameters. Here, we will simulate 200 trials from  participant having `d=1` and `c=0` (task performances) and a `meta-d=1.5` (metacognitive sensibility). The confidence ratings were provided using a 1-to-4 rating scale.\n\n\n```python\nfrom metadpy.utils import responseSimulation\nsimulation = responseSimulation(d=1, metad=1.5, c=0, \n                                nRatings=4, nTrials=200)\nsimulation\n```\n\n|    |   Stimuli |   Responses |   Accuracy |   Confidence |   nTrial |   Subject |\n|---:|----------:|------------:|-----------:|-------------:|---------:|----------:|\n|  0 |         1 |           1 |          1 |            4 |        0 |         0 |\n|  1 |         0 |           0 |          1 |            4 |        1 |         0 |\n|  2 |         1 |           1 |          1 |            2 |        2 |         0 |\n|  3 |         0 |           1 |          0 |            4 |        3 |         0 |\n|  4 |         0 |           0 |          1 |            3 |        4 |         0 |\n\n```python\nfrom metadpy.utils import trials2counts\nnR_S1, nR_S2 = trials2counts(\n    data=simulation, stimuli=\"Stimuli\", accuracy=\"Accuracy\",\n    confidence=\"Confidence\", nRatings=4)\n```\n\n## Data visualization\n\nYou can easily visualize metacognition results using one of the plotting functions. Here, we will use the `plot_confidence` and the `plot_roc` functions to visualize the metacognitive performance of our participant.\n\n```python\nimport matplotlib.pyplot as plt\nfrom metadpy.plotting import plot_confidence, plot_roc\n```\n\n```python\nfig, axs = plt.subplots(1, 2, figsize=(13, 5))\nplot_confidence(nR_S1, nR_S2, ax=axs[0])\nplot_roc(nR_S1, nR_S2, ax=axs[1])\n```\n\n![png](./docs/source/images/confidence_ROCAUC.png)\n\n# Signal detection theory (SDT)\n\n```python\nfrom metadpy.sdt import criterion, dprime, rates, roc_auc, scores\n```\n\nAll metadpy functions are registred as Pandas flavors (see [pandas-flavor](https://pypi.org/project/pandas-flavor/)), which means that the functions can be called as a method from the result data frame.\n\n```python\nsimulation.criterion()\n```\n\n5.551115123125783e-17\n\n```python\nsimulation.dprime()\n```\n\n0.9917006946949065\n\n```python\nsimulation.rates()\n```\n\n(0.69, 0.31)\n\n```python\nsimulation.roc_auc(nRatings=4)\n```\n\n0.695689287238583\n\n```python\nsimulation.scores()\n```\n\n(69, 31, 31, 69)\n\n# Estimating meta dprime using Maximum Likelyhood Estimates (MLE)\n\n```python\nfrom metadpy.mle import metad\n\nmetad(\n  data=simulation, nRatings=4, stimuli='Stimuli', accuracy='Accuracy',\n  confidence='Confidence', verbose=0\n  )\n```\n\n|    |   dprime |   meta_d |   m_ratio |   m_diff |\n|---:|---------:|---------:|----------:|---------:|\n|  0 | 0.970635 |  1.45925 |    1.5034 | 0.488613 |\n\n# Estimating meta-dprime using hierarchical Bayesian modeling\n\n## Subject level\n\n```python\nimport pymc as pm\nfrom metadpy.bayesian import hmetad\n```\n\n```python\nmodel, trace = hmetad(\n  data=simulation, nRatings=4, stimuli='Stimuli',\n  accuracy='Accuracy', confidence='Confidence'\n  )\n```\n\nAuto-assigning NUTS sampler...\nInitializing NUTS using jitter+adapt_diag...\nMultiprocess sampling (4 chains in 4 jobs)\nNUTS: [c1, d1, meta_d, cS1_hn, cS2_hn]\nSampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 10 seconds.\n\n```python\nimport arviz as az\naz.plot_trace(trace, var_names=['meta_d', 'cS2', 'cS1']);\n```\n \n![png](./docs/source/images/trace.png)\n\n```python\naz.summary(trace)\n```\n\n|        |   mean |    sd |   hdi_3% |   hdi_97% |   mcse_mean |   mcse_sd |   ess_bulk |   ess_tail |   r_hat |\n|:-------|-------:|------:|---------:|----------:|------------:|----------:|-----------:|-----------:|--------:|\n| meta_d |  1.384 | 0.254 |    0.909 |      1.86 |       0.004 |     0.003 |       3270 |       2980 |       1 |\n\n# References\n\n[1] Maniscalco, B., \u0026 Lau, H. (2014). Signal Detection Theory Analysis of Type 1 and Type 2 Data: Meta-d′, Response-Specific Meta-d′, and the Unequal Variance SDT Model. In The Cognitive Neuroscience of Metacognition (pp. 25–66). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-45190-4_3 \n\n[2] Maniscalco, B., \u0026 Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430. doi:10.1016/j.concog.2011.09.021\n\n[3] Fleming, S. M., \u0026 Lau, H. C. (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8. https://doi.org/10.3389/fnhum.2014.00443\n\n[4] Fleming, S.M. (2017) HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings, Neuroscience of Consciousness, 3(1) nix007, https://doi.org/10.1093/nc/nix007\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembodied-computation-group%2Fmetadpy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fembodied-computation-group%2Fmetadpy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembodied-computation-group%2Fmetadpy/lists"}