https://github.com/rbroc/benchmarks_paper

Code and data for Rocca, R., & Yarkoni, T., (2023): Putting psychology to the test: rethinking model evaluation through benchmarking and prediction, published in Advances in Methods and Practices in Psychological Science
https://github.com/rbroc/benchmarks_paper

Last synced: 28 days ago
JSON representation

Code and data for Rocca, R., & Yarkoni, T., (2023): Putting psychology to the test: rethinking model evaluation through benchmarking and prediction, published in Advances in Methods and Practices in Psychological Science

• Host: GitHub
• URL: https://github.com/rbroc/benchmarks_paper
• Owner: rbroc
• Created: 2020-08-10T13:01:25.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2022-09-10T09:22:55.000Z (over 2 years ago)
• Last Synced: 2024-10-24T11:52:08.824Z (3 months ago)
• Language: Jupyter Notebook
• Homepage:
• Size: 105 MB
• Stars: 2
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
### benchmarking-psychology

This repository contains code and data used for "Putting psychology to the test: rethinking model evaluation through benchmarking and prediction", published in Advances in Methods and Practices in Psychological Science (https://journals.sagepub.com/doi/pdf/10.1177/25152459211026864). 


This repo is organized as follows:

- `manylabs1_data` contains data from Many Labs 1 (Klein et al., 2014; data available at https://osf.io/wx7ck/);

- Minor preprocessing of the data (i.e. removal of unused columns and regex replacement in empty cells) was performed using `preprocess_data.py`;

- `manylabs1.ipynb` contains the re-analysis of Many Labs 1 data discussed in the section *Model evaluation in psychology*;

- Code for the simulation reported in Figure 3 can be found in `simulation.ipynb`

Authors: Roberta Rocca & Tal Yarkoni - University of Texas at Austin