https://github.com/hadley/mturkr
Tools to make MTurk tasks easy to run from R
https://github.com/hadley/mturkr
Last synced: 9 months ago
JSON representation
Tools to make MTurk tasks easy to run from R
- Host: GitHub
- URL: https://github.com/hadley/mturkr
- Owner: hadley
- Created: 2011-11-29T15:09:29.000Z (about 14 years ago)
- Default Branch: master
- Last Pushed: 2012-07-25T19:32:57.000Z (over 13 years ago)
- Last Synced: 2025-04-15T15:08:51.873Z (9 months ago)
- Language: R
- Homepage:
- Size: 209 KB
- Stars: 19
- Watchers: 1
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# mturkr package
The `mturkr` package makes it easy to construct Amazon Mechanical Turk (MTurk) human interface tasks (HITs). It makes it easy to create templated questions, submit to amazon, monitor, review and aggregate answers.
It echoes the design of Amazon's [command line tools](http://docs.amazonwebservices.com/AWSMechTurk/latest/AWSMturkCLT/), but mapping components of the API to data structures and file formats that R developers and more familiar with.
Important terms:
* HIT: a single question
* HIT Type or Task: a group of questions of the same form
* Assignment: one question assigned to a person
## Key functions
* `init_task` - initialises a task directory with basic directory structure
* `publish_task` - combines `questions.xml` and `template.csv` to and
publishes all HITs
* `view_task` - opens web browser pointing to one of your HITs
* `review_task` - downloads assignments needing review, and approves/rejects
according to value of status column
* `unpublish_task` - removes all published HITs (in case you've made a mistake
or were just testing)
Other useful functions include:
* `get_balance` to see how much money you have left.
Note that all these functions take a directory as their first argument (if you omit the directory it will use the last directory `mturkr` saw, so you can save some typing).
By default all work is done in the sandbox. When you are ready to publish your questions to real workers, set `Host: production` in your `DESCRIPTION` file.
## Design principles
A task is represented as a set of files in a directory (see below for details).
All functions are robust to broken connections and errors, and are hence idempotent (if a function has no errors, running it again will do nothing). If successful, each request appends a row to a csv file, otherwise `stop()`s on failure.
All functions have quiet option. If `FALSE` (the default), are pretty wordy - describing each HTTP request.
All tasks work in the sandbox until you specifically indicate otherwise.
## File structure
Each task lives in its own directory with files:
User created:
* `DESCRIPTION`: task metadata, dcf format. Uses mustache template.
* `questions.xml`: xml file for question. Uses mustache template.
* `quals-*/`: qualifications. See below for details.
* `template.csv`: template data. Each column becomes mustache key.
`mturkr` created:
* `hit-review.csv`: assignments needing review
* `hit-results.csv`: answers from approved assignments
## Useful amazon references
* Docs: http://aws.amazon.com/documentation/mturk/
* API: http://docs.amazonwebservices.com/AWSMechTurk/latest/AWSMturkAPI/
* Developer guide: http://docs.amazonwebservices.com/AWSMechTurk/latest/AWSMechanicalTurkRequester/