{"id":26109562,"url":"https://github.com/wzbsocialsciencecenter/otreeutils","last_synced_at":"2025-04-12T20:31:05.416Z","repository":{"id":57449841,"uuid":"75097281","full_name":"WZBSocialScienceCenter/otreeutils","owner":"WZBSocialScienceCenter","description":"Facilitate oTree experiment implementation with extensions for custom data models, surveys, understanding questions, timeout warnings and more.","archived":false,"fork":false,"pushed_at":"2022-03-07T10:45:23.000Z","size":318,"stargazers_count":17,"open_issues_count":0,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-10T07:17:49.376Z","etag":null,"topics":["experiments","otree","utilities"],"latest_commit_sha":null,"homepage":"https://datascience.blog.wzb.eu/2016/11/29/otreeutils-a-package-with-common-otree-utilityhelper-functions-and-classes/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WZBSocialScienceCenter.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-11-29T15:56:30.000Z","updated_at":"2023-12-17T23:57:19.000Z","dependencies_parsed_at":"2022-09-19T16:20:36.746Z","dependency_job_id":null,"html_url":"https://github.com/WZBSocialScienceCenter/otreeutils","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WZBSocialScienceCenter%2Fotreeutils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WZBSocialScienceCenter%2Fotreeutils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WZBSocialScienceCenter%2Fotreeutils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WZBSocialScienceCenter%2Fotreeutils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WZBSocialScienceCenter","download_url":"https://codeload.github.com/WZBSocialScienceCenter/otreeutils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248629364,"owners_count":21136241,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["experiments","otree","utilities"],"created_at":"2025-03-09T23:12:11.694Z","updated_at":"2025-04-12T20:31:05.387Z","avatar_url":"https://github.com/WZBSocialScienceCenter.png","language":"Python","readme":"# otreeutils\n\nMarch 2021, Markus Konrad \u003cmarkus.konrad@wzb.eu\u003e / \u003cpost@mkonrad.net\u003e / [Berlin Social Science Center](https://wzb.eu)\n\n**This project is currently not maintained.**\n\n## A package with common oTree utilities\n\nThis repository contains the package `otreeutils`. It features a set of common helper / utility functions and classes often needed when developing experiments with [oTree](http://www.otree.org/). So far, this covers the following use cases:\n\n* Easier creation of surveys:\n    * define all survey questions in a single data structure, let `otreeutils` create the required `Player` fields\n    * create a table of Likert scale inputs (\"Likert matrix\")\n    * create single Likert scale fields from given labels\n    * easy survey forms styling via CSS due to cleanly structured HTML output\n    * make survey forms with conditional inputs\n* Extensions to oTree's admin interface for [using custom data models](https://datascience.blog.wzb.eu/2016/10/31/using-custom-data-models-in-otree/), which include:\n    * Live session data view shows data from custom models\n    * Export page allows download of complete data with data from custom models\n    * Export page allows download in nested JSON format\n* Displaying and validating understanding questions\n* Displaying warnings to participants when a timeout occurs on a page (no automatic form submission after timeout)\n* More convenient development process by optional automatic fill-in of forms (saves you from clicking through many inputs during development)  \n* Setting custom URLs for pages (instead of default: the page's class name)\n\nThis screenshot shows an example survey page with a Likert matrix:\n\n![survey page with Likert matrix](img/likerttable.png)\n\nThe package is [available on PyPI](https://pypi.org/project/otreeutils/) and can be installed  via `pip install otreeutils`. \n\n**Compatibility note:** This package is compatible with oTree v3.3.x. If you have an older oTree version, check out the [CHANGES](CHANGES.md) file to see which version is compatible with your oTree version. You can install the exact version then with `pip install otreeutils==x.y.z` where `x.y.z` denotes an otreeutils version number.\n\n## Citation\n\nIf you used *otreeutils* in your published research, please cite it as follows:\n\n[Konrad, M. (2018). oTree: Implementing experiments with dynamically determined data quantity. *Journal of Behavioral and Experimental Finance.*](https://doi.org/10.1016/j.jbef.2018.10.006)\n\n\n## Examples\n\nThe repository contains three example apps which show the respective features and how they can be used in own experiments:\n\n* `otreeutils_example1` -- Understanding questions and timeout warnings\n* `otreeutils_example2` -- Surveys\n* `otreeutils_example3_market` -- Market: An example showing custom data models to collect a dynamically determined data quantity. Shows how otreeutils' admin extensions allow live data view and data export for these requirements. Companion code for [Konrad 2018](https://doi.org/10.1016/j.jbef.2018.10.006). See [its dedicated README page](https://github.com/WZBSocialScienceCenter/otreeutils/tree/master/otreeutils_example3_market). \n\n## Limitations\n\nThe admin interface extensions have still a limitation: Data export with all data from custom models is only possible with per app download option, not with the \"all apps\" option.\n\n## Requirements\n\nThis package requires oTree v3.3.x and optionally [pandas](http://pandas.pydata.org/). The requirements will be installed along with otreeutils when using `pip` (see below). \n\n## Installation and setup\n\nIn order to use otreeutils in your experiment implementation, you only need to do the following things:\n\n1. Either install the package from [PyPI](https://pypi.python.org/pypi/otreeutils) via *pip* (`pip install otreeutils`) or download/clone this github repository and copy the `otreeutils` folder to your oTree experiment directory.\n2. Edit your `settings.py` so that you add \"otreeutils\" to your `INSTALLED_APPS` list. **Don't forget this, otherwise the required templates and static files cannot be loaded correctly!**\n\n\n## API overview\n\nIt's best to have a look at the (documented) examples to see how to use the API.\n\n### `otreeutils.pages` module\n\n#### `ExtendedPage` class\n\nA common page extension to oTree's default `Page` class.\n All other page classes in `otreeutils` extend this class. Allows to define a custom page URL via `custom_name_in_url`, timeout warnings, a page title and provides a template variable `debug` with which you can toggle debug code in your templates / JavaScript parts.\n\nThe template variable `debug` (integer – 0 or 1) is toggled using an additional `APPS_DEBUG` variable in `settings.py`. See the `settings.py` of this repository. This is quite useful for example in order to fill in the correct questions on a page with understanding questions automatically in a debug session (so that it is easier to click through the pages).\n\nThere is also a page variable `debug_fill_forms_randomly`, which can be set for any page derived from the `ExtendedPage` class (i.e. also for survey pages -- see below). If you set this variable to `True`, then all form inputs on the page are automatically filled in with random values once you visit the page. This happens when you run the experiment in \"debug mode\", i.e. when `APPS_DEBUG` is set to `True`. By default, `debug_fill_forms_randomly` is set to `False`. You can enable this feature for a given page like this:\n\n```python\nfrom otreeutils.pages import ExtendedPage\n\nclass MyPage(ExtendedPage):\n    debug_fill_forms_randomly = True\n```\n\nThis saves time when you click through an experiment with many complex forms.\n\n#### `UnderstandingQuestionsPage` class\n\nBase class to implement understanding questions. A participant must complete all questions in order to proceed. You can display hints. Use it as follows:\n\n```python\nfrom otreeutils.pages import UnderstandingQuestionsPage\n\nclass SomeUnderstandingQuestions(UnderstandingQuestionsPage):\n    page_title = 'Set a page title'\n    questions = [\n        {\n            'question': 'What is π?',\n            'options': [1.2345, 3.14159],\n            'correct': 3.14159,\n            'hint': 'You can have a look at Wikipedia!'   # this is optional\n        },\n        # ...\n    ]\n```\n\nBy default, the performance of the participant is not recorded, but you can optionally provide a `form_model` and set a field in `form_field_n_wrong_attempts` which defines in which field the number of wrong attempts is written.\n\nIf you set `APPS_DEBUG` to `True`, the correct answers will already be filled in order to skip swiftly through pages during development.\n\n\n### `otreeutils.surveys` module\n\n#### `create_player_model_for_survey` function\n\nThis function allows to dynamically create a `Player` model class for a survey. It can be used as follows in `models.py`.\n\nAt first you define your questions per page in a survey definitions data structure, for example like this:\n\n```python\nfrom otreeutils.surveys import create_player_model_for_survey\n\n\nGENDER_CHOICES = (\n    ('female', 'Female'),\n    ('male', 'Male'),\n    ('no_answer', 'Prefer not to answer'),\n)\n\n\nSURVEY_DEFINITIONS = {\n    'SurveyPage1': {\n        'page_title': 'Survey Questions - Page 1',\n        'survey_fields': [\n            ('q1_a', {   # field name (which will also end up in your \"Player\" class and hence in your output data)\n                'text': 'How old are you?',   # survey question\n                'field': models.PositiveIntegerField(min=18, max=100),  # the same as in normal oTree model field definitions\n            }),\n            ('q1_b', {\n                'text': 'Please tell us your gender.',\n                'field': models.CharField(choices=GENDER_CHOICES),\n            }),\n            # ... more questions\n        ]\n    },\n    # ... more pages\n}\n```\n\nNote how `SURVEY_DEFINITIONS` is a dictionary which maps pages to the survey questions that should appear on each page. We will later create a `SurveyPage1` page class in `pages.py`.\n\nNow you create the `Player` class by passing the name of the module for which it will be created. This should be  the `models` module of your app, so in your case this is `'survey.models'` if your app is named `survey`. The second parameter is the survey definitions that we just created:\n\n```python\nPlayer = create_player_model_for_survey('otreeutils_example2.models', SURVEY_DEFINITIONS)\n```\n\nThe attributes (model fields, etc.) will be automatically created. When you run `otree resetdb`, you will see that the fields `q1_a`, `q1_b`, etc. will be generated in the database.\n\nYou may also add extra (non-survey) fields to your `Player` class, by passing a dict to the optional `other_fields` parameter:\n\n```python\nPlayer = create_player_model_for_survey('otreeutils_example2.models', SURVEY_DEFINITIONS, other_fields={\n    'treatment': models.IntegerField()\n})\n```\n\n##### Likert score inputs via `generate_likert_field` and `generate_likert_table` functions\n\nThe function `generate_likert_field` allows you to easily generate fields for a given Likert scale and can be used inside a survey definitions data structure:\n\n```python\nfrom otreeutils.surveys import generate_likert_field\n\nlikert_5_labels = (\n    'Strongly disagree',            # value: 1\n    'Disagree',                     # value: 2\n    'Neither agree nor disagree',   # ...\n    'Agree',\n    'Strongly agree'                # value: 5\n)\n\nlikert_5point_field = generate_likert_field(likert_5_labels)\n```\n\nThe object `likert_5point_field` is now a *function* to generate new fields of the specified Likert scale:\n\n```python\n# ...\n\nSURVEY_DEFINITIONS = {\n    'SurveyPage2': {\n        'page_title': 'A Likert 5-point scale example',\n        'survey_fields': [\n            ('q_otree_surveys', {  # most of the time, you'd add a \"help_text\" for a Likert scale question. You can use HTML:\n                'help_text': \"\"\"\n                    \u003cp\u003eConsider this quote:\u003c/p\u003e\n                    \u003cblockquote\u003e\n                        \"oTree is great to make surveys, too.\"\n                    \u003c/blockquote\u003e\n                    \u003cp\u003eWhat do you think?\u003c/p\u003e\n                \"\"\",\n                'field': likert_5point_field(),   # don't forget the parentheses at the end!\n            }),\n            ('q_just_likert', {\n                 'label': 'Another Likert scale input:',  # optional, no HTML\n                 'field': likert_5point_field(),  # don't forget the parentheses at the end!\n            }),\n        ]\n    },\n    # ... more pages\n}\n```\n\nThe function `generate_likert_table` allows you to easily generate a table of Likert scale inputs like a matrix with the Likert scale increments in the columns and your questions in the rows:\n\n```python\n# ...\n\nSURVEY_DEFINITIONS = {\n    'SurveyPage3': {\n        'page_title': 'A Likert scale table example',\n        'survey_fields': [\n            # create a table of Likert scale choices\n            # we use the same 5-point scale a before and specify four rows for the table,\n            # each with a tuple (field name, label)\n            generate_likert_table(likert_5_labels,\n                                  [\n                                      ('q_pizza_tasty', 'Tasty'),\n                                      ('q_pizza_spicy', 'Spicy'),\n                                      ('q_pizza_cold', 'Too cold'),\n                                      ('q_pizza_satiable', 'Satiable'),\n                                  ],\n                                  form_help_initial='\u003cp\u003eHow was your latest Pizza?\u003c/p\u003e',  # HTML to be placed on top of form\n                                  form_help_final='\u003cp\u003eThank you!\u003c/p\u003e'                     # HTML to be placed below form\n            )\n        ]\n    },\n    # ... more pages\n}\n```\n\nThere are several additional parameters that you can pass to `generate_likert_table()` which will control the display and behavior of the table:\n\n- `table_repeat_header_each_n_rows=\u003cinteger\u003e`: set to integer N \u003e 0 to repeat the table header after every N rows\n- `table_cols_equal_width=\u003cTrue/False\u003e`: adjust form columns so that they have equal width\n- `table_row_header_width_pct=\u003cnumber\u003e`: if form columns should have equal width, this specifies the width of the first column (the table row header) in percent (default: 25)\n- `table_rows_equal_height=\u003cTrue/False\u003e`: adjust form rows so that they have equal height\n- `table_rows_alternate=\u003cTrue/False\u003e`: alternate form rows between \"odd\" and \"even\" CSS classes (alternates background colors)\n- `table_rows_randomize=\u003cTrue/False\u003e`: randomize form rows\n- `table_rows_highlight=\u003cTrue/False\u003e`: highlight form rows on mouse-over\n- `table_cells_highlight=\u003cTrue/False\u003e`: highlight form cells on mouse-over\n- `table_cells_clickable=\u003cTrue/False\u003e`: make form cells clickable for selection (otherwise only the small radio buttons can be clicked)\n\n#### More options for surveys\n\nTo implement advanced features such as conditional input display, have a look at the example app `otreeutils_example2`.\n\n#### `SurveyPage` class\n\nYou can then create the survey pages which will contain the questions for the respective pages as defined before in `SURVEY_DEFINITIONS`:\n\n```python\n# (in pages.py)\n\nfrom otreeutils.surveys import SurveyPage, setup_survey_pages\n\n# Create the survey page classes; their names must correspond to the names used in the survey definition  \n\nclass SurveyPage1(SurveyPage):\n    pass\nclass SurveyPage2(SurveyPage):\n    pass\n# more pages ...\n\n# Create a list of survey pages.\n\nsurvey_pages = [\n    SurveyPage1,\n    SurveyPage2,\n    # more pages ...\n]\n```\n\nSince each `SurveyPage` is derived from the `ExtendedPage` class, you can also enable the automatic fill-in feature. This means that all form inputs on the page are automatically filled in with random values once you visit the page. That happens when you run the experiment in \"debug mode\", i.e. when `APPS_DEBUG` is set to `True`. By default, `debug_fill_forms_randomly` is set to `False`. You can enable this feature for a given survey page like this:\n\n```python\nclass SurveyPage3(SurveyPage):\n    debug_fill_forms_randomly = True\n```\n\nThis saves time when you click through an experiment with many survey fields.\n\n#### `setup_survey_pages` function\n\nNow all survey pages need to be set up. The `Player` class will be passed to all survey pages and the questions for each page will be set according to their order. \n\n```python\n# Common setup for all pages (will set the questions per page)\nsetup_survey_pages(models.Player, survey_pages)\n```\n\nFinally, we can set the `page_sequence` in order to use our survey pages:\n\n```python\npage_sequence = [\n    SurveyIntro,  # define some pages that come before the survey\n    # ...\n]\n\n# add the survey pages to the page sequence list\npage_sequence.extend(survey_pages)\n\n# we could add more pages after the survey here\n# ...\n```\n\n**Have a look into the example implementations provided as `otreeutils_example1` (understanding questions, simple page extensions), `otreeutils_example2` (surveys) and `otreeutils_example3_market` (custom data models).**  \n\n\n### `otreeutils.scripts` module\n\nThis module allows creating scripts that interface with oTree from the command line. Importing `otreeutils.scripts` makes sure that everything is correctly set up and the settings are loaded. An example might be a script which exports data from the current sessions for specific apps as JSON file:\n\n```python\nimport sys\n\nfrom otreeutils import scripts   # this is the most import line and must be included at the beginning\n\n\nif len(sys.argv) != 2:\n    print('call this script with a single argument: python %s \u003coutput.json\u003e' % sys.argv[0])\n    exit(1)\n\noutput_file = sys.argv[1]\n\napps = ['intro',\n        'my_app',\n        'outro']\n\nprint('loading data...')\n\n# get the data as hierarchical data structure. this is esp. useful if you use\n# custom data models\ncombined = scripts.get_hierarchical_data_for_apps(apps)\n\nprint('writing data to file', output_file)\n\nscripts.save_data_as_json_file(combined, output_file, indent=2)\n\nprint('done.')\n```\n\n### Custom data models and admin extensions\n\nIf you implement custom data models and want to use otreeutils' admin extensions you additionally need to follow these steps:\n\n#### 1. Install all dependencies\n\nMake sure that you install otreeutils with extra dependencies via `pip install otreeutils[admin]`.\n\n#### 2. Add configuration class to custom models\n\nFor each of the custom models that you want to include in the live data view or extended data export, you have to define a subclass called `CustomModelConf` like this:\n\n```python\nfrom otree.db.models import Model, ForeignKey   # import base Model class and ForeignKey\n\n# ...\n\nclass FruitOffer(Model):\n    amount = models.IntegerField(label='Amount', min=0, initial=0)\n\n    # ... more fields here ...\n\n    seller = ForeignKey(Player)\n\n\n    class CustomModelConf:\n        \"\"\"\n        Configuration for otreeutils admin extensions.\n        \"\"\"\n        data_view = {    # define this attribute if you want to include this model in the live data view\n            'exclude_fields': ['seller'],\n            'link_with': 'seller'\n        }\n        export_data = {  # define this attribute if you want to include this model in the data export\n            'exclude_fields': ['seller_id'],\n            'link_with': 'seller'\n        }\n\n``` \n\n#### 3. Add a custom urls module\n\nIn your experiment app, add a file `urls.py` and simply include the custom URL patters from otreeutils as follows:\n\n```python\nfrom otreeutils.admin_extensions.urls import urlpatterns\n\n# add more custom URL rules here if necessary\n# ...\n```\n\n#### 4. Import the `custom_export` function in `models.py`\n\nIn your experiment app, add the following line to `models.py`:\n\n```python\nfrom otreeutils.admin_extensions import custom_export\n```\n\nThis makes sure that the data from the custom data models can be exported via oTree's admin interface.\n\n#### 5. Update `settings.py` to load the custom URLs and channel routes\n\nAdd this line to your `settings.py`:\n\n```python\nROOT_URLCONF = '\u003cAPP_PACKAGE\u003e.urls'\n```\n\nInstead of `\u003cAPP_PACKAGE\u003e` write your app's package name (e.g. \"market\" if your app is named \"market\").\n\n**And don't forget to edit your settings.py so that you add \"otreeutils\" to your INSTALLED_APPS list!**\n\nThat's it! When you visit the admin pages, they won't really look different, however, the live data view will now support your custom models and in the data export view you can download the data *including* the custom models' data with the \"custom\" link. **So far, the \"all-apps\" download option will not include the custom models' data.**\n\n\n## License\n\nApache License 2.0. See LICENSE file.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwzbsocialsciencecenter%2Fotreeutils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwzbsocialsciencecenter%2Fotreeutils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwzbsocialsciencecenter%2Fotreeutils/lists"}