{"id":36632498,"url":"https://github.com/sbobek/tobii-pytracker","last_synced_at":"2026-01-12T09:39:59.454Z","repository":{"id":320336686,"uuid":"749700812","full_name":"sbobek/tobii-pytracker","owner":"sbobek","description":"Tobii Eyetracker usage and analysis with Python SDK (no Tobii Labs needed)","archived":false,"fork":false,"pushed_at":"2025-11-20T10:27:00.000Z","size":10722,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-20T12:18:10.583Z","etag":null,"topics":["artificial-intelligence","eye-tracking","eyetracking"],"latest_commit_sha":null,"homepage":"https://tobii-pytracker.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sbobek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-01-29T08:14:37.000Z","updated_at":"2025-11-20T10:27:04.000Z","dependencies_parsed_at":"2025-10-23T08:43:26.435Z","dependency_job_id":null,"html_url":"https://github.com/sbobek/tobii-pytracker","commit_stats":null,"previous_names":["sbobek/tobii-pytracker"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sbobek/tobii-pytracker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbobek%2Ftobii-pytracker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbobek%2Ftobii-pytracker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbobek%2Ftobii-pytracker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbobek%2Ftobii-pytracker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sbobek","download_url":"https://codeload.github.com/sbobek/tobii-pytracker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbobek%2Ftobii-pytracker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28337740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T06:09:07.588Z","status":"ssl_error","status_checked_at":"2026-01-12T06:05:18.301Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","eye-tracking","eyetracking"],"created_at":"2026-01-12T09:39:58.366Z","updated_at":"2026-01-12T09:39:59.441Z","avatar_url":"https://github.com/sbobek.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PyPI](https://img.shields.io/pypi/v/tobii-pytracker)](https://pypi.org/project/tobii-pytracker/)  ![License](https://img.shields.io/github/license/sbobek/tobii-pytracker)\r\n ![PyPI - Downloads](https://img.shields.io/pypi/dm/tobii-pytracker) [![Documentation Status](https://readthedocs.org/projects/tobii-pytracker/badge/?version=latest)](https://tobii-pytracker.readthedocs.io/en/latest/?badge=latest)\r\n# Toolkit for AI-enhanced Eye-tracking data collection\r\n\r\n\r\nA Python framework for conducting **eyetracking-based experiments** on **perception** and **reasoning** in machine learning (ML) tasks, as well as for **data enrichment**.\r\n\r\n\r\nThe framework integrates multiple data modalities commonly used as inputs for ML models and provides a platform for **rapid experiment design and execution**. It combines the **PsychoPy** platform for basic user input and **eye-tracking hardware calibration (and emulation)**, **state-of-the-art AI models** for data processing and analysis, and a **custom, extensible architecture** built on top of `tobii-pytracker`.\r\n\r\n\r\n- **Non-intrusive, multimodal data labeling**, including:\r\n  - Classical label assignment  \r\n  - ML-guided object annotations with **free speech input**, which is automatically transcribed into text and aligned with the regions the user refers to (based on **eye gaze area-of-interest detection**)\r\n\r\n- **Analytical platform** for experiment results, offering:\r\n  - Classical metrics such as **saccades**, **fixations**, and **heatmaps**  \r\n  - Advanced methods for **analysis and visualization**, including **process mining scanpath analysis** and **concept detection based on eye-gaze clusters**\r\n\r\n\r\n\r\n![Alt text](https://raw.githubusercontent.com/sbobek/tobii-pytracker/refs/heads/psychopy/pix/tobii-pytracker.svg)\r\n\r\n## Table of Contents\r\n- [Requirements](#Requirements)\r\n- [Installation](#Installation)\r\n- [Usage](#Usage)\r\n- [Datasets](#Datasets)\r\n- [Custom Model](#CustomModel)\r\n- [Configuration](#Configuration)\r\n\r\n## Requirements\r\n\r\n- Python 3.10.x - _(Python 3.10 version is __crucial__, the script won't work with any other version)_\r\n- Pip\r\n- [Tobii Pro Eye Tracker Manager](https://connect.tobii.com/s/etm-downloads?language=en_US) (only optional, for some extra features like firmware updates, etc.)\r\n\r\n## Installation\r\n\r\n1. Clone or download this repository to your local machine.\r\n\r\n    ```sh\r\n   conda create --name pytracker-env python=3.10\r\n   conda activate pytracker-env\r\n   git clone https://github.com/sbobek/tobii-pytracker.git\r\n   cd tobii-pytracker\r\n   pip install .\r\n   ```\r\n\r\n   Install psychopy, with no-deps, to keep the installation simple and lightweight.\r\n   Note that we need psychopy in a version at least 2024.1.4\r\n   \r\n   ```sh\r\n   pip install \"psychopy\u003e=2024.1.4,\u003c2025.1.0\" --no-deps\r\n   ```\r\n\r\n\r\n## Usage\r\n\r\n1. To run the script, use the following command (make sure you have activated virtual environment):\r\n\r\n    ```sh\r\n    tobii-pytracker\r\n    ```\r\n\r\n\u003csup\u003e*Make sure to connect eyetracker to your device before running the script with `--enable_eyetracker=True`, otherwise it will fail.\u003c/sup\u003e\r\n\r\n2. Additionally you can specify a few arguments:\r\n- `--config_file` - Path to YAML script config file _(default: configs/config.yaml)_\r\n- `--eyetracker_config_file` - Path to YAML eyetracker config file _(default: configs/eyetracker_config.yaml)_\r\n- `--enable_eyetracker` - Launch script with launchHubServer (requires connected eyetracker) _(default: False)_\r\n- `--enable_voice` - Start voice recording for Think-Aloud Protocol _(default: False)_\r\n- `--raw_data` - Record full Tobii raw samples instead of filtered gaze positions _(default: False)_\r\n- `--disable_psychopy` - Run headless (no GUI) and continuously record gaze + voice until stopped _(default: False)_\r\n- `--loop_count` - Number of stimuli to display before exit _(default: 10)_\r\n- `--log_level` - Logger level (\"info\", \"debug\", \"warning\", \"error\", \"critical\") _(default: info)_\r\n\r\nFor more information and usage you can run `tobii-pytracker --help`\r\n\r\n## Datasets\r\n\r\nDifferent datasets can be specified in the config file that will be processed and displayed in the PsychoPy window. They must be downloaded locally on the device where the script is run and follow these structure rules:\r\n\r\n1. __Image datasets__ *(.png, .jpg, .jpeg, .bmp, .gif)*:\r\n    ```.\r\n    ├── dataset\r\n    │   ├── category1\r\n    │   │   ├──file1.jpg\r\n    │   │   ├──file2.jpg\r\n    │   │   └──...\r\n    │   ├── category2\r\n    │   │   ├──file1.jpg\r\n    │   │   ├──file2.jpg\r\n    │   │   └──...\r\n    │   └── ...\r\n    └── ...\r\n    ```\r\n\r\n    \u003e NOTE: GUI will be created accordingly to this structure and class labels will be extracted from subfolder names within the dataset. If using custom model, remember to make them match class names.\r\n\r\n2. __Text datasets__ *(.csv)*:\r\n    ```.\r\n    ├── dataset.csv\r\n    └── ...\r\n    ```\r\n\r\n    \u003e NOTE: The first line in .csv file __MUST__ be a header. The GUI will take unique label values from a column with a name specified in config file.\r\n\r\n## Custom Model\r\n\r\nThe custom model for detecting bounding boxes fro possible area of interest is optional. \r\nEvery dataset class has its own bounding-box detection method (words for text, bins dor time-series, superpixels for images).\r\nHowever, you might want to change it with your custom implementation.\r\nIt is desired, when you want to get bounding box from external model at the runtime, because for instance screenshots have format or size which is not used by you detection model and cannot be use post-hoc.\r\n\r\nThe `CustomModel` class (located in `runtime_models/custom_model.py`) serves as an abstract base class. It defines the essential methods that any custom model implementation must provide. By inheriting from `CustomModel` and implementing these methods, your custom model can seamlessly interact with the rest of the toolkit:\r\n\r\n- `prepare_model(self)`: Load and prepare model for prediction\r\n- `predict(self, input_data)`: Run prediction with the loaded model\r\n- `process(self, path)`: Post-process the prediction and return formatted results\r\n\r\n### Creating Your Own Model Modules\r\nTo create your own model module:\r\n\r\n1. Create a new Python file within the specified directory (e.g., `custom_runtime_models/my_custom_model.py`).\r\n\r\n2. Import the CustomModel class:\r\n```python\r\nfrom runtime_models.custom_model import CustomModel\r\n```\r\n\r\n3. Define a new class that inherits from `CustomModel`:\r\n\r\n```python\r\nclass MyCustomModel(CustomModel):\r\n\r\n  def prepare_model(self):\r\n    # Load and prepare model for prediction\r\n    self.logger.debug(\"Preparing MyCustomModel...\")\r\n    self.model = ...\r\n    self.logger.debug(\"Model preparation done.\")\r\n\r\n  def predict(self, input_data):\r\n    # Run prediction with the loaded model\r\n    predictions = ...\r\n    return predictions\r\n\r\n  def process(self, data):\r\n    # Post-process the prediction and return formatted results\r\n    predictions = self.predict(data)\r\n    processed_predictions = ...\r\n    return processed_predictions\r\n```\r\n\r\nKeep in mind that main processing pipeline uses the `process()` method to save processed model predictions.\r\n\r\n## Configuration\r\n\r\nThe script requires two configuration files in YAML format: one for general settings and another for eye tracker settings.\r\nBy default, the configuration files are located in `configs`` directory.\r\n\r\n### General Configuration (config.yaml)\r\n\r\nThe general configuration file should include the following fields:\r\n\r\n```yaml\r\ndataset:\r\n  path: path/to/dataset\r\ndisplay:\r\n  monitor:\r\n    name: monitor_name\r\n    resolution:\r\n      - width\r\n      - height\r\n    width: monitor_width\r\n    distance: distance_from_monitor\r\n  gui:\r\n    button:\r\n      size:\r\n        - size_x\r\n        - size_y\r\n      color: color\r\n      text:\r\n        color: text_color\r\n        size: text_size\r\n    fixation_dot: \r\n      size: radius_of_a_dot\r\n      color: dot_color\r\n    aoe:\r\n      - size_x\r\n      - size_y\r\noutput:\r\n  folder: folder\r\n  file: file\r\n\r\nbbox_model:\r\n  folder: folder\r\n  module: module\r\n  class: class\r\n  filename: filename\r\n\r\n\r\ninstructions:\r\n  intro:\r\n    - \"Introductory message.\"\r\n    - \"Press SPACE to begin.\"\r\n  outro:\r\n    - \"Message for the end of the study.\"\r\n    - \"You may now close the window or press ESC to exit.\"\r\n```\r\n\r\n\u003e NOTE: If using a dataset with __text__ data, the path should be specified for a file in a `.csv` format, as well as additional `text` field:\r\n\r\n```yaml\r\ndataset:\r\n  path: path/to/file.csv\r\n  text:\r\n    label_column_name: label_column_name\r\n    text_column_name: text_column_name\r\n```\r\n\r\n### Eye Tracker Configuration (eyetracker_config.yaml)\r\n\r\nThe eye tracker configuration file should include the fields required by `launchHubServer` from the `psychopy.iohub` module. Refer to the [PsychoPy documentation](https://www.psychopy.org/api/iohub/device/eyetracker_interface/Tobii_Implementation_Notes.html#default-device-settings) for more details on the specific settings.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbobek%2Ftobii-pytracker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsbobek%2Ftobii-pytracker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbobek%2Ftobii-pytracker/lists"}