{"id":31934347,"url":"https://github.com/testomatio/pytestomatio","last_synced_at":"2025-10-14T06:25:42.498Z","repository":{"id":184440426,"uuid":"671891333","full_name":"testomatio/pytestomatio","owner":"testomatio","description":"Pytest plugin to analyze tests and link them automatically with Testomat.io test case management system","archived":false,"fork":false,"pushed_at":"2025-10-08T15:22:16.000Z","size":336,"stargazers_count":12,"open_issues_count":22,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-10-08T17:42:18.685Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/testomatio.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-28T11:34:53.000Z","updated_at":"2025-09-29T17:40:22.000Z","dependencies_parsed_at":"2023-11-07T15:27:41.132Z","dependency_job_id":"5f715d15-8a66-4f4f-99ba-167e8a7c9dc8","html_url":"https://github.com/testomatio/pytestomatio","commit_stats":null,"previous_names":["ypurek/pytest-analyzer","testomatio/pytestomatio"],"tags_count":91,"template":false,"template_full_name":null,"purl":"pkg:github/testomatio/pytestomatio","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testomatio%2Fpytestomatio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testomatio%2Fpytestomatio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testomatio%2Fpytestomatio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testomatio%2Fpytestomatio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/testomatio","download_url":"https://codeload.github.com/testomatio/pytestomatio/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/testomatio%2Fpytestomatio/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018120,"owners_count":26086281,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-14T06:25:40.379Z","updated_at":"2025-10-14T06:25:42.460Z","avatar_url":"https://github.com/testomatio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Support Ukraine Badge](https://bit.ly/support-ukraine-now)](https://github.com/support-ukraine/support-ukraine)\n\n# testomat.io plugin for pytest\nA powerful pytest plugin that integrates your tests with testomat.io platform for test management, reporting and analytics\n\n## Features\n- ✅ Sync tests with testomat.io\n- 📊 Real-time test execution reporting\n- 🏷️ Test labeling and categorization\n- 📁 Test run grouping and environments\n- 📎 Artifact management with S3 integration\n- 🔍 Advanced filtering and debugging\n\n## uses testomat.io API:\n\n- https://testomatio.github.io/check-tests/ - for sync\n- https://testomatio.github.io/reporter/ - for reporting\n\n## Table of Contents\n- [Installation](#installation)\n- [Quick Start](#quick-start)\n- [Advanced Usage](#advanced-usage)\n  - [Core commands](#core-commands)\n  - [Additional options](#additional-options)\n  - [Configuration](#configuration-with-environment-variables)\n  - [Test artifacts](#submitting-test-artifacts)\n- [Contributing](#contributing)\n\n## Installation\nPrerequisites:\n - Python 3.10+\n - Pytest 6.2.5+\n - Active [testomat.io](https://testomat.io) account \n\nInstall via pip:\n```bash\npip install pytestomatio\n```\n\n## Quick Start\n\n### Get your API token\n1. Login to testomat.io\n2. Create project or go to existing project\n3. Click on \"Import Tests from Source Code\"\n4. Copy your project token(starts with \"tstmt_\")\n### Sync tests\nSynchronize tests to testomat.io:\n```bash\nTESTOMATIO=your_token pytest --testomatio sync\n```\n### Report tests\nExecute tests and send results to testomat.io:\n```bash\nTESTOMATIO=your_token pytest --testomatio report\n```\n### Example of test\n\nTo make the experience more consistent, it uses standard pytest markers.  \ntestomat.io test id is a string value that starts with `@T` and has 8 symbols after.\n\n```python\nfrom pytest import mark\n\n\n@mark.testomatio('@T96c700e6')\ndef test_example():\n    assert 2 + 2 == 4\n```\n\n## Advanced usage\n\n### Core commands\n#### Synchronization\nSynchronize tests to testomat.io and get back test id.\n\n```bash\npytest --testomatio sync\n```\nClarification:\n- tests will not be executed\n- tests can be synced even without `@pytest.mark.testomatio('@T96c700e6')` decorator.\n- test title in testomat.io == test name in pytest\n- test suit title in testomat.io == test file name in pytest\n\n\n#### Clean up\nRemove all test ids from source code. Tests will not be executed\n\n```bash\npytest --testomatio remove\n```\n\n#### Report\n\nRun pytest and send test results into testomat.io.  \nTest can be sent to testomat.io without ids in your test code. If testomat.io failed to match tests by title, it will create\nnew tests for the run\n\n```bash\npytest --testomatio report\n```\n\n#### Debug\n\nRun pytest with debug parameter to get test data collected in metadata.json file\n\n```bash\npytest --testomatio debug\n```\n\n#### Launch\nCreate empty run and obtain its RUN_ID from testomat.io.\n\n```bash\npytest --testomatio launch\n```\n\n#### Finish\nFinish running or scheduled run on testomat.io. \n**TESTOMATIO_RUN_ID** environment variable is required.\n\n```bash\nTESTOMATIO_RUN_ID=***run_id*** pytest --testomatio finish\n```\n\n\n### Additional options\n#### Submitting Test Run Environment\nTo configure test environment, you can use additional option **testRunEnv**. The configured environment will be added to the test report. Use it with **report** command:\n\n```bash\npytest --testomatio report --testRunEnv \"windows11,chrome,1920x1080\"\n```\n\nEnvironment values are comma separated, please use double quotation.\n\n#### Clearing empty test suites\nTo automatically clean empty test suites on testomat.io you can use **no-empty** option. Use it with\n**sync** command:\n```bash\npytest --testomatio sync --no-empty\n```\n\n#### Disable detaching tests\nIf a test from a previous import was not found on next import it is marked as \"detached\".\nThis is done to ensure that deleted tests are not staying in Testomatio while deleted in codebase.\nTo disable this behaviour and don't mark anything as detached on import use **no-detach** option. Use it with **sync** command:\n\n```bash\npytest --testomatio sync --no-detach\n```\n\n#### Keeping original file structure\n By default, when importing tests, testomat.io does not preserve original file structure. Use option **keep-structure** with **sync** command to keep original file structure:\n```bash\npytest --testomatio sync --keep-structure\n```\n#### Import into given directory\nBy default, tests are imported into the root of the project.\nYou can use **directory** option to specify directory to use for test file structure. Use this option with **sync** command:\n ```bash\npytest --testomatio sync --directory imported_tests\n```\nNote: **keep-structure** option takes precedence over **directory** option. If both are used **keep-structure** will be used.\n#### Filter tests by id\nYou can filter tests by testomat.io id, using **test-id** option. You pass single or multiple ids to this option. Use this option with **report** command:\n ```bash\npytest --testomatio report --test-id \"Tc0880217|Tfd1c595c\"\n```\nNote: Test id should be started from letter \"T\"\n\n\n### Configuration with environment variables\nYou can use environment variable to control certain features of testomat.io\n\n#### Basic configuration\n| Env variable             | What it does                                                                                                                                                                                                          | Examples                                                                        |\n|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|\n| TESTOMATIO               | Provides token for pytestomatio to access and push data to testomat.io. Required for **sync** and **report** commands                                                                                                 | TESTOMATIO=tstmt_***** pytest --testomatio sync                                 |\n| TESTOMATIO_SYNC_LABELS   | Assign labels to a test case when you synchronise test from code with testomat.io. Labels must exist in project and their scope must be enabled for tests                                                             | TESTOMATIO_SYNC_LABELS=\"number:1,list:one,standalone\" pytest --testomatio report |\n| TESTOMATIO_CODE_STYLE    | Code parsing style for test synchronization. If you are not sure, don't set this variable. Default value is 'default'                                                                                                 | TESTOMATIO_CODE_STYLE=pep8 pytest --testomatio sync                             |\n| TESTOMATIO_CI_DOWNSTREAM | If set, pytestomatio will not set or update build url for a test run. This is useful in scenarios where build url is already set in the test run by Testomat.io for test runs that a created directly on Testomat.io. | TESTOMATIO_CI_DOWNSTREAM=true pytest --testomatio report                        |\n | TESTOMATIO_URL           | Customize testomat.io url                                                                                                                                                                                             | TESTOMATIO_URL=https://custom.com/ pytest --testomatio report                   |\n | BUILD_URL                | Overrides build url run tests                                                                                                                                                                                         | BUILD_URL=http://custom.com/ pytest --testomatio report                         |\n\n\n#### Test Run configuration\n| Env variable             | What it does                                                                                                               | Examples                                                           |\n|--------------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|\n| TESTOMATIO_TITLE         | Name of a test run to create on testomat.io                                                                                | TESTOMATIO_TITLE=\"Nightly Smoke Tests\" pytest --testomatio report  |\n| TESTOMATIO_RUN_ID        | Id of existing test run to use for sending test results to                                                                 | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report               |\n| TESTOMATIO_RUNGROUP_TITLE | Create a group (folder) for a test run. If group already exists, attach test run to it                                     | TESTOMATIO_RUNGROUP_TITLE=\"Release 2.0\" pytest --testomatio report |\n| TESTOMATIO_ENV           | Assign environment to a test run, env variant of **testRunEnv** option. Has a lower precedence than **testRunEnv** option. | TESTOMATIO_ENV=\"linux,chrome,1920x1080\" pytest --testomatio report |\n| TESTOMATIO_LABEL         | Assign labels to a test run. Labels must exist in project and their scope must be enabled for runs                         | TESTOMATIO_LABEL=\"smoke,regression\" pytest --testomatio report     |\n| TESTOMATIO_UPDATE_CODE         | Send code of your test to Testomat.io on each run. If not enabled(default) assumes the code is pushed using **sync** command | TESTOMATIO_UPDATE_CODE=True pytest --testomatio report             |\n| TESTOMATIO_EXCLUDE_SKIPPED         | Exclude skipped tests from the report                                                                                      | TESTOMATIO_EXCLUDE_SKIPPED=1 pytest --testomatio report            |\n| TESTOMATIO_PUBLISH            | Publish run after reporting and provide a public URL                                                                                                                             | TESTOMATIO_PUBLISH=true pytest --testomatio report                                                          |\n| TESTOMATIO_PROCEED            | Do not finalize the run                                                                                                                                                          | TESTOMATIO_PROCEED=1 pytest --testomatio report                                                             |\n| TESTOMATIO_SHARED_RUN         | Report parallel execution to the same run matching it by title. If the run was created more than 20 minutes ago, a new run will be created instead.                              | TESTOMATIO_TITLE=\"Run1\" TESTOMATIO_SHARED_RUN=1 pytest --testomatio report                                  |\n| TESTOMATIO_SHARED_RUN_TIMEOUT | Changes timeout of shared run. After timeout, shared run won`t accept other runs with same name, and new runs will be created. Timeout is set in minutes, default is 20 minutes. | TESTOMATIO_TITLE=\"Run1\" TESTOMATIO_SHARED_RUN=1 TESTOMATIO_SHARED_RUN_TIMEOUT=10 pytest --testomatio report |\n\n\n#### S3 Bucket configuration\n| Env variable         | Description                           |\n|----------------------|---------------------------------------|\n| S3_REGION            | Your S3 region                        |\n| S3_ACCESS_KEY_ID     | Your S3 access key ID                 |\n| S3_SECRET_ACCESS_KEY | Your S3 secret access key             |\n| S3_BUCKET            | Your S3 bucket name                   |\n| S3_ENDPOINT          | Your S3 endpoint                      |\n| S3_BUCKET_PATH       | Path to your bucket                   |\n| TESTOMATIO_PRIVATE_ARTIFACTS       | Store artifacts in a bucket privately |\n\n\n### pytest.ini\nIn case you are using private testomat.io service, create `pytest.ini` file in your project root directory. Specify\ntestomat.io url in it\n\n```ini\n[pytest]\ntestomatio_url = https://app.testomat.io\n\n```\n\n### Submitting Test Artifacts\n\ntestomat.io does not store any screenshots, logs or other artifacts.\n\nIn order to manage them it is advised to use S3 Buckets (GCP Storage).\nhttps://docs.testomat.io/usage/test-artifacts/\n\nAnalyser needs to be aware of the cloud storage credentials.\nThere are two options:\n1. Enable **Share credentials with testomat.io Reporter** option in testomat.io Settings -\u003e Artifacts.\n2. Use environment variables   `ACCESS_KEY_ID, SECRET_ACCESS_KEY, ENDPOINT, BUCKET, BUCKET_PATH, REGION`\n\nYou would need to decide when you want to upload your test artifacts to cloud storage\n\n1) Upload page screenshot when test fails, using fixtures [reference](https://docs.pytest.org/en/latest/example/simple.html#making-test-result-information-available-in-fixtures)\n\n```python\n# content of conftest.py\nimport pytest\nimport random\nimport os\nfrom typing import Dict\nfrom pytest import StashKey, CollectReport\nfrom playwright.sync_api import Page\n\nphase_report_key = StashKey[Dict[str, CollectReport]]()\n\n@pytest.hookimpl(wrapper=True, tryfirst=True)\ndef pytest_runtest_makereport(item, call):\n    rep = yield\n    item.stash.setdefault(phase_report_key, {})[rep.when] = rep\n    return rep\n\n\n@pytest.fixture(scope=\"function\")\ndef handle_artifacts(page: Page, request):\n    yield\n    report = request.node.stash[phase_report_key]\n    if (\"call\" not in report) or report[\"setup\"].failed or report[\"call\"].failed:\n        random_string = ''.join(random.choices(string.ascii_letters + string.digits, k=8))\n\n        filename = f\"{random_string}.png\"\n        screenshot_path = os.path.join(artifacts_dir, filename)\n        page.screenshot(path=screenshot_path)\n        # file_path - required, path to file to be uploaded\n        # file_bytes - required, bytes of the file to be uploaded\n        # key - required, file name in the s3 bucket\n        # bucket_name - optional,name of the bucket to upload file to. Default value is taken from testomat.io\n        artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)\n        # or\n        # artifact_url = pytest.testomatio.upload_file_object(file_bytes, key, bucket_name)\n        pytest.testomatio.add_artifacts(request.node, [artifact_url])\n    page.close()\n```\n\n⚠️ Please take into account s3_connector available only after **pytest_collection_modifyitems()** hook is executed.\n\n2) If you prefer to use pytest hooks - add `pytest_runtest_makereport` hook in your `conftest.py` file.\n\n```python\ndef pytest_runtest_makereport(item, call):\n    artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)\n    pytest.testomatio.add_artifacts([artifact_url])\n```\n\n### Cross-Platform Testing\nThe plugin supports reporting the same test multiple times in a single run. This is especially useful for Cross-Platform\ntesting, when you run the same test on different environments. \nTo use this feature you need to specify test run environment through \n**TESTOMATIO_ENV** environment variable or by using **--testRunEnv** option.\nExample:\n```bash\nTESTOMATIO=***api_key*** TESTOMATIO_RUN_ID=***run_id*** pytest --testomatio report --testRunEnv \"os:ubuntu, integration\"\nTESTOMATIO=***api_key*** TESTOMATIO_RUN_ID=***run_id*** pytest --testomatio report --testRunEnv \"os:windowns, integration\"\n```\nExecuting these commands will include the tests in the same run, but as separate instances. Each test will contain metadata with information about the test run environment.\n\n**Note**: Only key:value envs will be passed into tests metadata\n\n## Contributing\nUse python 3.12\n\n### Manual testing\n- Run unit tests\n- import into empty project\n- updated test - (resync)\n- test run\n- test run into a folder\n- test run labels, tags\n\n1. `pip install \".[dev]\"`\n1. `python ./smoke.py`\n1. Test things manually\n1. Verify no regression bugs\n1. `cz commit --prerelease beta`\n1. `cz bump`\n1. `git push remoteName branchName --tags`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftestomatio%2Fpytestomatio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftestomatio%2Fpytestomatio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftestomatio%2Fpytestomatio/lists"}