https://github.com/jsfehler/pytest-testrail2

A pytest plugin to upload results to TestRail.
https://github.com/jsfehler/pytest-testrail2

pytest python python3 testing testrail testrail-reporting

Last synced: 8 months ago
JSON representation

A pytest plugin to upload results to TestRail.

• Host: GitHub
• URL: https://github.com/jsfehler/pytest-testrail2
• Owner: jsfehler
• License: mit
• Created: 2022-11-23T16:19:16.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2023-02-10T16:27:42.000Z (over 3 years ago)
• Last Synced: 2025-03-16T00:14:11.232Z (about 1 year ago)
• Topics: pytest, python, python3, testing, testrail, testrail-reporting
• Language: Python
• Homepage:
• Size: 96.7 KB
• Stars: 2
• Watchers: 3
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.rst
  • Changelog: CHANGELOG.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
++++++++++++++++

pytest-testrail2

++++++++++++++++

Create and update testplans and/or testruns in TestRail based on pytest results.

Test functions marked with a TestRail case ID will have their results sent to TestRail.

This project preserves the functionality of the deprecated `pytest-testrail `_ project.

.. image:: https://img.shields.io/pypi/v/pytest-testrail2.svg

    :target: https://pypi.org/project/pytest-testrail2

    :alt: PyPI

.. image:: https://img.shields.io/pypi/pyversions/pytest-testrail2.svg

    :alt: PyPI - Python Version

    :target: https://github.com/jsfehler/pytest-testrail2

.. image:: https://img.shields.io/github/license/jsfehler/pytest-testrail2.svg

   :alt: GitHub

   :target: https://github.com/jsfehler/pytest-testrail2/blob/master/LICENSE

.. image:: https://github.com/jsfehler/pytest-testrail2/workflows/CI/badge.svg

   :target: https://github.com/jsfehler/pytest-testrail2/actions/workflows/main.yml

   :alt: Build status

Key features:

- Configurable: Handle results in whichever way makes sense for your project.

- Powerful: Handle complex use cases via a simple interface to the TestRail API.

- Flexible: Fully compatible with pytest.parametrize(), pytest-xdist, and pytest-rerunfailures.

Example

=======

.. code:: python

  import pytest

  @pytest.mark.case_id('C1950')

  def test_all_the_things():

    doit = True

    assert doit

Getting Started

===============

Installation

------------

Via pip:

.. code-block:: bash

   pip install pytest-testrail2

Plugin Configuration

--------------------

The following values are required:

- A valid TestRail instance URL.

- A valid email address for a user on the instance.

- A valid API key for the user.

They can be set on the command line via the follow flags:

.. code-block:: bash

  --tr-url=

  --tr-email=

  --tr-password=

Alternatively, they can be set via a pytest configuration file:

.. code-block:: ini

  [pytest]

    tr_url=

    tr_email=

    tr_password=

Marking Tests

-------------

case_id

~~~~~~~

The `case_id` marker takes a string which must match an existing TestRail testcase.

Only tests with this marker will be added to the TestRail testrun.

.. code:: python

  import pytest

  # This test's results will be uploaded.

  @pytest.mark.case_id('C1950')

  def test_all_the_things():

    ...

  # This test's results will not be uploaded.

  def test_all_the_other_things():

    ...

defect_ids

~~~~~~~~~~

The 'defect_ids' marker takes a list of strings. These will be used in the `defect`

field in TestRail. This is useful for tests with known failures.

Typically these are IDs for your bug tracking software.

.. code:: python

  import pytest

  @pytest.mark.case_id('C1950')

  @pytest.mark.defect_ids(['JS-7001', 'JS-9001'])

  def test_all_the_things():

    ...

Running Pytest

--------------

The `--testrail` command-line flag must be present to upload results:

.. code-block:: bash

  pytest --testrail

Options

=======

Setup

-----

- ``--testrail``

  Activate the TestRail plugin.

- ``--tr-url``

  Web address used to access a TestRail instance.

- ``--tr-email``

  E-mail address for an account on the TestRail instance.

- ``--tr-password``

  Password for an account on the TestRail instance.

- ``--tr-timeout``

  Timeout for connecting to a TestRail server.

- ``--tr-no-ssl-cert-check``

  Do not check for valid SSL certificate on TestRail host.

Testrun

-------

- ``--tr-run-id``

  ID of an existing testrun in TestRail.

  If specified, the testrun matching the ID will be used instead of creating a new testrun.

  If given, ``--tr-testrun-name`` will be ignored.

- ``--tr-testrun-name``

  Name used for a new testrun in TestRail.

- ``--tr-testrun-description``

  Description used for a new testrun in TestRail.

- ``--tr-testrun-assignedto-id``

  ID of the user to be assigned to the testrun.

- ``--tr-testrun-project-id``

  ID of the project the testrun will be created in.

- ``--tr-testrun-suite-id``

  ID of the suite the testrun will be created in.

- ``--tr-testrun-suite-include-all``

  Include all test cases in the specified testsuite for a new testrun.

- ``--tr-milestone-id``

  ID of milestone used in testrun creation.

- ``--tr-skip-missing``

  Skip pytest test functions with marks that are not present in a specified testrun.

Testplan

--------

- ``--tr-plan-id``

  ID of an existing testplan to use. If given, ``--tr-testrun-name`` will be ignored.

Publishing

----------

- ``--tr-version``

  Specify a version in testcase results.

- ``--tr-close-on-complete``

  On pytest completion, close the testrun.

- ``--tr-dont-publish-blocked``

  Do not publish results of "blocked" testcases (in TestRail).

- ``--tr-custom-comment``

  Custom text appended to comment for all testcase results.