{"id":18980722,"url":"https://github.com/sublimetext/unittesting","last_synced_at":"2025-04-04T23:08:37.341Z","repository":{"id":14562434,"uuid":"17278238","full_name":"SublimeText/UnitTesting","owner":"SublimeText","description":"Testing Sublime Text Packages","archived":false,"fork":false,"pushed_at":"2025-02-26T10:58:54.000Z","size":1061,"stargazers_count":113,"open_issues_count":14,"forks_count":31,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-28T22:13:25.409Z","etag":null,"topics":["actions","coverage-report","sublime-text","unittest"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SublimeText.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-02-28T07:04:32.000Z","updated_at":"2025-02-26T10:54:49.000Z","dependencies_parsed_at":"2024-04-17T19:44:20.133Z","dependency_job_id":"aafe3ac6-d153-4c59-ab67-e732588aa4b6","html_url":"https://github.com/SublimeText/UnitTesting","commit_stats":null,"previous_names":[],"tags_count":103,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SublimeText%2FUnitTesting","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SublimeText%2FUnitTesting/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SublimeText%2FUnitTesting/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SublimeText%2FUnitTesting/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SublimeText","download_url":"https://codeload.github.com/SublimeText/UnitTesting/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247261609,"owners_count":20910108,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actions","coverage-report","sublime-text","unittest"],"created_at":"2024-11-08T16:07:35.728Z","updated_at":"2025-04-04T23:08:37.312Z","avatar_url":"https://github.com/SublimeText.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"UnitTesting\n===========\n\n[![test](https://github.com/SublimeText/UnitTesting/actions/workflows/test.yaml/badge.svg)](https://github.com/SublimeText/UnitTesting/actions/workflows/test.yaml)\n[![codecov](https://codecov.io/gh/SublimeText/UnitTesting/branch/master/graph/badge.svg)](https://codecov.io/gh/SublimeText/UnitTesting)\n\nThis is a unittest framework for Sublime Text.\nIt runs unittest testcases on local machines and via Github Actions.\nIt also supports testing syntax_test files for the new [sublime-syntax](https://www.sublimetext.com/docs/3/syntax.html) \nformat and sublime-color-scheme files.\n\n## Sublime Text 4\n\nSublime Text 4 is now supported and testing works for Python 3.8 packages.\n\n\n## Preparation\n\n1. Install [UnitTesting](https://github.com/SublimeText/UnitTesting) via Package Control.\n2. Your package!\n3. TestCases should be placed in `test*.py` under the directory `tests` \n   (configurable, see below). The testcases are then loaded by [TestLoader.discover](https://docs.python.org/3.3/library/unittest.html#unittest.TestLoader.discover).\n\n[Here](https://github.com/randy3k/UnitTesting-example) are some small examples\n\n\n## Running Tests Locally\n\n### Command Palette\n\n1. Open `Command Palette` using \u003ckbd\u003ectrl+shift+P\u003c/kbd\u003e or menu item `Tools → Command Palette...`\n2. Choose a `Unittesting: ...` command to run and hit \u003ckbd\u003eEnter\u003c/kbd\u003e\n\nTo test any package...\n\n1. run `UnitTesting: Test Package` \n2. enter the package name in the input panel and hit enter.\n\nAn output panel pops up displaying progress and results of running tests. \n\nTo run only tests in particular files, enter `\u003cPackage name\u003e:\u003cfilename\u003e`.\n`\u003cfilename\u003e` should be a unix shell wildcard to match the file names.\n`\u003cPackage name\u003e:test*.py` is used by default.\n\nThe command `UnitTesting: Test Current Package` runs all tests \nof the current package the active view's file is part of.\nThe package is reloaded to pickup any code changes and then tests are executed.\n\nThe command `UnitTesting: Test Current Package with Coverage`\nruns tests for current package and generates a coverage report via [coverage](https://pypi.python.org/pypi/coverage).\nThe [.coveragerc](.coveragerc) file is used to control coverage configurations. \nIf it is missing, UnitTesting will ignore the `tests` directory.\n\n\u003e [!NOTE]\n\u003e\n\u003e As of Unittesting 1.8.0 the following commands have been replaced\n\u003e to enable more flexible usage and integration in build systems.\n\u003e\n\u003e unit_testing_current_package\n\u003e\n\u003e   ```json\n\u003e   { \"command\": \"unit_testing\", \"package\": \"$package_name\" }\n\u003e   ```\n\u003e\n\u003e unit_testing_current_file\n\u003e\n\u003e   ```json\n\u003e   { \"command\": \"unit_testing\", \"package\": \"$package_name\", \"pattern\": \"$file_name\" }\n\u003e   ```\n\n\n### Build System\n\nTo run tests via build system specify `unit_testing` build system `\"target\"`.\n\n```json\n{\n  \"target\": \"unit_testing\"\n}\n```\n\n\n### Project specific `Test Current Package` build command\n\nIt is recommended to add the following to _.sublime-project_ file \nso that \u003ckbd\u003ectrl\u003c/kbd\u003e+\u003ckbd\u003eb\u003c/kbd\u003e would invoke the testing action.\n\n```json\n\"build_systems\":\n[\n  {\n    \"name\": \"Test Current Package\",\n    \"target\": \"unit_testing\",\n    \"package\": \"$package_name\",\n    \"failfast\": true\n  }\n]\n```\n\n\n### Project specific `Test Current File` build command\n\nIt is recommended to add the following to _.sublime-project_ file \nso that \u003ckbd\u003ectrl\u003c/kbd\u003e+\u003ckbd\u003eb\u003c/kbd\u003e would invoke the testing action.\n\n```json\n\"build_systems\":\n[\n  {\n    \"name\": \"Test Current File\",\n    \"target\": \"unit_testing\",\n    \"package\": \"$package_name\",\n    \"pattern\": \"$file_name\",\n    \"failfast\": true\n  }\n]\n```\n\n\n## GitHub Actions\n\nUnittesting provides the following GitHub Actions, which can be combined\nin a workflow to design package tests.\n\n1. SublimeText/UnitTesting/actions/setup\n   \n   _Setup Sublime Text to run tests within._\n\n   \u003e This must always be the first step after checking out the package to test.\n\n2. SublimeText/UnitTesting/actions/run-color-scheme-tests\n  \n   _Test color schemes using [ColorSchemeUnit](https://packagecontrol.io/packages/ColorSchemeUnit)._\n\n3. SublimeText/UnitTesting/actions/run-syntax-tests\n\n   _Test sublime-syntax definitions using built-in syntax test functions of already running Sublime Text environment._\n\n   \u003e It is an alternative to [SublimeText/syntax-test-action](https://github.com/SublimeText/syntax-test-action) \n   \u003e or sublimehq's online syntax_test_runner\n\n4. SublimeText/UnitTesting/actions/run-tests\n\n   _Runs the `unit_testing` command to perform python unit tests._\n\n\u003e [!NOTE]\n\u003e \n\u003e actions are released in the branch [`v1`](https://github.com/SublimeText/UnitTesting/tree/v1). \n\u003e Minor changes will be pushed to the same branch unless there are breaking changes.\n\n\n### Color Scheme Tests\n\nTo integrate color scheme tests via [ColorSchemeUnit](https://packagecontrol.io/packages/ColorSchemeUnit) \nadd the following snippet to a workflow file\n(e.g. `.github/workflows/color-scheme-tests.yml`).\n\n```yaml\nname: ci-color-scheme-tests\n\non: [push, pull_request]\n\njobs:\n  run-syntax-tests:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: SublimeText/UnitTesting/actions/setup@v1\n      - uses: SublimeText/UnitTesting/actions/run-color-scheme-tests@v1\n```\n\n\n### Syntax Tests\n\nTo run only syntax tests add the following snippet to a workflow file\n(e.g. `.github/workflows/syntax-tests.yml`).\n\n```yaml\nname: ci-syntax-tests\n\non: [push, pull_request]\n\njobs:\n  run-syntax-tests:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: SublimeText/UnitTesting/actions/setup@v1\n      - uses: SublimeText/UnitTesting/actions/run-syntax-tests@v1\n```\n\n\u003e [!NOTE]\n\u003e\n\u003e If you are looking for syntax tests only, you may also checkout [SublimeText/syntax-test-action](https://github.com/SublimeText/syntax-test-action).\n\u003e Using this test makes most sense to just re-use an already set-up ST test environment.\n\n### Unit Tests\n\nTo run only python unit tests on all platforms and versions of Sublime Text\nadd the following snippet to a workflow file (e.g. `.github/workflows/unit-tests.yml`).\n\n```yaml\nname: ci-unit-tests\n\non: [push, pull_request]\n\njobs:\n  run-tests:\n    strategy:\n      fail-fast: false\n      matrix:\n        st-version: [3, 4]\n        os: [\"ubuntu-latest\", \"macOS-latest\", \"windows-latest\"]\n    runs-on: ${{ matrix.os }}\n    steps:\n      - uses: actions/checkout@v4\n      - uses: SublimeText/UnitTesting/actions/setup@v1\n        with:\n          package-name: Package Name   # if differs from repo name\n          sublime-text-version: ${{ matrix.st-version }}\n      - uses: SublimeText/UnitTesting/actions/run-tests@v1\n        with:\n          coverage: true\n          package-name: Package Name   # if differs from repo name\n      - uses: codecov/codecov-action@v4\n```\n\n\n### Run All Tests\n\n```yaml\nname: ci-tests\n\non: [push, pull_request]\n\njobs:\n  run-tests:\n    strategy:\n      fail-fast: false\n      matrix:\n        st-version: [3, 4]\n        os: [\"ubuntu-latest\", \"macOS-latest\", \"windows-latest\"]\n    runs-on: ${{ matrix.os }}\n    steps:\n      # checkout package to test\n      - uses: actions/checkout@v4\n\n      # setup test environment\n      - uses: SublimeText/UnitTesting/actions/setup@v1\n        with:\n          sublime-text-version: ${{ matrix.st-version }}\n\n      # run color scheme tests (only on Linux)\n      - if: ${{ matrix.os == 'ubuntu-latest' }}\n        uses: SublimeText/UnitTesting/actions/run-color-scheme-tests@v1\n      \n      # run syntax tests and check compatibility with new syntax engine (only on Linux)\n      - if: ${{ matrix.os == 'ubuntu-latest' }}\n        uses: SublimeText/UnitTesting/actions/run-syntax-tests@v1\n        with:\n          compatibility: true\n      \n      # run unit tests with coverage upload\n      - uses: SublimeText/UnitTesting/actions/run-tests@v1\n        with:\n          coverage: true\n          extra-packages: |\n            A File Icon:SublimeText/AFileIcon\n      - uses: codecov/codecov-action@v4\n```\n\nCheck [this](https://github.com/randy3k/UnitTesting-example) for further examples.\n\n\n## Options\n\n### Package Configuration\n\nUnitTesting is primarily configured by `unittesting.json` file in package root directory.\n\n```json\n{\n  \"verbosity\": 1,\n  \"coverage\": true\n}\n```\n\n### Build System Configuration\n\nOptions provided via build system configuration override `unittesting.json`.\n\n```json\n{\n  \"target\": \"unit_testing\",\n  \"package\": \"$package_name\",\n  \"verbosity\": 2,\n  \"coverage\": true\n}\n```\n\n### Command Arguments\n\nOptions passed as arguments to `unit_testing` command override `unittesting.json`.\n\n```py\nwindow.run_command(\"unit_testing\", {\"package\": \"$package_name\", \"coverage\": False})\n```\n\n### Available Options\n\n| name                        | description                                                  | default value |\n| --------------------------- | ------------------------------------------------------------ | ------------- |\n| tests_dir                   | the name of the directory containing the tests               | \"tests\"       |\n| pattern                     | the pattern to discover tests                                | \"test*.py\"    |\n| deferred                    | whether to use deferred test runner                          | true          |\n| condition_timeout           | default timeout in ms for callables invoked via `yield`      | 4000          |\n| failfast                    | stop early if a test fails                                   | false         |\n| output                      | name of the test output instead of showing \u003cbr\u003e in the panel | null          |\n| verbosity                   | verbosity level                                              | 2             |\n| warnings                    | The warnings filter controls python warnings treatment.      | \"default\"     |\n| capture_console             | capture stdout and stderr in the test output                 | false         |\n| reload_package_on_testing   | reloading package will increase coverage rate                | true          |\n| coverage                    | track test case coverage                                     | false         |\n| coverage_on_worker_thread   | (experimental)                                               | false         |\n| generate_html_report        | generate HTML report for coverage                            | false         |\n| generate_xml_report         | generate XML report for coverage                             | false         |\n\nValid `warnings` values are:\n\n| Value     | Disposition\n| --------- | -----------\n| \"default\" | print the first occurrence of matching warnings for each location (module + line number) where the warning is issued\n| \"error\"   | turn matching warnings into exceptions\n| \"ignore\"  | never print matching warnings\n| \"always\"  | always print matching warnings\n| \"module\"  | print the first occurrence of matching warnings for each module where the warning is issued (regardless of line number)\n| \"once\"    | print only the first occurrence of matching warnings, regardless of location\n\nsee also: https://docs.python.org/3/library/warnings.html#warning-filter\n\n## Writing Unittests\n\nUnitTesting is based on python's `unittest` library. \nAny valid unittest test case is allowed.\n\nExample:\n\n_tests/test_myunit.py_\n\n```py\nfrom unittest import TestCase\n\nclass MyTestCase(TestCase):\n\n  def test_something(self):\n    self.assertTrue(True)\n```\n\n\n### Deferred testing\n\nTests can be written using deferrable test cases to test results \nof asynchronous or long lasting sublime commands, which require yielding\ncontrol to sublime text runtime and resume test execution at a later point.\n\nIt is a kind of cooperative multithreading such as provided by `asyncio`,\nbut with a home grown [DeferringTextTestRunner][2] acting as event loop.\n\nThe idea was inspired by [Plugin UnitTest Harness](https://bitbucket.org/klorenz/sublimepluginunittestharness).\n\n[DeferrableTestCase][1] is used to write the test cases. They are executed by\nthe [DeferringTextTestRunner][2] and the runner expects not only regular test\nfunctions, but also generators. If the test function is a generator, it does\nthe following\n\n- if the yielded object is a callable, the runner will evaluate the\n  [callable][3] and check its returned value. If the result is not `None`, \n  the runner continues the generator, if not, the runner will wait until the\n  condition is met with the default timeout of 4s. The result of the callable\n  can be also retrieved from the `yield` statement. The yielded object could \n  also be a dictionary of the form \n\n  ```py\n  {\n    # required condition callable\n    \"condition\": callable,\n    # system timestamp when to start condition checks (default: `time.time()`)\n    \"start_time\": timestamp,\n    # optional the interval to invoke `condition()` (default: 17)\n    \"period\": milliseconds,\n    # optional timeout to wait for condition to be met (default: value from unittesting.json or 4000)\n    \"timeout\": milliseconds,\n    # optional message to print, if condition is not met within timeout\n    \"timeout_message\": \"Condition not fulfilled\"\n  }\n  ```\n\n  to specify various overrides such as poll interval or timeout in ms.\n\n- if the yielded object is an integer, say `x`, then it will [continue][4] the\n  generator after `x` ms.\n\n- `yield AWAIT_WORKER` would yield to a task in the worker thread.\n\n- otherwise, a single `yield` would yield to a task in the main thread.\n\nExample:\n\n```py\nimport sublime\nfrom unittesting import DeferrableTestCase\n\n\nclass TestCondition(DeferrableTestCase):\n\n    def test_condition1(self):\n        x = []\n\n        def append():\n            x.append(1)\n\n        def condition():\n            return len(x) == 1\n\n        sublime.set_timeout(append, 100)\n\n        # wait until `condition()` is true\n        yield condition\n\n        self.assertEqual(x[0], 1)\n\n    def test_condition2(self):\n        x = []\n\n        def append():\n            x.append(1)\n\n        def condition():\n            return len(x) == 1\n\n        sublime.set_timeout(append, 100)\n\n        # wait until `condition()` is true\n        yield {\n          \"condition\": condition,\n          \"period\": 200,\n          \"timeout\": 5000,\n          \"timeout_message\": \"Not enough items added to x\"\n        }\n\n        self.assertEqual(x[0], 1)\n```\n\nsee also [tests/test_defer.py](https://github.com/randy3k/UnitTesting-example/blob/master/tests/test_defer.py).\n\n## Helper TestCases\n\nUnitTesting provides some helper test case classes, \nwhich perform common tasks such as overriding preferences, setting up views, etc.\n\n- DeferrableViewTestCase\n- OverridePreferencesTestCase\n- TempDirectoryTestCase\n- ViewTestCase\n\nUsage and some examples are available via docstrings, which are displayed as hover popup by [LSP](https://packagecontrol.io/packages/LSP) and e.g. [LSP-pyright](https://packagecontrol.io/packages/LSP-pyright).\n\n## Credits\n\nThanks [guillermooo](https://github.com/guillermooo) and [philippotto](https://github.com/philippotto) for their early efforts in AppVeyor and Travis CI macOS support (though these services are not supported now).\n\n\n[1]: https://github.com/SublimeText/UnitTesting/blob/60e15d42d6ff96156408aec1999d6a16ddcf8e03/unittesting/core/py33/case.py#L22\n[2]: https://github.com/SublimeText/UnitTesting/blob/60e15d42d6ff96156408aec1999d6a16ddcf8e03/unittesting/core/py33/runner.py#L21\n[3]: https://github.com/SublimeText/UnitTesting/blob/60e15d42d6ff96156408aec1999d6a16ddcf8e03/unittesting/core/py33/runner.py#L65\n[4]: https://github.com/SublimeText/UnitTesting/blob/60e15d42d6ff96156408aec1999d6a16ddcf8e03/unittesting/core/py33/runner.py#L72\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsublimetext%2Funittesting","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsublimetext%2Funittesting","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsublimetext%2Funittesting/lists"}