{"id":21721370,"url":"https://github.com/informaticsmatters/squonk2-data-manager-job-tester","last_synced_at":"2026-02-13T10:05:29.671Z","repository":{"id":44677940,"uuid":"447668585","full_name":"InformaticsMatters/squonk2-data-manager-job-tester","owner":"InformaticsMatters","description":"A test utility for Data Manager Job repositories","archived":false,"fork":false,"pushed_at":"2025-06-02T14:29:10.000Z","size":179,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-19T18:46:17.949Z","etag":null,"topics":["squonk2"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InformaticsMatters.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-01-13T16:20:47.000Z","updated_at":"2025-06-02T14:25:48.000Z","dependencies_parsed_at":"2024-07-31T09:07:07.677Z","dependency_job_id":"a53ac89a-649e-4db0-8c9f-f6cf5359117a","html_url":"https://github.com/InformaticsMatters/squonk2-data-manager-job-tester","commit_stats":{"total_commits":113,"total_committers":2,"mean_commits":56.5,"dds":0.008849557522123908,"last_synced_commit":"edc867cecd59543110603950d306e98d32fffa52"},"previous_names":["informaticsmatters/data-manager-job-tester"],"tags_count":64,"template":false,"template_full_name":null,"purl":"pkg:github/InformaticsMatters/squonk2-data-manager-job-tester","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InformaticsMatters%2Fsquonk2-data-manager-job-tester","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InformaticsMatters%2Fsquonk2-data-manager-job-tester/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InformaticsMatters%2Fsquonk2-data-manager-job-tester/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InformaticsMatters%2Fsquonk2-data-manager-job-tester/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InformaticsMatters","download_url":"https://codeload.github.com/InformaticsMatters/squonk2-data-manager-job-tester/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InformaticsMatters%2Fsquonk2-data-manager-job-tester/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272024519,"owners_count":24860527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-25T02:00:12.092Z","response_time":1107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["squonk2"],"created_at":"2024-11-26T02:16:10.089Z","updated_at":"2026-02-13T10:05:29.589Z","avatar_url":"https://github.com/InformaticsMatters.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Informatics Matters Job Tester (\"jote\")\n=======================================\n\n.. image:: https://badge.fury.io/py/im-jote.svg\n   :target: https://badge.fury.io/py/im-jote\n   :alt: PyPI package (latest)\n\n.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-tester/actions/workflows/build.yaml/badge.svg\n   :target: https://github.com/InformaticsMatters/squonk2-data-manager-job-tester/actions/workflows/build.yaml\n   :alt: Build\n\n.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-tester/actions/workflows/publish.yaml/badge.svg\n   :target: https://github.com/InformaticsMatters/squonk2-data-manager-job-tester/actions/workflows/publish.yaml\n   :alt: Publish\n\nThe **Squonk2 Job Tester** (``jote``) is a Python utility used to run *unit tests*\nthat are defined in Data Manager *job implementation repositories* against\nthe job's container image, images that are typically built from the same\nrepository.\n\n``jote`` is designed to run job implementations in a file-system\nenvironment that replicates what they find when they're run by the Data Manager.\nBut jobs *are not* running in the same operating-system environment, e.g. they\nare not bound by the same processor and memory constraints they'll encounter in\nthe Data Manager, which runs in `Kubernetes`_.\n\nTo use ``jote`` you will need to install ``docker-compose`` (v1 or v2).\n\nA successful test should give you confidence that it *should* work in the\nData Manger but without writing a lot of tests you'll never be completely\nconfident that it will always run successfully.\n\n``jote`` is a tool we designed to provide *us* with confidence that we can\ndeploy jobs to a Data Manager instance and know that they're basically fit\nfor purpose. Jobs that have no tests will not normally be deployed to the\nData Manager.\n\nTo use a job in Squonk you need to create at least one **manifest file** and\none **job definition file**. These reside in the ``data-manager``\ndirectory of the repository you're going to test. ``jote`` expects the\ndefault manifest file to be called ``manifest.yaml`` but you can use a\ndifferent name and have more than one.\n\n    If you want to provide your own Squonk jobs and corresponding\n    job definitions our **Virtual Screening** repository\n    (https://github.com/InformaticsMatters/virtual-screening) is a good\n    place to start. The repository is host to a number of job-based\n    container images and several *manifests* and *job definition files*.\n\nHere's an example **manifest** from a recent **Virtual Screening** repository::\n\n    ---\n    kind: DataManagerManifest\n    kind-version: '2021.1'\n\n    job-definition-files:\n    - im-virtual-screening.yaml\n    - rdkit.yaml\n    - xchem.yaml\n\nEach Manifest must list at least one file. To be included in Squonk every\njob must contain at least one test. ``jote`` runs the tests but also ensures\nthe repository structure is as expected and applies strict rules for the\nformatting of the YAML files.\n\nBoth ``jote`` and the Data Manager rely on the schemas that can be found\nin our **Job Decoder** repository\n(https://github.com/InformaticsMatters/data-manager-job-decoder).\n\nHere's a snippet from a job definition file illustrating a\njob (``max-min-picker``) that has a test called ``simple-execution``.\n\nThe test defines an input option (a file) and some other command options.\nThe ``checks`` section is used to define the exit criteria of the test.\nIn this case the container must exit with code ``0`` and the file\n``diverse.smi`` must be found in the generated test directory, i.e\nit must *exist* and contain ``100`` lines. ``jote`` will fail the test unless\nthese checks are satisfied::\n\n    jobs:\n      [...]\n      max-min-picker:\n        [...]\n        tests:\n          simple-execution:\n            inputs:\n              inputFile: data/100000.smi\n            options:\n              outputFile: diverse.smi\n              count: 100\n            checks:\n              exitCode: 0\n              outputs:\n              - name: diverse.smi\n                checks:\n                - exists: true\n                - lineCount: 100\n\n.. _kubernetes: https://kubernetes.io/\n\nRunning tests\n-------------\n\nRun ``jote`` from the root of a clone of the Data Manager Job implementation\nrepository that you want to test::\n\n    jote\n\nYou can display the utility's help with::\n\n    jote --help\n\nJote container network\n----------------------\n\n``jote`` tests are executed on the network ``data-manager_jote``. This is\ndefined in the docker-compose file that it generates to run your tests.\n\nBuilt-in variables\n------------------\n\nJob definition command-expansion provided by the **job decoder**\nrelies on a number of *built in* variables. Some are provided by the\nData Manager when the job runs under its control\n(i.e. ``DM_INSTANCE_DIRECTORY``) others are provided by ``jote`` to simplify\ntesting.\n\nThe set of variables injected into the command expansion by ``jote``\nare: -\n\n- ``DM_INSTANCE_DIRECTORY``. Set to the path of the simulated instance\n  directory created by ``jote``, normally created by the Data Manager\n- ``CODE_DIRECTORY``. Set to the root of the repository that you're running\n  the tests in. This is a convenient variable to locate your out-of-container\n  nextflow workflow file, which is likely to be in the root of your repository\n\nIgnoring tests\n--------------\n\nOccasionally you may want to disable some tests because they need some work\nbefore they're complete. To allow you to continue testing other jobs under\nthese circumstances you can mark individual tests and have them excluded\nby adding an ``ignore`` declaration::\n\n    jobs:\n      [...]\n      max-min-picker:\n        [...]\n        tests:\n          simple-execution:\n            ignore:\n            [...]\n\nYou don't have to remove the ``ignore`` declaration to run the test in ``jote``.\nIf you want to see whether an ignored test now works you can run ``jote``\nfor specific tests by using ``--test`` and naming the ignored test you want\nto run. When a test is named explicitly it is run, regardless of whether\n``ignore`` has been set or not.\n\nTest run levels\n---------------\n\nTests can be assigned a ``run-level``. Run-levels are numerical value (1..100)\nthat can be used to group your tests. You can use the ``run-level``\nas an indication of execution time, with short tests having low values and\ntime-consuming tests with higher values.\n\nBy default all tests that have no run-level defined and those with\na run-level of ``1`` are executed.  If you set the run-level for longer-running\ntests to a higher value, e.g. ``5``, these will be skipped. To run these more\ntime-consuming tests you specify the run-level when running ``jote``\nusing ``--run-level 5``.\n\n    When you give ``jote`` a run-level only tests up to and including the\n    level, and those without any run-level, will be run.\n\nYou define the run-level in the root block of the job's test specification::\n\n    jobs:\n      [...]\n      max-min-picker:\n        [...]\n        tests:\n          simple-execution:\n            run-level: 5\n            [...]\n\nTest timeouts\n-------------\n\n``jote`` lets each test run for 10 minutes before cancelling (and failing) them.\nIf you expect that your test needs to run for more than 10 minutes you must\nuse the ``timeout-minutes`` property in the job definition to define your own\ntest-specific value::\n\n    jobs:\n      [...]\n      max-min-picker:\n        [...]\n        tests:\n          simple-execution:\n            timeout-minutes: 120\n            [...]\n\nYou should try and avoid creating too many long-running tests. If you cannot,\nconsider whether it's a appropriate to use ``run-level`` to avoid ``jote``\nrunning them by default.\n\nTest groups\n-----------\n\nTests are normally executed and the environment torn-down between them.\nIf you have tests that depend on the results from a prior test you can run\ntests as a **group**, which preserves the project directory between the tests.\n\nTo run a sequence of test (as a **group**) you need to define a ``test-group``\nin your Job Definition file and then refer to that group in your test. Here,\nwe define a test group called ``experiment-a``, at the top of the\ndefinition file::\n\n    test-groups:\n    - name: experiment-a\n\n\nWe then place a test in that group with a ``run-group`` declaration\nin the corresponding test block::\n\n    jobs:\n      max-min-picker:\n        [...]\n        tests:\n          test-a:\n            run-groups:\n            - name: experiment-a\n              ordinal: 1\n\nWe need to provide an ``ordinal`` value. This numeric value (from 1 ..N)\nputs the test in a specific position in the test sequence. When tests are\nplaced in a ``run-group`` you have to order your tests, i.e. declare that\n``test-a`` follows ``test-b``. This is done with unique ordinals for each\ntest in the ``run-group``. A test with ordinal ``1`` will run before a test\nwith ordinal ``2``. Ordinals have to be unique within a ``run-group``.\n\nYou can run the tests for a specific group by using  the ``--run-group``\noption::\n\n    jote --run-group experiment-a\n\nRunning additional containers (group testing)\n---------------------------------------------\n\nTest groups provide an ability to launch additional support containers during\ntesting. You might want to start a background database for example, that can\nbe used by tests in your ``test-group``. To take advantage of this feature\nyou just need to provide a ``docker-compose`` file (in the Job definition\n``data-manager`` directory) and name that file in you r``test-groups``\ndeclaration.\n\nHere we declare a docker-compose file called\n``docker-compose-experiment-a.yaml``::\n\n    test-groups:\n    - name: experiment-a\n      compose:\n        file: docker-compose-experiment-a.yaml\n\nThe compose filename must begin ``docker-compose`` and end ``.yaml``.\n\nThe compose file is run before any tests in the corresponding test group\nhave been run and will be stopped after the last test in the group.\n\nThe compose file you provide is run in a *detached* state so ``jote`` does\nnot wait for the containers to start (or initialise). As the first test\nin the test group can begin very soon after the compose file is started\nyou can minimise the risk that your containers are not ready for the tests\nby adding a fixed delay between ``jote`` starting the compose file and\nrunning the first test::\n\n    test-groups:\n    - name: experiment-a\n      compose:\n        file: docker-compose-experiment-a.yaml\n        delay-seconds: 10\n\nNextflow test execution\n-----------------------\n\nJob image types can be ``simple`` or ``nextflow``. Simple jobs are executed in\nthe container image you've built and should behave much the same as they do\nwhen run within the Data Manager. Nextflow jobs on the other hand are executed\nusing the shell, relying on Docker as the execution run-time for the processes\nin your workflow.\n\nBe aware that nextflow tests run by ``jote`` run under different conditions\ncompared to when it runs under the Data Manager's control. In the Data Manager\nnextflow jobs will be executed within a Kubernetes environment. When run by ``jote``\nnextflow is expected using the operating system shell. This introduces a\nvariability that you need to take into account - i.e. under ``jote`` the\nnextflow controller runs in the shell, and *are not* executed in the same\nenvironment or under the same memory or processor constraints.\n\nYou might need to provide a custom nextflow configuration file\nfor your tests to run successfully. You do this by adding a ``nextflow-config-file``\ndeclaration in the test. Here, we name the file ``nextflow-test.config``::\n\n    jobs:\n      max-min-picker:\n        [...]\n        tests:\n          simple-load:\n            nextflow-config-file: nextflow-test.config\n            [...]\n\nThe config file must be located in the Job repository's ``data-manager``\ndirectory.\n\nPrior to running the corresponding test ``jote`` copies it to the\nJob's project directory as the file ``nextflow.config`` (a standard file\nexpected by nextflow).\n\n``jote`` *will not* let you have a nextflow config in your home directory\nas any settings found there would be merged with the file ``jote`` writes,\npotentially disturbing the execution behaviour.\n\n.. note::\n   It's your responsibility to install a suitable nextflow that's available\n   for shell execution. ``jote`` expects to be able to run nextflow when\n   executing the corresponding ``command`` that's defined in the job\n   definition.\n\nInstallation\n============\n\n``jote`` is published on `PyPI`_ and can be installed from there::\n\n    pip install im-jote\n\nThis is a Python 3 utility, so try to run it from a recent (ideally 3.10)\nPython environment.\n\nTo use the utility you will need to have installed `Docker`_, `docker-compose`,\n and, if you want to test nextflow jobs, `nextflow`_.\n\n.. _PyPI: https://pypi.org/project/im-jote/\n.. _Docker: https://docs.docker.com/get-docker/\n.. _nextflow: https://www.nextflow.io/\n\nGet in touch\n------------\n\n- Report bugs, suggest features or view the source code `on GitHub`_.\n\n.. _on GitHub: https://github.com/informaticsmatters/squonk2-data-manager-job-tester\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finformaticsmatters%2Fsquonk2-data-manager-job-tester","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finformaticsmatters%2Fsquonk2-data-manager-job-tester","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finformaticsmatters%2Fsquonk2-data-manager-job-tester/lists"}