{"id":24443784,"url":"https://github.com/endlessm/eos-openqa-tests","last_synced_at":"2026-04-10T10:02:14.621Z","repository":{"id":54687968,"uuid":"140021797","full_name":"endlessm/eos-openqa-tests","owner":"endlessm","description":"Integration tests to run on Endless OS in OpenQA","archived":false,"fork":false,"pushed_at":"2023-02-04T03:29:36.000Z","size":214405,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-04-30T16:49:52.553Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/endlessm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-07-06T19:15:07.000Z","updated_at":"2025-04-05T21:05:50.000Z","dependencies_parsed_at":"2023-02-17T14:01:21.678Z","dependency_job_id":null,"html_url":"https://github.com/endlessm/eos-openqa-tests","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/endlessm/eos-openqa-tests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endlessm%2Feos-openqa-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endlessm%2Feos-openqa-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endlessm%2Feos-openqa-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endlessm%2Feos-openqa-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/endlessm","download_url":"https://codeload.github.com/endlessm/eos-openqa-tests/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endlessm%2Feos-openqa-tests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31637747,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T07:40:12.752Z","status":"ssl_error","status_checked_at":"2026-04-10T07:40:11.664Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-20T22:17:34.581Z","updated_at":"2026-04-10T10:02:14.586Z","avatar_url":"https://github.com/endlessm.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"Endless OS OpenQA tests\n===\n\nThis repository contains a set of tests for Endless OS which get run on our\nOpenQA instance (https://openqa.endlessm.com/) for every OS image build\nproduced by our image builder.\n\nDocumentation\n---\n\nThe bulk of the documentation for these tests is in the wiki, which provides an\noverview and a guide to writing new tests:\n * [EOS OpenQA overview](https://phabricator.endlessm.com/w/software/build/openqa/)\n * [Writing new tests](https://phabricator.endlessm.com/w/software/build/openqa/writing-tests/)\n\nOpenQA itself provides various pieces of documentation:\n * [Documentation as a single page](http://open.qa/docs/)\n * [Documentation overview](http://open.qa/documentation/)\n * [Test API](http://open.qa/api/testapi/)\n\nTest naming\n---\n\nAny test named with a leading underscore is part of the setup/teardown\narchitecture, rather than testing a specific feature. Some combination of these\ntests is run as part of every test job.\n\nAny test with `desktop_` as a prefix is a simple smoketest for launching a\nspecific application from the desktop and checking its basic functionality\n(typically, that it starts and its main window looks correct).\n\nVariables\n---\n\nEach test scheduled by OpenQA is essentially a dictionary of variables which\ncontrol how the test is run. Many of these variables are well-known to OpenQA,\nand control which OS image and worker are used to run the test, and how the VM\nis set up.\n\nSome of these variables are documented in the\n[OpenQA documentation](http://open.qa/docs/#_test_suites) and\n[os-autoinst documentation](https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc).\n\nOther variables are well-known to the test harness, and control which test\nmodules are loaded from the `tests/` directory. These affect the control flow\nin `main.pm`, and are documented below. Common OpenQA/os-autoinst variables have\nbeen included too, for convenience.\n\n| Variable | Values allowed | Default value | Description |\n| -------- | -------------- | ------------- | ----------- |\n| `ARCH` | `x86_64`, `arm` | Undefined | Architecture the image was built for and the test will be run on. Understood by os-autoinst. |\n| `BOOTFROM` | String | Undefined | Sets the boot order for QEMU. See documentation for `qemu -boot` for accepted values and their meanings. Understood by os-autoinst. |\n| `BUILD` | String | Undefined | `EIB_BUILD_VERSION` from eos-image-builder. Understood by OpenQA. |\n| `DISTRI` | String | Undefined | `EIB_PRODUCT` from eos-image-builder. Understood by OpenQA. |\n| `EOS_IMAGE_FLATPAK_APPS` | Space-separated list of flatpak refs | Empty | List of flatpak apps preinstalled on the image. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_FLATPAK_REMOTES` | Space-separated list of flatpak remote names | Empty | List of flatpak remotes preconfigured on the image. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_FLATPAK_RUNTIMES` | Space-separated list of flatpak refs | Empty | List of flatpak runtimes preinstalled on the image. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_FLATPAK_LOCALES` | Space-separated list of flatpak locales | Empty | List of locales downloaded when pulling flatpak refs on the image. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_LANGUAGE` | `LANG` variable value | Undefined | Language the image is configured to use by default. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_OSTREE_COLLECTION_ID` | OSTree collection ID | Undefined | Collection ID of the collection–ref used for the OS. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_OSTREE_COMMIT` | OSTree commit checksum | Undefined | Commit checksum of the deployed OS. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_OSTREE_DATE` | ISO 8601 date/time | Undefined | Timestamp of the commit of the deployed OS. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_OSTREE_REF` | OSTree ref name | Undefined | Ref name of the collection–ref used for the OS. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_OSTREE_REMOTE` | OSTree remote name | Undefined | Remote name of the refspec used for the OS. Specific to eos-openqa-tests. |\n| `EOS_IMAGE_TYPE` | `boot`, `full`, `iso` | Undefined | Type of image this test is (or is to be) installed from; the type of image the SUT is booted from. From the `images` dictionary in the `manifest.json` file. Specific to eos-openqa-tests. |\n| `EOS_PLATFORM` | String | Undefined | `EIB_PLATFORM` from eos-image-builder. Specific to eos-openqa-tests. |\n| `FBE_TEST` | Space separated list of test names, string | Undefined | The given tests should be run, in order, in the install/first boot phase of `main.pm`, instead of installation and the normal `_fbe.pm` test. Each test name should refer to an existing `tests/$(name).pm` module in eos-openqa-tests. Specific to eos-openqa-tests. |\n| `FLAVOR` | String | Undefined | `${EIB_PERSONALITY}_{boot,full,iso}{,_update}`,  from eos-image-builder. Understood by OpenQA. |\n| `HDD_1_DECOMPRESS_URL` | URI | Undefined | URI of a `.img.xz` or `.img.gz` file to download and boot the system under test from. Understood by OpenQA. |\n| `ISO_URL` | URI | Undefined | URI of a `.iso` file to download and boot the system under test from. Understood by OpenQA. |\n| `LIVE` | Boolean | False | The test is running on a live system rather than being installed. Specific to eos-openqa-tests. |\n| `OS_UPDATE_TO` | OS version number string | Undefined | EOS should be updated after installation, and the tests should expect that the update results in running the given OS version (as checked against `/etc/os-release`). If this variable is not set, the OS will not be updated. Specific to eos-openqa-tests. |\n| `OS_UPDATE_TO_STAGE` | `dev`|`demo`|`prod` | Undefined | Should be set exactly when `OS_UPDATE_TO` is set. It provides the stage of the OS version which an update should result in — this may differ from the stage for the original image, for example in the case of updating to master. It will be used to run `eos-stage-ostree` before updating. Specific to eos-openqa-tests. |\n| `POSTINSTALL` | Space separated list of test names, string | Undefined | The given tests should be run, in order, in the post-install phase of `main.pm`, after logging in to GDM. Each test name should refer to an existing `tests/$(name).pm` module in eos-openqa-tests. Specific to eos-openqa-tests. |\n| `PUBLISH_HDD_1` | Filename for HDD image, string | Undefined | Publicly publish a qcow2 image of the VM’s HDD after the test completes successfully, using the given filename for it. Understood by os-autoinst. |\n| `START_AFTER_TEST` | Test name, string | Undefined | Schedule this test to be run only once the given test has completed, as an ordering constraint. Understood by OpenQA. |\n| `STORE_HDD_1` | Filename for HDD image, string | Undefined | As for `PUBLISH_HDD_1`, but store the image privately, so it can only be used by other tests within this test job. Understood by os-autoinst. |\n| `VERSION` | String | Undefined | `EIB_BRANCH` from eos-image-builder. Understood by os-autoinst. |\n\nUpdate tests\n---\n\nOne of the key goals of the OpenQA testing is to automatically test OS updates\nbetween different releases of Endless OS. Specifically, we care about the cases:\n\n * For a new minor release 3.`$old` on an old branch, we want to run all the\n   tests on 3.`$old`, and we want to update to and run the update tests on\n   3.`$latest`.\n * For a new release 3.`$latest`, we want to run all the tests on 3.`$latest`,\n   and we want to update from and run the update tests on 3.`$latest-1`,\n   3.`$latest-2`, 3.`$latest-3`, etc.\n\n‘The update tests’ may be all tests, or a critical subset of them.\n\nThe `OS_UPDATE_TO` variable controls whether an OS update is performed after\ninstallation. So in those cases above:\n\n * For a new minor release 3.`$old`, trigger tests on 3.`$old` without\n   `OS_UPDATE_TO`; and trigger tests on 3.`$old` with `OS_UPDATE_TO=3.$latest`.\n * For a new release 3.`$latest`, trigger tests on 3.`$latest` without\n   `OS_UPDATE_TO`; and trigger tests on each of 3.`$latest-1..n` with\n   `OS_UPDATE_TO=3.$latest`.\n\nIt’s expected that tests aren’t triggered from before the most recent checkpoint,\nsince the update code in `_os_update.pm` will not successfully update through\na checkpoint, and checkpoints provide an obvious place to break long update\nchains. The idea is that if a system passes the tests after upgrading to version\n3.`$checkpoint`, it should pass the tests after upgrading onwards from that\nversion. This behaviour could, however, be changed in future by expanding\n`_os_update.pm` to deal with checkpoints.\n\nNote that these update tests don’t cover testing the situation where the user\ninstalled 3.`$x`, then stepwise update 3.`$x` → 3.`$x+1` → 3.`$x+2` → 3.`$x+3`\n→ …. We will always end up testing the largest update step possible. In order to\ntest stepwise updates we’d need to keep intermediate qcow2 images around from\nhistoric test runs. We do not currently do that.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fendlessm%2Feos-openqa-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fendlessm%2Feos-openqa-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fendlessm%2Feos-openqa-tests/lists"}