{"id":14128476,"url":"https://github.com/os-autoinst/os-autoinst","last_synced_at":"2025-06-22T23:33:04.142Z","repository":{"id":2094414,"uuid":"3034889","full_name":"os-autoinst/os-autoinst","owner":"os-autoinst","description":"OS-level test automation","archived":false,"fork":false,"pushed_at":"2024-10-29T14:04:49.000Z","size":34996,"stargazers_count":152,"open_issues_count":19,"forks_count":199,"subscribers_count":25,"default_branch":"master","last_synced_at":"2024-10-29T17:13:18.347Z","etag":null,"topics":["hacktoberfest","perl","test-automation","test-execution","virtualization"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/os-autoinst.png","metadata":{"files":{"readme":"README.asciidoc","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-12-22T16:07:47.000Z","updated_at":"2024-10-29T14:04:54.000Z","dependencies_parsed_at":"2023-12-21T20:29:26.257Z","dependency_job_id":"4596223a-ba46-4376-bc61-cbc6f675b7bb","html_url":"https://github.com/os-autoinst/os-autoinst","commit_stats":{"total_commits":4241,"total_committers":174,"mean_commits":"24.373563218390803","dds":0.8101862768215043,"last_synced_commit":"6d02300de5720874406b3dbe9e8d4e3983dc88ad"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/os-autoinst%2Fos-autoinst","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/os-autoinst%2Fos-autoinst/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/os-autoinst%2Fos-autoinst/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/os-autoinst%2Fos-autoinst/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/os-autoinst","download_url":"https://codeload.github.com/os-autoinst/os-autoinst/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228571844,"owners_count":17938772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","perl","test-automation","test-execution","virtualization"],"created_at":"2024-08-15T16:01:45.288Z","updated_at":"2024-12-07T06:31:23.099Z","avatar_url":"https://github.com/os-autoinst.png","language":"Perl","funding_links":[],"categories":["Perl"],"sub_categories":[],"readme":"= os-autoinst image:https://github.com/os-autoinst/os-autoinst/workflows/ci/badge.svg?branch=master[link=https://github.com/os-autoinst/os-autoinst/actions] image:https://codecov.io/gh/os-autoinst/os-autoinst/branch/master/graph/badge.svg[link=https://codecov.io/gh/os-autoinst/os-autoinst]\n:author: openSUSE Team at SUSE\n:toc:\n\nThe OS-autoinst project aims at providing a means to run fully\nautomated tests.  Especially to run tests of basic and low-level\noperating system components such as bootloader, kernel, installer and\nupgrade, which can not easily and safely be tested with other\nautomated testing frameworks.  However, it can just as well be used to\ntest firefox and openoffice operation on top of a newly installed OS.\n\nos-autoinst can be executed alone, but is currently designed to be\nexecuted together with openQA, the web user interface that allows to\nrun more than one os-autoinst instance at the same time.\n\nMore information on os-autoinst and openQA can be found on\nhttp://os-autoinst.github.io/openQA/\n\n== Getting started\n\nUnder openSUSE the `os-autoinst` package can be installed from the official\nrepository or from our\nlink:https://build.opensuse.org/project/show/devel:openQA[devel repository].\nFor further details, have a look at the\nlink:http://open.qa/docs/#_development_version_repository[openQA documentation].\n\nFor building os-autoinst manually checkout the build instructions below.\n\nThe main executable `isotovideo` can read test parameters from the command\nline or read test parameters from a file named `vars.json`. This file stores\nthe values of the different variables that will configure the behavior of the\ntest execution.\n\nA container is provided and can be pulled and the main execution can be called\nin one step, for example using the `podman` container engine for tests defined\nin the current directory on x86_64 if your environment supports KVM\nvirtualization acceleration:\n\n[source,sh]\n----\npodman run --rm -it -v .:/tests registry.opensuse.org/devel/openqa/containers/isotovideo:qemu-kvm casedir=/tests\n----\n\nUse the image variant ending with `qemu-x86` on x86_64 if no KVM support is\navailable.\n\nTake a look on\nhttps://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3Ddevel%3AopenQA\nfor all available container images.\n\nAdditional test variables can be supplied on the command line. There are some\nvariables used by os-autoinst itself and other that are used by the tests. A\nminimal command line can look like this:\n\n[source,sh]\n----\nisotovideo distri=opensuse casedir=/full/path/for/tests iso=/full/path/for/iso\n----\n\nAs alternative or completementary a corresponding `vars.json` with additional\nparameters could be:\n\n[source, javascript]\n-------------------------------------------------------------------\n{\n   \"DISTRI\" :      \"opensuse\",\n   \"CASEDIR\" :     \"/full/path/for/tests\",\n   \"NAME\" :        \"test-name\",\n   \"ISO\" :         \"/full/path/for/iso\",\n   \"VNC\" :         \"91\",\n   \"BACKEND\" :     \"qemu\",\n   \"DESKTOP\" :     \"kde\"\n}\n-------------------------------------------------------------------\n\nBe advised that the file `vars.json` is also modified by `os-autoinst` so make\nsure to backup handcrafted versions of this file.\n\nFor more concrete instructions read on in the \"How to run test cases\" section\nbelow. Find sections about \"How to contribute\" or \"Build instructions\" further\nbelow.\n\n== How to run test cases\n\nThis following instructions shows how to run test cases. First one needs to\nclone the test distribution. Checkout\nlink:https://github.com/os-autoinst/os-autoinst-distri-example[os-autoinst-distri-example]\nfor an example of a minimal test distribution.\n\nExample for openSUSE's tests:\n\n-----------------------------------------------------------------------------\nmkdir distri \u0026\u0026 cd distri\ngit clone git@github.com:os-autoinst/os-autoinst-distri-opensuse.git opensuse\ncd opensuse/products/opensuse\ngit clone git@github.com:os-autoinst/os-autoinst-needles-opensuse.git needles\n-----------------------------------------------------------------------------\n\nExample for openQA's self-tests (\"openQA-in-openQA\" test):\n\n-----------------------------------------------------------------------------\nmkdir distri \u0026\u0026 cd distri\ngit clone git@github.com:os-autoinst/os-autoinst-distri-openQA.git openqa\ncd openqa\ngit clone git@github.com:os-autoinst/os-autoinst-needles-openQA.git needles\n-----------------------------------------------------------------------------\n\nThen create a working directory for the test execution, e.g.:\n\n-----------------------------------------------------\nmkdir /tmp/os-autoinst-run \u0026\u0026 cd /tmp/os-autoinst-run\n-----------------------------------------------------\n\nCreate a minimal `vars.json` config file within that directory, e.g.:\n\n.vars.json\n[source,json]\n---------------------------------------------------------\n{\n   \"ARCH\" : \"x86_64\",\n   \"BACKEND\" : \"qemu\",\n   \"CASEDIR\" : \"/path/to/os-autoinst-distri-opensuse\",\n   \"DESKTOP\" : \"gnome\",\n   \"DISTRI\" : \"opensuse\",\n   \"ISO\" : \"/path/to/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20160715-Media.iso\",\n   \"PRODUCTDIR\" : \"/path/to/os-autoinst-distri-opensuse/products/opensuse\",\n   \"VNC\" : 90,\n}\n---------------------------------------------------------\n\nYou will need to correct the file paths to point to real locations. Some of the\nvariables you can use are listed link:doc/backend_vars.asciidoc[here]. Test case\nspecific variables are listed in the distri directories e.g.\nlink:https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/variables.md[os-autoinst-distri-opensuse/variables].\n\nThen you can run the `isotovideo` script within the created working directory.\nWhen doing a manual build, that script can be found at the top-level of the\nos-autoinst Git checkout.\n\nAll of these examples were using the QEMU backend which is usually the easiest\nbackend to handle and therefore recommended. If you need to develop and test\nother backends, have a look at the\nlink:doc/backends.md[backend-specific documentation].\n\nWhen using the QEMU backend it is possible to access the system under test via\nVNC:\n\n----------------------------------------\nvncviewer localhost:91 -ViewOnly -Shared\n----------------------------------------\n\nRun `isotovideo` with the environment variable `RUN_VNCVIEWER` set to autostart\na VNC viewer on the right port.\n\nRun `isotovideo` with the environment variable `RUN_DEBUGVIEWER` to start the\ninternal debug screenshot viewer updated with an always recent screenshot of the\ntest run.\n\n== Develop test modules\n\nIndividual test modules are written with one test module per file using the test\nAPI in Perl code. Experimental support for test modules in the Python\nprogramming language is provided.\n\nFind more details about how to write tests on\nhttp://open.qa/docs/#_how_to_write_tests\n\n== Verifying a runtime environment\n\nTo check if your hardware is able to successfully execute os-autoinst based\ntests one can execute openQA tests, all the development tests or simply call\nsomething like\n\n----\npodman run --pull=always --rm -it --entrypoint '' registry.opensuse.org/devel/openqa/containers/os-autoinst_dev:latest /bin/sh -c 'git -C /opt clone --depth 1 https://github.com/os-autoinst/os-autoinst \u0026\u0026 make -C /opt/os-autoinst/ test-perl-testsuite TESTS=t/99-full-stack.t'\n----\n\nwhich only requires the container runtime environment \"podman\" and will run a\ncontainer based os-autoinst full-stack test, here without KVM hardware\naccelerated virtualization support.\n\n== How to contribute\n\nIf you want to contribute to this project, please clone and send pull requests\nvia https://github.com/os-autoinst/os-autoinst.\n\nMore information on the contribution can be found on\nhttp://os-autoinst.github.io/openQA/contact/, too.\n\nIssues are tracked on https://progress.opensuse.org/projects/openqav3/.\n\nFor an overview of the architecture, see\nlink:doc/architecture.md[doc/architecture.md].\n\n=== Rules for commits\n\n* Every commit is checked by our CI system as soon as you create a pull request\n  but you *should* run the os-autoinst tests locally. Checkout the build\n  instructions for further details.\n\n* For git commit messages use the rules stated on\n  http://chris.beams.io/posts/git-commit/[How to Write a Git Commit Message] as\n  a reference\n\n* Every pull request is reviewed in a peer review to give feedback on possible\n  implications and how we can help each other to improve\n\nIf this is too much hassle for you feel free to provide incomplete pull requests\nfor consideration or create an issue with a code change proposal.\n\n=== Deprecation approach\n\nIn case you want to deprecate functionality consider the use of the function\n`backend::baseclass::handle_deprecate_backend`.\n\n== Build instructions\n\n=== Installing dependencies\n\nOn openSUSE one can install the package `os-autoinst-devel` which provides all\nthe dependencies to build and run os-autoinst for the corresponding version of\nthe sources. To build a current version of `os-autoinst` it is recommended to\ninstall `os-autoinst-devel` from\nlink:https://build.opensuse.org/project/show/devel:openQA[devel:openQA] as the\ndistribution-provided packages might be too old or miss dependencies. This is\nparticularly true for openSUSE Leap. Also see\nlink:https://open.qa/docs/#_development_version_repository[the openQA docs].\n\nThe required dependencies are also declared in `dependencies.yaml`. (The names\nlisted within that file are specific to openSUSE but can be easily transferred\nto other distributions.)\n\n=== Conducting the build\n\nSimply call\n\n----\nmake\n----\n\nin the top folder which automatically creates a build directory and builds the\ncomplete project.\n\nCall\n\n----\nmake help\n----\n\nto list all available targets.\n\nThe above commands use a convenience Makefile calling `cmake`. For packaging,\nwhen using an IDE or to conduct the steps manually it is suggested to use CMake\ndirectly and do the following: Create a build directory outside of the source\ndirectory. The following commands need to be invoked within that directory.\n\nConfigure build:\n----\ncmake $path_to_os_autoinst_checkout\n----\n\nYou can specify any of the standard CMake variables, e.g.\n`-DCMAKE_BUILD_TYPE=Debug` and `-DCMAKE_INSTALL_PREFIX=/custom/install/prefix`.\n\nThe following examples assume that GNU Make is used. It is possible to generate\nfor a different build tool by adding e.g. `-G Ninja` to the CMake arguments.\n\nBuild executables and libraries:\n----\nmake symlinks\n----\n\nThis target also creates symlinks of the built executables and libraries within\nthe source directory so `isotovideo` can find them.\n\nRun all tests:\n----\nmake check\n----\n\nBy default CTest is invoked in verbose mode because prove already provides\ncondensed output. Add `-DVERBOSE_CTEST=OFF` to the CMake arguments to avoid\nthat.\n\nRun all Perl tests (`*.t` files found within the `t` and `xt` directories):\n----\nmake test-perl-testsuite\n----\n\nRun individual tests by specifying them explicitly:\n----\nmake test-perl-testsuite TESTS=\"t/15-logging.t t/28-signalblocker.t\"\n----\n\nRun perl author tests:\n----\nmake test-local-author-perl\n----\n\nRun all author tests:\n----\nmake test-local\n----\n\nNotice that the user needs to include the test directory for each test (either t\nfor normal or xt for developer-centric tests) when specifying individual tests.\n\nAdd additional arguments to the `prove` invocation, e.g. enable verbose output:\n----\nmake test-perl-testsuite PROVE_ARGS=-v\n----\n\nGather coverage data while running tests:\n----\nmake test-perl-testsuite WITH_COVER_OPTIONS=1\n----\n\nGenerate a coverage report from the gathered coverage data:\n----\nmake coverage\n----\n\nIf no coverage data has been gathered so far the `coverage` target will invoke\nthe testsuite automatically.\n\nReset gathered coverage data:\n----\nmake coverage-reset\n----\n\nInstall files for packaging:\n----\nmake install DESTDIR=…\n----\n\nAutomatically tidy all perl files:\n----\ntools/tidyall\n----\n\nTidy all changed perl files:\n----\ntools/tidyall --git\n----\n\nFurther notes:\n\n* When using the `test-perl-testsuite` target, `ctest` is not used (and\n  therefore `ctest` specific tweaks have no effect).\n* One can always run Perl tests manually via `prove` after the build has been\n  conducted with `make symlinks`. Note that some tests need to be invoked within\n  the `t` directory. An invocation like\n  `prove -vI.. -I../external/os-autoinst-common/lib 28-signalblocker.t` is\n  supposed to work.\n* It is also possible to run `ctest` within the build directory directly instead\n  of using the mentioned targets.\n* All mentioned variables to influence the test execution (`TESTS`,\n  `WITH_COVER_OPTIONS`, …) can be combined and can also be used with the\n  `coverage` target.\n\n== Running isotovideo as CI check\nWe provide a container to run `isotovideo` which can be used to run QEMU-based\ntests directly in a CI runner. Checkout this\nhttps://github.com/os-autoinst/os-autoinst-distri-example/blob/main/.github/workflows/isotovideo-action.yml[example workflow]\nfor how it can be used. The\nhttps://github.com/os-autoinst/os-autoinst-distri-example/blob/main/README.md#local-testing-and-ci-environment[README of the example test distribution]\nalso contains further details.\n\n== Standalone tool for image search\nThe script `imgsearch` in the repository's script folder allows to use the\nfuzzy image comparison independently of the normal test execution. Invoke the\nscript with no parameters to show its usage. There is also an\nlink:https://github.com/os-autoinst/os-autoinst/blob/master/t/imgsearch/expected-output.json[example file]\nshowing what output you can expect. There is one key for each file to be\nsearched. The best matching image to be found will show up as `match` and the\nother images under `candidates`. If no image matches well enough, `match` will\nbe `null`.\n\nTo use the script the previously shown build instructions need to be executed\n(including the invocation of the `symlinks` target).\n\n== History of os-autoinst\n\nAt a time Bernhard M. Wiedemann who later joined was on the openSUSE testing\nteam and was assigned the task of testing the installer. Which meant tedious\nand dull work of waiting for 4GB ISO files to download when it's not even\nclear if those things even boot. And as the Perl founder Larry Wall states,\nimportant traits of programmers are laziness, impatience and hybris. Which\nquickly led to developing os-autoinst to automate installations ;) See\nhttps://lizards.opensuse.org/2010/04/29/making-of-the-opensuse-install-video/\nand https://lizards.opensuse.org/2010/05/25/automated-opensuse-testing/ for\nBernhard's blog posts.\n\n== Further notes\n\nWhen using the QEMU backend, also ensure your user running os-autoinst has\naccess to `/dev/kvm`.\n\n-----------------------------------------------------------\nmodprobe kvm-intel || modprobe kvm-amd\nchgrp kvm /dev/kvm ; chmod g+rw /dev/kvm # maybe redundant\n# optionally use a new user; just to keep things separate\nuseradd -m USERNAME -G kvm\npasswd USERNAME # and/or add ~USERNAME/.ssh/authorized_keys\n-----------------------------------------------------------\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fos-autoinst%2Fos-autoinst","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fos-autoinst%2Fos-autoinst","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fos-autoinst%2Fos-autoinst/lists"}