{"id":19122418,"url":"https://github.com/rhinstaller/kickstart-tests","last_synced_at":"2025-05-05T17:05:22.936Z","repository":{"id":37432565,"uuid":"46139980","full_name":"rhinstaller/kickstart-tests","owner":"rhinstaller","description":"Kickstart-driven tests for the Anaconda installer","archived":false,"fork":false,"pushed_at":"2025-04-09T12:03:53.000Z","size":2858,"stargazers_count":59,"open_issues_count":145,"forks_count":36,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-19T10:28:31.538Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rhinstaller.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2015-11-13T18:33:01.000Z","updated_at":"2025-04-16T06:24:04.000Z","dependencies_parsed_at":"2024-03-25T08:42:31.170Z","dependency_job_id":"9577070e-b9e1-4d99-ae39-6fdaa0fbde1a","html_url":"https://github.com/rhinstaller/kickstart-tests","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhinstaller%2Fkickstart-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhinstaller%2Fkickstart-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhinstaller%2Fkickstart-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rhinstaller%2Fkickstart-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rhinstaller","download_url":"https://codeload.github.com/rhinstaller/kickstart-tests/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252542062,"owners_count":21764907,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T05:21:15.811Z","updated_at":"2025-05-05T17:05:22.929Z","avatar_url":"https://github.com/rhinstaller.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"Kickstart Test Documentation\n****************************\n\nKickstart tests are one way of testing the Anaconda Installer, by running an automated installation based on a kickstart file and checking the results.\n\n:Authors:\n   Chris Lumens \u003cclumens@redhat.com\u003e\n   Martin Kolman \u003cmkolman@redhat.com\u003e\n\nChapter 1. How to run kickstart tests in a container\n====================================================\n\nThis is the canonical way to run tests, as it requires very little setup, does\nnot do any permanent changes to your system, and exactly reproduces results\nfrom CI runs.\n\nClone the kickstart-tests repository and enter its directory::\n\n  git clone https://github.com/rhinstaller/kickstart-tests\n  cd kickstart-tests\n\nThe launch script downloads a current Fedora Rawhide boot.iso, downloads and\nstarts the runner container, and runs a set of tests in it::\n\n  containers/runner/launch keyboard [test2 test3 ...]\n\nPlease see the `runner documentation`_ for further details, like how to run all\ntests or some test types, running the container manually, using a different\nboot.iso, enabling caching, and more.\n\nChapter 2. How to run kickstart tests manually on the host\n==========================================================\n\n*Warning*: This is deprecated now.\n\nSetting up\n----------\n\nFirst you need to install the needed dependencies:\n\n- livemedia-creator\n- Python bindings for libvirt\n- libguestfs-tools\n- virt-install\n- parallel\n- createrepo\n- python3-rpmfluff\n- squid\n- scp\n- genisoimage\n- make\n\nYou also need to start libvirt service to be able to use virt-install::\n\n  sudo systemctl start virtqemud\n\nThen clone the kickstart-tests repository::\n\n  git clone https://github.com/rhinstaller/kickstart-tests\n\nAnd you also need a rawhide boot.iso (provided you want to run the kickstart tests on Rawhide)::\n\n  wget https://download.fedoraproject.org/pub/fedora/linux/development/rawhide/Server/x86_64/os/images/boot.iso\n\nPlease note that due to the dynamic nature of Rawhide the boot.iso might not always work.\n\nRunning a test\n--------------\n\nLets just run a simple test to check that everything works correctly – for example the simple tmpfs kickstart command test. First change directory to the kickstart-tests folder::\n\n  cd kickstart-tests\n\nThen run the single test::\n\n  scripts/run_kickstart_tests.sh -i ../boot.iso -k 2 tmpfs-fixed_size.sh\n\nAbout the parameters:\n\n  -i   sets the path to the boot.iso\n  -k   sets if logs from the run should be kept, as for the values:\n\n - 0 = keep nothing (the default)\n - 1 = keep log files\n - 2 = keep log files and disk images (will take up a lot of space)\n\n  -u   use updates image given by URL or local file path\n  -b   use additional installer boot options\n\nAnd at the end name of the kickstart test script to run.\n\nThe -k 2 option is especially useful if you are doing more complicated post-install test validation in you kickstart test script that needs to check contents of the disk image/images.\n\nThe results\n-----------\n\nIf everything worked out, you should be greeted by a successful test result similar to this one::\n\n\n    ===========================================================================\n    tmpfs-fixed_size.ks on computer.hostname\n    ===========================================================================\n    PYTHONPATH=\n    ...................................................\n    Domain LiveOS-1710fd05-898c-4cf2-b4e1-67d40aaf5f3d has been undefined\n\n    Pool kstest-tmpfs-fixed_size.RI8HWHMF destroyed\n\n    Pool kstest-tmpfs-fixed_size.RI8HWHMF has been undefined\n\n\n    RESULT:tmpfs-fixed_size:SUCCESS\n    2017-06-06 16:46:34,477: install_log = /var/tmp/kstest-tmpfs-fixed_size.RI8HWHMF/virt-install.log\n    2017-06-06 16:46:34,513: Running virt-install.\n    2017-06-06 16:46:35,903: Processing logs from ('127.0.0.1', 53130)\n    2017-06-06 16:55:06,646: Install finished. Or at least virt shut down.\n    2017-06-06 16:55:06,650: Shutting down LiveOS-1710fd05-898c-4cf2-b4e1-67d40aaf5f3d\n    error: Failed to destroy domain LiveOS-1710fd05-898c-4cf2-b4e1-67d40aaf5f3d\n    error: Requested operation is not valid: the domain is not running\n    2017-06-06 16:55:06,777: Shutting down log processing\n    2017-06-06 16:55:06,778: unmounting the iso\n    2017-06-06 16:55:06,812: Disk Image install successful\n    2017-06-06 16:55:06,812: SUMMARY\n    2017-06-06 16:55:06,812: -------\n    2017-06-06 16:55:06,813: Logs are in /var/tmp/kstest-tmpfs-fixed_size.RI8HWHMF\n    2017-06-06 16:55:06,813: Disk image(s) at /var/tmp/kstest-tmpfs-fixed_size.RI8HWHMF/disk-a.img,cache=unsafe\n    2017-06-06 16:55:06,813: Results are in /var/tmp/kstest-tmpfs-fixed_size.RI8HWHMF\n\nChapter 3. A test definition\n============================\n\nA kickstart test consists of two files:\n\n- \u003cTEST_NAME\u003e.sh - a file defining installer boot options and procedures to set\n  up test-specific environment (eg http server for providing the kickstart\n  file, special virtual networks, iscsi targets for test, etc). This file name\n  is used to specify the kickstart test to be run.\n\n- \u003cTEST_NAME\u003e.ks.in - the kickstart file belonging to the test, containing\n  variables that would be preprocessed (as described in following chapters) to\n  generate the actual kicstart file passed to installer. By default, the file\n  with the same name as the .sh file is used. This can be overriden (eg to\n  share kickstarts among tests that differ only in boot options) in .sh file\n  using KICKSTART_NAME=\u003cANOTHER_TEST_NAME\u003e variable. For example by defining\n\n  ::\n\n    KICKSTART_NAME=network-device-default\n\n  in network-device-default-httpks.sh test, the test will use kickstart\n  network-device-default.ks.in.\n\n  NOTE: possible redefinintions of KICKSTART_NAME value in files included in\n  the the .sh file (eg to reuse .sh file of another test) are ignored.\n\n  NOTE: The fragments (%ksappend) mechanism does not work together with\n  KICKSTART_NAME setting (%ksappend is not applied).\n\nChapter 4. Environment Variables\n================================\n\nA lot of tests need configuration.  This is information that is required by\ntests but typically cannot be hard coded.  Typically, this configuration is\na package repository needed for testing an installation method.  It is up to\nthe user running the tests to do whatever local setup is required and set\nthese configuration parameters.\n\nConfiguration parameters come from the environment.  All environment variables\nstarting with KSTEST_ will be grabbed by run_kickstart_tests.sh and\nautomatically substituted in to the kickstart file before it is run.  In the\nkickstart file, the target of a substitution is any string starting with\n@KSTEST_ and ending with another @.  This is similar to how the autotools work.\n\nConfiguration parameters may also come from special shell scripts that are\nsourced during run_kickstart_tests.sh.  It will first look at the defaults in\nscripts/defaults.sh.  Next, if platform is specified using -p PLATFORM option,\nthe scripts/defaults-PLATFORM.sh file is sourced.  Finally it will source any\nuser-specific defaults in ~/.kstests.defaults.sh.  These take precedence over\nthe local environment.  Environment variables set on the command line have the\nhighest priority.\n\nNote that not every test needs every setting.  You can determine which are\nrequired for the test you are running by simply running \"grep KSTEST_\" on it.\n\nThe following environment variables are currently supported:\n\n- KSTEST_HTTP_ADDON_REPO - This variable is a URL that points to an addon\n  repository.  It is only needed if you are testing that functionality, not\n  if you are testing something else that just happens to use the url command.\n  It will be set up for you automatically with a web server and auto-generated\n  packages.  There is no need to specify this variable.\n\n- KSTEST_LIVEIMG_CHECKSUM - This variable is the checksum of the image given\n  by KSTEST_LIVEIMG_URL.  It is only needed if you are testing the liveimg\n  command.  It will be set up for you automatically.  There is no need to\n  specify this variable.\n\n- KSTEST_LIVEIMG_URL - This variable is a URL that points to an install.img\n  that is used by the liveimg command.  It is only needed if you are testing\n  that command.  It will be set up for you automatically based on the boot.iso\n  specified on the command line.  There is no need to specify this variable.\n\n- KSTEST_NFS_ADDON_REPO - This variable points to an NFS server and path where\n  an addon repository can be found.  This is different from KSTEST_NFS_PATH\n  and KSTEST_NFS_SERVER.  Those are used with the nfs command.  This variable\n  is used with the repo command, and its format is different.  Here, it takes\n  the form of nfs://\u003cserver\u003e:\u003cpath\u003e.  See the kickstart documentation.  You\n  will need to set up your own NFS server.\n\n- KSTEST_NFS_PATH - This variable points to the path of a package repository\n  on the NFS server given by KSTEST_NFS_SERVER.  It is only needed if you are\n  testing the nfs command and installation method.  You will need to set up\n  your own NFS server.\n\n- KSTEST_NFS_SERVER - This variable points at an NFS server, and is only needed\n  if you are testing the nfs command and installation method.  You will need to\n  set up your own NFS server.\n\n- KSTEST_OSTREE_REPO - This variable points at the atomic repo, and is only\n  needed if you are testing the ostreesetup command and installation method.\n  You will need to set up your own repo.\n\n- KSTEST_FTP_URL - This variable is used by FTP tests. It is set to a Fedora\n  mirror in Texas, USA in scripts/defaults.sh. This is potentially slow and\n  you may want to point it at a local mirror.\n\n- KSTEST_URL - This variable is used by all tests that don't test installation\n  method and instead just use the default.  It is set to the Fedora mirrors in\n  scripts/defaults.sh.  This is potentially slow if you are running a lot of\n  tests, and you may want to point it at a local mirror.\n\n- KSTEST_OS_NAME - This variable is read from the input boot.iso and it\n  contains a name of the OS. Possible names can be \"fedora\", \"rhel\".\n\n- KSTEST_OS_VERSION - This variable is read from the input boot.iso and it\n  contains version of the OS. For example Fedora 26 has\n  KSTEST_OS_VERSION = 26, Fedora rawhide has \"Rawhide\", and RHEL 7.3 has\n  KSTEST_OS_VERSION = 7.3 .\n\n- KSTEST_EXTRA_BOOTOPTS - This variable is used in functions.sh to pass\n  additional kernel command line options. For example, setting this to `inst.text`\n  enables Anaconda's text mode (instead of the default GUI). Multiple values\n  separated by semicolon can be passed.\n\n- KSTEST_OSINFO_NAME - This variable is used as the default to be used in\n  case of failed virt-install os detection (--osinfo name=).  It may by useful\n  to specify for a new platform (os release) that is not detectable yet, for\n  example to make sure the expected network device naming scheme is used.\n\nChapter 5. Sharing common code in kickstart (.ks.in) files\n==========================================================\n\nTo include kickstart or code snippets into test kickstart file during its\npre-processing (just after KSTEST_ variables are substituted) use\n@KSINCLUDE@ \u003cFILE_NAME\u003e directive.\n\nFor example to include post-lib-network.sh which is a library with functions\nfor checking test results of network tests, include it in ks.in test file:\n\n::\n\n  %post\n\n  @KSINCLUDE@ post-lib-network.sh\n\n  check_device_connected ens4 yes\n\n  %end\n\nThe including is flat, only one level is supported. Do not use @KSINCLUDE@ in\nincluded files, the results could be unexpected.\n\nChapter 6. Networking tests\n===========================\n\nThis section contains tips for creating kicstart tests for network\nconfiguration.  In some test cases special or additional network devices and\nvirtual networks for test/virt-install instance are defined in prepare() and\nprepare_network() functions of .sh test file.\n\nNetwork device names\n--------------------\n\nNetwork device names used in guest may differ for tested os variants (eg RHEL\nvs Fedora).  Actual naming scheme to be used by the tests is defined in\nnetwork-device-names.cfg snippet which is sourced both in .sh files for boot\noptions network configuration (via functions.sh) and .ks.in files for kickstart\nnetwork configuration (via @KSTEST_ substitution). The variables used in .sh and\n.ks.in files have the form of KSTEST_NETDEV\u003cINDEX\u003e where \u003cINDEX\u003e is the\nnumerical index of the device, starting from 1.\n\nStatic IP configuration\n-----------------------\n\nFor tests using static IP configuration, separate NATed network is created in\nprepare() function for each test so IP address collisions between tests running\nin parallel are prevented. Static configuration generated during network\ncreation is referred to in kickstart using @KSTEST_ substitiution described\nabove.\n\nAllocating device MAC addresses\n-------------------------------\n\nFor tests requiring definition of MAC address assigned to the device the\naddress is statically assigned in prepare_network() function.  For kvm/qemu\nvirtual machines it must start with 52:54:00.\n\nhttpks tests\n------------\n\nThe tests containing httpks in its name are fetching kickstart from https\nserver (prepare() function of .sh test file) instead of including it via initrd\ninject into initramfs - which is the default approach used in tests.  The\nreason is that using the inject method the network devices are not initialized\nin time of parsing kickstart and obtaining information from sysfs (mostly\ngetting hw address) fails which results in incomplete ifcfg file generated.\n\nChapter 7. Continuous Integration structure\n===========================================\n\nRegular test runs\n-----------------\nEvery night, the `scenarios workflow`_ runs all tests on all our supported\noperating systems/repositories, like \"Fedora Rawhide\" or \"RHEL 9\". They\nessentially call the runner container's ``launch`` script documented above with\nthe desired parameters.\n\nThe actual set of tests run in a scenario is defined using `TESTTYPE` tags in\n`skip-testtypes file`_.\n\nThe ``rawhide`` and ``daily-iso`` scenarios can in principle run on any host\nthat has enough resources. The ``rhelX`` tests however needs to run on RHEL\ninternal infrastructure.\n\nCurrently all scenarios run on `self-hosted GitHub action runners`_, which are\nrunning in our upshift cluster. See our internal ``builders.git`` repository\nfor details and the launch/setup playbooks. These have little magic, though,\nthey mostly just create an OpenStack instance and install/configure the action\nrunner binary as a service. All the actual test logic is contained in the\nworkflow files and the runner container.\n\nThe results can be viewed on the `GitHub Daily run workflows page`_. Each run\nhas an artifact attached with the detailed log files. This is currently not\nvery comfortable, and we are actively looking for a better solution how to\npublish the test result history.\n\nThese tests are expected to succeed normally. On failures, rhinstaller\nmaintainers get a \"failed workflow\" notification email and should investigate\nthe cause.\n\nSometimes tests fail due to networking/infrastructure flakes. To avoid this\nkind of noise, the nightly runs use the ``--retry`` option to re-run a test\nwhich failed due to an unspecific reason (i.e. not due to a skip or a syntax\nerror in the kickstart file, etc.). The test log will still show both results\nright after each other, so that the original failure can be examined; but if\nthe retry works, the test as a whole counts as success.\n\nPull requests\n-------------\nPRs are gated to avoid introducing broken or unstable tests, and to validate\nchanges to existing tests. To keep PRs open to the whole community, we want to\navoid running them in self-hosted internal infrastructure (if we did, we'd need\nto restrict running the tests to avoid exfiltrating secrets from the internal\nRed Hat network).\n\nWe are using `TMT`_ for PR gating. Their test result will be required on newly\ncreated PRs, however, only committers have privileged to execute `tmt workflow`_\nby comment ``/test-tmt`` on the PR. After a while status on PR will have a link to\nTMT run.\n\nPR runs do *not* auto-retry test failures. This avoids introducing unstable\ntests, and PRs usually just run a few tests so that flakes are much less likely\nto ruin the result.\n\nTo test a PR on all supported os versions (including rhel) there is a\n`test-os-variants`_ workflow running the tests on a comment in a PR.\nRunning it requires admin repository permissions.\n\nService jobs\n------------\n* The `container-autoupdate`_ workflow refreshes the runner container\n  every week, and pushes it to `quay.io/rhinstaller/kstest-runner`_.\n  Developers, CI, and the ``launch`` script usually download it from there.\n\n* The `daily-boot-iso`_ workflow creates a ``boot.iso`` out of current Fedora\n  Rawhide and various COPRs every night, so that we can test updates to\n  anaconda, dnf, or blivet before they get released. This is consumed by the\n  ``daily-iso`` scenario.\n\nThese jobs don't have any particular infrastructure requirements. They run on\nGitHub's infrastructure and can be run manually by a developer.\n\n.. _runner documentation: ./containers/runner/README.md\n.. _containers: ./containers\n.. _self-hosted GitHub action runners: https://docs.github.com/en/free-pro-team@latest/actions/hosting-your-own-runners\n.. _scenarios workflow: .github/workflows/scenarios-permian.yml\n.. _skip-testtypes file: ./containers/runner/skip-testtypes\n.. _GitHub Daily run workflows page: https://github.com/rhinstaller/kickstart-tests/actions?query=workflow%3A%22Daily+run%22\n.. _tmt workflow: ./.github/workflows/testingfarm.yml\n.. _TMT: https://docs.testing-farm.io/\n.. _Travis: https://travis-ci.com/\n.. _.travis.yml: ./.travis.yml\n.. _container-autoupdate: ./.github/workflows/container-autoupdate.yml\n.. _test-platforms: ./.github/workflows/test-platforms.yml\n.. _quay.io/rhinstaller/kstest-runner: https://quay.io/repository/rhinstaller/kstest-runner\n.. _daily-boot-iso: ./.github/workflows/daily-boot-iso.yml\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhinstaller%2Fkickstart-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frhinstaller%2Fkickstart-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhinstaller%2Fkickstart-tests/lists"}