Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/savoirfairelinux/cukinia

A simple on-target system test framework for Linux
https://github.com/savoirfairelinux/cukinia

embedded firmware linux microtest stdout tdd testing validation

Last synced: 1 day ago
JSON representation

A simple on-target system test framework for Linux

Awesome Lists containing this project

README 

        
![cukinia logo](./doc/cukinia_logo.png?raw=true)

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)



# Cukinia - a Linux firmware validation framework



Cukinia is designed to help Linux-based embedded firmware developers

run simple system-level validation tests on their firmware.



Cukinia integrates well with embedded firmware generation frameworks

such as Buildroot and Yocto, and can be run manually or by your

favourite continuous integration framework.



## Project objectives



Cukinia works if it offers the following value:



* It is very simple to use

* It requires no dependencies other than busybox

* It integrates easily with CI/CD pipelines

* It helps developers creating better software



## Usage



``cukinia [options] [config file]``



Useful options:



* `-f junitxml`: format results as JUnit XML (useful for Jenkins & others)

* `-f csv`: format results as CSV text

    * `--no-header`: omit CSV header line

* `-o file`: output results to file instead of stdout



## Screenshot



![Screenshot](doc/screenshot.png)



## Basic config



To run Cukinia, create a configuration describing your tests, and

invoke it. By default, cukinia reads ``/etc/cukinia/cukinia.conf``.

Alternatively, a config file can be passed to cukinia as its argument.



A cukinia config file supports the following statements:



### Test statements



* ``cukinia_user ``: Validates that user exists

* ``cukinia_group ``: Validates that group exists

* ``cukinia_user_memberof  ``: Validate that user is member of groups

* ``cukinia_kmod ``: Validates that kernel module is loaded

* ``cukinia_kconf  ``: Validates that kernel config

  symbol is set to given tristate value

* ``cukinia_kversion ``: Validate kernel version (only check maj.min, e.g 5.14)

* ``cukinia_process  [user]``: Validates that process runs (optional user)

* ``cukinia_kthread ``: Validates that kernel thread runs

* ``cukinia_python_pkg ``: Validates that Python package is installed

* ``cukinia_test ``: Validates that test(1) expression is true

* ``cukinia_http_request ``: Validates that url returns a 200 code

* ``cukinia_cmd ``: Validates that arbitrary command returns true

* ``cukinia_cmdline ``: Validates kernel cmdline contains param (optional value)

* ``cukinia_listen4  ``: Validates that tcp/udp port is open locally

* ``cukinia_mount   [fstype] [options]``: Validate the

  presence of a mount on the system

* ``cukinia_symlink  ``: Validate the target of a symlink

* ``cukinia_systemd_failed``: Raise a failure if a systemd unit is in failed state

* ``cukinia_systemd_unit ``: Validate systemd unit is active

* ``cukinia_i2c  [device_address] [driver_name]``: This checks i2c bus or (optional) device, and (optionally) verifies it uses the indicated driver

* ``cukinia_gpio_libgpiod -i [input_pins] -l [output_low_pins] -h [output_high_pins]

  -g [gpiochip](default:gpiochip0)``: Validate the gpio configuration via libgpiod

  (ex: cukinia_gpio_libgpiod -i "0 3 4" -l "10" -h "2 50" -g gpiochip1)

* ``cukinia_gpio_sysfs -i [input_pins] -l [output_low_pins] -h [output_high_pins]

  -g [gpiochip](default:gpiochip0)``: Validate the gpio configuration via sysfs

  (ex: cukinia_gpio_sysfs -i "20 34" -h "3 99 55")

* ``cukinia_knoerror ``: Validate kernel has booted without important

  errors (the priority argument is the log level number to check)

* ``cukinia_sysctl  ``: Validate kernel sysctl parameter is set to value

* ``cukinia_netif_has_ip  [-4|-6] [flags]``: Validate that interface has ip config parameters

  * example: `cukinia_netif_has_ip eth0 -4 dynamic`

  * example: `cukinia_netif_has_ip eth0 -6 "scope global"`

* ``cukinia_netif_is_up ``: Validate network interface state is up

* ``cukinia_dns_resolve ``: Validate that hostname can be resolved

* ``not``: Can prefix any test to invert the issue it will produce (a

  ``[!]`` is appended to the default test description)

* ``verbose``: Can prefix any test to preserve stdout/stderr

* ``as ``: Can prefix any test to change its textual description

* ``id ``: Can prefix any test to add a test id in the different outputs



### Condition statements



* `when `: Can prefix any test to `` it

* `unless `: Just like `when`, but the opposite



If the condition is not met, the test status will be reported as SKIP.



A few examples using `when` and `unless`:

``` bash

on_eval_board() { grep -q EVK /sys/firmware/devicetree/base/model; }

arch_is_arm64() { test "$(uname -m)" = "aarch64"; }



unless "on_eval_board" \

  as "Custom LED controller was detected" \

    cukinia_test -d /sys/class/leds/superled



when "arch_is_arm64" \

  unless "on_eval_board" \

    cukinia_kmod some_driver 

```

* `on  `: Can execute a statemement conditionally to test result

For each test result some statements can be executed:

    * `retry  [after ]`: retry the test `count` times after

    optional `interval` second(s), month(m), hour(h) or day(d) between each attempt (default to 0 second).

    The `count` is the number of retry attempts, the first attempt is not counted.

    Then if count is 3, the test will be executed 4 times if the condition is not met.

        examples:

        ``` shell

        on success retry 3 after 2s cukinia_systemd_unit some_unit

        on failure retry 3 cukinia_systemd_unit some_unit

        ```



### Utility statements



* ``cukinia_conf_include ``: Includes files as additional config files

* ``cukinia_run_dir ``: Runs all executables in directory as individual tests

* ``cukinia_log ``: Logs message to stdout

* ``_ver2int ``: Convert numeric version string to int, for use with

  e.g. ``cukinia_test $(_ver2int ${kernel_version}) -gt $(_ver2int 4.19.7)``



### Logging customization



* ``logging prefix "string"``: prefix logs with "string"

* ``logging class "string"``: change the junitxml class name to "string" for the next tests

* ``logging suite "string"``: change the junitxml test suite to "string" for the next tests



### Useful variables



* ``$cukinia_tests``: number of tests attempted

* ``$cukinia_failures``: number of tests that failed



### Environment variables



* ``$CUKINIA_ALWAYS_PASS``: if set, every test will succeed



### Example cukinia.conf



```shell



# Ensure our basic users are present

cukinia_user appuser1

cukinia_user appuser2



# This should always be the case

cukinia_test -f /etc/passwd



# If this user exists, then something went wrong

not cukinia_user baduser



# Those config snippets are deployed by our packages

cukinia_conf_include /etc/cukinia/conf.d/*.conf



# Is our embedded webservice up?

as "Checking webapp" cukinia_http_request http://localhost:8080/sanitycheck



# Run executable tests for myapp1

cukinia_run_dir /etc/cukinia/myapp1.d/



# Check for misc. mount points

cukinia_mount sysfs /sys

cukinia_mount /dev/sda1 /boot ext4 rw sync



# Check for ssh and dns servers

cukinia_listen4 tcp 22

cukinia_listen4 udp 53



# Check the link interfaces point to /tmp/interfaces

cukinia_symlink /etc/network/interfaces /tmp/interfaces



# Add a id linked to the test

id "SWR_001" as "Checking systemd units" cukinia_systemd_failed



# End

cukinia_log "ran $cukinia_tests tests, $cukinia_failures failures"

```



## More advanced config



A config file is actually a POSIX shell script that is sourced by

cukinia, so any logic can be used in a test file scenario. This is

useful for example to make certain groups of tests depend on

preliminary checks:



```shell



if cukinia_test -x /usr/bin/myapp; then

	cukinia_user myuser

	cukinia_process myapp myuser

	cukinia_http_request http://localhost:8080/testme

else

	cukinia_log "$(_colorize red "myapp not found :(")"

fi



```



## License



`Copyright (C) 2017-2024 Savoir-faire Linux, Inc.`



Cukinia is released under the Apache 2 license.