{"id":19362701,"url":"https://github.com/smartdevicelink/sdl_atf","last_synced_at":"2025-04-23T12:33:07.870Z","repository":{"id":42078954,"uuid":"52923610","full_name":"smartdevicelink/sdl_atf","owner":"smartdevicelink","description":"Automated Tests for SmartDeviceLink","archived":false,"fork":false,"pushed_at":"2022-04-13T14:38:21.000Z","size":1541,"stargazers_count":4,"open_issues_count":15,"forks_count":35,"subscribers_count":29,"default_branch":"master","last_synced_at":"2025-04-02T15:21:17.730Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/smartdevicelink.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-03-02T01:30:48.000Z","updated_at":"2025-01-17T15:07:03.000Z","dependencies_parsed_at":"2022-09-22T00:00:43.464Z","dependency_job_id":null,"html_url":"https://github.com/smartdevicelink/sdl_atf","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartdevicelink%2Fsdl_atf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartdevicelink%2Fsdl_atf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartdevicelink%2Fsdl_atf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartdevicelink%2Fsdl_atf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/smartdevicelink","download_url":"https://codeload.github.com/smartdevicelink/sdl_atf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250435251,"owners_count":21430247,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T07:30:01.027Z","updated_at":"2025-04-23T12:33:02.856Z","avatar_url":"https://github.com/smartdevicelink.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Automated Test Framework (ATF)\n\n## Dependencies:\nLibrary                | License\n---------------------- | -------------\n**Lua libs**           |\nliblua5.2              | MIT\njson4lua               | MIT\nlua-stdlib             | MIT\nlua-lpeg               |\n**Qt libs**            |\nQt5.9 WebSockets       | LGPL 2.1\nQt5.9 Network          | LGPL 2.1\nQt5.9 Core             | LGPL 2.1\nQt5.9 Test             | LGPL 2.1\n**Other libs**         |\nlpthread               | LGPL\nOpenSSL (ssl, crypto)  | OpenSSL License\nlibxml2                | MIT\nldoc                   | MIT/X11\ndocker-ce              | Apache\nscreen                 | GNU\nrsync                  | GNU\n\n## Get source code\n```\n$ git clone --recurse-submodules https://github.com/smartdevicelink/sdl_atf.git\n$ git clone https://github.com/smartdevicelink/sdl_atf_test_scripts.git\n```\n\n## Compilation\n**1.** Install 3rd-parties developers libraries\n- Run the following commands:\n```\n$ sudo apt-get install lua5.2 liblua5.2-dev libxml2-dev lua-lpeg-dev\n$ sudo apt-get install openssl libssl-dev net-tools\n```\n\n**2.** Install Qt5.9+\n- Run the following command:\n```\n$ sudo apt-get install libqt5websockets5 libqt5websockets5-dev\n```\n\n**3.** Build ATF\n\nCMake 3.15 or newer is required to generate the build files and can be downloaded from [here](https://cmake.org/download/).\n\n- Create a build folder outside of the `sdl_atf` folder, e.g. `atf_build`\n- Assume there is the following directory structure:\n```\n..\n ├sdl_atf/\n ├sdl_atf_test_scripts/\n └atf_build\n```\n- Go into `atf_build` folder and run commands:\n```\n$ cmake ../sdl_atf\n$ make\n$ make install\n```\n- When build process is finished navigate to `atf_build/bin` folder\n\n**4.** Create symlinks to folders in scripts repository\n\n```\n$ ln -s ../../sdl_atf_test_scripts/files\n$ ln -s ../../sdl_atf_test_scripts/test_sets\n$ ln -s ../../sdl_atf_test_scripts/test_scripts\n$ ln -s ../../sdl_atf_test_scripts/user_modules\n```\n\n**5.** Install dependencies for local parallel mode\n\nSteps below are required only in case if local parallel mode is going to be used.\n- Install Docker, e.g. [how-to](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04)\n- Install additional tools:\n```\n$ sudo apt-get install screen rsync\n```\n- Create docker image\n```\n$ cd atf_parallels/docker\n$ ./build.sh \u003cubuntu_version\u003e\n```\n\u003cb\u003eNote:\u003c/b\u003e accepted values are `18` and `20` (will be processed as `18.04`, and `20.04` correspondingly).\nIf version is not specified `20` will be used by default.\n\n## Settings\n\n### Defaults\n\nBy default ATF reads the following parameters from `Default` configuration (`./modules/configuration/`).\n\nConfig File           | Config Parameter                | Cmd Argument      | Description\n----------------------|---------------------------------|-------------------|-----------------------------------------\nbase_config.lua       | config.SDL                      |                   | SDL binary name\nbase_config.lua       | config.reportPath               | --report          | Path to reports and logs\nbase_config.lua       | config.pathToSDL                | --sdl-core        | Path to SDL binaries\nbase_config.lua       | config.pathToSDLSource          | --sdl-src         | Path to SDL Core\nbase_config.lua       | config.pathToSDLMobileInterface | --sdl-mobile-api  | Path to SDL MOBILE API\nbase_config.lua       | config.pathToSDLHMIInterface    | --sdl-hmi-api     | Path to SDL HMI API\nconnection_config.lua | config.remoteConnection.enabled |                   | Defines if remote connection is enabled\n\n### Priorities\n\n1. Command line argument\n2. Specific config\n3. Base config\n\nWhere 1 is max and 3 is min priority.\n\nThis means if some particular option is defined in all 3 places ATF will use the one from highest priority place.\n\nE.g. if path to SDL binaries is defined through `--sdl-core` command line argument (1), in specific config (2) and also in base config (3), ATF will use the value from command line argument.\n\n## Run\n\nBeing in `atf_build/bin` folder run the command:\n```\n./start.sh TEST [OPTIONS]...\n```\n\n- TEST - test target, is one of the following:\n  - test script\n  - test set\n  - folder with test scripts\n- [OPTION] - is one or more of available options:\n  - --sdl-core \u0026lt;path\u0026gt;  - path to SDL binaries\n  - --config \u0026lt;folder\u0026gt;  - name of the folder with configuration\n  - --sdl-src \u0026lt;path\u0026gt;   - path to SDL Core\n  - --sdl-mobile-api \u0026lt;path\u0026gt;   - path to SDL MOBILE API\n  - --sdl-hmi-api \u0026lt;path\u0026gt;   - path to SDL HMI API\n  - --report \u0026lt;path\u0026gt;    - path to report and logs\n  - --sdl-log [ACTION]       - how to collect SDL logs:\n    'yes' - (default) always save, 'no' - do not save, 'fail' - save if script failed or aborted\n  - --sdl-core-dump [ACTION] - how to collect SDL core dumps:\n    'yes' - (default) always save, 'no' - do not save, 'fail' - save if script failed or aborted\n  - --parallels              - force to use local parallel mode\n    - -j|--jobs \u0026lt;n\u0026gt;        - number of simultaneous jobs to start\n    - --third-party \u0026lt;path\u0026gt; - path to SDL third party\n    - --tmp \u0026lt;path\u0026gt;         - path to temporary folder\n    - --copy-atf-ts              - force copying of ATF test scripts instead of creating symlinks\n\nIn case if folder is specified:\n   - only scripts which name starts with number will be taken into account (e.g. 001, 002 etc.)\n   - if there are sub-folders scripts will be run recursively\n\nBesides execution of .lua scripts ATF also does auxiliary actions:\n  - clean up SDL and ATF folders before running of each script\n  - backup and restore SDL important files\n  - create report with all required logs for each script including SDL core dumps (in local parallel mode)\n\n### Modes\n\n#### Local consecutive\n\nTest scripts will be run locally one by one.\n\n*Example 1* - Single script:\n```\n./start.sh ./test_scripts/Smoke/Policies/001_PTU_all_flows.lua\n```\n\n*Example 2* - Multiple scripts from some folder:\n```\n./start.sh ./test_scripts/Smoke/Policies\n```\n\n*Example 3* - Test set:\n```\n./start.sh ./test_sets/smoke_tests.txt\n```\n\n#### Local parallel\n\nTest scripts will be run locally in isolated environments and, if required, in several threads.\n\nTest targets described in examples 1 - 3 can be executed in parallel mode by adding --parallels option.\n\nIn addition number of parallel jobs can be defined by setting `-j` (or `--jobs`) option.\n\n*Example 4* - Single script in parallel mode:\n```\n./start.sh ./test_scripts/Smoke/Policies/001_PTU_all_flows.lua --parallels\n```\n\n*Example 5* - Multiple scripts from some folder in parallel mode split into 4 threads:\n```\n./start.sh ./test_scripts/Smoke/Policies -j 4\n```\n\n*Example 6* - Test set in parallel mode split into 2 threads:\n```\n./start.sh ./test_sets/smoke_tests.txt -j 2\n```\n\nIn case if number of jobs is more than one `--parallels` option can be omitted.\n\n#### Remote consecutive\n\nTest scripts will be run using remote connection.\nIn this mode `RemoteTestingAdapterServer` should be run on the same host as SDL\n\nTest targets described in examples 1 - 3 can be executed in remote mode.\n\nAppropriate parameters needs to be defined in configuration files:\n\nConfig Parameter                | Description\n------------------------------- | -----------------------------------\nconfig.pathToSDL                | Path to SDL on a remote host\nconfig.remoteConnection.enabled | Set to `true` for remote connection\nconfig.remoteConnection.url     | SDL host URL\nconfig.mobileHost               | Mobile host URL\nconfig.sdl_logs_host            | SDL host URL to receive log\n\nThe best approach is to use predefined configuration (e.g. `remote_linux`) as base and update some parameters in it.\n\n*Example 7* - Test set in remote connection mode:\n```\n./start.sh ./test_sets/smoke_tests.txt --config remote_linux\n```\n\n### Advanced options\n\n1. For a big tests sets (\u003e1000 scripts) Report and Logs can be very huge (\u003e10Gb). Most of the space is occupied by SDL logs. In order to turn them off `--sdl-log` or `--sdl-core-dump` options with `no` or `fail` value can be specified.\n2. Some scripts (old policy ones) create the same temporary files inside `files`, `test_scripts` or `user_modules` folders. In case of parallel mode the same temporary file can be used by different scripts at the same time. This leads to incorrect results or even aborts. In order to mitigate this issue `--copy-atf-ts` option can be specified. It tells ATF to copy mentioned folders for each job instead of creating symlinks.\n3. By default scripts for vehicle data executed for all available parameters. However there is possibility to restrict parameters to be tested by defining `VD_PARAMS` environment variable. E.g. `export VD_PARAMS=gps,speed` will allow to run the tests only for `gps` and `speed` vehicle data parameters.\n\n## Documentation generation\n\n### Download and install [ldoc](stevedonovan.github.io/ldoc/manual/doc.md.html)\n```\n$ sudo apt install luarocks\n$ sudo luarocks install luasec\n$ sudo luarocks install penlight\n$ sudo luarocks install ldoc\n$ sudo luarocks install discount\n```\n\n### Generate ATF documentation\n```\n$ cd sdl_atf\n$ ldoc -c docs/config.ld .\n```\n\n### Open documentation\n```\n$ chromium-browser docs/html/index.html\n```\n\n## Run Unit Tests\n```\n$ make test\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmartdevicelink%2Fsdl_atf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmartdevicelink%2Fsdl_atf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmartdevicelink%2Fsdl_atf/lists"}