https://github.com/simonjwright/scripted_testing
Supports functional testing using Tcl scripts.
https://github.com/simonjwright/scripted_testing
Last synced: about 1 month ago
JSON representation
Supports functional testing using Tcl scripts.
- Host: GitHub
- URL: https://github.com/simonjwright/scripted_testing
- Owner: simonjwright
- Created: 2019-04-07T14:08:44.000Z (about 6 years ago)
- Default Branch: alire
- Last Pushed: 2023-04-25T10:33:37.000Z (almost 2 years ago)
- Last Synced: 2025-03-10T16:55:22.689Z (about 1 month ago)
- Language: Ada
- Size: 170 KB
- Stars: 2
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: COPYING3
Awesome Lists containing this project
- awesome-ada - scripted-testing - Supports functional testing using Tcl scripts. (Frameworks / Unit Test, Testing)
README
# Scripted Testing
## Introduction
To set the scene, the Ada software under test (SUT) forms part of a system. Generally, the system will be constructed using a layered approach, and the other parts of the system that interact with the SUT are higher-level (which can call interface subprograms presented by the SUT) or lower-level (whose interfaces can be called by the SUT, or which can provide callbacks to the SUT).
This package is intended for testing at a level between unit testing (using, for example, [AUnit](https://github.com/AdaCore/aunit)) and integration testing (using real hardware). Unit testing tends to be fragile, requiring a lot of rework for even minor functional changes in the requirements.
The scripting language supported is [Tcl](http://www.tcl.tk), and specifically the Ada binding [Tash](https://github.com/simonjwright/tcladashell). The reason for choosing Tcl rather than Python or Lua is that Tcl's interface is entirely string-based; this is important, considering the need to specify values of enumerated types.
## Genesis
The project from which this project is derived is the author's [ColdFrame](https://github.com/simonjwright/coldframe); see that project's [stubbing facilities](https://simonjwright.github.io/coldframe/stubs.html).
ColdFrame is a code generation framework, and has its own approach to stubbing (mocking) subprograms. This package has a different, but related, approach.
## Facilities
The package provides facilities to write new commands in Tcl to
* call the interface subprograms presented by the SUT,
* set up values to be returned by calls the SUT makes to lower-level system components,
* call callbacks provided by lower-level system components,
* delay for appropriate periods, and
* check that the proper calls have been made to lower-level system components, with the required values.
It's assumed that the interface subprograms of the lower-level subsystems are stubbed so that:
* `in` and `in out` parameters can be recorded for later checking,
* `out` (and `in out`) parameter values and function `return` values can be provided to be returned to the SUT,
* exceptions can be raised when required,
* the number of calls to the subprogram can be checked.
The components of the package are _Commands_, _Actions_, and an _Action Queue_.
### Commands
A Command implements a Tcl command.
It creates an Action to be executed at run time, and posts it on the Action Queue.
Some commands are provided by this package. Other commands are to be provided to support the specific application to be tested: typically, call\_procedure param1 param2 ... (where the parameters are those required by _procedure_) and check\_subprogram parameter value (to check the value passed to _procedure_ in _parameter_ on the last call).
Commands have to be `Register`ed with this package, because once Tcl has been started (using `Start`, which doesn't return) no more Commands can be added. Registration would normally be done during elaboration of the package in which the Command is defined (see `test/test-first.adb`).
### Actions
An Action carries the data required to enact the command at run time.
When it is executed, it performs the required action.
* If all is well, it completes normally.
* If some condition fails, it raises an exception (`Execution_Failure`) with a message stating the problem.
* Any other exceptions are propagated.
### Action Queue
When the Action Queue is started (using the `go` command), it repeatedly picks the next Action and executes it, until either the end of the queue is reached (which implies that the script has succeeded) or an exception is propagated (which implies that the script has failed).
The commands provided by this package are
-
echo "string" - outputs string to the terminal at run time. Useful to report the script's progress.
-
wait duration - delays for duration.
-
mark name - notes the time at which the command was executed.
-
wait_from_mark name duration - delays until duration after the named mark. It is an error if the indicated time has already passed. The mark can be re-used.
-
go - start executing the script.
## Building
This is an [Alire](https://alire.ada.dev/docs/#introduction) crate; to build, say `alr build`.
See a worked example in the [`demo/`](demo/) directory.