Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/apergos/pytest-docker

small test suite manager in python, using docker containers as 'test hosts'
https://github.com/apergos/pytest-docker

Last synced: 9 days ago
JSON representation

small test suite manager in python, using docker containers as 'test hosts'

Awesome Lists containing this project

README 

        
Configuring your tests



You will set up a top level config.py containing a python

dict 'opts'  which can contain an entry for each phase of testing.



You may also specifiy job-level configuration in a file config.py

containing a dict jobopts that in all other respects has the

same structure as the top level configuration.



JOB PHASES



These two phases are outside of any test suite job.



prep    -- run once on hosts before any other setup, before

           the test suite is executed

cleanup -- run on hosts after the entire test suite is complete



These phases are done once in each test suite job.



data    -- data files to be put into place on hosts before

           each test job in the suite

main    -- the scripts or programs to be tested

collect -- collect test output from the hosts

           after each test job; this stanza contains default

           attributes for the per job collct scripts

verify  -- verify the output from the hosts

           after each test job; this stanza contains default

           attributes for the per job verify scripts

mod     -- scripts that modify the data or files on the hosts,

           run before a given job; this stanza contains default

           attributes for the per job mod scripts



The following two phases must be present in the top level config

and must not be present in the per job config, since they are run

before and after the entire suite of jobs has run: prep, cleanup



The following phases may have stanzas in the top level config or

in the job level config or both: data, mod, main, collect, verify



For each section, the script or filename will be specified,

and a set of attributes provided for it.



Here is an example section:



jobopts = {

     ...

    'main': {

        'rsync-dump.py': {

            'remote': '/usr/local/bin/',

            'hosts': [ 'dataset2', 'dataset1001' ],

            'execute': 'python'

        }

    },

    ...

}



This tells us that in the 'main' phase for the specific job,

there will be one script tested, 'rsync-dump.py'.

It will be copied into /usr/local/bin/ on the hosts

(containers) names dataset2 and dataset1001.

It will be executed, by invoking python .



Additional attributes:

'untar' -- True if a file to be copied to remote host needs

	   to be untarred after copying



FIXME there are more attributes, document them here.



DEFAULTS



All files listed in a stanza for a job-level configuration

or in the top level configuration will be used (run or

installed or whatever else needs to be done with them), if

they are present in the appropriate subdirectory for

that stanza in tests/jobs.



If an attribute is not specified for a file in th job level

directory, it will be taken from the top level config,

if a declaration is present for the file there; if neither

of these holds but there is a wildcard entry, i.e. an entry

for '*' in the stanza in the top level file, the attribute

value will be taken from there if given.



mod, data, main, collect and verify are done per job, so the

entry for e.g. 'mod' in the top level config.py contains defaults

that apply to all mods. Below is an example of a wild card stanza

and a top level attribute definition for a specific script.



    'mod': {

        '*' {

            'remote': '/root/'

        },

        'mod-1.sh': {

            'execute': 'bash'

        }

    }



This says, for all mods where these attributes are not specified

in a job-specific config file, copy them into the /root directory

on hosts (containers) dataset2 and dataset1001.  And for every job

that runs mod-1.sh, use the bash interpreter to run it.



Addtionally you can add a section 'default' and put attributes there

which will apply to all items in all stanzas if they are

not otherwise specified.



A good example of an attribute that could go into 'default'

is 'hosts', which might be the same set of hosts for copying

data, for mods, for running the script to be tested, etc.



If you have specified an attribute value for an item in the top level

config and you don't want that attribute present at all in the job

level config, you can set the value of the attribute to None in the

job level config, and it will be treated as absent.



GLOBALS



A section 'globals' should be defined in the top level config

which will contain entries for the following:

'password' -- the root password to all the docker containers

'tmp' -- directory on the remote hosts where files can be

    temporarily copied; for example, data files before they are untarred



EXAMPLES



See the file config.py.sample for an example top-level config, and

job.py.sample for an example per job config.