https://github.com/awncorp/test-auto
Test Automation, Docs Generation
https://github.com/awncorp/test-auto
automation documentation framework perl perl5 testing
Last synced: 5 months ago
JSON representation
Test Automation, Docs Generation
- Host: GitHub
- URL: https://github.com/awncorp/test-auto
- Owner: awncorp
- License: other
- Created: 2019-09-24T23:47:23.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-06-06T22:08:07.000Z (almost 6 years ago)
- Last Synced: 2025-01-01T11:27:31.388Z (about 1 year ago)
- Topics: automation, documentation, framework, perl, perl5, testing
- Language: Perl
- Homepage: http://metacpan.org/release/Test-Auto/
- Size: 204 KB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README
- Changelog: CHANGES
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
NAME
Test::Auto - Test Automation
ABSTRACT
Test Automation, Docs Generation
SYNOPSIS
#!/usr/bin/env perl
use Test::Auto;
use Test::More;
my $test = Test::Auto->new(
't/Test_Auto.t'
);
# automation
# my $subtests = $test->subtests->standard;
# ...
# done_testing;
DESCRIPTION
This package aims to provide, a standard for documenting Perl 5
software projects, a framework writing tests, test automation, and
documentation generation.
REASONING
This framework lets you write documentation in test files using
pod-like comment blocks. By using a particular set of comment blocks
(the specification) this framework can run certain kinds of tests
automatically. For example, we can automatically ensure that the
package the test is associated with is loadable, that the test file
comment blocks meet the specification, that any super-classes or
libraries are loadable, and that the functions, methods, and routines
are properly documented.
LIBRARIES
This package uses type constraints from:
Test::Auto::Types
SCENARIOS
This package supports the following scenarios:
exports
use Test::Auto;
use Test::More;
my $subtests = testauto 't/Test_Auto.t';
# automation
# $subtests->standard;
# ...
# done_testing;
This package automatically exports the testauto function which uses the
"current file" as the automated testing source.
ATTRIBUTES
This package has the following attributes:
data
data(Data)
This attribute is read-only, accepts (Data) values, and is optional.
file
file(Str)
This attribute is read-only, accepts (Str) values, and is required.
FUNCTIONS
This package implements the following functions:
testauto
testauto(Str $file) : Subtests
This function is exported automatically and returns a
Test::Auto::Subtests object for the test file given.
testauto example #1
# given: synopsis
my $subtests = testauto 't/Test_Auto.t';
METHODS
This package implements the following methods:
document
document() : Document
This method returns a Test::Auto::Document object.
document example #1
# given: synopsis
my $document = $test->document;
parser
parser() : Parser
This method returns a Test::Auto::Parser object.
parser example #1
# given: synopsis
my $parser = $test->parser;
subtests
subtests() : Subtests
This method returns a Test::Auto::Subtests object.
subtests example #1
# given: synopsis
my $subtests = $test->subtests;
SPECIFICATION
# [required]
=name
=abstract
=tagline
=includes
=synopsis
=description
# [optional]
=libraries
=inherits
=integrates
=attributes
# [repeatable; optional]
=scenario $name
=example $name
# [repeatable; optional]
=method $name
=signature $name
=example-$number $name # [repeatable]
# [repeatable; optional]
=function $name
=signature $name
=example-$number $name # [repeatable]
# [repeatable; optional]
=routine $name
=signature $name
=example-$number $name # [repeatable]
# [repeatable; optional]
=type $name
=type-library $name
=type-composite $name # [optional]
=type-parent $name # [optional]
=type-coercion-$number $name # [optional]
=type-example-$number $name # [repeatable]
The specification is designed to accommodate typical package
declarations. It is used by the parser to provide the content used in
the test automation and document generation. Note: when code blocks are
evaluated "redefined" warnings are now automatically disabled.
name
=name
Path::Find
=cut
The name block should contain the package name. This is tested for
loadability.
tagline
=tagline
Path Finder
=cut
The tagline block should contain a tagline for the package. This is
optional but if present is concatenated with the name during POD
generation.
abstract
=abstract
Find Paths using Heuristics
=cut
The abstract block should contain a subtitle describing the package.
This is tested for existence.
includes
=includes
function: path
method: children
method: siblings
method: new
=cut
The includes block should contain a list of function, method, and/or
routine names in the format of $type: $name. Empty lines are ignored.
This is tested for existence. Each function, method, and/or routine is
tested to be documented properly. Also, the package must recognize that
each exists.
synopsis
=synopsis
use Path::Find 'path';
my $path = path; # get path using cwd
=cut
The synopsis block should contain the normative usage of the package.
This is tested for existence. This block should be written in a way
that allows it to be evaled successfully and should return a value.
description
=description
interdum posuere lorem ipsum dolor sit amet consectetur adipiscing elit duis
tristique sollicitudin nibh sit amet
=cut
The description block should contain a thorough explanation of the
purpose of the package. This is tested for existence.
libraries
=libraries
Types::Standard
Types::TypeTiny
=cut
The libraries block should contain a list of packages, each of which is
itself a Type::Library. These packages are tested for loadability, and
to ensure they are type library classes.
inherits
=inherits
Path::Tiny
=cut
The inherits block should contain a list of parent packages. These
packages are tested for loadability.
integrates
=integrates
Path::Find::Upable
Path::Find::Downable
=cut
The integrates block should contain a list of packages that are
involved in the behavior of the main package. These packages are not
automatically tested.
scenarios
=scenario export-path-make
quisque egestas diam in arcu cursus euismod quis viverra nibh
=example export-path-make
# given: synopsis
package main;
use Path::Find 'path_make';
path_make 'relpath/to/file';
=cut
There are situation where a package can be configured in different
ways, especially where it exists without functions, methods or routines
for the purpose of configuring the environment. The scenario directive
can be used to automate testing and documenting package usages and
configurations.Describing a scenario requires two blocks, i.e. scenario
$name and example $name. The scenario block should contain a
description of the scenario and its purpose. The example block must
exist when documenting a method and should contain valid Perl code and
return a value. The block may contain a "magic" comment in the form of
given: synopsis or given: example $name which if present will include
the given code example(s) with the evaluation of the current block.
Each scenario is tested and must be recognized to exist by the main
package.
attributes
=attributes
cwd: ro, req, Object
=cut
The attributes block should contain a list of package attributes in the
form of $name: $is, $presence, $type, where $is should be ro
(read-only) or rw (read-wire), and $presence should be req (required)
or opt (optional), and $type can be any valid Type::Tiny expression.
Each attribute declaration must be recognized to exist by the main
package and have a type which is recognized by one of the declared type
libraries.
methods
=method children
quis viverra nibh cras pulvinar mattis nunc sed blandit libero volutpat
=signature children
children() : [Object]
=example-1 children
# given: synopsis
my $children = $path->children;
=example-2 children
# given: synopsis
my $filtered = $path->children(qr/lib/);
=cut
Describing a method requires at least three blocks, i.e. method $name,
signature $name, and example-1 $name. The method block should contain a
description of the method and its purpose. The signature block should
contain a method signature in the form of $signature : $return_type,
where $signature is a valid typed signature and $return_type is any
valid Type::Tiny expression. The example-$number block is a repeatable
block, and at least one block must exist when documenting a method. The
example-$number block should contain valid Perl code and return a
value. The block may contain a "magic" comment in the form of given:
synopsis or given: example-$number $name which if present will include
the given code example(s) with the evaluation of the current block.
Each method is tested and must be recognized to exist by the main
package.
functions
=function path
lectus quam id leo in vitae turpis massa sed elementum tempus egestas
=signature children
path() : Object
=example-1 path
package Test::Path::Find;
use Path::Find;
my $path = path;
=cut
Describing a function requires at least three blocks, i.e. function
$name, signature $name, and example-1 $name. The function block should
contain a description of the function and its purpose. The signature
block should contain a function signature in the form of $signature :
$return_type, where $signature is a valid typed signature and
$return_type is any valid Type::Tiny expression. The example-$number
block is a repeatable block, and at least one block must exist when
documenting a function. The example-$number block should contain valid
Perl code and return a value. The block may contain a "magic" comment
in the form of given: synopsis or given: example-$number $name which if
present will include the given code example(s) with the evaluation of
the current block. Each function is tested and must be recognized to
exist by the main package.
routines
=routine algorithms
sed sed risus pretium quam vulputate dignissim suspendisse in est ante
=signature algorithms
algorithms() : Object
=example-1 algorithms
# given: synopsis
$path->algorithms
=example-2 algorithms
package Test::Path::Find;
use Path::Find;
Path::Find->algorithms;
=cut
Typically, a Perl subroutine is declared as a function or a method.
Rarely, but sometimes necessary, you will need to describe a subroutine
where the invocant is either a class or class instance. Describing a
routine requires at least three blocks, i.e. routine $name, signature
$name, and example-1 $name. The routine block should contain a
description of the routine and its purpose. The signature block should
contain a routine signature in the form of $signature : $return_type,
where $signature is a valid typed signature and $return_type is any
valid Type::Tiny expression. The example-$number block is a repeatable
block, and at least one block must exist when documenting a routine.
The example-$number block should contain valid Perl code and return a
value. The block may contain a "magic" comment in the form of given:
synopsis or given: example-$number $name which if present will include
the given code example(s) with the evaluation of the current block.
Each routine is tested and must be recognized to exist by the main
package.
types
=type Path
Path
=type-parent Path
Object
=type-library Path
Path::Types
=type-composite Path
InstanceOf["Path::Find"]
=type-coercion-1 Path
# can coerce from Str
'./path/to/file'
=type-example-1 Path
require Path::Find;
Path::Find::path('./path/to/file')
=cut
When developing Perl programs, or type libraries, that use Type::Tiny
based type constraints, testing and documenting custom type constraints
is often overlooked. Describing a custom type constraint requires at
least two blocks, i.e. type $name and type-library $name. While it's
not strictly required, it's a good idea to also include at least one
type-example-1 $name. The optional type-parent block should contain the
name of the parent type. The type-composite block should contain a type
expression that represents the derived type. The type-coercion-$number
block is a repeatable block which is used to validate type coercion.
The type-coercion-$number block should contain valid Perl code and
return the value to be coerced. The type-example-$number block is a
repeatable block, and it's a good idea to have at least one block must
exist when documenting a type. The type-example-$number block should
contain valid Perl code and return a value. Each type is tested and
must be recognized to exist within the package specified by the
type-library block.
AUTOMATION
$test->standard;
This is the equivalent of writing:
$test->package;
$test->document;
$test->libraries;
$test->inherits;
$test->attributes;
$test->methods;
$test->routines;
$test->functions;
$test->types;
This framework provides a set of automated subtests based on the
package specification, but not everything can be automated so it also
provides you with powerful hooks into the framework for manual testing.
my $subtests = $test->subtests;
$subtests->synopsis(sub {
my ($tryable) = @_;
ok my $result = $tryable->result, 'result ok';
$result; # for automated testing after the callback
});
The code examples documented can be automatically evaluated (evaled)
and returned using a callback you provide for further testing. Because
the code examples are returned as Test::Auto::Try objects (see
Data::Object::Try), this makes capturing and testing exceptions simple,
for example:
my $subtests = $test->subtests;
$subtests->synopsis(sub {
my ($tryable) = @_;
# catch exception thrown by the synopsis
$tryable->catch('Path::Find::Error', sub {
return $_[0];
});
# test the exception
ok my $result = $tryable->result, 'result ok';
ok $result->isa('Path::Find::Error'), 'exception caught';
$result;
});
Additionally, another manual testing hook (with some automation) is the
example method. This hook evaluates (evals) a given example and returns
the result as a Test::Auto::Try object (see Data::Object::Try). The
first argument is the example ID (or number), for example:
my $subtests = $test->subtests;
$subtests->example(-1, 'children', 'method', sub {
my ($tryable) = @_;
ok my $result = $tryable->result, 'result ok';
$result; # for automated testing after the callback
});
Finally, the lesser-used but useful manual testing hook is the scenario
method. This hook evaluates (evals) a documented scenario and returns
the result as a Test::Auto::Try object (see Data::Object::Try), for
example:
my $subtests = $test->subtests;
$subtests->scenario('export-path-make', sub {
my ($tryable) = @_;
ok my $result = $tryable->result, 'result ok';
$result; # for automated testing after the callback
});
The test automation and document generation enabled through this
framework makes it easy to maintain source/test/documentation parity.
This also increases reusability and reduces the need for complicated
state and test setup.
AUTHOR
Al Newkirk, awncorp@cpan.org
LICENSE
Copyright (C) 2011-2019, Al Newkirk, et al.
This is free software; you can redistribute it and/or modify it under
the terms of the The Apache License, Version 2.0, as elucidated in the
"license file"
.
PROJECT
Wiki
Project
Initiatives
Milestones
Issues