{"id":14985694,"url":"https://github.com/mandelsoft/spiff","last_synced_at":"2025-10-24T09:49:07.853Z","repository":{"id":17720399,"uuid":"82348283","full_name":"mandelsoft/spiff","owner":"mandelsoft","description":"In-domain YAML templating engine spiff++","archived":false,"fork":false,"pushed_at":"2025-10-08T17:47:58.000Z","size":2771,"stargazers_count":45,"open_issues_count":8,"forks_count":16,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-10-09T01:38:41.578Z","etag":null,"topics":["bosh","bosh-template","golang","json","json-template","kubernetes-deployment","kubernetes-manifests","lambda-expressions","template-engine","yaml","yaml-template"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mandelsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-02-17T23:55:56.000Z","updated_at":"2025-07-30T01:41:41.000Z","dependencies_parsed_at":"2024-01-18T09:05:00.987Z","dependency_job_id":"ebfd6f19-4421-409e-a106-70e750a8ed5b","html_url":"https://github.com/mandelsoft/spiff","commit_stats":{"total_commits":485,"total_committers":43,"mean_commits":"11.279069767441861","dds":0.3237113402061855,"last_synced_commit":"16b5302bb9426b754d8cc26304acebc80345b341"},"previous_names":[],"tags_count":48,"template":false,"template_full_name":null,"purl":"pkg:github/mandelsoft/spiff","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mandelsoft%2Fspiff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mandelsoft%2Fspiff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mandelsoft%2Fspiff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mandelsoft%2Fspiff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mandelsoft","download_url":"https://codeload.github.com/mandelsoft/spiff/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mandelsoft%2Fspiff/sbom","scorecard":{"id":614813,"data":{"date":"2025-08-11","repo":{"name":"github.com/mandelsoft/spiff","commit":"c5d636ea32344f72e76ed5098c4f77b6c92df80b"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.4,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 1/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: goCommand not pinned by hash: scripts/test:7","Warn: goCommand not pinned by hash: scripts/test:16","Info:   0 out of   2 goCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v1.7.0-beta-7 not signed: https://api.github.com/repos/mandelsoft/spiff/releases/201996914","Warn: release artifact v1.7.0-beta-6 not signed: https://api.github.com/repos/mandelsoft/spiff/releases/179858604","Warn: release artifact v1.7.0-beta-5 not signed: https://api.github.com/repos/mandelsoft/spiff/releases/104255639","Warn: release artifact v1.7.0-beta-4 not signed: https://api.github.com/repos/mandelsoft/spiff/releases/93183422","Warn: release artifact v1.7.0-beta-3 not signed: https://api.github.com/repos/mandelsoft/spiff/releases/44916126","Warn: release artifact v1.7.0-beta-7 does not have provenance: https://api.github.com/repos/mandelsoft/spiff/releases/201996914","Warn: release artifact v1.7.0-beta-6 does not have provenance: https://api.github.com/repos/mandelsoft/spiff/releases/179858604","Warn: release artifact v1.7.0-beta-5 does not have provenance: https://api.github.com/repos/mandelsoft/spiff/releases/104255639","Warn: release artifact v1.7.0-beta-4 does not have provenance: https://api.github.com/repos/mandelsoft/spiff/releases/93183422","Warn: release artifact v1.7.0-beta-3 does not have provenance: https://api.github.com/repos/mandelsoft/spiff/releases/44916126"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'","Warn: branch protection not enabled for branch 'dev'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 1 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"10 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GO-2023-2402 / GHSA-45x7-px36-x8w8","Warn: Project is vulnerable to: GO-2024-3321 / GHSA-v778-237x-gjrc","Warn: Project is vulnerable to: GO-2025-3487 / GHSA-hcg3-q754-cr77","Warn: Project is vulnerable to: GO-2023-1988 / GHSA-2wrh-6pvc-2jm9","Warn: Project is vulnerable to: GO-2023-2102 / GHSA-4374-p667-p6c8","Warn: Project is vulnerable to: GHSA-qppj-fm5r-hxr3","Warn: Project is vulnerable to: GO-2024-2687 / GHSA-4v7x-pqxf-cx7m","Warn: Project is vulnerable to: GO-2024-3333","Warn: Project is vulnerable to: GO-2025-3503 / GHSA-qxp5-gwg8-xv66","Warn: Project is vulnerable to: GO-2025-3595 / GHSA-vvgc-356p-c3xw"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-21T03:37:05.547Z","repository_id":17720399,"created_at":"2025-08-21T03:37:05.547Z","updated_at":"2025-08-21T03:37:05.547Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280776485,"owners_count":26388950,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-24T02:00:06.418Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bosh","bosh-template","golang","json","json-template","kubernetes-deployment","kubernetes-manifests","lambda-expressions","template-engine","yaml","yaml-template"],"created_at":"2024-09-24T14:11:30.007Z","updated_at":"2025-10-24T09:49:07.782Z","avatar_url":"https://github.com/mandelsoft.png","language":"Go","readme":"```\n                                              _  __  __             \n                                    ___ _ __ (_)/ _|/ _|  _     _   \n                                   / __| '_ \\| | |_| |_ _| |_ _| |_ \n                                   \\__ \\ |_) | |  _|  _|_   _|_   _|\n                                   |___/ .__/|_|_| |_|   |_|   |_|  \n                                       |_|\n\n```\n\n---\n\n**NOTE**: *Active development on spiff is currently paused, including Pull Requests. `spiff++` is a fork of spiff that provides a compatible extension to spiff based on the latest version offering a rich set of new features not yet available in spiff. All fixes provided by the original spiff project will be incorporated into spiff++, also. Because there will be no way back to the spiff source base a new independent spiff++ repository has been created to continue development of spiff++.*\n---\n\n*spiff* is a command line tool and declarative in-domain hybrid YAML templating system. While regular templating systems process a template file by substituting the template expressions by values taken from \nexternal data sources, in-domain means that the templating engine knows about the syntax and structure of the processed template. It therefore can take the values for the template expressions directly\nfrom the document processed, including those parts denoted by the template expressions itself.\n\nFor example:\n```yaml\nresource:\n  name: bosh deployment\n  version: 25\n  url: (( \"http://resource.location/bosh?version=\" version ))\n  description: (( \"This document describes a \" name \" located at \" url ))\n```\n\nInstead of using only external value sources *spiff* provides a merging mechanism\nto merge a template with any number of merging stubs to produce a final document.\n\nIt is a command line tool and declarative YAML templating system, specially designed for generating deployment\nmanifests (for example BOSH, [Kubernetes](https://github.com/kubernetes) or\n[Landscaper](https://github.com/gardener/landscaper) manifests).\n\nBesides the CLI there is a golang library enabling the usage of the spiff\ntemplate processing in any GO program (for example [Landscaper](https://github.com/gardener/landscaper)).\n\nThe templating engine offers enabling access to the filesystem based on a\nconfigurable [virtual filesystem](https://github.com/mandelsoft/vfs)\nor the process system to execute commands and incorporate the output into the\ntemplate processing. \n\nContents:\n- [Installation](#installation)\n- [Usage](#usage)\n- [Feature Flags](#feature-flags)\n- [Libraries](#libraries)\n- [dynaml Templating Language](#dynaml-templating-language)\n\t- [(( foo ))](#-foo-)\n\t- [(( foo.bar.[1].baz ))](#-foobar1baz-)\n\t- [(( foo.[bar].baz ))](#-foobarbaz-)\n\t- [(( list.[1..3] ))](#-list13-)\n\t- [(( tag::foo ))](#-tagfoo-)\n\t- [(( 1.2e4 ))](#-12e4-)\n\t- [(( \"foo\" ))](#-foo-)\n\t- [(( [ 1, 2, 3 ] ))](#--1-2-3--)\n\t- [(( { \"alice\" = 25 } ))](#--alice--25--)\n\t- [(( ( \"alice\" = 25 ) alice ))](#--alice--25---alice-)\n\t- [(( foo bar ))](#-foo-bar-)\n\t\t- [(( \"foo\" bar ))](#-foo-bar--1)\n\t\t- [(( [1,2] bar ))](#-12-bar-)\n\t\t- [(( map1 map2 ))](#-map1-map2-)\n\t- [(( auto ))](#-auto-)\n\t- [(( merge ))](#-merge-)\n\t\t- [\u003c\u003c: (( merge ))](#--merge-)\n\t\t\t- [merging maps](#merging-maps)\n\t\t\t- [merging lists](#merging-lists)\n\t\t- [- \u003c\u003c: (( merge on key ))](#----merge-on-key-)\n\t\t- [\u003c\u003c: (( merge replace ))](#--merge-replace-)\n\t\t\t- [merging maps](#merging-maps-1)\n\t\t\t- [merging lists](#merging-lists-1)\n\t\t- [\u003c\u003c: (( foo )) ](#--foo-)\n\t\t\t- [merging maps](#merging-maps-2)\n\t\t\t- [merging lists](#merging-lists-2)\n\t\t- [\u003c\u003c: (( merge foo ))](#--merge-foo-)\n\t\t\t- [merging maps](#merging-maps-3)\n\t\t\t- [merging lists](#merging-lists-3)\n\t\t- [\u003c\u003c: (( merge none ))](#--merge-none-)\n\t- [(( a || b ))](#-a--b-)\n\t- [(( 1 + 2 * foo ))](#-1--2--foo-)\n\t- [(( \"10.10.10.10\" - 11 ))](#-10101010---11-)\n\t- [(( a \u003e 1 ? foo :bar ))](#-a--1--foo-bar-)\n\t- [(( 5 -or 6 ))](#-5--or-6-)\n\t- [Functions](#functions)\n\t\t- [(( format( \"%s %d\", alice, 25) ))](#-format-s-d-alice-25-)\n\t\t- [(( join( \", \", list) ))](#-join---list-)\n\t\t- [(( split( \",\", string) ))](#-split--string-)\n\t\t- [(( trim(string) ))](#-trimstring-)\n\t\t- [(( element(list, index) ))](#-elementlist-index-)\n\t\t- [(( element(map, key) ))](#-elementmap-key-)\n\t\t- [(( compact(list) ))](#-compactlist-)\n\t\t- [(( uniq(list) ))](#-uniqlist-)\n\t\t- [(( contains(list, \"foobar\") ))](#-containslist-foobar-)\n\t\t- [(( index(list, \"foobar\") ))](#-indexlist-foobar-)\n\t\t- [(( lastindex(list, \"foobar\") ))](#-lastindexlist-foobar-)\n\t\t- [(( basename(path) ))](#-basenamepath-)\n\t\t- [(( dirname(path) ))](#-dirnamepath-)\n\t\t- [(( parseurl(\"http://github.com\") ))](#-parseurlhttpgithubcom-)\n\t\t- [(( sort(list) ))](#-sortlist-)\n\t\t- [(( replace(string, \"foo\", \"bar\") ))](#-replacestring-foo-bar-)\n\t\t- [(( substr(string, 1, 2) ))](#-substrstring-1-2-)\n\t\t- [(( match(\"(f.*)(b.*)\", \"xxxfoobar\") ))](#-matchfb-xxxfoobar-)\n\t\t- [(( keys(map) ))](#-keysmap-)\n\t\t- [(( length(list) ))](#-lengthlist-)\n\t\t- [(( base64(string) ))](#-base64string-)\n\t\t- [(( hash(string) ))](#-hashstring-)\n\t\t- [(( bcrypt(\"password\", 10) ))](#-bcryptpassword-10-)\n\t\t- [(( bcrypt_check(\"password\", hash) ))](#-bcrypt_checkpassword-hash-)\n\t\t- [(( md5crypt(\"password\") ))](#-md5cryptpassword-)\n\t\t- [(( md5crypt_check(\"password\", hash) ))](#-md5crypt_checkpassword-hash-)\n\t\t- [(( decrypt(\"secret\") ))](#-decryptsecret-)\n\t\t- [(( rand(\"[:alnum:]\", 10) ))](#-randalnum-10-)\n\t\t- [(( type(foobar) ))](#-typefoobar-)\n\t\t- [(( defined(foobar) ))](#-definedfoobar-)\n\t\t- [(( valid(foobar) ))](#-validfoobar-)\n\t\t- [(( require(foobar) ))](#-requirefoobar-)\n\t\t- [(( stub(foo.bar) ))](#-stubfoobar-)\n\t\t- [(( tagdef(\"tag\", value) ))](#-tagdeftag-valiue-)\n\t\t- [(( eval(foo \".\" bar ) ))](#-evalfoo--bar--)\n\t\t- [(( env( HOME\" ) ))](#-envHOME--)\n\t\t- [(( static_ips(0, 1, 3) ))](#-static_ips0-1-3-)\n\t\t- [(( ipset(ranges, 3, 3,4,5,6) ))](#-ipsetranges-3-3456-)\n\t\t- [(( list_to_map(list, \"key\") ))](#-list_to_maplist-key-)\n\t\t- [(( makemap(fieldlist) ))](#-makemapfieldlist-)\n\t\t- [(( makemap(key, value) ))](#-makemapkey-value-)\n\t\t- [(( merge(map1, map2) ))](#-mergemap1-map2-)\n\t\t- [(( intersect(list1, list2) ))](#-intersectlist1-list2-)\n\t\t- [(( reverse(list) ))](#-reverselist-)\n\t\t- [(( parse(yamlorjson) ))](#-parseyamlorjson-)\n\t\t- [(( asjson(expr) ))](#-asjsonexpr-)\n\t\t- [(( asyaml(expr) ))](#-asjsonexpr-)\n\t\t- [(( catch(expr) ))](#-catchexpr-)\n\t\t- [(( validate(value,\"dnsdomain\") ))](#-validatevaluednsdomain-)\n\t\t- [(( check(value,\"dnsdomain\") ))](#-checkvaluednsdomain-)\n\t\t- [(( error(\"message\") ))](#-errormessage-)\n\t\t- [Math](#math)\n\t\t- [Conversions](#conversions)\n\t\t- [Accessing External Content](#accessing-external-content)\n\t\t    - [(( read(\"file.yml\") ))](#-readfileyml-)\n\t\t    - [(( exec(\"command\", arg1, arg2) ))](#-execcommand-arg1-arg2-)\n            - [(( pipe(data, \"command\", arg1, arg2) ))](#-pipedata-command-arg1-arg2-)\n\t\t    - [(( write(\"file.yml\", data) ))](#-writefileyml-data-)\n\t\t    - [(( tempfile(\"file.yml\", data) ))](#-tempfilefileyml-data-)\n\t\t    - [(( lookup_file(\"file.yml\", data) ))](#-lookup_filefileyml-list-)\n\t\t    - [(( mkdir(\"dir\", 0755) ))](#-mkdirdir-0755-)\n\t\t    - [(( list_files(\".\") ))](#-list_files-)\n\t\t    - [(( archive(files, \"tar\") ))](#-archivefiles-tar-)\n\t\t- [Semantic Versioning Functions](#semantic-versioning-functions)\n\t\t    - [(( semver(\"v1.2-beta.1\") ))](#-semverv12-beta1-)\n\t\t    - [(( semverrelease(\"v1.2.3-beta.1\") ))](#-semverreleasev123-beta1-)\n\t\t    - [(( semvermajor(\"1.2.3-beta.1\") ))](#-semvermajor123-beta1-)\n\t\t    - [(( semverminor(\"1.2.3-beta.1\") ))](#-semverminor123-beta1-)\n\t\t    - [(( semverpatch(\"1.2.3-beta.1\") ))](#-semverpatch123-beta1-)\n\t\t    - [(( semverprerelease(\"1.2.3-beta.1\") ))](#-semverprerelease123-beta1-)\n\t\t    - [(( semvermetadata(\"1.2.3+demo\") ))](#-semvermetadata123demo-)\n\t\t    - [(( semvercmp(\"1.2.3\", \"1.2.3-beta.1\") ))](#-semvercmp123-123-beta1-)\n\t\t    - [(( semvermatch(\"1.2.3\", \"~1.2\") ))](#-semvermatch123-12-)\n\t\t    - [(( semversort(\"1.2.3\", \"1.2.1\") ))](#-semversort123-121-)\n\t\t- [X509 Functions](#x509-functions)\n\t\t    - [(( x509genkey(spec) ))](#-x509genkeyspec-)\n\t\t    - [(( x509publickey(key) ))](#-x509publickeykey-)\n\t\t    - [(( x509cert(spec) ))](#-x509certspec-)\n\t\t- [Wireguard Functions](#wireguard-functions)\n            - [(( wggenkey() ))](#-wggenkey-)\n        \t- [(( wgpublickey(key) ))](#-wgpublickey-)\n\t- [(( lambda |x|-\u003ex \":\" port ))](#-lambda-x-x--port-)\n\t    - [Positional versus Named Argunments](#positional-versus-named-arguments)\n\t    - [Scopes and Lambda Expressions](#scopes-and-lambda-expressions)\n\t    - [Optional Parameters (( |x,y=2|-\u003e x * y ))](#optional-parameters)\n\t    - [Variable Argument Lists (( |x,y...|-\u003e x y ))](#variable-argument-lists)\n\t    - [Currying (( function*(1) ))](#currying)\n\t- [(( catch[expr|v,e|-\u003ev] ))](#-catchexprve-v-)\n\t- [(( sync[expr|v,e|-\u003edefined(v.field),v.field|10] ))](#-syncexprve-definedvfieldvfield10-)\n\t- [Inline List Expansion (( [a, list..., b] ))](#inline-list-expansion)\n\t- [Mappings](#mappings)\n\t\t- [(( map[list|elem|-\u003edynaml-expr] ))](#-maplistelem-dynaml-expr-)\n\t\t- [(( map[list|idx,elem|-\u003edynaml-expr] ))](#-maplistidxelem-dynaml-expr-)\n\t\t- [(( map[map|key,value|-\u003edynaml-expr] ))](#-mapmapkeyvalue-dynaml-expr-)\n\t\t- [(( map{map|elem|-\u003edynaml-expr} ))](#-mapmapelem-dynaml-expr-)\n\t\t- [(( map{list|elem|-\u003edynaml-expr} ))](#-maplistelem-dynaml-expr-)\n\t\t- [(( select[expr|elem|-\u003edynaml-expr] ))](#-selectexprelem-dynaml-expr-)\n\t\t- [(( select{map|elem|-\u003edynaml-expr} ))](#-selectmapelem-dynaml-expr-)\n\t- [Aggregations](#aggregations)\n\t\t- [(( sum[list|initial|sum,elem|-\u003edynaml-expr] ))](#-sumlistinitialsumelem-dynaml-expr-)\n\t\t- [(( sum[list|initial|sum,idx,elem|-\u003edynaml-expr] ))](#-sumlistinitialsumidxelem-dynaml-expr-)\n\t\t- [(( sum[map|initial|sum,key,value|-\u003edynaml-expr] ))](#-summapinitialsumkeyvalue-dynaml-expr-)\n\t- [Projections](#projections)\n\t    - [(( expr.[*].value ))](#-exprvalue-)\n\t\t- [(( list.[1..2].value ))](#-list12value-)\n\t- [Markers](#markers)\n\t    - [(( \u0026temporary ))](#-temporary-)\n\t    - [(( \u0026local ))](#-local-)\n    \t- [(( \u0026dynamic ))](#-dynamic-)\n    \t- [(( \u0026inject ))](#-inject-)\n    \t- [(( \u0026default ))](#-default-)\n    \t- [(( \u0026state ))](#-state-)\n    \t- [(( \u0026tag:name ))](#-tagname-)\n    - [Tags](#tags)\n        - [(( \u0026tag:name(value) ))](#-tagnamevalue-)\n        - [(( tag::foo ))](#-tagfoo-)\n        - [(( tag::. ))](#-tag-)\n        - [(( foo.bar::alice ))](#-foobaralice-)\n        - [Path Resolution for Tags](#path-resolution-for-tags)\n        - [Tags in Multi-Document Streams](#tags-in-multi-document-streams)\n\t- [Templates](#templates)\n\t\t- [\u003c\u003c: (( \u0026template ))](#--template-)\n\t\t- [(( *foo.bar ))](#-foobar-)\n\t- [Scope References](#scope-references)\n\t    - [_](#_)\n\t    - [__](#__)\n\t    - [___](#___)\n\t    - [__ctx.OUTER](#__ctxouter)\n\t- [Special Literals](#special-literals)\n\t- [Access to evaluation context](#access-to-evaluation-context)\n\t- [Operation Priorities](#operation-priorities)\n\t- [String Interpolation](#string-interpolation)\n\t- [YAML-based Control Structures](#yaml-based-control-structures)\n\t    - [in Maps](#control-structures-in-maps)\n\t    - [in Lists](#control-structures-in-lists)\n\t    - [`\u003c\u003cif:`](#if)\n\t    - [`\u003c\u003cswitch:`](#switch)\n\t    - [`\u003c\u003ctype:`](#type)\n\t    - [`\u003c\u003cfor:`](#for)\n\t        - [Lists as Iteration Result](#lists-as-iteration-result)\n\t        - [Maps as Iteration Result](#maps-as-iteration-result)\n\t    - [`\u003c\u003cmerge:`](#merge)\n- [Structural Auto-Merge](#structural-auto-merge)\n- [Bringing it all together](#bringing-it-all-together)\n- [Useful to Know](#useful-to-know)\n- [Error Reporting](#error-reporting)\n- [Using _spiff_ as Go Library](#using-spiff-as-go-library)\n\n\n# Installation\n\nOfficial release executable binaries can be downloaded via [Github releases](https://github.com/mandelsoft/spiff/releases) for Darwin, Linux and PowerPC machines (and virtual machines).\n\nSome of spiff's dependencies have changed since the last official release, and spiff will not be updated to keep up with these dependencies. THose dependencies are either fixed or copied into the local code base.\n\n# Usage\n\n### `spiff merge template.yml [template2.yml ...]`\n\nMerge a bunch of template files into one manifest, printing it out.\n\nSee 'dynaml templating language' for details of the template file, or examples/ subdir for more complicated examples.\n\nExample:\n\n```\nspiff merge cf-release/templates/cf-deployment.yml my-cloud-stub.yml\n```\n\nIt is possible to read one file from standard input by using the file\nname `-`. It may be used only once. This allows using spiff as part of a\npipeline to just process a single stream or to process a stream based on\nseveral templates/stubs.\n\nThe template file (first argument) may be a multiple-document stream\ncontaining multiple YAML documents separated by a line containing only `---`.\nEach YAML document will be processed independently with the given stub files.\nThe result is the stream of processed documents in the same order.\nIf a document's root node is marked as temporary, the document is omitted\nfrom the output stream.\nFor example, this can be used to generate *kubernetes* manifests to be used\nby `kubectl`.\n\nThe ` merge` command offers several options:\n\n- The option `--partial`. If this option is\n  given spiff handles incomplete expression evaluation. All errors are ignored\n  and the unresolvable parts of the yaml document are returned as strings.\n  \n- With the option `--json` the output will be in JSON format instead of YAML.\n\n- The option `--path \u003cpath\u003e` can be used to output a nested path, instead of the \n  the complete processed document.\n  \n- If the output is a list, the option `--split` outputs every list element as\n  separate documen. The _yaml_ format uses as usual `---` as separator line.\n  The _json_ format outputs a sequence of _json_ documents, one per line.\n  \n- With `--select \u003cfield path\u003e` it is possible to select a dedicated field of the\n  processed document for the output\n  \n- With `--evaluate \u003cdynaml expression\u003e` it is possible to evaluate a given dynaml\n  expression on the processed document for the output. The expression is evaluated\n  before the selection path is applied, which will then work on the evaluation\n  result.\n  \n- The option `--state \u003cpath\u003e` enables the state support of _spiff_. If the\n  given file exists it is put on top of the configured stub list for the\n  given file exists it is put on top of the configured stub list for the\n  merge processing. Additionally to the output of the processed document\n  it is filtered for nodes marked with the [`\u0026state` marker](#-state-).\n  This filtered document is then stored under the denoted file, saving the old\n  state file with the `.bak` suffix. This can be used together with a manual\n  merging as offered by the [state](libraries/state/README.md) utility library.\n  \n- With option `--bindings \u003cpath\u003e` a yaml file can be specified, whose content\n  is used to build additional bindings for the processing. The yaml document must\n  consist of a map. Each key is used as additional binding. The bindings document\n  is not processed, the values are used as defined.\n\n- With option `--tag \u003ctag\u003e:\u003cpath\u003e` a yaml file can be specified, whose content\n  is used as value for a predefined global tag (see [Tags](#tags)).\n  Tags can be accessed by reference expressions of the form `\u003ctag\u003e::\u003cref\u003e`.\n  In contrast to bindings tagged content does not compete with the nodes\n  in the document, it uses another reference namespace.\n\n- With option `--define \u003ckey\u003e=\u003cvalue\u003e` (shorthand`-D`) additional binding values\n  can be specified on the command line overriding binding values from the\n  binding file. The option may occur multiple times.\n\n  If the *key* contains dots (`.`), it will be interpreted as path expression to \n  describe fields in deep map values. A dot (and a `\\` before a dot) can be escaped\n  by `\\` to keep it in the field name.\n  \n- The option `--preserve-escapes` will preserve the escaping for dynaml\n  expressions and list/map merge directives. This option can be used\n  if further processing steps of a processing result with *spiff* is intended.\n\n- The option `--preserve-temporary` will preserve the fields marked as temporary\n  in the final document.\n  \n- The option `--features=\u003cfeaturelist\u003e` will enable this given features. New\n  features that are incompatible with the old behaviour must be explicitly \n  enabled. Typically those feature do not break the common behavior but introduce\n  a dedicated interpretation for yaml values that were used as regular values\n  before.\n  \nThe folder [libraries](libraries/README.md) offers some useful\nutility libraries. They can also be used as an example for the power\nof this templating engine.\n\n\n### `spiff diff manifest.yml other-manifest.yml`\n\nShow structural differences between two deployment manifests.\nHere streams with multiple documents are supported, also.\nTo indicate no difference the number of documents in both streams must be\nidentical and each document in the first stream must have no difference\ncompared to the document with the same index in the second stream.\nFound differences are shown for each document separately.\n\nUnlike basic diffing tools and even `bosh diff`, this command has semantic\nknowledge of a deployment manifest, and is not just text-based. For example,\nif two manifests are the same except they have some jobs listed in different\norders, `spiff diff` will detect this, since job order matters in a manifest.\nOn the other hand, if two manifests differ only in the order of their\nresource pools, for instance, then it will yield and empty diff since\nresource pool order doesn't actually matter for a deployment.\n\nAlso unlike `bosh diff`, this command doesn't modify either file.\n\nIt's tailed for checking differences between one deployment and the next.\n\nTypical flow:\n\n```sh\n$ spiff merge template.yml [templates...] \u003e deployment.yml\n$ bosh download manifest [deployment] current.yml\n$ spiff diff deployment.yml current.yml\n$ bosh deployment deployment.yml\n$ bosh deploy\n```\n\n### `spiff convert --json manifest.yml `\n\nThe `convert` sub command can be used to convert input files to json or\njust to normalize the order of the fields.\nAvailable options are `--json`, `--path`, `--split` or `--select` according\nto their meanings for the `merge` sub command.\n\n### `spiff encrypt secret.yaml`\n\nThe `encrypt` sub command can be used to encrypt or decrypt data\naccording to the [`encrypt`](#-decryptsecret-) dynaml function.\nThe password can be given as second argument or it is taken from the\nenvironment variable `SPIFF_ENCRYPTION_KEY`. The last argument can be used\nto pass the encryption method (see [`encrypt` function](#-decryptsecret-))\n\nThe data is taken from the specified file. If `-` is given, it is read from\nstdin.\n\nIf the option `-d` is given, the data is decrypted, otherwise the data is\nread as yaml document and the encrypted result is printed. \n\n# Feature Flags\n\nNew features that are incompatible with the old behaviour must be explicitly \nenabled. Typically those features do not break the common behavior but introduce\na dedicated interpretation for yaml values that were used as regular values\nbefore and can therefore break existing use cases.\n  \nThe following feature flags are currently supported:\n\n| Feature | Since | State | Meaning |\n|---------|-------|-------|---------|\n| `interpolation` | 1.7.0-beta-1 | alpha | [dynaml as part of yaml strings](#string-interpolation) |\n| `control` | 1.7.0-beta-4 | alpha | [yaml based control structures](#yaml-based-control-structures) | \n\nActive feature flags can be queried using the *dynaml* function\n`features()` as list of strings. If this function is called with a string\nargument, it returns whether the given feature is currenty enabled.\n\nFeatures can be enabled by command line using the `--features` option,\nby the go library using the `WithFeatures` function or generally\nby setting the environment variable `SPIFF_FEATURES` to a feature list.\nThis setting is alwas used as default. By using the `Plain()` spiff\nsettings from the go library all environment variables are ignored.\n\nA feature can be specified by name or by name prepended with the prefix `no`\nto disable it.\n \n# Libraries\n\nThe [libraries](libraries/README.md) folder contains some useful _spiff_ template\nlibraries. These are basically just stubs that are added to the merge file list\nto offer the utility functions for the merge processing.\n\n# dynaml Templating Language\n\nSpiff uses a declarative, logic-free templating language called 'dynaml'\n(dynamic yaml).\n\nEvery dynaml node is guaranteed to resolve to a YAML node. It is *not*\nstring interpolation. This keeps developers from having to think about how\na value will render in the resulting template.\n\nA dynaml node appears in the .yml file as a string denoting an expression\nsurrounded by two parentheses `(( \u003cdynaml\u003e ))`. They can be used as the\nvalue of a map or an entry in a list. The expression might span multiple\nlines. In any case the yaml string value *must not* end with a newline\n(for example using `|-`)\n\nIf a parenthesized value should not be interpreted as an *dynaml* expression and\nkept as it is in the output, it can be escaped by an exclamation mark directly\nafter the openeing brackets.\n\nFor example, `((! .field ))` maps to the string value `(( .field ))` and\n`((!! .field ))` maps to the string value `((! .field ))`.\n\nThe following is a complete list of dynaml expressions:\n\n\n## `(( foo ))`\n\nLook for the nearest 'foo' key (i.e. lexical scoping) in the current\ntemplate and bring it in.\n\ne.g.:\n\n```yaml\nfizz:\n  buzz:\n    foo: 1\n    bar: (( foo ))\n  bar: (( foo ))\nfoo: 3\nbar: (( foo ))\n```\n\nThis example will resolve to:\n\n```yaml\nfizz:\n  buzz:\n    foo: 1\n    bar: 1\n  bar: 3\nfoo: 3\nbar: 3\n```\n\nThe following will not resolve because the key name is the same as the value to be merged in:\n```yaml\nfoo: 1\n\nhi:\n  foo: (( foo ))\n```\n\n## `(( foo.bar.[1].baz ))`\n\nLook for the nearest 'foo' key, and from there follow through to `.bar.[1].baz`.\n\nA path is a sequence of steps separated by dots. A step is either a word for\nmaps, or digits surrounded by brackets for list indexing. The index might be negative (a minus followed by digits). Negative indices are taken from then end\nof the list (effective index = index + length(list)).\n\nA path that cannot be resolved lead to an evaluation error. If a reference is\nexpected to sometimes not be provided, it should be\nused in combination with '||' (see [below](#-a--b-)) to guarantee resolution.\n\n**Note**: The dynaml grammer has been reworked to enable the usual index syntax,\nnow. Instead of `foo.bar.[1]` it is possible now to use `foo.bar[1]`.\n\n**Note**: References are always within the template or stub, and order does not\nmatter. You can refer to another dynamic node and presume it's resolved, and the\nreference node will just eventually resolve once the dependent node resolves.\n\ne.g.:\n\n```yaml\nproperties:\n  foo: (( something.from.the.stub ))\n  something: (( merge ))\n```\n\nThis will resolve as long as 'something' is resolveable, and as long as it\nbrings in something like this:\n\n```yaml\nfrom:\n  the:\n    stub: foo\n```\n\nIf the path starts with a dot (`.`) the path is always evaluated from the root\nof the document. If the document root is a list, the first map level is used to resolve the path expression if it starts with `.__map`. This can be used to avoid the need \nfor using the own list index (like `.[1].path`), which might change if\nlist entries are added.\n\nList entries consisting of a map with `name` field can directly be addressed\nby their name value as path component. \n\n**Note**: This also works for the absolute paths for list documents.\n\ne.g.:\n\nThe age of alice in\n\n```yaml\nlist:\n - name: alice\n   age: 25\n```\n\ncan be referenced by using the path `list.alice.age`, instead of `list[0].age`.\n\nBy default a field with name `name` is used as key field. If another field\nshould be used as key field, it can be marked in one list entry as key by\nprefixing the field name with the keyword `key:`. This keyword is removed\nfrom by the processing and will not be part of the final processing result.\n\ne.g.:\n\n```yaml\nlist:\n - key:person: alice\n   age: 25\n\nalice: (( list.alice ))\n```\n\nwill be resolved to\n\n```yaml\nlist:\n - person: alice\n   age: 25\n\nalice:\n  person: alice\n  age: 25\n```\n\nThis new key field will also be observed during the merging of lists.\n\nIf the selected key field starts with a `!`, the key feature is disabled.\nThe exclamation mark is removed from the effective field name, also.\n\nIf the values for the key field are not unqiue, it is disables, also.\n\n## `(( foo.[bar].baz ))`\n\nLook for the nearest 'foo' key, and from there follow through to the\nfield(s) described by the expression `bar` and then to .baz.\n\nThe index may be an integer constant (without spaces) as described in the\nlast section. But it might also be an arbitrary dynaml expression (even\nan integer, but with spaces). If the expression evaluates to a string,\nit lookups the dedicated field. If the expression evaluates to an integer,\nthe array element with this index is addressed. The dot (`.`) in front of the index operator is optional.\n\ne.g.:\n\n```yaml\nproperties:\n  name: alice\n  foo: (( values.[name].bar ))\n  values:\n    alice:\n      bar: 42\n```\n\nThis will resolve `foo` to the value `42`. The dynamic index may also be at\nthe end of the expression (without `.bar`).\n\nBasically this is the simplier way to express something like\n[eval(\"values.\" name \".bar\")](#-eval-foo--bar--)\n\nIf the expression evaluates to a list, the list elements (strings or integers)\nare used as path elements to access deeper fields.\n\ne.g.:\n\n```yaml\nproperties:\n  name:\n   - foo\n   - bar\n  foo: (( values.[name] ))\n  values:\n    foo:\n      bar: 42\n```\n\nresolves `foo` again to the value `42`.\n\n**Note**: The index operator is usable on the root element (`.[index]`), also.\n\nIt is possible, to specify multiple comma separated indicies to successive lists\n(`foo[0][1]` is equivalent to `foo[0,1]). In such case the indices may not be again lists.\n\n## `(( list.[1..3] ))`\n\nThe slice expression can be used to extract a dedicated sub list from a list\nexpression. The range *start* `..` *end* extracts a list of the length\n*end-start+1* with the elements from\nindex *start* to *end*. If the start index is negative the slice is taken\nfrom the end of the list from *length+start* to *length+end*. If the end\nindex is lower than the start index, the result is an empty array.\n\ne.g.:\n\n```yaml\nlist:\n  - a\n  - b\n  - c\nfoo: (( list.[1..length(list) - 1] ))\n```\n\nThe start or end index might be omitted. It is then selected according to the \nactual size of the list. Therefore `list.[1..length(list)]` is equivalent\nto `list.[1..]`.\n\nevaluates `foo` to the list `[b,c]`.\n\n## `(( 1.2e4 ))`\n\nNumber literatls are supported for integers and floating point values.\n\n## `(( \"foo\" ))`\n\nString literal. All [json string encodings](https://www.json.org/) are supported\n(for exmple `\\n`, `\\\"` or `\\uxxxx`).\n\n## `(( [ 1, 2, 3 ] ))`\n\nList literal. The list elements might again be expressions. There is a special list literal `[1 .. -1]`, that can be used to resolve an increasing or descreasing number range to a list.\n\ne.g.:\n\n```yaml\nlist: (( [ 1 .. -1 ] ))\n```\n\nyields\n\n```yaml\nlist:\n  - 1\n  - 0\n  - -1\n```\n\n## `(( { \"alice\" = 25 } ))`\n\nThe map literal can be used to describe maps as part of a dynaml expression. Both,\nthe key and the value, might again be expressions, whereby the key expression must\nevaluate to a string. This way it is possible to create maps with non-static keys.\nThe assignment operator `=` has been chosen instead of the regular colon `:`\ncharacter used in yaml, because this would result in conflicts with the yaml\nsyntax.\n\nA map literal might consist of any number of field assignments separated by a\ncomma `,`.\n\ne.g.:\n\n```yaml\nname: peter\nage: 23\nmap: (( { \"alice\" = {}, name = age } ))\n```\n\nyields\n\n```yaml\nname: peter\nage: 23\nmap:\n  alice: {}\n  peter: 23\n```\n\nAnother way to compose lists based on expressions are the functions\n[`makemap`](#-makemapkey-value-) and [`list_to_map`](#-list_to_maplist-key-).\n\n## `(( ( \"alice\" = 25 ) alice ))`\n\nAny expression may be preluded by any number of explicit _scope literals_. A\nscope literal describes a map whose values are available for relative reference \nresolution of the expression (static scope). It creates an additional local\nbinding for given names.\n\nA scope literal might consist of any number of field assignments separated by a\ncomma `,`. The key as well as the value are given by expressions, whereas the\nkey expression must evaluate to a string. All expressions are evaluated in the\nnext outer scope, this means later settings in a scope _cannot_ use earlier\nsettings in the same scope literal. \n\ne.g.:\n\n```yaml\nscoped: (( ( \"alice\" = 25, \"bob\" = 26 ) alice + bob ))\n```\n\nyields\n\n```yaml\nscoped: 51\n```\n\nA field name might also be denoted by a symbol (_`$`name_).\n\n## `(( foo bar ))`\n\nConcatenation expression used to concatenate a sequence of dynaml expressions.\n\n### `(( \"foo\" bar ))`\n\nConcatenation (where bar is another dynaml expr). Any sequences of simple values (string, integer and boolean) can be concatenated, given by any dynaml expression.\n\ne.g.:\n\n```yaml\ndomain: example.com\nuri: (( \"https://\" domain ))\n```\n\nIn this example `uri` will resolve to the value `\"https://example.com\"`.\n\n### `(( [1,2] bar ))`\n\nConcatenation of lists as expression (where bar is another dynaml expr). Any sequences of lists can be concatenated, given by any dynaml expression.\n\ne.g.:\n\n```yaml\nother_ips: [ 10.0.0.2, 10.0.0.3 ]\nstatic_ips: (( [\"10.0.1.2\",\"10.0.1.3\"] other_ips ))\n```\n\nIn this example `static_ips` will resolve to the value `[ 10.0.1.2, 10.0.1.3, 10.0.0.2, 10.0.0.3 ] `.\n\nIf the second expression evaluates to a value other than a list (integer, boolean, string or map), the value is appended to the first list.\n\ne.g.:\n\n```yaml\nfoo: 3\nbar: (( [1] 2 foo \"alice\" ))\n```\nyields the list `[ 1, 2, 3, \"alice\" ]` for `bar`.\n\n### `(( map1 map2 ))`\n\nConcatenation of maps as expression. Any sequences of maps can be concatenated, given by any dynaml expression. Thereby entries will be merged. Entries with the same key are overwritten from left to right.\n\ne.g.:\n\n```yaml\nfoo:\n  alice: 24\n  bob: 25\n\nbar:\n  bob: 26\n  paul: 27\n\nconcat: (( foo bar ))\n```\n\nyields\n\n```yaml\nfoo:\n  alice: 24\n  bob: 25\n\nbar:\n  bob: 26\n  paul: 27\n\nconcat:\n  alice: 24\n  bob: 26\n  paul: 27\n```\n\n## `(( auto ))`\n\nContext-sensitive automatic value calculation.\n\nIn a resource pool's 'size' attribute, this means calculate based on the total\ninstances of all jobs that declare themselves to be in the current resource\npool.\n\ne.g.:\n\n```yaml\nresource_pools:\n  - name: mypool\n    size: (( auto ))\n\njobs:\n  - name: myjob\n    resource_pool: mypool\n    instances: 2\n  - name: myotherjob\n    resource_pool: mypool\n    instances: 3\n  - name: yetanotherjob\n    resource_pool: otherpool\n    instances: 3\n```\n\nIn this case the resource pool size will resolve to '5'.\n\n## `(( merge ))`\n\nBring the current path in from the stub files that are being merged in.\n\ne.g.:\n\n```yaml\nfoo:\n  bar:\n    baz: (( merge ))\n```\n\nWill try to bring in `foo.bar.baz` from the first stub, or the second, etc.,\nreturning the value from the last stub that provides it.\n\nIf the corresponding value is not defined, it will return nil. This then has the\nsame semantics as reference expressions; a nil merge is an unresolved template.\nSee [`||`](#-a--b-).\n\n### `\u003c\u003c: (( merge ))`\n\nMerging of maps or lists with the content of the same element found in some stub.\n\n** Attention **\nThis form of `merge` has a compatibility propblem. In versions before 1.0.8, this expression\nwas never parsed, only the existence of the key `\u003c\u003c:` was relevant. Therefore there are often\nusages of `\u003c\u003c: (( merge ))` where `\u003c\u003c: (( merge || nil ))` is meant. The first variant would\nrequire content in at least one stub (as always for the merge operator). Now this expression\nis evaluated correctly, but this would break existing manifest template sets, which use the\nfirst variant, but mean the second. Therfore this case is explicitly handled to describe an\noptional merge. If really a required merge is meant an additional explicit qualifier has to\n\n**Note**: Instead of using a `\u003c\u003c:` insert field to place merge expressions it is\npossible now to use `\u003c\u003c\u003c:`, also, which allows to use regular yaml parsers for\nspiff-like yaml documents. `\u003c\u003c:` is kept for backward compatibility.\nbe used (`(( merge required ))`).\n\nIf the merge key should not be interpreted as regular key instead of a merge\ndirective, it can be escaped by an excalamtion mark (`!`).\n\nFor example, a map key `\u003c\u003c\u003c!` will result in a string key `\u003c\u003c\u003c` and `\u003c\u003c\u003c!!`\nwill result in a string key `\u003c\u003c\u003c!`\n\n#### Merging maps\n\n**values.yml**\n```yaml\nfoo:\n  a: 1\n  b: 2\n```\n\n**template.yml**\n```yaml\nfoo:\n  \u003c\u003c: (( merge ))\n  b: 3\n  c: 4\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nfoo:\n  a: 1\n  b: 2\n  c: 4\n```\n\n#### Merging lists\n\n**values.yml**\n```yaml\nfoo:\n  - 1\n  - 2\n```\n\n**template.yml**\n```yaml\nfoo:\n  - 3\n  - \u003c\u003c: (( merge ))\n  - 4\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nfoo:\n  - 3\n  - 1\n  - 2\n  - 4\n```\n\n### `- \u003c\u003c: (( merge on key ))`\n\n`spiff` is able to merge lists of maps with a key field. Those lists are handled like maps with the value of the key field as key. By default the key `name` is used. But with the selector `on` an arbitrary key name can be specified for a list-merge expression.\n\ne.g.:\n\n```yaml\nlist:\n  - \u003c\u003c: (( merge on key ))\n  - key: alice\n    age: 25\n  - key: bob\n    age: 24\n```\n\nmerged with\n\n```yaml\nlist:\n  - key: alice\n    age: 20\n  - key: peter\n    age: 13\n```\n\nyields\n\n```yaml\nlist:\n  - key: peter\n    age: 13\n  - key: alice\n    age: 20\n  - key: bob\n    age: 24\n```\n\nIf no insertion of new entries is desired (as requested by the insertion merge expression), but only overriding of existent entries, one existing key field can be prefixed with the tag `key:` to indicate a non-standard key name, for example `- key:key: alice`.\n\n### `\u003c\u003c: (( merge replace ))`\n\nReplaces the complete content of an element by the content found in some stub instead of doing a deep merge for the existing content.\n\n#### Merging maps\n\n**values.yml**\n```yaml\nfoo:\n  a: 1\n  b: 2\n```\n\n**template.yml**\n```yaml\nfoo:\n  \u003c\u003c: (( merge replace ))\n  b: 3\n  c: 4\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nfoo:\n  a: 1\n  b: 2\n```\n\n#### Merging lists\n\n**values.yml**\n```yaml\nfoo:\n  - 1\n  - 2\n```\n\n**template.yml**\n```yaml\nfoo:\n  - \u003c\u003c: (( merge replace ))\n  - 3\n  - 4\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nfoo:\n  - 1\n  - 2\n```\n\n### `\u003c\u003c: (( foo ))`\n\nMerging of maps and lists found in the same template or stub.\n\n#### Merging maps\n\n```yaml\nfoo:\n  a: 1\n  b: 2\n\nbar:\n  \u003c\u003c: (( foo )) # any dynaml expression\n  b: 3\n```\n\nyields:\n\n```yaml\nfoo:\n  a: 1\n  b: 2\n\nbar:\n  a: 1\n  b: 3\n```\n\nThis expression just adds new entries to the actual list. It does not merge\nexisting entries with the content described by the merge expression.\n\n#### Merging lists\n\n```yaml\nbar:\n  - 1\n  - 2\n\nfoo:\n  - 3\n  - \u003c\u003c: (( bar ))\n  - 4\n```\n\nyields:\n\n```yaml\nbar:\n  - 1\n  - 2\n\nfoo:\n  - 3\n  - 1\n  - 2\n  - 4\n```\n\nA common use-case for this is merging lists of static ips or ranges into a list of ips. Another possibility is to use a single [concatenation expression](#-12-bar-).\n\n### `\u003c\u003c: (( merge foo ))`\n\nMerging of maps or lists with the content of an arbitrary element found in some stub (Redirecting merge). There will be no further (deep) merge with the element of the same name found in some stub. (Deep merge of lists requires maps with field `name`)\n\nRedirecting merges can be used as direct field value, also. They can be combined with replacing merges like `(( merge replace foo ))`.\n\n#### Merging maps\n\n**values.yml**\n```yaml\nfoo:\n  a: 10\n  b: 20\n\nbar:\n  a: 1\n  b: 2\n```\n\n**template.yml**\n```yaml\nfoo:\n  \u003c\u003c: (( merge bar))\n  b: 3\n  c: 4\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nfoo:\n  a: 1\n  b: 2\n  c: 4\n```\n\nAnother way doing a merge with another element in some stub could also be done the traditional way:\n\n**values.yml**\n```yaml\nfoo:\n  a: 10\n  b: 20\n\nbar:\n  a: 1\n  b: 2\n```\n\n**template.yml**\n```yaml\nbar:\n  \u003c\u003c: (( merge ))\n  b: 3\n  c: 4\n\nfoo: (( bar ))\n```\n\nBut in this scenario the merge still performs the deep merge with the original element name. Therefore\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nbar:\n  a: 1\n  b: 2\n  c: 4\nfoo:\n  a: 10\n  b: 20\n  c: 4\n```\n\n#### Merging lists\n\n**values.yml**\n```yaml\nfoo:\n  - 10\n  - 20\n\nbar:\n  - 1\n  - 2\n```\n\n**template.yml**\n```yaml\nfoo:\n  - 3\n  - \u003c\u003c: (( merge bar ))\n  - 4\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nfoo:\n  - 3\n  - 1\n  - 2\n  - 4\n```\n\n### `\u003c\u003c: (( merge none ))`\n\nIf the reference of an redirecting merge is set to the constant `none`,\nno merge is done at all. This expressions always yields the nil value.\n\ne.g.: for\n\n**template.yml**\n```yaml\nmap:\n  \u003c\u003c: (( merge none ))\n  value: notmerged\n```\n\n**values.yml**\n```yaml\nmap:\n  value: merged\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\nmap:\n  value: notmerged\n```\n\nThis can be used for explicit field merging using the `stub` function\nto access dedicated parts of upstream stubs.\n\ne.g.:\n\n**template.yml**\n```yaml\nmap:\n  \u003c\u003c: (( merge none ))\n  value: ((  \"alice\"  \"+\" stub(map.value) ))\n```\n\n**values.yml**\n```yaml\nmap:\n  value: bob\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\ntest:\n  value: alice+bob\n```\n\nThis also works for dedicated fields:\n\n**template.yml**\n```yaml\nmap:\n  value: ((  merge none // \"alice\"  \"+\" stub() ))\n```\n\n**values.yml**\n```yaml\nmap:\n  value: bob\n```\n\n`spiff merge template.yml values.yml` yields:\n\n```yaml\ntest:\n  value: alice+bob\n```\n\n## `(( a || b ))`\n\nUses a, or b if a cannot be resolved.\n\ne.g.:\n\n```yaml\nfoo:\n  bar:\n    - name: some\n    - name: complicated\n    - name: structure\n\nmything:\n  complicated_structure: (( merge || foo.bar ))\n```\n\nThis will try to merge in `mything.complicated_structure`, or, if it cannot be\nmerged in, use the default specified in `foo.bar`.\n\nThe operator `//` additionally checks, whether `a` can be solved to a valid \nvalue (not equal `~`).\n\n## `(( 1 + 2 * foo ))`\n\nDynaml expressions can be used to execute arithmetic integer and floating-point calculations. Supported operations are `+`, `-`, `*`, and `/`.\nThe modulo operator (`%`) only supports integer operands.\n\ne.g.:\n\n**values.yml**\n```yaml\nfoo: 3\nbar: (( 1 + 2 * foo ))\n```\n\n`spiff merge values.yml` yields `7` for `bar`. This can be combined with [concatentions](#-foo-bar-) (calculation has higher priority than concatenation in dynaml expressions):\n\n```yaml\nfoo: 3\nbar: (( foo \" times 2 yields \" 2 * foo ))\n```\nThe result is the string `3 times 2 yields 6`.\n\n## `(( \"10.10.10.10\" - 11 ))`\n\nBesides arithmetic on integers it is also possible to use addition and subtraction on ip addresses and cidrs.\n\ne.g.:\n\n```yaml\nip: 10.10.10.10\nrange: (( ip \"-\" ip + 247 + 256 * 256 ))\n```\n\nyields\n\n```yaml\nip: 10.10.10.10\nrange: 10.10.10.10-10.11.11.1\n```\n\nSubtraction also works on two IP addresses or cidrs to calculate the number of\nIP addresses between two IP addresses.\n\ne.g.:\n\n```yaml\ndiff: (( 10.0.1.0 - 10.0.0.1 + 1 ))\n```\n\nyields the value 256. IP address constants can be directly used in dynaml\nexpressions. They are implicitly converted to strings and back to IP\naddresses if required by an operation.\n\nMultiplication and division can be used to handle IP range shifts on CIDRs.\nWith division a network can be partioned. The network size is increased\nto allow at least a dedicated number of subnets below the original CIDR.\nMultiplication then can be used to get the n-th next subnet of the same\nsize.\n\ne.g.:\n\n```yaml\nsubnet: (( \"10.1.2.1/24\" / 12 ))  # first subnet CIDR for 16 subnets\nnext: (( \"10.1.2.1/24\" / 12 * 2)) # 2nd next (3rd) subnet CIDRS\n```\n\nyields\n\n```yaml\nsubnet: 10.1.2.0/28\nnext: 10.1.2.32/28\n```\n\nAdditionally there are functions working on IPv4 CIDRs:\n\n```yaml\ncidr: 192.168.0.1/24\nrange: (( min_ip(cidr) \"-\" max_ip(cidr) ))\nnext: (( max_ip(cidr) + 1 ))\nnum: (( min_ip(cidr) \"+\" num_ip(cidr) \"=\" min_ip(cidr) + num_ip(cidr) ))\ncontains: (( contains_ip(cidr, \"192.168.0.2\") ))\n```\n\nyields\n\n```yaml\ncidr: 192.168.0.1/24\nrange: 192.168.0.0-192.168.0.255\nnext: 192.168.1.0\nnum: 192.168.0.0+256=192.168.1.0\ncontains: true\n```\n\n## `(( a \u003e 1 ? foo :bar ))`\n\nDynaml supports the comparison operators `\u003c`, `\u003c=`, `==`, `!=`, `\u003e=` and `\u003e`. The comparison operators work on\ninteger values. The checks for equality also work on lists and maps. The result is always a boolean value. To negate a condition the unary not opertor (`!`) can be used.\n\nAdditionally there is the ternary conditional operator `?:`, that can be used to evaluate expressions depending on a condition. The first operand is used as condition. The expression is evaluated to the second operand, if the condition is true, and to the third one, otherwise.\n\ne.g.:\n\n```yaml\nfoo: alice\nbar: bob\nage: 24\nname: (( age \u003e 24 ? foo :bar ))\n```\n\nyields the value `bob` for the property `name`.\n\nAn expression is considered to be `false` if it evaluates to\n- the boolean value `false`\n- the integer value 0\n- an empty string, map or list\n\nOtherwise it is considered to be `true`\n\n\n**Remark**\n\nThe use of the symbol `:` may collide with the yaml syntax, if the complete expression is not a quoted string value.\n\nThe operators `-or` and `-and` can be used to combine comparison operators to compose more complex conditions.\n\n**Remark:**\n\nThe more traditional operator symbol `||` (and `\u0026\u0026`) cannot be used here, because the operator `||` already exists in dynaml with a different semantic, that does not hold for logical operations. The expression `false || true` evaluates to `false`, because it yields the first operand, if it is defined, regardless of its value. To be as compatible as possible this cannot be changed and the bare symbols `or` and `and` cannot be be used, because this would invalidate the concatenation of references with such names.\n\n## `(( 5 -or 6 ))`\n\nIf both sides of an `-or` or `-and` operator evaluate to integer values, a bit-wise operation is executed and the result is again an integer. Therefore the expression `5 -or 6` evaluates to `7`.\n\n## Functions\n\nDynaml supports a set of predefined functions. A function is generally called like\n\n```yaml\nresult: (( functionname(arg, arg, ...) ))\n```\n\nAdditional functions may be defined as part of the yaml document using [lambda expressions](#-lambda-x-x--port-). The function name then is either a grouped expression or the path to the node hosting the lambda expression.\n\n### `(( format( \"%s %d\", alice, 25) ))`\n\nFormat a string based on arguments given by dynaml expressions. There is a second flavor of this function: `error` formats an error message and sets the evaluation to failed.\n\n\n### `(( join( \", \", list) ))`\n\nJoin entries of lists or direct values to a single string value using a given separator string. The arguments to join can be dynaml expressions evaluating to lists, whose values again are strings or integers, or string or integer values.\n\ne.g.:\n\n```yaml\nalice: alice\nlist:\n  - foo\n  - bar\n\njoin: (( join(\", \", \"bob\", list, alice, 10) ))\n```\n\nyields the string value `bob, foo, bar, alice, 10` for `join`.\n\n### `(( split( \",\", string) ))`\n\nSplit a string for a dedicated separator. The result is a list.\nInstead of a separator string an integer value might be given,\nwhich splits the give string into list of length limited strings.\nThe length is counted in runes, not bytes.\n\ne.g.:\n\n```yaml\nlist: (( split(\",\" \"alice, bob\") ))\nlimited: (( split(4, \"1234567890\") ))\n\n```\n\nyields:\n\n```yaml\nlist:\n  - alice\n  - ' bob'\nlimited:\n  - \"1234\"\n  - \"5678\"\n  - \"90\"\n\n```\n\nAn optional 3rd argument might be specified. It limits the number of returned\nlist entries. The value -1 leads to an unlimited list length.\n\nIf a [regular expression](https://github.com/google/re2/wiki/Syntax) should\nbe used as separator string, the function `split_match` can be used.\n\n\n### `(( trim(string) ))`\n\nTrim a string or all elements of a list of strings. There is an optional second string argument. It can be used to specify a set of characters that will be cut. The default cut set consists of a space and a tab character.\n\ne.g.:\n\n```yaml\nlist: (( trim(split(\",\" \"alice, bob\")) ))\n```\n\nyields:\n\n```yaml\nlist:\n  - alice\n  - bob\n```\n\n### `(( element(list, index) ))`\n\nReturn a dedicated list element given by its index.\n\ne.g.:\n\n```yaml\nlist: (( trim(split(\",\" \"alice, bob\")) ))\nelem: (( element(list,1) ))\n```\n\nyields:\n\n```yaml\nlist:\n  - alice\n  - bob\nelem: bob\n```\n\n### `(( element(map, key) ))`\n\nReturn a dedicated map field given by its key.\n\n```yaml\nmap:\n  alice: 24\n  bob: 25\nelem: (( element(map,\"bob\") ))\n```\n\nyields:\n\n```yaml\nmap:\n  alice: 24\n  bob: 25\nelem: 25\n```\n\nThis function is also able to handle keys containing dots (.).\n\n### `(( compact(list) ))`\n\nFilter a list omitting empty entries.\n\ne.g.:\n\n```yaml\nlist: (( compact(trim(split(\",\" \"alice, , bob\"))) ))\n```\n\nyields:\n\n```yaml\nlist:\n  - alice\n  - bob\n```\n\n### `(( uniq(list) ))`\n\nUniq provides a list without dupliates.\n\ne.g.:\n\n```yaml\nlist:\n- a\n- b\n- a\n- c\n- a\n- b\n- 0\n- \"0\"\nuniq: (( uniq(list) ))\n```\n\nyields for field `uniq`:\n\n```yaml\nuniq:\n- a\n- b\n- c\n- 0\n```\n\n### `(( contains(list, \"foobar\") ))`\n\nChecks whether a list contains a dedicated value. Values might also be lists or maps.\n\ne.g.:\n\n```yaml\nlist:\n  - foo\n  - bar\n  - foobar\ncontains: (( contains(list, \"foobar\") ))\n```\n\nyields:\n\n```yaml\nlist:\n  - foo\n  - bar\n  - foobar\ncontains: true\n```\n\nThe function `contains` also works on strings to look for sub strings or maps to look for a key. In those cases the element must be a string.\n\ne.g.:\n\n```yaml\ncontains: (( contains(\"foobar\", \"bar\") ))\n```\n\nyields `true`.\n\n### `(( basename(path) ))`\n\nThe function `basename` returns the name of the last element of a path.\nThe argument may either be a regular path name or a URL.\n\ne.g.:\n\n```yaml\npathbase:  (( basename(\"alice/bob\") ))\nurlbase:  (( basename(\"http://foobar/alice/bob?any=parameter\") ))\n```\n\nyields:\n\n```yaml\npathbase:  bob\nurlbase:  bob\n```\n\n### `(( dirname(path) ))`\n\nThe function `dirname` returns the parent directory of a path.\nThe argument may either be a regular path name or a URL.\n\ne.g.:\n\n```yaml\npathbase:  (( dirname(\"alice/bob\") ))\nurlbase:  (( dirname(\"http://foobar/alice/bob?any=parameter\") ))\n```\n\nyields:\n\n```yaml\npathbase:  alice\nurlbase:  /alice\n```\n\n### `(( parseurl(\"http://github.com\") ))`\n\nThis function parses a URL and yield a map with all elements of an URL.\nThe fields `port`, `userinfo`and `password` are optional.\n\ne.g.:\n\n```yaml\nurl:  (( parseurl(\"https://user:pass@github.com:443/mandelsoft/spiff?branch=master\u0026tag=v1#anchor\") ))\n```\n\nyields:\n\n```yaml\nurl:\n  scheme: https\n  host: github.com\n  port: 443\n  path: /mandelsoft/spiff\n  fragment: anchor\n  query: branch=master\u0026tag=v1\n  values:\n    branch: [ master ]\n    tag: [ v1 ]\n  userinfo:\n    username: user\n    password: pass\n```\n\n\n\n### `(( index(list, \"foobar\") ))`\n\nChecks whether a list contains a dedicated value and returns the index of the first match.\nValues might also be lists or maps. If no entry could be found `-1` is returned.\n\ne.g.:\n\n```yaml\nlist:\n  - foo\n  - bar\n  - foobar\nindex: (( index(list, \"foobar\") ))\n```\n\nyields:\n\n```yaml\nlist:\n  - foo\n  - bar\n  - foobar\nindex: 2\n```\n\nThe function `index` also works on strings to look for sub strings.\n\ne.g.:\n\n```yaml\nindex: (( index(\"foobar\", \"bar\") ))\n```\n\nyields `3`.\n\n### `(( lastindex(list, \"foobar\") ))`\n\nThe function `lastindex` works like [`index`](#-indexlist-foobar-) but the index of the last occurence is returned.\n\n### `(( sort(list) ))\n\nThe function `sort` can be used to sort integer or string lists. The sort\noperation is stable.\n\ne.g.:\n\n```yaml\nlist:\n  - alice\n  - foobar\n  - bob\n\nsorted: (( sort(list) ))\n\n```\n\nyields for `sorted`\n\n```yaml\n- alice\n- bob\n- foobar\n\n```\n\nIf other types should be sorted, especially complex types like lists or maps, or\na different comparison rule is required, a\ncompare function can be specified as an optional second argument. The compare\nfunction must be a lambda expression taking two arguments. The result type\nmust be `integer`or `bool`  indicating whether _a_ is less then _b_. If an\ninteger is returned it should be\n- negative, if _a\u003cb_\n- zero, if _a==b_ and\n- positive if _a\u003eb_\n\ne.g.:\n\n```yaml\nlist:\n  - alice\n  - foobar\n  - bob\n\nsorted: (( sort(list, |a,b|-\u003elength(a) \u003c length(b)) ))\n\n```\n\nyields for `sorted`\n\n```yaml\n- bob\n- alice\n- foobar\n\n```\n\n\n### `(( replace(string, \"foo\", \"bar\") ))`\n\nReplace all occurences of a sub string in a string by a replacement string. With an optional\nfourth integer argument the number of substitutions can be limited (-1 mean unlimited).\n\ne.g.:\n\n```yaml\nstring: (( replace(\"foobar\", \"o\", \"u\") ))\n```\n\nyields `fuubar`.\n\nIf a regular expression should be used as search string the function \n`replace_match` can be used. Here the search string is evaluated as [regular\nexpression](https://github.com/google/re2/wiki/Syntax). It may conatain sub expressions.\nThese matches can be used in the [replacement string](https://golang.org/pkg/regexp/#Regexp.Expand)\n\ne.g.:\n\n```yaml\nstring: (( replace_match(\"foobar\", \"(o*)b\", \"b${1}\") ))\n```\n\nyields `fbooar`.\n\nThe replacement argument might also be a lambda function. In this case, for\nevery match the function is called to determine the replacement value.\nThe single input argument is a list of actual sub expression matches.\n\ne.g.:\n\n```yaml\nstring: (( replace_match(\"foobar-barfoo\", \"(o*)b\", |m|-\u003eupper(m.[1]) \"b\" ) ))\n```\n\nyields `fOObar-barfoo`.\n\n### `(( substr(string, 1, 2) ))`\n\nExtract a stub string from a string, starting from a given start index up to an optional end index (exclusive). If no end index is given the sub struvt up to the end of the string is extracted.\nBoth indices might be negative. In this case they are taken from the end of the string.\n\ne.g.:\n\n```yaml\nstring: \"foobar\"\nend1: (( substr(string,-2) ))\nend2: (( substr(string,3) ))\nrange: (( substr(string,1,-1) ))\n```\n\nevaluates to\n\n```yaml\nstring: foobar\nend1: ar\nend2: bar\nrange: ooba\n```\n\n### `(( match(\"(f.*)(b.*)\", \"xxxfoobar\") ))`\n\nReturns the match of a [regular expression](https://github.com/google/re2/wiki/Syntax)\nfor a given string value. The match is a list of the matched values for the sub\nexpressions contained in the regular expression. Index 0 refers to the match of\nthe complete regular expression. If the string value does not match an empty\nlist is returned.\n\ne.g.:\n\n```yaml\nmatches: (( match(\"(f.*)*(b.*)\", \"xxxfoobar\") ))\n```\n\nyields:\n\n```yaml\nmatches:\n- foobar\n- foo\n- bar\n```\n\nA third argument of type integer may be given to request a multi match of a\nmaximum of *n* repetitions. If the value is negative all repetions are reported.\nThe result is a list of all matches, each in the format described above.\n\n### `(( keys(map) ))`\n\nDetermine the sorted list of keys used in a map.\n\ne.g.:\n\n```yaml\nmap:\n  alice: 25\n  bob: 25\nkeys: (( keys(map) ))\n```\n\nyields:\n\n```yaml\nmap:\n  alice: 25\n  bob: 25\nkeys:\n  - alice\n  - bob\n```\n\n### `(( length(list) ))`\n\nDetermine the length of a list, a map or a string value.\n\ne.g.:\n\n```yaml\nlist:\n  - alice\n  - bob\nlength: (( length(list) ))\n```\n\nyields:\n\n```yaml\nlist:\n  - alice\n  - bob\nlength: 2\n```\n\n### `(( base64(string) ))`\n\nThe function `base64` generates a base64 encoding of a given string. `base64_decode` decodes a base64 encoded string.\n\ne.g.:\n\n```yaml\nbase64: (( base64(\"test\") ))\ntest: (( base64_decode(base64)))\n```\n\nevaluates to\n\n```yaml\nbase54: dGVzdA==\ntest: test\n```\n\nAn optional second argument can be used to specify the maximum line length.\nIn this case the result will be multi-line string.\n\n### `(( hash(string) ))`\n\nThe function `hash` generates several kinds of hashes for the given string.\nBy default as `sha256` hash is generated. An optional second argument specifies\nthe hash type. Possible types are `md4`, `md5`, `sha1`, `sha224`, `sha256`, \n`sha384`, `sha2512`, `sha512/224`or `sha512/256`.\n\n`md5`hashes can still be generated by the deprecated finctio `md5(string)`.\n\ne.g.:\n\n```yaml\ndata: alice\n\nhash:\n  deprecated: (( md5(data) ))\n  md4: (( hash(data,\"md4\") ))\n  md5: (( hash(data,\"md5\") ))\n  sha1: (( hash(data,\"sha1\") ))\n  sha224: (( hash(data,\"sha224\") ))\n  sha256: (( hash(data,\"sha256\") ))\n  sha384: (( hash(data,\"sha384\") ))\n  sha512: (( hash(data,\"sha512\") ))\n  sha512_224: (( hash(data,\"sha512/224\") ))\n  sha512_256: (( hash(data,\"sha512/256\") ))\n```\n\nevaluates to\n\n```yaml\ndata: alice\nhash:\n  deprecated: 6384e2b2184bcbf58eccf10ca7a6563c\n  md4: 616c69636531d6cfe0d16ae931b73c59d7e0c089c0\n  md5: 6384e2b2184bcbf58eccf10ca7a6563c\n  sha1: 522b276a356bdf39013dfabea2cd43e141ecc9e8\n  sha224: 38b7e5d5651aaf85694a7a7c6d5db1275af86a6df93a36b8a4a2e771\n  sha256: 2bd806c97f0e00af1a1fc3328fa763a9269723c8db8fac4f93af71db186d6e90\n  sha384: 96a5353e625adc003a01bdcd9b21b21189bdd9806851829f45b81d3dfc6721ee21f6e0e98c4dd63bc559f66c7a74233a\n  sha512: 408b27d3097eea5a46bf2ab6433a7234a33d5e49957b13ec7acc2ca08e1a13c75272c90c8d3385d47ede5420a7a9623aad817d9f8a70bd100a0acea7400daa59\n  sha512_224: c3b8cfaa37ae15922adf3d21606e3a9836ba2a9d7838b040b7c96fd7\n  sha512_256: ad0a339b08dc090fe3b16eae376f7e162836e8728da9c45466842e19508d7627\n```\n\n### `(( bcrypt(\"password\", 10) ))`\n\nThe function `bcrypt` generates a bcrypt password hash for the given string\nusing the specified cost factor (defaulted to 10, if missing).\n\ne.g.:\n\n```yaml\nhash: (( bcrypt(\"password\", 10) ))\n```\n\nevaluates to\n\n```yaml\nhash: $2a$10$b9RKb8NLuHB.tM9haPD3N.qrCsWrZy8iaCD4/.cCFFCRmWO4h.koe\n```\n\n### `(( bcrypt_check(\"password\", hash) ))`\n\nThe function `bcrypt_check` validates a password against a given bcrypt hash.\n\ne.g.:\n\n```yaml\nhash: $2a$10$b9RKb8NLuHB.tM9haPD3N.qrCsWrZy8iaCD4/.cCFFCRmWO4h.koe\nvalid: (( bcrypt_check(\"password\", hash) ))\n```\n\nevaluates to\n\n```yaml\nhash: $2a$10$b9RKb8NLuHB.tM9haPD3N.qrCsWrZy8iaCD4/.cCFFCRmWO4h.koe\nvalid: true\n```\n\n### `(( md5crypt(\"password\") ))`\n\nThe function `md5crypt` generates an Apache MD5 encrypted password hash for the\ngiven string.\n\ne.g.:\n\n```yaml\nhash: (( md5crypt(\"password\") ))\n```\n\nevaluates to\n\n```yaml\nhash: $apr1$3Qc1aanY$16Sb5h7U1QrcqwZbDJIYZ0\n```\n\n### `(( md5crypt_check(\"password\", hash) ))`\n\nThe function `md5crypt_check` validates a password against a given Apache MD5 encrypted hash.\n\ne.g.:\n\n```yaml\nhash: $2a$10$b9RKb8NLuHB.tM9haPD3N.qrCsWrZy8iaCD4/.cCFFCRmWO4h.koe\nvalid: (( bcrypt_check(\"password\", hash) ))\n```\n\nevaluates to\n\n```yaml\nhash: $apr1$B77VuUUZ$NkNFhkvXHW8wERSRoi74O1\nvalid: true\n```\n\n### `(( decrypt(\"secret\") ))`\n\nThis function can be used to store encrypted secrets in a spiff yaml file.\nThe processed result will then contain the decrypted value.\nAll node types can be encrypted and decrypted, including complete maps and lists.\n\nThe password for the decryption can either be given as second argument, or\n(the preferred way) it can be specified by the environment variable\n`SPIFF_ENCRYPTION_KEY`. \n\nAn optional last argument may select the encryption method. The only method\nsupported so far is `3DES`. Other methods may be added for dedicated\nspiff versions by using the encryption method registration offered by the spiff\nlibrary.\n\nA value can be encrypted by using the `encrypt(\"secret\")` function.\n\ne.g.:\n\n```yaml\npassword: this a very secret secret and may never be exposed to unauthorized people\nencrypted: (( encrypt(\"spiff is a cool tool\", password) ))\ndecrypted: (( decrypt(encrypted, password) ))\n```\n\nevaluated to something like\n\n```yaml\ndecrypted: spiff is a cool tool\nencrypted: d889f9e4cc7ae13effcbc8bb8cd0c38d1fb2197738444f753c48796d7946083e6639e5a1bf8f77648f2a1ddf37023c65ff57d52d0519d1d92cbcf87d3e263cba\npassword: this a very secret secret and may never be exposed to unauthorized people\n```\n\n### `(( rand(\"[:alnum:]\", 10) ))`\n\nThe function `rand` generates random values. The first argument \ndecides what kind of values are requested. With no argument it generates\na positive random number in the `int64` range.\n\n| argument type | result |\n| ------------- | ------ |\n| int | integer value in the range [0,_n_) for positive _n_ and (_n_,0] for negative _n_ |\n| bool | boolean value |\n| string | one rune string, where the rune is in the given character range, any combination of character classes or character ranges usable for [regexp](https://github.com/google/re2/wiki/Syntax) can be used. If an additional length argument is specified the resulting string will have the given length.\n\ne.g.:\n\n```yaml\nint:   (( rand() ))\nint10: (( rand(10) ))\nneg10:   (( rand(-10) ))\nbool: (( rand(true) ))\nstring: (( rand(\"[:alpha:][:digit:]-\", 10) ))\nupper: (( rand(\"A-Z\", 10) ))\npunct: (( rand(\"[:punct:]\", 10) ))\nalnum: (( rand(\"[:alnum:]\", 10) ))\n```\n\nevaluates to\n\n```yaml\nint: 8037669378456096839\nint10: 7\nneg10: -5\nbool: true\nstring: ghhjAYPMlD\nupper: LBZQFRSURL\nalnum: 0p6KS7EhAj\npunct: '\u0026{;,^])\"(#'\n```\n\n### `(( type(foobar) ))`\n\nThe function `type` yields a string denoting the type of the given expression.\n\ne.g.:\n\n```yaml\ntemplate:\n  \u003c\u003c: (( \u0026template ))\n  \ntypes:\n  - int: (( type(1) ))\n  - float: (( type(1.0) ))\n  - bool: (( type(true) ))\n  - string: (( type(\"foobar\") ))\n  - list:   (( type([]) ))\n  - map:    (( type({}) ))\n  - lambda: (( type(|x|-\u003ex) ))\n  - template: (( type(.template) ))\n  - nil: (( type(~) ))\n  - undef: (( type(~~) ))\n```\n\nevaluates types to\n\n```yaml\ntypes:\n- int: int\n- float: float\n- bool: bool\n- string: string\n- list: list\n- map: map\n- lambda: lambda\n- template: template\n```\n\n### `(( defined(foobar) ))`\n\nThe function `defined` checks whether an expression can successfully be evaluated. It yields the boolean value `true`, if the expression can be evaluated, and `false` otherwise.\n\ne.g.:\n\n```yaml\nzero: 0\ndiv_ok: (( defined(1 / zero ) ))\nzero_def: (( defined( zero ) ))\nnull_def: (( defined( null ) ))\n```\n\nevaluates to\n\n```yaml\nzero: 0\ndiv_ok: false\nzero_def: true\nnull_def: false\n```\n\nThis function can be used in combination of the [conditional operator](#-a--1--foo-bar-) to evaluate expressions depending on the resolvability of another expression.\n\n### `(( valid(foobar) ))`\n\nThe function `valid` checks whether an expression can successfully be evaluated and evaluates to a defined value not equals to `nil`. It yields the boolean value `true`, if the expression can be evaluated, and `false` otherwise.\n\ne.g.:\n\n```yaml\nzero: 0\nempty:\nmap: {}\nlist: []\ndiv_ok: (( valid(1 / zero ) ))\nzero_def: (( valid( zero ) ))\nnull_def: (( valid( ~ ) ))\nempty_def: (( valid( empty ) ))\nmap_def: (( valid( map ) ))\nlist_def: (( valid( list ) ))\n```\n\nevaluates to\n\n```yaml\nzero: 0\nempty: null\nmap: {}\nlist: []\ndiv_ok:   false\nzero_def: true\nnull_def: false\nempty_def: false\nmap_def:  true\nlist_def: true\n```\n\n### `(( require(foobar) ))`\n\nThe function `require` yields an error if the given argument is undefined or `nil`, otherwise it yields the given value.\n\ne.g.:\n\n```yaml\nfoo: ~\nbob: (( foo || \"default\" ))\nalice: (( require(foo) || \"default\" ))\n```\n\nevaluates to\n\n```yaml\nfoo: ~\nbob: ~\nalice: default\n```\n\n### `(( stub(foo.bar) ))`\n\nThe function `stub` yields the value of a dedicated field found in the first\nupstream stub defining it.\n\ne.g.:\n\n**template.yml**\n```yaml\nvalue: (( stub(foo.bar) ))\n```\nmerged with stub\n\n**stub.yml**\n```yaml\nfoo:\n  bar: foobar\n```\n\nevaluates to\n\n```yaml\nvalue: foobar\n```\n\nThe argument passed to this function must either be a reference literal or\nan expression evaluating to either a string denoting a reference or a string\nlist denoting the list of path elements for the reference.\nIf no argument or an undefined (`~~`) is given, the actual field path is used.\n\nPlease note, that a given sole reference will not be evaluated as expression,\nif its value should be used, it must be transformed to an expression, for example \nby denoting `(ref)` or `[] ref` for a list expression.\n  \nAlternatively the `merge` operation could be used, for example `merge foo.bar`. The difference is that `stub` does not merge, therefore the field will still be merged (with the original path in the document).\n\n\n### `(( tagdef(\"tag\", value) ))`\n\nThe function `tagdef` can be used to define dynamic tags (see [Tags](#tags)).\nIn contrast to the tag marker this function allows to specify the tag name \nand its intended value by an expression. Therefore, it can be used in composing\nelements like `map` or `sum` to create dynamic tag with calculated values.\n\nAn optional third argument can be used to specify the intended scope\n(`local` or `global`). By default a local tag is created. Local tags are visible\nonly at the actual processing level (template or sub), while global tags,\nonce defined, can be used in all further processing levels (stub or template).\n\nAlternatively the tag name can be prefixed with a start (`*`) to declare\na global tag.\n\nThe specified tag value will be used as result for the function.\n\ne.g.:\n\n**template.yml**\n```yaml\nvalue: (( tagdef(\"tag:alice\", 25) ))\nalice: (( tag:alice::. ))\n```\n\nevaluates to\n\n```yaml\nvalue: 25\nalice: 25\n```\n\n### `(( eval(foo \".\" bar ) ))`\n\nEvaluate the evaluation result of a string expression again as dynaml expression. This can, for example, be used to realize indirections.\n\ne.g.: the expression in\n\n```yaml\nalice:\n  bob: married\n\nfoo: alice\nbar: bob\n\nstatus: (( eval( foo \".\" bar ) ))\n```\n\ncalculates the path to a field, which is then evaluated again to yield the value of this composed field:\n\n```yaml\nalice:\n  bob: married\n\nfoo: alice\nbar: bob\n\nstatus: married\n```\n\n### `(( env(\"HOME\" ) ))`\n\nRead the value of an environment variable whose name is given as dynaml expression. If the environment variable is not set the evaluation fails.\n\nIn a second flavor the function `env` accepts multiple arguments and/or list arguments, which are joined to a single list. Every entry in this list is used as name of an environment variable and the result of the function is a map of the given given variables as yaml element. Hereby non-existent environment variables are omitted.\n\n### `(( parse(yamlorjson) ))`\n\nParse a yaml or json string and return the content as yaml value. It can therefore be used for\nfurther dynaml evaluation.\n\ne.g.:\n\n```yaml\n\njson: |\n   { \"alice\": 25 }\nresult: (( parse( json ).alice ))\n```\n\nyields the value `25` for the field `result`.\n\nThe function `parse` supports an optional second argument, the _parse mode_.\nHere the same modes are possible as for the [read function](#-readfileyml-).\nThe default parsing mode is `import`, the content is just parsed and there is\nno further evaluation during this step.\n\n### `(( asjson(expr) ))`\n\nThis function transforms a yaml value given by its argument to a _json_ string.\nThe corresponding function `asyaml` yields the yaml value as _yaml document_ string.\n\ne.g.:\n\n```yaml\ndata:\n  alice: 25\n\nmapped:\n  json: (( asjson(.data) ))\n  yaml: (( asyaml(.data) ))\n```\n\nresolves to\n\n```yaml\ndata:\n  alice: 25\n\nmapped:\n  json: '{\"alice\":25}'\n  yaml: |+\n    alice: 25\n```\n\n### `(( catch(expr) ))`\n\nThis function executes an expression and yields some evaluation info map.\nIt always succeeds, even if the expression fails. The map includes the \nfollowing fields:\n\n| name  | type   | meaning |\n| ----- | ------ | ------- |\n| `valid` | bool   | expression is valid |\n| `error` | string | the error message text of the evaluation |\n| `value` | any    | the value of the expression, if evaluation was successful |\n\n\ne.g.:\n\n```yaml\ndata:\n  fail: (( catch(1 / 0) ))\n  valid: (( catch( 5 * 5) ))\n```\n\nresolves to \n\n```yaml\ndata:\n  fail:\n    error: division by zero\n    valid: false\n  valid:\n    error: \"\"\n    valid: true\n    value: 25\n```\n\n### `(( static_ips(0, 1, 3) ))`\n\nGenerate a list of static IPs for a job.\n\ne.g.:\n\n```yaml\njobs:\n  - name: myjob\n    instances: 2\n    networks:\n    - name: mynetwork\n      static_ips: (( static_ips(0, 3, 4) ))\n```\n\nThis will create 3 IPs from `mynetwork`s subnet, and return two entries, as\nthere are only two instances. The two entries will be the 0th and 3rd offsets\nfrom the static IP ranges defined by the network.\n\nFor example, given the file **bye.yml**:\n\n```yaml\nnetworks: (( merge ))\n\njobs:\n  - name: myjob\n    instances: 3\n    networks:\n    - name: cf1\n      static_ips: (( static_ips(0,3,60) ))\n```\n\nand file **hi.yml**:\n\n```yaml\nnetworks:\n- name: cf1\n  subnets:\n  - cloud_properties:\n      security_groups:\n      - cf-0-vpc-c461c7a1\n      subnet: subnet-e845bab1\n    dns:\n    - 10.60.3.2\n    gateway: 10.60.3.1\n    name: default_unused\n    range: 10.60.3.0/24\n    reserved:\n    - 10.60.3.2 - 10.60.3.9\n    static:\n    - 10.60.3.10 - 10.60.3.70\n  type: manual\n```\n\n```\nspiff merge bye.yml hi.yml\n```\n\nreturns\n\n\n```yaml\njobs:\n- instances: 3\n  name: myjob\n  networks:\n  - name: cf1\n    static_ips:\n    - 10.60.3.10\n    - 10.60.3.13\n    - 10.60.3.70\nnetworks:\n- name: cf1\n  subnets:\n  - cloud_properties:\n      security_groups:\n      - cf-0-vpc-c461c7a1\n      subnet: subnet-e845bab1\n    dns:\n    - 10.60.3.2\n    gateway: 10.60.3.1\n    name: default_unused\n    range: 10.60.3.0/24\n    reserved:\n    - 10.60.3.2 - 10.60.3.9\n    static:\n    - 10.60.3.10 - 10.60.3.70\n  type: manual\n```\n.\n\nIf **bye.yml** was instead\n\n```yaml\nnetworks: (( merge ))\n\njobs:\n  - name: myjob\n    instances: 2\n    networks:\n    - name: cf1\n      static_ips: (( static_ips(0,3,60) ))\n```\n\n```\nspiff merge bye.yml hi.yml\n```\n\ninstead returns\n\n```yaml\njobs:\n- instances: 2\n  name: myjob\n  networks:\n  - name: cf1\n    static_ips:\n    - 10.60.3.10\n    - 10.60.3.13\nnetworks:\n- name: cf1\n  subnets:\n  - cloud_properties:\n      security_groups:\n      - cf-0-vpc-c461c7a1\n      subnet: subnet-e845bab1\n    dns:\n    - 10.60.3.2\n    gateway: 10.60.3.1\n    name: default_unused\n    range: 10.60.3.0/24\n    reserved:\n    - 10.60.3.2 - 10.60.3.9\n    static:\n    - 10.60.3.10 - 10.60.3.70\n  type: manual\n```\n\n`static_ips`also accepts list arguments, as long as all transitivly contained elements are either again lists or integer values. This allows to abbreviate the list of IPs as follows:\n\n```\n  static_ips: (( static_ips([1..5]) ))\n```\n\n### `(( ipset(ranges, 3, 3,4,5,6) ))`\n\nWhile the function [static_ips](#-static_ips0-1-3-) for historical reasons\nrelies on the structure of a bosh manifest\nand works only at dedicated locations in the manifest, the function *ipset*\noffers a similar calculation purely based on its arguments. So, the available\nip ranges and the required numbers of IPs are passed as arguments.\n\nThe first (ranges) argument can be a single range as a simple string or a\nlist of strings. Every string might be\n- a single IP address\n- an explicit IP range described by two IP addresses separated by a dash (-)\n- a CIDR\n\nThe second argument specifies the requested number of IP addresses in the\nresult set.\n\nThe additional arguments specify the indices of the IPs to choose (starting\nfrom 0) in the given ranges. Here again lists of indices might be used.\n\ne.g.:\n\n```yaml\nranges:\n  - 10.0.0.0 - 10.0.0.255\n  - 10.0.2.0/24\nipset: (( ipset(ranges,3,[256..260]) ))\n```\n\nresolves *ipset* to `[ 10.0.2.0, 10.0.2.1, 10.0.2.2 ]`.\n\nIf no IP indices are specified (only two arguments), the IPs are chosen\nstarting from the beginning of the first range up to the end of the last\ngiven range, without indirection.\n\n### `(( list_to_map(list, \"key\") ))`\n\nA list of map entries with explicit name/key fields will be mapped to a map with the dedicated keys. By default the key field `name` is used, which can changed by the optional second argument. An explicitly denoted key field in the list will also be taken into account.\n\ne.g.:\n\n```yaml\nlist:\n  - key:foo: alice\n    age: 24\n  - foo: bob\n    age: 30\n\nmap: (( list_to_map(list) ))\n```\n\nwill be mapped to\n\n```yaml\nlist:\n  - foo: alice\n    age: 24\n  - foo: bob\n    age: 30\n\nmap:\n  alice:\n    age: 24\n  bob:\n    age: 30\n```\n\nIn combination with templates and lambda expressions this can be used to generate maps with arbitrarily named key values, although dynaml expressions are not allowed for key values.\n\n### `(( makemap(fieldlist) ))`\n\nIn this flavor `makemap` creates a map with entries described by the given field list.\nThe list is expected to contain maps with the entries `key` and `value`, describing\ndedicated map entries.\n\ne.g.:\n\n```yaml\nlist:\n  - key: alice\n    value: 24\n  - key: bob\n    value: 25\n  - key: 5\n    value: 25\n\nmap: (( makemap(list) ))\n```\n\nyields\n\n\n```yaml\nlist:\n  - key: alice\n    value: 24\n  - key: bob\n    value: 25\n  - key: 5\n    value: 25\n\nmap:\n  \"5\": 25\n  alice: 24\n  bob: 25\n```\n\nIf the key value is a boolean or an integer it will be mapped to a string.\n\n### `(( makemap(key, value) ))`\n\nIn this flavor `makemap` creates a map with entries described by the given argument\npairs. The arguments may be a sequence of key/values pairs (given by separate arguments).\n\ne.g.:\n\n```yaml\nmap: (( makemap(\"peter\", 23, \"paul\", 22) ))\n```\n\nyields\n\n\n```yaml\nmap:\n  paul: 22\n  peter: 23\n```\n\nIn contrast to the previous `makemap` flavor, this one could also be handled by\n[map literals](#--alice--25--).\n\n### `(( merge(map1, map2) ))`\n\nBeside the keyword ` merge` there is also a function called `merge` (It must always be followed by an opening bracket). It can be used to merge severals maps taken from the actual document analogous to the stub merge process. If the maps are specified by reference expressions, they cannot contain\nany _dynaml_ expressions, because they are always evaluated in the context of the actual document before evaluating the arguments.\n\ne.g.:\n\n```yaml\nmap1:\n  alice: 24\n  bob: (( alice ))\nmap2:\n  alice: 26\n  peter: 8\nresult: (( merge(map1,map2) ))\n```\n\nresolves `result` to\n\n```yaml\nresult:\n  alice: 26\n  bob: 24  # \u003c---- expression evaluated before mergeing\n```\n\nAlternatively map [templates](#templates) can be passed (without evaluation operator!). In this case the _dynaml_ expressions from the template are evaluated while merging the given documents as for regular calls of _spiff merge_.\n\ne.g.:\n\n```yaml\nmap1:\n  \u003c\u003c: (( \u0026template ))\n  alice: 24\n  bob: (( alice ))\nmap2:\n  alice: 26\n  peter: 8\nresult: (( merge(map1,map2) ))\n```\n\nresolves `result` to\n\n```yaml\nresult:\n  alice: 26\n  bob: 26\n```\n\nA map might also be given by a [map expression](#--alice--25--). Here it is possible to specify\ndynaml expressions using the usual syntax:\n\ne.g.:\n\n```yaml\nmap1:\n  alice: 24\n  bob: 25\n\nmap2:\n  alice: 26\n  peter: 8\n\nresult: (( merge(map1, map2, { \"bob\"=\"(( carl ))\", \"carl\"=100 }) ))\n```\n\nresolves `result` to\n\n```yaml\nresult:\n  alice: 26\n  bob: 100\n```\n\nInstead of multiple arguments a single list argument can be given. The list\nmust contain the maps to be merged.\n\nNested merges have access to all outer bindings. Relative references are first\nsearched in the actual document. If they are not found there all outer bindings\nare used to lookup the reference, from inner to outer bindings. Additionally the\n[context (`__ctx`)](#access-to-evaluation-context) offers a field `OUTER`,\nwhich is a list of all outer documents of the nested merges, which can be used\nto lookup absolute references.\n\ne.g.:\n\n```yaml\ndata:\n  alice:\n    age: 24\n\ntemplate:\n  \u003c\u003c: (( \u0026template ))\n  bob:  25\n  outer1: (( __ctx.OUTER.[0].data )) # absolute access to outer context\n  outer2: (( data.alice.age ))       # relative access to outer binding\n  sum: (( .bob + .outer2 ))\n\nmerged: (( merge(template) ))\n```\n\nresolves `merged` to\n\n```yaml\nmerged:\n  bob: 25\n  outer1:\n    alice:\n      age: 24\n  outer2: 24\n  sum: 49\n```\n\n### `(( intersect(list1, list2) ))`\n\nThe function `intersect` intersects multiple lists. A list may contain entries\nof any type.\n\ne.g.:\n\n```yaml\nlist1:\n- - a\n- - b\n- a\n- b\n- { a: b }\n- { b: c }\n- 0\n- 1\n- \"0\"\n- \"1\"\nlist2:\n- - a\n- - c\n- a\n- c\n- { a: b }\n- { b: b }\n- 0\n- 2\n- \"0\"\n- \"2\"\nintersect: (( intersect(list1, list2) ))\n```\n\nresolves `intersect` to\n\n```yaml\nintersect:\n- - a\n- a\n- { a: b }\n- 0\n- \"0\"\n```\n\n### `(( reverse(list) ))`\n\nThe function `reverse` reverses the order of a list. The list may contain entries\nof any type.\n\ne.g.:\n\n```yaml\nlist:\n- - a\n- b\n- { a: b }\n- { b: c }\n- 0\n- 1\nreverse: (( reverse(list) ))\n```\n\nresolves `reverse` to\n\n```yaml\nreverse:\n- 1\n- 0\n- { b: c }\n- { a: b }\n- b\n- - a\n```\n\n### `(( validate(value,\"dnsdomain\") ))`\n\nThe function `validate` validates an expression using a set of validators.\nThe first argument is the value to validate and all other arguments are\nvalidators that must succeed to accept the value. If at least one validator\nfails an appropriate error message is generated that explains the fail reason.\n\nA validator is denoted by a string or a list containing the validator type\nas string and its arguments. A validator can be negated with a preceeding\n`!` in its name.\n\nThe following validators are available:\n\n| Type | Arguments | Meaning |\n| ---- | --------- | ------- |\n| `empty` | none | empty list, map or string |\n| `dnsdomain` | none | dns domain name |\n| `wildcarddnsdomain` | none | wildcard dns domain name |\n| `dnslabel` | none | dns label |\n| `dnsname` | none | dns domain or wildcard domain |\n| `ip` | none | ip address |\n| `cidr` | none | cidr | \n| `publickey` | none | public key in pem format |\n| `privatekey` | none | private key in pem format |\n| `certificate` | none | certificate in pem format |\n| `ca`|  none | certificate for CA |\n| `semver` | optional list of constraints | validate semver version against constraints |\n| `type`| list of accepted type keys | at least one [type key](#-typefoobar-) must match |\n| `valueset` | list argument with values | possible values |\n| `value` or `=` | value | check dedicated value |\n| `gt` or `\u003e` | value | greater than (number/string) |\n| `lt` or `\u003c` | value | less than (number/string) |\n| `ge` or `\u003e=` | value | greater or equal to (number/string) |\n| `le` or `\u003c=` | value | less or equal to (number/string) |\n| `match` or `~=` | regular expression | string value matching regular expression |\n| `list` | optional list of entry validators | is list and entries match given validators |\n| `map` | [[ \u0026lt;key validator\u0026gt;, ] \u0026lt;entry validator\u0026gt; ] | is map and keys and entries match given validators |\n| `mapfield` | \u0026lt;field name\u0026gt; [ , \u0026lt;validator\u0026gt;] | required entry in map |\n| `optionalfield` | \u0026lt;field name\u0026gt; [ , \u0026lt;validator\u0026gt;] | optional entry in map |\n| `and` | list of validators | all validators must succeed |\n| `or` | list of validators | at least one validator must succeed |\n| `not` or `!` | validator | negate the validator argument(s) |\n\nIf the validation succeeds the value is returned.\n\ne.g.:\n\n```yaml\ndnstarget: (( validate(\"192.168.42.42\", [ \"or\", \"ip\", \"dnsdomain\" ]) ))\n```\n\nevaluates to\n\n```yaml\ndnstarget: 192.168.42.42\n```\n\nIf the validation fails an error explaining the failure reason is generated.\n\ne.g.:\n\n```yaml\ndnstarget: (( validate(\"alice+bob\", [ \"or\", \"ip\", \"dnsdomain\" ]) ))\n```\n\nyields the following error:\n\n```\n*condition 1 failed: (is no ip address: alice+bob and is no dns domain: [a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]) \n```\n\nA validator might also be a lambda expression taking at least one argument and returning\na boolean value. This way it is possible to provide own validators as part\nof the yaml document.\n\ne.g.:\n\n```yaml\nval: (( validate( 0, |x|-\u003e x \u003e 1 ) ))\n```\n\nIf more than one parameter is declared the additional arguments\nmust be specified as validator arguments. The first argument is always\nthe value to check.\n\ne.g.:\n\n```yaml\nval: (( validate( 0, [|x,m|-\u003e x \u003e m, 5] ) ))\n```\n\nThe lambda function may return a list with 1, 2 or 3 elements, also.\nThis can be used to provide appropriate messages.\n\n| Index | Meaning |\n| ----- | ------- | \n| 0 | the first index always is the match result, it must be evaluatable as boolean |\n| 1 | if two elements are given, the second index is the message describing the actual result |\n| 2 | here index 1 decribes the success message and 2 the failure message |\n\ne.g.:\n\n```yaml\nval: (( validate( 6, [|x,m|-\u003e [x \u003e m, \"is larger than \" m, \"is less than or equal to \" m], 5] ) ))\n```\n\nJust to mention, the validator specification might be given inline as shown\nin the examples above, but as reference expressions, also. The `not`, `and` \nand `or` validators accept deeply nested validator specifications.\n\ne.g.:\n\n```yaml\ndnsrecords:\n   domain: 1.2.3.4\nvalidator:\n  - map\n  - - or                              # key validator\n    - dnsdomain\n    - wildcarddnsdomain\n  - ip                                # entry validator\n\nval: (( validate( map, validator)  ))\n```\n\n### `(( check(value,\"dnsdomain\") ))`\n\nThe function `check` can be used to match a yaml structure against a yaml\nbased value checker. Hereby the same check description already described for \n[validate](#-validatevaluednsdomain-) can be used. The result of the call is\na boolean value indicating the match result. It does not fail if the check\nfails.\n \n### `(( error(\"message\") ))`\n\nThe function `error` can be used to cause explicit evaluation failures with\na dedicated message.\n\nThis can be used, for example, to reduce a complex\nprocessing error to a meaningful message by appending the error function\nas default for the potentially failing comples expression.\n\ne.g.:\n\n```yaml\nvalue: (( \u003csome complex potentially failing expression\u003e || error(\"this was an error my friend\") ))\n```\n\nAnother scenario could be omitting a descriptive message for missing required\nfields by using an error expression as (default) value for a field intended to\nbe defined in an upstream stub.\n\n### Math\n\n*dynaml* support various math functions:\n\nreturning integers: `ceil`, `floor`, `round` and `roundtoeven`\n\nreturning floats or integers: `abs`\n\nreturning floats: `sin`,`cos`, `sinh`, `cosh`, `asin`, `acos`, `asinh`,`acosh`,\n           `sqrt`, `exp`, `log`, `log10`,\n\n### Conversions\n\n*dynaml* supports various type conversions between `integer`, `float`, `bool`\nand `string` values by appropriate functions.\n\ne.g.:\n\n\n```yaml\nvalue: (( integer(\"5\") ))\n```\n\nconverts a string to an integer value.\n\nConverting an integer to a string accepts an optional additional integer\nargument for specifying the base for conversion, for example `string(55,2)`\nwill result in `\"110111\"`. The default base is 10. The base must be between\n2 and 36.\n\n\n### Accessing External Content\n\n_Spiff_ supports access to content outside of the template and sub files. It is\npossible to read files, execute commands and pipelines. All those functions exist\nin two flavors.\n- A cached flavor executes the operation ones and caches the result\n  for subsequent identical operations. This speeds up the processing, especially\n  for command executions.\n- If the result evolves over time, it might be useful to always get the latest \n  content. This is the case if the [`sync`](#-syncexpr-condition.value-10-) \n  function is used, which is intended to synchronize the template processing\n  with a dedicate state (provided by external content). Here the caching \n  operations would not be useful, therefore there is a second uncached flavor.\n  Every function is available with the suffix `_uncached` (for example \n  `read_uncached()`)\n\n#### `(( read(\"file.yml\") ))`\n\nRead a file and return its content. There is support for three content types:\n`yaml` files,`text` files and `binary` files. Reading in binary mode will\nresult in a base64 encoded multi-line string.\n\nIf the file suffix is `.yml`, `.yaml` or `.json`,\nby default the yaml type is used. If the file should be read as `text`, this\ntype must be explicitly specified.\nIn all other cases the default is `text`, therefore reading a binary file\n(for example an archive) urgently requires specifying the `binary` mode.\n  \nAn optional second parameter can be used to explicitly specifiy the desired\nreturn type: `yaml` or `text`. For _yaml_ documents some addtional \ntypes are supported: `multiyaml`, `template`, `templates`, `import` and\n`importmulti`.\n\n##### yaml documents\n\nA yaml document will be parsed and the tree is returned. The  elements of the\ntree can be accessed by regular dynaml expressions.\n\nAdditionally the yaml file may again contain dynaml expressions. All included\ndynaml expressions will be evaluated in the context of the reading expression.\nThis means that the same file included at different places in a yaml document\nmay result in different sub trees, depending on the used dynaml expressions.\n\nIf is possible to read a multi-document yaml, also. If the type `multiyaml`\nis given, a list node with the yaml document root nodes is returned.\n\nThe yaml or json document can also read as _template_ by specifying the type\n`template`. Here the result will be a template value, that can be used like\nregular inline templates. If `templates` is specified, a multi-document is\nmapped to a list of templates.\n\nIf the read type is set to `import`, the file content is read as yaml document\nand the root node is used to substitute the expression. Potential dynaml\nexpressions contained in the document will not be evaluated with the actual\nbinding of the expression together with the read call,\nbut as it would have been part of the original file.\nTherefore this mode can only be used, if there is no further processing\nof the read result or the delivered values are unprocessed.\n\nThis can be used together with a chained reference\n (for examle `(( read(...).selection ))`) to delect a dedicated fragment of\nthe imported document. Then, the evaluatio will be done for the selected\nportion, only. Expressions and references in the other parts are not\nevalauted and at all and cannot lead to error.\n\ne.g.: \n\n**template.yaml**\n\n```yaml\nages:\n  alice: 25\n\ndata: (( read(\"import.yaml\", \"import\").first ))\n``` \n\n**import.yaml**\n\n```yaml\nfirst:\n  age: (( ages.alice ))\n\nsecond:\n  age: (( ages.bob ))\n```\n\nwill not fail, because the `second` section is never evaluated.\n\nThis mode should be taken with caution, because it often leads to unexpected\nresults.\n\nThe read type `importmulti` can be used to import multi-document yaml files as a \nlist of nodes.\n\n##### text documents\n\nA text document will be returned as single string.\n\n##### binary documents\n\nIt is possible to read binary documents, also. The content cannot be used\nas a string (or yaml document), directly. Therefore the read mode `binary` has\nto be specified. The content is returned as a base64 encoded multi-line string\nvalue.\n\n#### `(( exec(\"command\", arg1, arg2) ))`\n\nExecute a command. Arguments can be any dynaml expressions including reference expressions evaluated to lists or maps. Lists or maps are passed as single arguments containing a yaml document with the given fragment.\n\nThe result is determined by parsing the standard output of the command. It might be a yaml document or a single multi-line string or integer value. A yaml document should start with the document prefix `---`. If the command fails the expression is handled as undefined.\n\ne.g.\n\n```yaml\narg:\n  - a\n  - b\nlist: (( exec( \"echo\", arg ) ))\nstring: (( exec( \"echo\", arg.[0] ) ))\n\n```\n\nyields\n\n```yaml\narg:\n- a\n- b\nlist:\n- a\n- b\nstring: a\n```\n\nAlternatively `exec` can be called with a single list argument completely describing the command line.\n\nThe same command will be executed once, only, even if it is used in multiple expressions.\n\n#### `(( pipe(data, \"command\", arg1, arg2) ))`\n\nExecute a command and feed its standard input with dedicated data. \nThe command argument must be a string. Arguments\nfor the command can be any dynaml expressions including reference expressions\nevaluated to lists or maps. Lists or maps are passed as single arguments\ncontaining a yaml document with the given fragment.\n\nThe input stream is generated from the given data. If this is a simple type its\nstring representation is used. Otherwise a yaml document is generated from the\ninput data. The result is determined by parsing the standard output of the\ncommand. It might be a yaml document or a single multi-line string or integer\nvalue. A yaml document should start with the document prefix `---`. If\nthe command fails the expression is handled as undefined.\n\ne.g.\n\n```yaml\ndata:\n  - a\n  - b\nlist: (( pipe( data, \"tr\", \"a\", \"z\") ))\n```\n\nyields\n\n```yaml\narg:\n- a\n- b\nlist:\n- z\n- b\n```\n\nAlternatively `pipe` can be called with data and a list argument completely describing the command line.\n\nThe same command will be executed once, only, even if it is used in multiple expressions.\n\n#### `(( write(\"file.yml\", data) ))`\n\nWrite a file and return its content. If the result can be parsed as yaml document,\nthe document is returned. An optional 3rd argument can be used to pass the\nwrite options.\nThe option arguments might be an integer denoting file permissions (default is `0644`)\nor a comma separated string with options. Supported options are\n- `binary`: data is base64 decoded before writing\n- _integer_ string: file permissions, a leading `0` is indicating an octal value.\n\n#### `(( tempfile(\"file.yml\", data) ))`\n\nWrite a a temporary file and return its path name. An optional 3rd argument can\nbe used to pass write options. It basically behavies\nlike [`write`](#-writefileyml-data-) \n\n_Attention_: A temporary file only exists during the merge processing. It will\nbe deleted afterwards. \n\nIt can be used, for example, to provide a temporary file argument for the\n[`exec`](#-execcommand-arg1-arg2-) function.\n\n#### `(( lookup_file(\"file.yml\", list) ))`\n\nLookup a file is a list of directories. The result is a list of existing\nfiles. With `lookup_dir` it is possible to lookup a directory, instead.\n\nIf no existing files can be found the empty list is returned.\n\nIt is possible to pass multiple list or string arguments to compose the\nsearch path.\n\n#### `(( mkdir(\"dir\", 0755) ))`\n\nCreate a directory and all its intermediate directories if they do not\nexist yet.\n\nThe permission part is optional (default 0755). The path of the directory\nmight be given by atring like value or as a list of path components.\n\n#### `(( list_files(\".\") ))`\n\nList files in a directory. The result is a list of existing\nfiles. With `list_dirs` it is possible to list directories, instead.\n\n#### `(( archive(files, \"tar\") ))`\n\nCreate an archive of the given type (default is `tar`) containing the listed\nfiles. The result is the base64 encoded archive.\n\nSupported archive types are `tar` and `targz`.\n\n`files` might be a list or map of file entries. In case of a map, the map key\nis used as default for the file path. A file entry is a map with the \nfollowing fields:\n\n| field | type | meaning |\n|-------|------|---------|\n| `path`| string | optional for maps, the file path in the archive, defaulted by the map key |\n| `mode` | int or int string | file mode or write options. It basically behavies like the option argument for [`write`](#-writefileyml-data-). |\n| `data` | any | file content, yaml will be marshalled as yaml document. If `mode` indicates binary mode, a string value will be base64 decoded. |\n| `base64` | string | base64 encoded binary data |\n\ne.g.:\n\n```yaml\nyaml:\n  alice: 26\n  bob: 27\n\nfiles:\n  \"data/a/test.yaml\":\n    data: (( yaml ))\n  \"data/b/README.md\":\n    data: |+\n      ### Test Docu\n\n      **Note**: This is a test\n\narchive: (( archive(files,\"targz\") ))\n\ncontent: (( split(\"\\n\", exec_uncached(\"tar\", \"-tvf\", tempfile(archive,\"binary\"))) ))\n```\n\nyields:\n\n```yaml\narchive: |-\n  H4sIAAAAAAAA/+zVsQqDMBAG4Mx5igO3gHqJSQS3go7tUHyBqIEKitDEoW9f\n  dLRDh6KlJd/yb8ll+HOd8SY1qbfOJw8zDmQHiIhayjURcZuIQhOeSZlphVwL\n  glwsAXvM8mJ23twJ4qfnbB/3I+I4pmboW1uA0LSZmgJETr89VXCUtf9Neq1O\n  5blKxm6PO972X/FN/7nKVej/EaIogto6D+XUzpQydpm8ZayA+tY76B0YWHYD\n  DV9CEATBf3kGAAD//5NlAmIADAAA\ncontent:\n- -rw-r--r-- 0/0              22 2019-03-18 09:01 data/a/test.yaml\n- -rw-r--r-- 0/0              41 2019-03-18 09:01 data/b/README.md\n\nfiles:\n  data/a/test.yaml:\n    data:\n      alice: 26\n      bob: 27\n  data/b/README.md:\n    data: |+\n      ### Test Docu\n\n      **Note**: This is a test\n\nyaml:\n  alice: 26\n  bob: 27\n\n```\n### Semantic Versioning Functions\n\n*Spiff* supports handling of semantic version names. It supports all functionality\nfrom the [Masterminds Semver Package](https://github.com/Masterminds/semver/blob/v3.1.1/README.md)\naccepting versions with or without a leading `v`.\n\n#### `(( semver(\"v1.2-beta.1\") ))`\n\nCheck whether a given string is a semantic version and return \nits normalized form (without leading `v` and complete release part with major,\nminor and and patch version number).\n\ne.g.:\n\n```yaml\nnormalized: (( semver(\"v1.2-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nnormalized: 1.2.0-beta.1\n```\n\n#### `(( semverrelease(\"v1.2.3-beta.1\") ))`\n\nReturn the release part of a semantic version omitting metadata and prerelease\ninformation.\n\ne.g.:\n\n```yaml\nrelease: (( semverrelease(\"v1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nrelease: v1.2.3\n```\n\nIf an additional string argument is given this function replaces\nthe release by the release of the given semantic version preserving\nmetadata and prerelease information.\n\ne.g.:\n\n```yaml\nnew: (( semverrelease(\"1.2.3-beta.1\", \"1.2.1) ))\n```\n\nresolves to\n\n```yaml\nnew: 1.2.1-beta.1\n```\n\n#### `(( semvermajor(\"1.2.3-beta.1\") ))`\n\nDetermine the major version number of the given semantic version.\nThe result is an integer.\n\ne.g.:\n\n```yaml\nmajor: (( semvermajor(\"1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nmajor: 1\n```\n\nThe function `semverincmajor` can be used to increment the\nmajor version number and reset the minor version, patch version \nand release suffixes.\n\ne.g.:\n\n```yaml\nnew: (( semverincmajor(\"1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nnew: 2.0.0\n```\n\n#### `(( semverminor(\"1.2.3-beta.1\") ))`\n\nDetermine the minor version number of the given semantic version.\nThe result is an integer.\n\ne.g.:\n\n```yaml\nminor: (( semverminor(\"1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nminor: 2\n```\n\nThe function `semverincminor` can be used to increment the\nminor version number and reset the patch version \nand release suffixes.\n\ne.g.:\n\n```yaml\nnew: (( semverincmajor(\"v1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nnew: v1.3.0\n```\n\n#### `(( semverpatch(\"1.2.3-beta.1\") ))`\n\nDetermine the patch version number of the given semantic version.\nThe result is an integer.\n\ne.g.:\n\n```yaml\npatch: (( semverpatch(\"1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\npatch: 3\n```\n\nThe function `semverincpatch` can be used to increment the\npatch version number or reset the release suffixes.\nIf there are rlease suffixes, they are removed and the release\ninfo is kept unchanged, otherwise the patch version number\nis increased.\n\ne.g.:\n\n```yaml\nfinal: (( semverincpatch(\"1.2.3-beta.1\") ))\nnew: (( semverincpatch(final) ))\n```\n\nresolves to\n\n```yaml\nfinal: 1.2.3\nnew: 1.2.4\n```\n\n#### `(( semverprerelease(\"1.2.3-beta.1\") ))`\n\nDetermine the prerelease of the given semantic version.\nThe result is a string.\n\ne.g.:\n\n```yaml\nprerelease: (( semverprerelease(\"1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\nprerelease: beta.1\n```\n\nIf an additional string argument is given this function sets, replaces or clears\n(if set to empty string) the prerelease\n\ne.g.:\n\n```yaml\nnew: (( semverprerelease(\"1.2.3-beta.1\", \"beta.2) ))\n```\n\nresolves to\n\n```yaml\nnew: 1.2.3-beta.2\n```\n\n#### `(( semvermetadata(\"1.2.3+demo\") ))`\n\nDetermine the metadata of the given semantic version.\nThe result is a string.\n\ne.g.:\n\n```yaml\nmetadata: (( semvermetadata(\"1.2.3+demo\") ))\n```\n\nresolves to\n\n```yaml\nmetadata: demo\n```\n\nIf an additional string argument is given this function sets, replaces or clears\n(if set to empty string) the metadata.\n\ne.g.:\n\n```yaml\nnew: (( semvermetadata(\"1.2.3-test\", \"demo) ))\n```\n\nresolves to\n\n```yaml\nnew: 1.2.3+demo\n```\n\n#### `(( semvercmp(\"1.2.3\", 1.2.3-beta.1\") ))`\n\nCompare two semantic versions. A prerelease is always *smaller* than \nthe final release. The result is an integer with the following values:\n\n| result | meaning |\n|--------|---------|\n| -1 | first version is before the second version |\n| 0 | both versions are equal |\n| 1 | first versuon is after the second one |\n\ne.g.:\n\n```yaml\ncompare: (( semvercmp(\"1.2.3\", \"1.2.3-beta.1\") ))\n```\n\nresolves to\n\n```yaml\ncompare: 1\n```\n\n#### `(( semvermatch(\"1.2.3\", \"~1.2\") ))`\n\nMatch the given semantic version against a list of contraints.\nThe result is a boolean. It is possible to specify any number of version\nconstraints. If no constraint is given, the function just checks whether \nthe given string is a semantic version.\n\ne.g.:\n\n```yaml\nmatch: (( semvermatch(\"1.2.3\", \"~1.2\") ))\n```\n\nresolves to\n\n```yaml\nmatch: true\n```\n\nThe complete list of possible constraints specification can be found\n[here](https://github.com/Masterminds/semver/blob/v3.1.1/README.md#checking-version-constraints).\n\n#### `(( semversort(\"1.2.3\", \"1.2.1\") ))`\n\nSort a list of versions in ascending order. A leading `v` is preserved.\n\ne.g.:\n\n```yaml\nsorted: (( semversort(\"1.2.3\", \"1.2.1\") ))\n```\n\nresolves to\n\n```yaml\nsorted:\n  - 1.2.1\n  - 1.2.3\n```\n\nThe list of versions to be sorted may also be specified with a single list\nargument.\n\n### X509 Functions\n\n*Spiff* supports some useful functions to work with _X509_ certificates and keys.\nPlease refer also to the [Useful to Know](#useful-to-know) section to find some\ntips for providing state.\n\n#### `(( x509genkey(spec) ))`\n\nThis function can be used generate private RSA or ECDSA keys. The result will\nbe a PEM encoded key as multi line string value. If a key size (integer or string)\nis given as argument, an RSA key will be generated with the given key size\n(for example 2048). Given one of the string values\n\n- \"P224\"\n- \"P256\"\n- \"P384\"\n- \"P521\"\n\nthe function will generate an appropriate ECDSA key.\n\ne.g.:\n\n```yaml\nkeys:\n  key: (( x509genkey(2048) ))\n```\n\nresolves to something like\n\n```yaml\nkey: |+\n    -----BEGIN RSA PRIVATE KEY-----\n    MIIEpAIBAAKCAQEAwxdZDfzxqz4hlRwTL060pm1J12mkJlXF0VnqpQjpnRTq0rns\n    CxMxvSfb4crmWg6BRaI1cEN/zmNcT2sO+RZ4jIOZ2Vi8ujqcbzxqyoBQuMNwdb32\n    ...\n    oqMC9QKBgQDEVP7FDuJEnCpzqddiXTC+8NsC+1+2/fk+ypj2qXMxcNiNG1Az95YE\n    gRXbnghNU7RUajILoimAHPItqeeskd69oB77gig4bWwrzkijFXv0dOjDhQlmKY6c\n    pNWsImF7CNhjTP7L27LKk49a+IGutyYLnXmrlarcNYeCQBin1meydA==\n    -----END RSA PRIVATE KEY-----\n```\n\n#### `(( x509publickey(key) ))`\n\nFor a given key or certificate in PEM format (for example generated with the [x509genkey](#-x509genkeyspec-)\nfunction) this function extracts the public key and returns it again in PEM format as a\nmulti-line string.\n\ne.g.:\n\n```yaml\nkeys:\n  key: (( x509genkey(2048) ))\n  public: (( x509publickey(key)\n```\n\nresolves to something like\n\n```yaml\nkey: |+\n    -----BEGIN RSA PRIVATE KEY-----\n    MIIEpAIBAAKCAQEAwxdZDfzxqz4hlRwTL060pm1J12mkJlXF0VnqpQjpnRTq0rns\n    CxMxvSfb4crmWg6BRaI1cEN/zmNcT2sO+RZ4jIOZ2Vi8ujqcbzxqyoBQuMNwdb32\n    ...\n    oqMC9QKBgQDEVP7FDuJEnCpzqddiXTC+8NsC+1+2/fk+ypj2qXMxcNiNG1Az95YE\n    gRXbnghNU7RUajILoimAHPItqeeskd69oB77gig4bWwrzkijFXv0dOjDhQlmKY6c\n    pNWsImF7CNhjTP7L27LKk49a+IGutyYLnXmrlarcNYeCQBin1meydA==\n    -----END RSA PRIVATE KEY-----\npublic: |+\n    -----BEGIN RSA PUBLIC KEY-----\n    MIIBCgKCAQEAwxdZDfzxqz4hlRwTL060pm1J12mkJlXF0VnqpQjpnRTq0rnsCxMx\n    vSfb4crmWg6BRaI1cEN/zmNcT2sO+RZ4jIOZ2Vi8ujqcbzxqyoBQuMNwdb325Bf/\n   ...\n    VzYqyeQyvvRbNe73BXc5temCaQayzsbghkoWK+Wrc33yLsvpeVQBcB93Xhus+Lt1\n    1lxsoIrQf/HBsiu/5Q3M8L6klxeAUcDbYwIDAQAB\n    -----END RSA PUBLIC KEY-----\n```\n\nTo generate an ssh public key an optional additional format argument can be set\nto `ssh`. The result will then be a regular public key format usable for ssh.\nThe default format is `pem` providing the pem output format shown above.\n\nRSA keys are by default marshalled in PKCS#1 format(`RSA PUBLIC KEY`) in pem.\nIf the generic *PKIX* format (`PUBLIC KEY`) is required the format\nargument `pkix` must be given.\n\nUsing the format `ssh` this function can also be used to convert a pem formatted\npublic key into an ssh key, \n\n#### `(( x509cert(spec) ))`\n\nThe function `x509cert` creates locally signed certificates, either a self signed\none or a certificate signed by a given ca. It returns a PEM encoded certificate\nas a multi-line string value.\n\nThe single _spec_ parameter take a map with some optional and non optional\nfields used to specify the certificate information. It can be an\n[inline map expression](#--alice--25--) or any map reference into the rest of\nthe yaml document.\n\nThe following map fields are observed:\n\n| Field Name  | Type | Required | Meaning |\n| ------------| ---- | -------- | ------- |\n| `commonName` | string | optional |  Common Name field of the subject |\n| `organization` | string or string list | optional |  Organization field of the subject |\n| `country` | string or string list | optional |  Country field of the subject |\n| `isCA` | bool | optional |  CA option of certificate |\n| `usage` | string or string list | required |  usage keys for the certificate (see below) |\n| `validity` | integer | optional |  validity interval in hours |\n| `validFrom` | string | optional |  start time in the format \"Jan 1 01:22:31 2019\" |\n| `hosts` | string or string list | optional |  List of DNS names or IP addresses |\n| `privateKey` | string | required or publicKey |  private key to generate the certificate for |\n| `publicKey` | string | required or privateKey|  public key to generate the certificate for |\n| `caCert` | string | optional|  certificate to sign with |\n| `caPrivateKey` | string | optional|  priavte key for `caCert` |\n\nFor self-signed certificates, the `privateKey`field must be set. `publicKey`\nand the `ca` fields should be omitted. If the `caCert`field is given, the `caKey`\nfield is required, also. If the `privateKey`field is given together with the\n`caCert`, the public key for the certificate is extracted from the private key.\n\nAdditional fields are silently ignored.\n\nThe following usage keys are supported (case is ignored):\n\n| Key |  Meaning |\n| ------------| ---- |\n| `Signature` | x509.KeyUsageDigitalSignature |\n| `Commitment` | x509.KeyUsageContentCommitment |\n| `KeyEncipherment` | x509.KeyUsageKeyEncipherment |\n| `DataEncipherment` | x509.KeyUsageDataEncipherment |\n| `KeyAgreement` | x509.KeyUsageKeyAgreement |\n| `CertSign` | x509.KeyUsageCertSign |\n| `CRLSign` | x509.KeyUsageCRLSign |\n| `EncipherOnly` | x509.KeyUsageEncipherOnly |\n| `DecipherOnly` | x509.KeyUsageDecipherOnly |\n| `Any` | x509.ExtKeyUsageAny |\n| `ServerAuth` | x509.ExtKeyUsageServerAuth |\n| `ClientAuth` | x509.ExtKeyUsageClientAuth |\n| `codesigning` | x509.ExtKeyUsageCodeSigning |\n| `EmailProtection` | x509.ExtKeyUsageEmailProtection |\n| `IPSecEndSystem` | x509.ExtKeyUsageIPSECEndSystem |\n| `IPSecTunnel` | x509.ExtKeyUsageIPSECTunnel |\n| `IPSecUser` | x509.ExtKeyUsageIPSECUser |\n| `TimeStamping` | x509.ExtKeyUsageTimeStamping |\n| `OCSPSigning` | x509.ExtKeyUsageOCSPSigning |\n| `MicrosoftServerGatedCrypto` | x509.ExtKeyUsageMicrosoftServerGatedCrypto |\n| `NetscapeServerGatedCrypto` | x509.ExtKeyUsageNetscapeServerGatedCrypto |\n| `MicrosoftCommercialCodeSigning` | x509.ExtKeyUsageMicrosoftCommercialCodeSigning |\n| `MicrosoftKernelCodeSigning` | x509.ExtKeyUsageMicrosoftKernelCodeSigning |\n\n\ne.g.:\n\n```yaml\nspec:\n  \u003c\u003c: (( \u0026local ))\n  ca:\n    organization: Mandelsoft\n    commonName: Uwe Krueger\n    privateKey: (( data.cakey ))\n    isCA: true\n    usage:\n      - Signature\n      - KeyEncipherment\n\ndata:\n  cakey: (( x509genkey(2048) ))\n  cacert: (( x509cert(spec.ca) ))\n```\n\ngenerates a self-signed root certificate and resolves to something like\n\n```yaml\ncakey: |+\n    -----BEGIN RSA PRIVATE KEY-----\n    MIIEpAIBAAKCAQEAwxdZDfzxqz4hlRwTL060pm1J12mkJlXF0VnqpQjpnRTq0rns\n    CxMxvSfb4crmWg6BRaI1cEN/zmNcT2sO+RZ4jIOZ2Vi8ujqcbzxqyoBQuMNwdb32\n    ...\n    oqMC9QKBgQDEVP7FDuJEnCpzqddiXTC+8NsC+1+2/fk+ypj2qXMxcNiNG1Az95YE\n    gRXbnghNU7RUajILoimAHPItqeeskd69oB77gig4bWwrzkijFXv0dOjDhQlmKY6c\n    pNWsImF7CNhjTP7L27LKk49a+IGutyYLnXmrlarcNYeCQBin1meydA==\n    -----END RSA PRIVATE KEY-----\ncacert: |+\n    -----BEGIN CERTIFICATE-----\n    MIIDCjCCAfKgAwIBAgIQb5ex4iGfyCcOa1RvnKSkMDANBgkqhkiG9w0BAQsFADAk\n    MQ8wDQYDVQQKEwZTQVAgU0UxETAPBgNVBAMTCGdhcmRlbmVyMB4XDTE4MTIzMTE0\n    ...\n    pOUBE3Tgim5rnpa9K9RJ/m8IVqlupcONlxQmP3cCXm/lBEREjODPRNhU11DJwDdJ\n    5fd+t5SMEit2BvtTNFXLAwz48EKTxsDPdnHgiQKcbIV8NmgUNPHwXaqRMBLqssKl\n    Cyvds9xGtAtmZRvYNI0=\n    -----END CERTIFICATE-----\n```\n#### `(( x509parsecert(cert) ))`\n\nThis function parses a certificate given in PEM format and returns a map\nof fields:\n\n| Field Name  | Type | Required | Meaning |\n| ------------| ---- | -------- | ------- |\n| `commonName` | string | optional |  Common Name field of the subject |\n| `organization` | string list | optional |  Organization field of the subject |\n| `country` | string list | optional |  Country field of the subject |\n| `isCA` | bool | always |  CA option of certificate |\n| `usage` | string list | always |  usage keys for the certificate (see below) |\n| `validity` | integer | always |  validity interval in hours |\n| `validFrom` | string | always |  start time in the format \"Jan 1 01:22:31 2019\" |\n| `validUntil` | string | always |  start time in the format \"Jan 1 01:22:31 2019\" |\n| `hosts` | string list | optional |  List of DNS names or IP addresses |\n| `dnsNames` | string list | optional |  List of DNS names |\n| `ipAddresses` | string list | optional |  List of IP addresses |\n| `publicKey` | string | always|  public key to generate the certificate for |\n\ne.g.:\n\n```yaml\ndata:\n  \u003c\u003c: (( \u0026temporary ))\n  spec:\n    commonName: test\n    organization: org\n    validity: 100\n    isCA: true\n    privateKey: (( gen.key ))\n    hosts:\n      - localhost\n      - 127.0.0.1\n\n    usage:\n     - ServerAuth\n     - ClientAuth\n     - CertSign\n\n  gen:\n    key: (( x509genkey() ))\n    cert: (( x509cert(spec) ))\n\ncert: (( x509parsecert(data.gen.cert) ))\n```\n\nresolves to\n\n```yaml\ncert:\n  commonName: test\n  dnsNames:\n  - localhost\n  hosts:\n  - 127.0.0.1\n  - localhost\n  ipAddresses:\n  - 127.0.0.1\n  isCA: true\n  organization:\n  - org\n  publickey: |+\n    -----BEGIN RSA PUBLIC KEY-----\n    MIIBCgKCAQEA+UIZQUTa/j+WlXC394bccBTltV+Ig3+zB1l4T6Vo36pMBmU4JIkJ\n    ...\n    TCsrEC5ey0cCeFij2FijOJ5kmm4cK8jpkkb6fLeQhFEt1qf+QqgBw3targ3LnZQf\n    uE9t5MIR2X9ycCQSDNBxcuafHSwFrVuy7wIDAQAB\n    -----END RSA PUBLIC KEY-----\n  usage:\n  - CertSign\n  - ServerAuth\n  - ClientAuth\n  validFrom: Mar 11 15:34:36 2019\n  validUntil: Mar 15 19:34:36 2019\n  validity: 99  # yepp, that's right, there has already time passed since the creation\n```\n\n### Wireguard Functions\n\nspiff supports some useful functions to work with _wireguard_ keys.\nPlease refer also to the [Useful to Know](#useful-to-know) section to find some\ntips for providing state.\n\n#### `(( wggenkey() ))`\n\nThis function can be used generate private wireguard key. The result will\nbase64 encoded.\n\ne.g.:\n\n```yaml\nkeys:\n  key: (( wggenkey() ))\n```\n\nresolves to something like\n\n```yaml\nkey: WH9xNVJuSuh7sDVIyUAlmxc+woFDJg4QA6tGUVBtGns=\n```\n\n#### `(( wgpublickey(key) ))`\n\nFor a given key (for example generated with the [wggenkey](#-wggenkey-)\nfunction) this function extracts the public key and returns it again in base64 format-\n\ne.g.:\n\n```yaml\nkeys:\n  key: (( wggenkey() ))\n  public: (( wgpublickey(key)\n```\n\nresolves to something like\n\n```yaml\nkey: WH9xNVJuSuh7sDVIyUAlmxc+woFDJg4QA6tGUVBtGns=\npublic: n405KfwLpfByhU9pOu0A/ENwp0njcEmmQQJvfYHHQ2M=\n```\n\n## `(( lambda |x|-\u003ex \":\" port ))`\n\nLambda expressions can be used to define additional anonymous functions. They\ncan be assigned to yaml nodes as values and referenced with path expressions\nto call the function with approriate arguments in other dynaml expressions.\nFor the final document they are mapped to string values.\n\nThere are two forms of lambda expressions. While\n\n```yaml\nlvalue: (( lambda |x|-\u003ex \":\" port ))\n```\n\nyields a function taking one argument by directly taking the elements from the dynaml expression,\n\n```yaml\nstring: \"|x|-\u003ex \\\":\\\" port\"\nlvalue: (( lambda string ))\n```\n\nevaluates the result of an expression to a function. The expression must evaluate to a function or string. If the expression is evaluated to a string it parses the function from the string.\n\nSince the evaluation result of a lambda expression is a regular value, it can also be passed as argument to function calls and merged as value along stub processing.\n\nA complete example could look like this:\n\n```yaml\nlvalue: (( lambda |x,y|-\u003ex + y ))\nmod: (( lambda|x,y,m|-\u003e(lambda m)(x, y) + 3 ))\nvalue: (( .mod(1,2, lvalue) ))\n```\n\nyields\n\n```yaml\nlvalue: lambda |x,y|-\u003ex + y\nmod: lambda|x,y,m|-\u003e(lambda m)(x, y) + 3\nvalue: 6\n```\nIf a complete expression is a lambda expression the keyword `lambda` can be omitted.\n\nLambda expressions evaluate to lambda values, that are used as final values\nin yaml documents processed by _spiff_. \n\n**Note**: If the final document still contains lambda values, they are transferred\nto a textual representation. It is not guaranteed that this representation can \ncorrectly be parsed again, if the document is re-processed by _spiff_. Especially\nfor complex scoped and curried functions this is not possible.\n\nTherefore function nodes should always be _temporary_ or _local_ to be available\nduring processing or merging, but being omitted for the final document.\n\n\n### Positional versus Named Arguments\n\nA typical function call uses positional arguments. Here the given arguments\nsatisfy the declared function parameters in the given order.\nFor lambda values it is also possible to use named arguments in the call\nexpression. Here an argument is assigned to a dedicated parameter as declared\nby the lambda expression. The order of named arguments can be arbitrarily chosen.\n\ne.g.:\n\n```yaml\nfunc: (( |a,b,c|-\u003e{$a=a, $b=b, $c=c } ))\nresult: (( .func(c=1, b=2, a=1) ))\n```\n\nIt is also posible to combine named with positional arguments. Hereby the\npositional arguments must follow the named ones.\n\ne.g.:\n\n```yaml\nfunc: (( |a,b,c|-\u003e{$a=a, $b=b, $c=c } ))\nresult: (( .func(c=1, 1, 2) ))\n```\n\nThe same argument MUST NOT be satified by both, a named and a positional \nargument.\n\nInstead of using the parameter name it is also possible to use the parameter\nindex, instead.\n\ne.g.:\n\n```yaml\nfunc: (( |a,b,c|-\u003e{$a=a, $b=b, $c=c } ))\nresult: (( .func(3=1, 1) ))\n```\n\nAs such, this feature seems to be quite useless, but it shows its power if\ncombined with [optional parameters](#optional-parameters) or \n[currying](#currying) as shown in the next paragraphs.\n\n### Scopes and Lambda Expressions\n\nA lambda expression might refer to absolute or relative nodes of the actual yaml document of the call. Relative references are evaluated in the context of the function call. Therefore\n\n```yaml\nlvalue: (( lambda |x,y|-\u003ex + y + offset ))\noffset: 0\nvalues:\n  offset: 3\n  value: (( .lvalue(1,2) ))\n```\n\nyields `6` for `values.value`.\n\n\nBesides the specified parameters, there is an implicit name (`_`), that can be\nused to refer to the function itself. It can be used to define self recursive\nfunction. Together with the logical and conditional operators a fibunacci\nfunction can be defined:\n\n```yaml\nfibonacci: (( lambda |x|-\u003e x \u003c= 0 ? 0 :x == 1 ? 1 :_(x - 2) + _( x - 1 ) ))\nvalue: (( .fibonacci(5) ))\n```\n\nyields the value `8` for the `value` property.\n\nBy default reference expressions in a lambda expression are evaluated in the\nstatic scope of the lambda dedinition followed by the static yaml scope of the\ncaller. Absolute references are always evalated in the document scope of the\ncaller.\n\nThe name `_` can also be used as an anchor to refer to the static definition\nscope of the lambda expression in the yaml document that was used to define\nthe lambda function. Those references are always interpreted as relative\nreferences related to the this static yaml document scope. There is no\ndenotation for accessing the root element of this definition scope.\n\nRelative names can be used to access the static \ndefinition scope given inside the dynaml expression (outer scope literals and\nparameters of outer lambda parameters)\n\ne.g.:\n\n````yaml\nenv:\n  func: (( |x|-\u003e[ x, scope, _, _.scope ] ))\n  scope: definition\n\ncall:\n   result: (( env.func(\"arg\") ))\n   scope: call\n````\n\nyields the `result` list:\n\n```yaml\ncall:\n  result:\n  - arg\n  - call\n  - (( lambda|x|-\u003e[x, scope, _, _.scope] ))  # the lambda expression as lambda value\n  - definition\n```\n\nThis also works across multiple stubs. The definition context is the stub the\nlambda expression is defined in, even if it is used in stubs down the chain.\nTherefore it is possible to use references in the lambda expression, not visible\nat the caller location, they carry the static yaml document scope of their \ndefinition with them.\n\nInner lambda expressions remember the local binding of outer lambda expressions.\nThis can be used to return functions based on arguments of the outer function.\n\ne.g.:\n\n```yaml\nmult: (( lambda |x|-\u003e lambda |y|-\u003e x * y ))\nmult2: (( .mult(2) ))\nvalue: (( .mult2(3) ))\n```\n\nyields `6` for property `value`.\n\n### Optional Parameters\n\nTrailing parameters may be defaulted in the lambda expression by assigning\nvalues in the declaration. Those parameter are then optional, it is not\nrequired to specify arguments for those parameters in function calls. \n\ne.g.:\n\n```yaml\nmult: (( lambda |x,y=2|-\u003e x * y ))\nvalue: (( .mult(3) ))\n```\n\nyields `6` for property `value`.\n\nIt is possible to default all parameters of a lambda expression. The function\ncan then be called without arguments. There might be no non-defaulted parameters\nafter a defaulted one.\n\nA call with positional arguments may only omit arguments for optional parameters\nfrom right to left. If there should be an explicit argument for the right most\nparameter, arguments for all parameters must be specified or\n[named arguments](#positional-versus-named-arguments) must be used.\nHere the desired optional parameter can explicitly be set prior to the regular\npositional arguments.\n\ne.g.:\n\n```yaml\nfunc:  (( |a,b=1,c=2|-\u003e{$a=a, $b=b, $c=c } ))\nresult: (( .func(c=3, 2) ))\n```\n\nevaluates `result` to\n\n```yaml\nresult:\n  a: 2\n  b: 1\n  c: 3\n```\n\nThe expression for the default does not need to be a constant value or even\nexpression, it might refer to other nodes in the yaml document. The default\nexpression is always evaluated in the scope of the lambda expression declaration\nat the time the lambda expression is evaluated.\n\ne.g.:\n\n**stub.yaml**\n```yaml\ndefault: 2\nmult: (( lambda |x,y=default * 2|-\u003e x * y ))\n```\n\n**template.yaml**\n```yaml\nmult: (( merge ))\nscope:\n  default: 3\n  value: (( .mult(3) ))\n```\n\nevaluates `value`to 12\n\n### Variable Argument Lists\n\nThe last parameter in the parameter list of a lambda expression may be a \n_varargs_ parameter consuming additional argument in a fnction call.\nThis parameter is always a list of values, one entry per additional argument.\n\nA _varargs_ parameter is denoted by a `...` following the last parameter name.\n\ne.g.:\n\n```yaml\nfunc: (( |a,b...|-\u003e [a] b ))\nresult: (( .func(1,2,3) ))\n```\n \nyields the list `[1, 2, 3]` for property `result`.\n\nIf no argument is given for the _varargs_ parameter its value is the empty list.\n\nThe `...` operator can also be used for [inline list expansion](#inline-list-expansion).\n\nIf a vararg parameter should be set by a [named argument](#positional-versus-named-arguments)\nits value must be a list.\n\n### Currying\n\nUsing the _currying_ operator (`*(`) a lambda function may be transformed to\nanother function with less parameters by specifying leading argument values.\n\nThe result is a new function taking the missing arguments (currying) and using\nthe original function body with a static binding for the specified parameters.\n\ne.g.:\n\n```yaml\nmult: (( lambda |x,y|-\u003e x * y ))\nmult2: (( .mult*(2) ))\nvalue: (( .mult2(3) ))\n```\n\nCurrying may be combined with [defaulted parameters](#optional-parameters).\nBut the resulting function does not default the leading parameters, it\nis just a new function with less parameters pinning the specified ones.\n\nIf the original function uses a [variable argument list](#variable-argument-lists),\nthe currying may span any number of the variable argument part, but once at\nleast one such argument is given, the parameter for the variable part is satisfied.\nIt cannot be extended by a function call of the curried function.\n\ne.g.:\n\n```yaml\nfunc: (( |a,b...|-\u003ejoin(a,b) ))\nfunc1: (( .func*(\",\",\"a\",\"b\")))\n#invalid: (( .func1(\"c\") ))\nvalue: (( .func1() ))\n```\n\nevaluates `value` to `\"a,b\"`.\n\nIt is also possible to use currying for builtin functions, like \n[`join`](#-join---list-).\n\ne.g.:\n\n```yaml\nstringlist: (( join*(\",\") ))\nvalue: (( .stringlist(\"a\", \"b\")  ))\n```\n\nevaluates `value` to `\"a,b\"`.\n\nThere are several builtin functions acting on unevaluated or unevaluatable\narguments, like [`defined`](#-definedfoobar-). For these functions currying is\nnot possible.\n\nUsing positional arguments currying is only possible from right to left.\nBut currying can also be done for [named arguments](#positional-versus-named-arguments).\nHere any parameter combination, regardless of the position in the parameter\nlist, can be preset. The resulting function then has the unsatisfied parameters\nin their original order. Switching the parameter order is not possible.\n\ne.g.:\n\n```yaml\nfunc: (( |a,b=1,c=2|-\u003e{$a=a, $b=b, $c=c } ))\ncurry: (( .func(c=3, 2) ))\n\nresult: (( .curry(5) ))\n```\n\nevalutes `result` to\n\n```yaml\nresult:\n  a: 2\n  b: 5\n  c: 3\n```\n\nThe resulting function keeps the parameter `b`. Hereby the default value will\nbe kept. Therefore it can just be called without argument (`.curry()`), which \nwould produce\n\n```yaml\nresult:\n  a: 2\n  b: 1\n  c: 3\n```\n\n**Attention**: \n\nFor compatibility reasons currying is also done, if a lambda function without\ndefaulted parameters is called with less arguments than declared parameters.\n\nThis behaviour is **deprecated** and will be removed in the future. It is\nreplaced by the currying operator.\n\ne.g.:\n\n```yaml\nmult: (( lambda |x,y|-\u003e x * y ))\nmult2: (( .mult(2) ))\nvalue: (( .mult2(3) ))\n```\n\nevaluates `value` to 6.\n\n## `(( catch[expr|v,e|-\u003ev] ))`\n\nThis expression evaluates an expression (`expr`) and then\nexecutes a lambda function with the evaluation state of the expression.\nIt always succeeds, even if the expression fails.\nThe lambda function may take one or two arguments, the first\nis always the evaluated value (or `nil` in case of an error).\nThe optional second argument gets the error message the evaluation of\nthe expression failed (or `nil` otherwise)\n\nThe result of the function is the result of the whole\nexpression. If the function fails, the complete expression fails.\n\ne.g.:\n\n```yaml\ndata:\n  fail: (( catch[1 / 0|v,e|-\u003e{$value=v, $error=e}] ))\n  valid: (( catch[5 * 5|v,e|-\u003e{$value=v, $error=e}] ))\n```\n\nresolves to \n\n```yaml\ndata:\n  fail:\n    error: division by zero\n    value: null\n  valid:\n    error: null\n    value: 25\n```\n\n## `(( sync[expr|v,e|-\u003edefined(v.field),v.field|10] ))`\n\nIf an expression `expr` may return different results for different evaluations,\nit is possible to synchronize the final output with a dedicated condition\non the expression value. Such an expression could, for example, be an\nuncached `read`, `exec` or `pipe` call.\n\nThe second element must evaluate to a lambda value, given by either a\nregular expression or by a lambda literal as shown in the title.\nIt may take one or two arguments, the actual value of the value expression\nand optionally an error message in case of a failing evaluation.\nThe result of the evaluation of the lamda expression decides whether \nthe state of the evaluation of the value expression is acceptable (`true`)\nor not (`false`).\n\nIf the value is accepted, an optional third expression is used to determine\nthe final result of the `sync[]` expression. It might be given as an expression\nevaluating to a lambda value, or by a comma separated expression using the\nsame binding as the preceeding lambda literal.\nIf not given, the value of the synched expression is returned. \n\nIf the value is not acceptable, the evaluation is repeated until a timeout\napplies. The timeout in seconds is given by an optional fourth expression\n(default is 5 min). Either the fourth, or the both, the\nthird and the fourth elements may be omitted.\n\nThe lambda values might be given as literal, or by expression, leading to the\nfollowing flavors:\n\n- `sync[expr|v,e|-\u003econd,value|10]`\n- `sync[expr|v,e|-\u003econd|valuelambda|10]`\n- `sync[expr|v,e|-\u003econd|v|-\u003evalue|10]`\n- `sync[expr|condlambda|valuelambda|10]`\n- `sync[expr|condlambda|v|-\u003evalue|10]`\n\nwith or without the timeout expression.\n\ne.g.:\n\n```yaml\ndata:\n  alice: 25\nresult: (( sync[data|v|-\u003edefined(v.alice),v.alice] ))\n```\n\nresolves to \n\n```yaml\ndata:\n  alice: 25\nresult: 25\n```\n\nThis example is quite useless, because the sync expression is a constant. It\njust demonstrates the usage.\n\n## Mappings\n\nMappings are used to produce a new list from the entries of a _list_ or _map_,\nor a new map from entries of a _map_ containing the entries processed by a\ndynaml expression. The expression is\ngiven by a [lambda function](#-lambda-x-x--port-). There are two basic forms of\nthe mapping function: It can be inlined as in `(( map[list|x|-\u003ex \":\" port] ))`, \nor it can be determined by a regular dynaml expression evaluating to a lambda\nfunction as in `(( map[list|mapping.expression))` (here the mapping is taken\nfrom the property `mapping.expression`, which should hold an approriate lambda\nfunction).\n\nThe mapping comes in two target flavors: with `[]` or `{}` in the syntax. The first\nflavor always produces a _list_ from the entries of the given source. The\nsecond one takes only a map source and produces a filtered or transformed _map_.\n\nAdditionally the mapping uses three basic mapping behaviours:\n- _transforming the values using the keyword `map`_. Here the result of the lambda\n  function is used as new value to replace the original one. Or\n- _filtering using the keywork `select`_. Here the result of the lambda\n  function is used as a boolean to decide whether the entry should be kept\n  (`true`) or omitted (`false`).\n- _composing_ using the keyword `sum`. Here always the list flavor is used,\n  but the result type and content is completely determined by the parameterization\n  of the statement by successively aggregating one entry after the other into an\n  arbitrary initial value. \n\n**Note**: The special reference `_` is not set for inlined lambda functions as part of\nthe mapping syntax. Therefore the mapping statements (and all other statements using\ninlined lambda functions as part of their syntax) can be used inside regular lambda\nfunctions without hampering the meaning of this special refrence for the surrounding\nexplicit lambda expression.\n\n### `(( map[list|elem|-\u003edynaml-expr] ))`\n\nExecute a mapping expression on members of a list to produce a new (mapped) list. The first expression (`list`) must resolve to a list. The last expression (`x \":\" port`) defines the mapping expression used to map all members of the given list. Inside this expression an arbitrarily declared simple reference name (here `x`) can be used to access the actually processed list element.\n\ne.g.\n\n```yaml\nport: 4711\nhosts:\n  - alice\n  - bob\nmapped: (( map[hosts|x|-\u003ex \":\" port] ))\n```\n\nyields\n\n```yaml\nport: 4711\nhosts:\n- alice\n- bob\nmapped:\n- alice:4711\n- bob:4711\n```\n\nThis expression can be combined with others, for example:\n\n```yaml\nport: 4711\nlist:\n  - alice\n  - bob\njoined: (( join( \", \", map[list|x|-\u003ex \":\" port] ) ))\n\n```\n\nwhich magically provides a comma separated list of ported hosts:\n\n```yaml\nport: 4711\nlist:\n  - alice\n  - bob\njoined: alice:4711, bob:4711\n```\n\n### `(( map[list|idx,elem|-\u003edynaml-expr] ))`\n\nIn this variant, the first argument `idx` is provided with the index and the\nsecond `elem` with the value for the index.\n\ne.g.\n\n```yaml\nlist:\n  - name: alice\n    age: 25\n  - name: bob\n    age: 24\n\nages: (( map[list|i,p|-\u003ei + 1 \". \" p.name \" is \" p.age ] ))\n```\n\nyields\n\n```yaml\nlist:\n  - name: alice\n    age: 25\n  - name: bob\n    age: 24\n\nages:\n- 1. alice is 25\n- 2. bob is 24\n```\n\n### `(( map[map|key,value|-\u003edynaml-expr] ))`\n\nMapping of a map to a list using a mapping expression. The expression may have access to the key and/or the value. If two references are declared, both values are passed to the expression, the first one is provided with the key and the second one with the value for the key. If one reference is declared, only the value is provided.\n\ne.g.\n\n```yaml\nages:\n  alice: 25\n  bob: 24\n\nkeys: (( map[ages|k,v|-\u003ek] ))\n\n```\n\nyields\n\n```yaml\nages:\n  alice: 25\n  bob: 24\n\nkeys:\n- alice\n- bob\n```\n\n### `(( map{map|elem|-\u003edynaml-expr} ))`\n\nUsing `{}` instead of `[]` in the mapping syntax, the result is again a map\nwith the old keys and the new entry values. As for a list mapping additionally\na key variable can be specified in the variable list.\n\n```yaml\npersons:\n  alice: 27\n  bob: 26\nolder: (( map{persons|x|-\u003ex + 1} ) ))\n```\n\njust increments the value of all entries by one in the field `older`:\n\n```yaml\nolder:\n  alice: 28\n  bob: 27\n```\n\n**Remark**\n\nAn alternate way to express the same is to use `sum[persons|{}|s,k,v|-\u003es { k = v + 1 }]`.\n\n### `(( map{list|elem|-\u003edynaml-expr} ))`\n\nUsing `{}` instead of `[]` together with a list in the mapping syntax, the result is again a map\nwith the list elements as key and the mapped entry values. For this all list entries must be strings.\nAs for a list mapping additionally an index variable can be specified in the variable list.\n\n```yaml\npersons:\n  - alice\n  - bob\nlength: (( map{persons|x|-\u003elength(x)} ) ))\n```\n\njust creates a map mapping the list entries to their length:\n\n```yaml\nlength:\n  alice: 5\n  bob: 3\n```\n\n### `(( select[expr|elem|-\u003edynaml-expr] ))`\n\nWith `select` a map or list can be filtered by evaluating a boolean expression\nfor every entry. An entry is selected if the expression evaluates to true\nequivalent value. (see [conditions](#-a--1--foo-bar-)).\n\nBasically it offers all the mapping flavors available for `map[]`\n\ne.g.\n\n```yaml\nlist:\n  - name: alice\n    age: 25\n  - name: bob\n    age: 26\n\n\nselected: (( select[list|v|-\u003ev.age \u003e 25 ] ))\n```\n\nevaluates selected to\n\n```yaml\nselected:\n- name: bob\n  age: 26\n```\n\n**Remark**\n\nAn alternate way to express the same is to use `map[list|v|-\u003ev.age \u003e 25 ? v :~]`.\n\n### `(( select{map|elem|-\u003edynaml-expr} ))`\n\nUsing `{}` instead of `[]` in the mapping syntax, the result is again a map\nwith the old keys filtered by the given expression.\n\n```yaml\npersons:\n  alice: 25\n  bob: 26\nolder: (( select{persons|x|-\u003ex \u003e 25} ))\n```\n\njust keeps all entries with a value greater than 25 and omits all others:\n\n```yaml\nselected:\n  bob: 26\n```\n\nThis flavor only works on _maps_.\n\n**Remark**\n\nAn alternate way to express the same is to use `sum[persons|{}|s,k,v|-\u003ev \u003e 25 ? s {k = v} :s]`.\n\n\n## Aggregations\n\nAggregations are used to produce a single result from the entries of a _list_ or _map_ aggregating the entries by a dynaml expression. The expression is given by a [lambda function](#-lambda-x-x--port-). There are two basic forms of the aggregation function: It can be inlined as in `(( sum[list|0|s,x|-\u003es + x] ))`, or it can be determined by a regular dynaml expression evaluating to a lambda function as in `(( sum[list|0|aggregation.expression))` (here the aggregation function  is taken from the property `aggregation.expression`, which should hold an approriate lambda function).\n\n\n### `(( sum[list|initial|sum,elem|-\u003edynaml-expr] ))`\n\nExecute an aggregation expression on members of a list to produce an aggregation result. The first expression (`list`) must resolve to a list. The second expression is used as initial value for the aggregation. The last expression (`s + x`) defines the aggregation expression used to aggregate all members of the given list. Inside this expression an arbitrarily declared simple reference name (here `s`) can be used to access the intermediate aggregation result and a second reference name (here `x`) can be used to access the actually processed list element.\n\ne.g.\n\n```yaml\nlist:\n  - 1\n  - 2\nsum: (( sum[list|0|s,x|-\u003es + x] ))\n```\n\nyields\n\n```yaml\nlist:\n  - 1\n  - 2\nsum: 3\n```\n\n### `(( sum[list|initial|sum,idx,elem|-\u003edynaml-expr] ))`\n\nIn this variant, the second argument `idx` is provided with the index and the\nthird `elem` with the value for the index.\n\ne.g.\n\n```yaml\nlist:\n  - 1\n  - 2\n  - 3\n\nprod: (( sum[list|0|s,i,x|-\u003es + i * x ] ))\n```\n\nyields\n\n```yaml\nlist:\n  - 1\n  - 2\n  - 3\n\nprod: 8\n```\n\n### `(( sum[map|initial|sum,key,value|-\u003edynaml-expr] ))`\n\nAggregation of the elements of a map to a single result using an aggregation expression. The expression may have access to the key and/or the value. The first argument is always the intermediate aggregation result. If three references are declared, both values are passed to the expression, the second one is provided with the key and the third one with the value for the key. If two references are declared, only the second one is provided with the value of the map entry.\n\ne.g.\n\n```yaml\nages:\n  alice: 25\n  bob: 24\n\nsum: (( map[ages|0|s,k,v|-\u003es + v] ))\n\n```\n\nyields\n\n```yaml\nages:\n  alice: 25\n  bob: 24\n\nsum: 49\n```\n\n## Projections\n\nProjections work over the elements of a list or map yielding a result list. Hereby every element is mapped by an optional subsequent reference expression. This may contain again projections, dynamic references or lambda calls. Basically this is a simplified form of the more general [mapping](#mappings) yielding a list working with a lambda function using only a reference expression based on the elements.\n\n### `(( expr.[*].value ))`\n\nAll elements of a map or list given by the expression `expr` are dereferenced with the subsequent reference expression (here `.expr`). If this expression works on a map the elements are ordered accoring to their key values. If the subsequent reference expression is omitted, the complete value list isreturned. For a list expression this means the identity operation.\n\ne.g.:\n\n```yaml\nlist:\n  - name: alice\n    age: 25\n  - name: bob\n    age: 26\n  - name: peter\n    age: 24\n\nnames: (( list.[*].name ))\n```\n\nyields for `names`:\n\n```yaml\nnames:\n  - alice\n  - bob\n  - peter\n```\n\nor for maps:\n\n```yaml\nnetworks:\n  ext:\n    cidr: 10.8.0.0/16\n  zone1:\n    cidr: 10.9.0.0/16\n\ncidrs: (( .networks.[*].cidr ))\n```\n\nyields for `cidrs`:\n\n```yaml\ncidrs:\n  - 10.8.0.0/16\n  - 10.9.0.0/16\n```\n\n### `(( list.[1..2].value ))`\n\nThis projection flavor only works for lists. The projection is done for a dedicated slice of the initial list.\n\ne.g.:\n\n```yaml\nlist:\n  - name: alice\n    age: 25\n  - name: bob\n    age: 26\n  - name: peter\n    age: 24\n\nnames: (( list.[1..2].name ))\n```\n\nyields for `names`:\n\n```yaml\nnames:\n  - bob\n  - peter\n```\n\n## Inline List Expansion\n\nIn argument lists or list literals the _list expansion operator_ (`...`) can be\nused.  It is a postfix operator on any list expression. It substituted\nthe list expression by a sequence of the list members. It can be be used\nin combination with static list argument denotation.\n\ne.g.:\n\n```yaml\nlist:\n  - a\n  - b\n  \nresult: (( [ 1, list..., 2, list... ]  ))\n```\n\nevaluates `result` to\n\n```yaml\nresult:\n  - 1\n  - a\n  - b\n  - 2\n  - a\n  - b\n```\n\nThe following example demonstrates the usage in combination with the\n[_varargs_ operator](#variable_argument_lists) in functions:\n\n```yaml\nfunc: (( |a,b...|-\u003e [a] b ))\n\nlist:\n  - a\n  - b\n\na: (( .func(1,2,3) ))\nb: (( .func(\"x\",list..., \"z\") ))\nc: (( [ \"x\", .func(list...)..., \"z\" ] ))\n```\n\nevaluates the following results:\n\n```yaml\na:\n- 1\n- 2\n- 3\nb:\n- x\n- a\n- b\n- z\nc:\n- x\n- a\n- b\n- z\n```\n\nPlease note, that the list expansion might span multiple arguments (including the\n[_varargs_ parameter](#variable-argument-lists)) in lambda function calls.\n\n## Markers\n\nNodes of the yaml document can be marked to enable dedicated behaviours for this\nnode. Such markers are part of the _dynaml_ syntax and may be prepended to\nany dynaml expression. They are denoted by the `\u0026` character directly followed \nby a marker name. If the expression is a combination of markers and regular\nexpressions, the expression follows the marker list enclosed in brackets\n(for example `(( \u0026temporary( a + b ) ))`).\n\n**Note**: Instead of using a `\u003c\u003c:` insert field to place markers it is possible\nnow to use `\u003c\u003c\u003c:`, also, which allows to use regular yaml parsers for spiff-like\nyaml documents. `\u003c\u003c:` is kept for backward compatibility.\n\n### `(( \u0026temporary ))`\n\nMaps, lists or simple value nodes can be marked as *temporary*. Temporary nodes\nare removed from the final output document, but are available during merging and\ndynaml evaluation.\n\ne.g.:\n\n```yaml\ntemp:\n  \u003c\u003c: (( \u0026temporary ))\n  foo: bar\n\nvalue: (( temp.foo ))\n```\n\nyields:\n\n```yaml\nvalue: bar\n```\nAdding `- \u003c\u003c: (( \u0026temporary ))` to a list can be used to mark a list as temporary.\n\nThe temporary marker can be combined with regular dynaml expressions to tag plain fields. Hereby the\nparenthesised expression is just appended to the marker\n\ne.g.:\n\n```yaml\ndata:\n  alice: (( \u0026temporary ( \"bar\" ) ))\n  foo: (( alice ))\n```\n\nyields:\n\n```yaml\ndata:\n  foo: bar\n```\n\nThe temporary marker can be combined with the [template marker](#templates) to omit templates from the final output.\n\n### `(( \u0026local ))`\n\nThe marker `\u0026local` acts similar to `\u0026temporary` but local nodes are always\nremoved from a stub directly after resolving dynaml expressions. Such nodes\nare therefore not available for merging and they are not used for further\nmerging of stubs and finally the template.\n\n### `(( \u0026dynamic ))`\n\nThis marker can be used to mark a template expression (direct or referenced)\nto enforce the re-evaluation of the template in the usage context whenever the\nnode is used to override or inject a node value along the processing chain.\nIt can also be used together with\n[`\u0026inject`](#-inject-) or [`\u0026default`](#-default-).\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata: 1\n```\n\nmerged with\n\n**stub.yaml**\n```yaml\nid: (( \u0026dynamic \u0026inject \u0026template(__ctx.FILE) ))\n```\n\nwill resolve to\n\n```yaml\nid: template.yaml\ndata: 1\n```\n\nThe original template is kept along the merge chain and is evaluated\nseparately in the context of the very stub or template it is used.\n\nUsing this marker for nodes not evaluationg to a template value is\nnot possible.\n\n### `(( \u0026inject ))`\n\nThis marker requests the marked item to be injected into the next stub level,\neven is the hosting element (list or map) does not requests a merge.\nThis only works if the next level stub already contains the hosting element.\n\ne.g.:\n\n**template.yaml**\n```yaml\nalice:\n foo: 1\n```\n\n**stub.yaml**\n```yaml\nalice:\n  bar: (( \u0026inject(2) ))\n  nope: not injected\nbob:\n  \u003c\u003c: (( \u0026inject ))\n  foobar: yep\n\n```\n\nis merged to\n\n```yaml\nalice:\n  foo: 1\n  bar: 2\nbob:\n  foobar: yep\n```\n\n### `(( \u0026default ))`\n\nNodes marked as *default* will be used as default values\nfor downstream stub levels. If no such entry is set there it will behave like\n`\u0026inject` and implicitly add this node, but existing settings will not be\noverwritten.\n\nMaps (or lists) marked as *default* will be considered as values. \nThe map is used as a whole as default if no such field is defined downstream.\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata: { }\n```\n\n**stub.yaml**\n```yaml\ndata:\n  foobar:\n    \u003c\u003c: (( \u0026default ))\n    foo: claude\n    bar: peter\n```\n\nis merged to\n\n```yaml\ndata:\n  foobar:\n    foo: claude\n    bar: peter\n```\n\nTheir entries will neither be used for overwriting existing downstream values\nnor for defaulting non-existng fields of a not defaulted map field.\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata:\n  foobar:\n    bar: bob\n```\n\n**stub.yaml**\n```yaml\ndata:\n  foobar:\n    \u003c\u003c: (( \u0026default ))\n    foo: claude\n    bar: peter\n```\n\nis merged to\n\n```yaml\ndata:\n  foobar:\n    bar: bob\n```\n\nIf sub sequent defaulting is desired, the fields of a default map must again be\nmarked as default.\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata:\n  foobar:\n    bar: bob\n```\n\n**stub.yaml**\n```yaml\ndata:\n  foobar:\n    \u003c\u003c: (( \u0026default ))\n    foo: (( \u0026default (\"claude\") ))\n    bar: peter\n```\n\nis merged to\n\n```yaml\ndata:\n  foobar:\n    foo: claude\n    bar: bob\n```\n\n**Note**: The behaviour of list entries marked as *default* is undefined.\n\n\n### `(( \u0026state ))`\n\nNodes marked as *state* are handled during the merge processing as if the\nmarker would not be present. But there will be a special handling for enabled\nstate processing [(option `--state \u003cpath\u003e`)](#usage) at the end of the\ntemplate processing.\nAdditionally to the regular output a document consisting only of state nodes\n(plus all nested nodes) will be written to a state file. This file will be used\nas top-level stub for further merge processings with enabled state support.\n\nThis enables to keep state between two merge processings. For regular\nmerging sich nodes are only processed during the first processing. Later\nprocessings will keep the state from the first one, because those nodes\nwill be overiden by the state stub added to the end of the sub list.\n\nIf those nodes additionally disable merging (for example using \n`(( \u0026state(merge none) ))`) dynaml expressions in sub level nodes may\nperform explicit merging using the function `stub()` to refer to\nvalues provided by already processed stubs (especially the implicitly added\nstate stub). For an example please refer to the \n[state library](libraries/state/README.md).\n\n### `(( \u0026template ))`\n\nNodes marked as *template* will not be evaluated at the place of their\noccurrence. Instead, they will result in a template value stored as value for\nthe node. They can later be instantiated inside a _dynaml_ expression\n(see [below](#templates)).\n\n### `(( \u0026tag:name ))`\n\nThe tag marker can be used to assign a logical name to a node value.\nThis name can then be used in tagged reference expressions to refer to this\nnode value (see [below](#tags)).\n\nA tagged reference has the form `\u003ctagname\u003e::\u003cpath\u003e`. The `\u003cpath\u003e` may denote any\nsub node of a tagged node. If the value of a complete node\n(or a simple value node) should be used, the `\u003cpath\u003e` must denote the root path\n(`.`).\n\n## Tags\n\nTags can be used to label node values in multi-document streams\n(used as template). After defined for a document the tag can then be used to\nreference node values from the actual or previous document(s) of a\ndocument sequence in a multi-document stream. Tags can be added for complex or\nsimple value nodes. A tagged reference may be used to refer to the tagged\nvalue as a whole or sub structure.\n\n### `(( \u0026tag:name(value) ))`\n\nThis syntax is used to tag a node whose value is defined by a dynaml expression.\nIt can also be used to denote tagged simple value nodes. (As usual the value\npart is optional for adding markers to structured values\n(see [Markers](#markers)).)\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata:\n  \u003c\u003c: (( \u0026tag:persons ))\n  alice: (( \u0026tag:alice(25)\n```\n\nIf the name is prefixed with a star (`*`), the tag is defined globally.\nGobal tags surive stub processing and their value is visible in subsequent\nstub (and template) processings.\n\nA tag name may consist of multiple components separated by a colon (`:`).\n\nTags can also be defined dynamically by the dynaml\nfunction [tagdef](#-tagdeftag-valiue-).\n\n### `(( tag::foo ))`\n\nReference a sub path of the value of a tagged node.\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata:\n  \u003c\u003c: (( \u0026tag:persons ))\n  alice: 25\n\ntagref: (( persons::alice ))\n```\n\nresolves `tagref` to `25`\n\n### `(( tag::. ))`\n\nReference the whole (structured) value of tagged node.\n\ne.g.:\n\n**template.yaml**\n```yaml\ndata:\n  alice: (( \u0026tag:alice(25) ))\n\ntagref: (( alice::. ))\n```\n\nresolves `tagref` to `25`\n\n### `(( foo.bar::alice ))`\n\nTag names may be structured. A tag name consists of a non-empty list of \ntag components separated by a dot or colon (`:`). A tag component may\ncontain ASCII letters or numbers, starting wit a letter.\nMulti-component tags are subject to [Tag Resolution](#path-resolution-for-tags).\n\n### Path Resolution for Tags\n\nA tag reference always contains a tag name and a path separated by a double \ncolon (`::`).\nThe standard use-case is to describe a dedicated sub node for a tagged\nnode value.\n\nfor example, if the tag `X` describes the value\n\n```yaml\ndata:\n  alice: 25\n  bob: 24\n```\n\nthe tagged reference `X::data.alice` describes the value `25`.\n\nFor tagged references with a path other than `.` (the whole tag value),\nstructured tags feature a more sophisticated resolution mechanism. A structured\ntag consist of multiple tag components separated by a colon (`:`), for\nexample `lib:mylib`. Therefore, tags span a tree of namespaces or scopes\nused to resolve path references. A tag-less reference just uses \nthe actual document or binding to resolve a path expression.\n\nEvaluation of a path reference for a tag tries to resolve the path in the\nfirst tag tree level where the path is available (breadth-first search).\nIf this level contains multiple tags that could resolve the given path, the\nresolution fails because it cannot be unambigiously resolved. \n\nFor example:\n\n```yaml\ntags:\n  - \u003c\u003c: (( \u0026tag:lib:alice ))\n    data: alice.alice\n  - \u003c\u003c: (( \u0026tag:lib:alice:v1))\n    data: alice.v1\n  - \u003c\u003c: (( \u0026tag:lib:bob))\n    other: bob\nusage:\n   data: (( lib::data ))\n```\n\neffectively resolves `usage.data` to `lib:alice::data` and therefore to the value\n`alice.alice`.\n\nTo achieve this all matching sub tags are orderd by their number of\ntag components. The first sub-level tag containing such a\ngiven path is selected. For this level, the matching tag must be non-ambigious.\nThere must only be one tag with this level containing a matching path.\nIf there are multiple ones the evaluation fails. In the above example this would\nbe the case if tag `lib:bob` would contain a field `data` instead of or\nadditional to `other`.\n\nThis feature can be used in library stubs to provide qualified names for their\nelements that can be used with merging the containing document nodes into\nthe template.\n\n### Tags in Multi-Document Streams\n\nIf the template file is a multi-document stream the tags are preserved during\nthe complete processing. This means tags defined in a earlier document can be used \nin all following documents, also. But the tag names must be unique across all\ndocuments in a multi-document stream.\n\ne.g.:\n\n**template.yaml**\n```yaml\n\u003c\u003c: (( \u0026temporary ))\ndata:\n  \u003c\u003c: (( \u0026tag:persons ))\n  alice: 25\n  bob: 24\n---\nalice: (( persons::alice ))\n---\nbob: (( persons::bob ))\n```\n\nresolves to\n\n```yaml\n---\nalice: 25\n---\nbob: 24\n```\n\nTags defined by tag markers are available for stubs and templates.\nGlobal tags are available down the stub processing to the templates.\nLocal tags are only avaialble on the processing level they are declared.\n \nAdditionally to the tags explicitly set by tag markers, there are implicit\ndocument tags given by the document index during the processing of a \n(multi-document) template. The implicit document tags are qualified with the\nprefix `doc.`. This prefix should not be used to own tags in the documents\n\ne.g.:\n\n**template.yaml**\n```yaml\n\u003c\u003c: (( \u0026temporary ))\ndata:\n  \u003c\u003c: (( \u0026tag:persons ))\n  alice: 25\n  bob: 24\n---\nalice: (( persons::alice ))\nprev: (( doc.1::. ))\n---\nbob: (( persons::bob ))\nprev: (( doc.2::. ))\n```\n\nresolves to\n\n```yaml\n---\nalice: 25\nprev:\n  data:\n    alice: 25\n    bob: 24\n---\nbob: 24\nprev:\n  alice: 25\n  prev:\n    data:\n      alice: 25\n      bob: 24\n```\n\nIf the given document index is negative it denotes the document relative to the\none actually processed (so, the tag `doc.-1` denotes the previous document).\nThe index `doc.0` can be used to denote the actual document. Here always a path\nmust be specified, it is not possible to refer to the complete document\n(with `.`). \n\n## Templates\n\nA map can be tagged by a dynaml expression to be used as template. Dynaml expressions in a template are not evaluated at its definition location in the document, but can be inserted at other locations using dynaml.\nAt every usage location it is evaluated separately.\n\n### `\u003c\u003c: (( \u0026template ))`\n\nThe dynaml expression `\u0026template` can be used to tag a map node as template:\n\ne.g.:\n\n```yaml\nfoo:\n  bar:\n    \u003c\u003c: (( \u0026template ))\n    alice: alice\n    bob: (( verb \" \" alice ))\n```\n\nThe template will be the value of the node `foo.bar`. As such it can be overwritten as a whole by settings in a stub during the merge process. Dynaml expressions in the template are not evaluated. A map can have only a single `\u003c\u003c` field. Therefore it is possible to combine the template marker with an expression just by adding the expression in parenthesis.\n\nAdding `- \u003c\u003c: (( \u0026template ))` to a list it is also possible to define list templates.\nIt is also possible to convert a single expression value into a simple template by adding the template\nmarker to the expression, for example `foo: (( \u0026template (expression) ))`\n\nThe template marker can be combined with the [temporary marker](#-temporary-) to omit templates from the final output.\n\n**Note**: Instead of using a `\u003c\u003c:` insert field to place the template marker it is\npossible now to use `\u003c\u003c\u003c:`, also, which allows to use regular yaml parsers for\nspiff-like yaml documents. `\u003c\u003c:` is kept for backward compatibility.\n\n### `(( *foo.bar ))`\n\nThe dynaml expression `*\u003creference expression\u003e` can be used to evaluate a template somewhere in the yaml document.\nDynaml expressions in the template are evaluated in the context of this expression.\n\ne.g.:\n\n```yaml\nfoo:\n  bar:\n    \u003c\u003c: (( \u0026template ))\n    alice: alice\n    bob: (( verb \" \" alice ))\n\n\nuse:\n  subst: (( *foo.bar ))\n  verb: loves\n\nverb: hates\n```\n\nevaluates to\n\n```yaml\nfoo:\n  bar:\n    \u003c\u003c: (( \u0026template ))\n    alice: alice\n    bob: (( verb \" \" alice ))\n\nuse:\n  subst:\n    alice: alice\n    bob: loves alice\n  verb: loves\n\nverb: hates\n```\n\n## Scope References\n\n### `_`\n\nThe special reference `_` (_self_) can be used inside of _lambda functions_\nand _templates_. They refer to the containing element (the lambda function or\ntemplate).\n\nAdditionally it can be used to lookup relative reference expressions\nstarting with the defining document scope of the element skipping intermediate\nscopes.\n\ne.g.:\n\n```yaml\nnode:\n  data:\n    scope: data\n  funcs:\n    a: (( |x|-\u003escope ))\n    b: (( |x|-\u003e_.scope ))\n    c: (( |x|-\u003e_.data.scope ))\n    scope: funcs\n\ncall:\n  scope: call\n\n  a: (( node.funcs.a(1) ))\n  b: (( node.funcs.b(1) ))\n  c: (( node.funcs.c(1) ))\n\n```\n\nevaluates `call` to\n\n```yaml\ncall:\n  a: call\n  b: funcs\n  c: data\n  scope: call\n```\n\n### `__`\n\nThe special reference `__` can be used to lookup references as relative\nreferences starting with the document node hosting the actually evaluated\n_dynaml_ expression skipping intermediate scopes.\n \nThis can, for example be\nused to relatively access a lambda value field besides the actual field in\na map. The usage of plain function names is reserved for builtin functions\nand are not used as relative references.\n\nThis special reference is also available in expressions in _templates_ and\nrefer to the map node in the template hosting the actually evaluated expression.\n\ne.g.:\n\n```yaml\ntemplates:\n  templ:\n    \u003c\u003c: (( \u0026template ))\n    self: (( _ ))\n    value: (( ($self=\"value\") __.self ))\n    result: (( scope ))\n    templ: (( _.scope ))\n\n  scope: templates\n\n\nresult:\n  inst: (( *templates.templ ))\n  scope: result\n```\n\nevaluates `result` to\n\n```yaml\nresult:\n  inst:\n    result: result\n    templ: templates\n    \n    self:\n      \u003c\u003c: (( \u0026template ))\n      result: (( scope ))\n      self: (( _ ))\n      templ: (( _.scope ))\n      value: (( ($self=\"value\") __.self ))\n    value:\n      \u003c\u003c: (( \u0026template ))\n      result: (( scope ))\n      self: (( _ ))\n      templ: (( _.scope ))\n      value: (( ($self=\"value\") __.self ))\n  scope: result\n```\n\nor with referencing upper nodes:\n\n```yaml\ntemplates:\n  templ:\n    \u003c\u003c: (( \u0026template ))\n    alice: root\n    data:\n      foo: (( ($bob=\"local\") __.bob ))\n      bar: (( ($alice=\"local\") __.alice ))\n      bob: static\n\n\nresult: (( *templates.templ ))\n```\n\nevaluates `result`  to\n\n```yaml\nresult:\n  alice: root\n  data:\n    bar: root\n    foo: static\n    \n    bob: static\n```\n\n\n### `___`\n\nThe special reference `___` can be used to lookup references in the outer most\nscope. It can therefore be used to access processing bindings specified for a\ndocument processing via command line or API. If no bindings are specified\nthe document root is used.\n\nCalling `spiff merge template.yaml --bindings bindings.yaml` with a binding of\n\n**bindings.yaml**\n```yaml\ninput1: binding1\ninput2: binding2\n``` \n\nand the template\n\n**template.yaml**\n```yaml\ninput1: top1\nmap:\n  input: map\n  input1: map1\n  \n  results:\n    frommap: (( input1 ))\n    fromroot: (( .input1 ))\n    frombinding1: (( ___.input1 ))\n    frombinding2: (( input2 ))\n```\n\nevaluates `map.results`  to\n\n```yaml\n  results:\n    frombinding1: binding1\n    frombinding2: binding2\n    frommap: map1\n    fromroot: top1\n```\n\n### `__ctx.OUTER`\n\nThe context field `OUTER` is used for nested [merges](#-mergemap1-map2-). \nIt is a list of documents, index 0 is the next outer document, and so on.\n\n## Special Literals\n\n### `(( {} ))`\n\nProvides an empty map.\n\n### `(( [] ))`\n\nProvides an empty list. Basically this is not a dedicated literal, but just a regular list expression without a value.\n\n### `(( ~ ))`\n\nProvides the *null* value.\n\n### `(( ~~ ))`\n\nThis literal evaluates to an *undefined* expression. The element (list entry or map field) carrying this value, although defined, will be removed from the document and handled as undefined for further merges and the evaluation of referential expressions.\n\ne.g.:\n\n```yaml\nfoo: (( ~~ ))\nbob: (( foo || ~~ ))\nalice: (( bob || \"default\"))\n```\n\nevaluates to\n\n```yaml\nalice: default\n```\n\n## Access to evaluation context\n\nInside every dynaml expression a virtual field `__ctx` is available. It allows access to information about the actual evaluation context. It can be accessed by a relative reference expression.\n\nThe following fields are supported:\n\n| Field Name  | Type | Meaning |\n| ------------| ---- | ------- |\n| `VERSION` | string |current version of *spiff*  |\n| `FILE` | string | name of actually processed template file  |\n| `DIR`  | string | name of directory of actually processed template file  |\n| `RESOLVED_FILE` | string | name of actually processed template file with resolved symbolic links |\n| `RESOLVED_DIR`  | string | name of directory of actually processed template file with resolved symbolic links |\n| `PATHNAME` | string | path name of actually processed field |\n| `PATH` | list[string] | path name as component list |\n| `OUTER` | yaml doc | outer documents for nested [merges](#-mergemap1-map2-), index 0 is the next outer document  |\n| `BINDINGS` | yaml doc |  the external bindings for the actual processing (see also [___](#___)) |\n\nIf external bindings are specified they are the last elements in `OUTER`.\n\ne.g.:\n\n**template.yml**\n```yaml\nfoo:\n  bar:\n    path: (( __ctx.PATH ))\n    str: (( __ctx.PATHNAME ))\n    file: (( __ctx.FILE ))\n    dir: (( __ctx.DIR ))\n```\n\nevaluates to\n\ne.g.:\n\n```yaml\nfoo:\n  bar:\n    dir: .\n    file: template.yml\n    path:\n    - foo\n    - bar\n    - path\n    str: foo.bar.str\n```\n\n## Operation Priorities\n\nDynaml expressions are evaluated obeying certain priority levels. This means operations with a higher priority are evaluated first. For example the expression `1 + 2 * 3` is evaluated in the order `1 + ( 2 * 3 )`. Operations with the same priority are evaluated from left to right (in contrast to version 1.0.7). This means the expression `6 - 3 - 2` is evaluated as `( 6 - 3 ) - 2`.\n\nThe following levels are supported (from low priority to high priority)\n\n1. `||`, `//`\n2. White-space separated sequence as concatenation operation (`foo bar`)\n3. `-or`, `-and`\n4. `==`, `!=`, `\u003c=`, `\u003c`, `\u003e`, `\u003e=`\n5. `+`, `-`\n6. `*`, `/`, `%`\n7. Grouping `( )`, `!`, constants, references (`foo.bar`), `merge`, `auto`, `lambda`, `map[]`, and [functions](#functions)\n\nThe complete grammar can be found in [dynaml.peg](dynaml/dynaml.peg).\n\n## String Interpolation\n\nFeature state: alpha\n\n**Attention:** This is an alpha feature. It must be enabled on the command\nline with the `--interpolation` or `--features=interpolation` option.\nAlso for the spiff library it must\nexplicitly be enabled. By adding the key `interpolation` to the feature list\nstored in the environment variable `SPIFF_FEATURES` this feature will be enabled\nby default.\n\nTypically a complete value can either be a literal or a dynaml expression.\nFor string literals it is possible to use an interpolation syntax to embed\ndynaml expressions into strings.\n\nFor example\n\n```yaml\ndata: test\ninterpolation: this is a (( data ))\n```\n\nreplaces the part between the double brackets by the result\nof the described expression evaluation. Here the brackets can be escaped\nby the usual escaping (`((!`) syntax.\n\nThose string literals will implicitly be converted to complete flat dynaml\nexpressions. The example above will therefore be converted into\n\n`(( \"this is a \" data ))`\n\nwhich is the regular dynaml equivalent. The escaping is very ticky, and\nmay be there are still problems. Quotes inside an embedded dynaml expression\ncan be escaped to enable quotes in string literals.\n\nIncomplete or partial interpolation expressions will be ignored and \njust used a s string.\n\nStrings inside a dynaml expression are NOT directly interpolated again, thus\n\n```yaml\ndata: \"test\"\ninterpolated: \"this is a (( length(\\\"(( data ))\\\") data ))\"\n```\n \nwill resolve `interpolation` to `this is 10test` and not to `this is 4test`.\n \nBut if the final string after the expression evaluation again describes a string\ninterpolation it will be processed, again.\n\n```yaml\ndata: test\ninterpolation: this is a (( \"(( data ))\" data ))\n```\n\nwill resolve `interpolation` to `this is testtest`.\n\nThe embedded dynaml expression must be concatenatable with strings.\n\n## YAML-based Control Structures\n\nFeature state: alpha\n\nIn addition to describe conditions and loops with *dynaml* expressions\nit is also possible to use elements of the document structure to embed\ncontrol structures.\n\nSuch a YAML-based control structure is always described as a map in YAML/JSON.\nThe syntactical elements are expressed as map fields starting with `\u003c\u003c`.\nAdditionally, depending on the control structure, regular fields are possible.\nControl structures finally represent a value for the containing node.\nThey may contain marker expressions (`\u003c\u003c`), also.\n\ne.g.:\n\n```yaml\ntemp:\n  \u003c\u003c: (( \u0026temporary ))\n  \u003c\u003cif: (( features(\"control\") ))\n  \u003c\u003cthen:\n    alice: 25\n    bob: 26\n```\n\nresolves to\n\n```yaml\nfinal:\n  alice: 25\n  bob: 26\n```\n\nTu use this alpha feature the feature flag `control` must be enabled.\n\nPlease be aware: Control structure maps typically are always completely resolved\nbefore they are evaluated.\n\n### Control Structures in Maps\n\nA control structure itself is always a dedicated map node in a document.\nIt is substituted by a regular value node determined by the execution\nof the control structure.\n\nThe fields of a control structure map are not subject to overwriting by stubs,\nbut the complete structure can be overwritten.\n\nIf used as value for a map field the resulting value is just used as effective\nvalue for this field.\n\nIf a map should be enriched by maps resulting from multiple control structures\nthe special control structure [`\u003c\u003cmerge:`](#merge) can be used. It allows to\nspecify a list of maps which should be merged with the actual control structure\nmap to finally build the result value.\n\n### Control Structures in Lists\n\nA control structure can be used as list value, also.\nIn this case there is a dedicated interpretation of the resulting\nvalue of the control structure. If it is NOT a list value,\nfor convenience, the value is directly used as list entry and substitutes the\ncontrol structure map.\n\nIf the resulting value is again a list, it is inserted into the containing\nlist at the place of occurrence of the control structure.\nSo, if a list value should be used as dedicated entry in a list, the \nresult of a control structure must be a list with the intended\nlist as entry.\n\ne.g.:\n\n```yaml\nlist:\n - \u003c\u003cif: (( features(\"control\") ))\n   \u003c\u003cthen: alice\n - \u003c\u003cif: (( features(\"control\") ))\n   \u003c\u003cthen:\n   - - peter\n - \u003c\u003cif: (( features(\"control\") ))\n   \u003c\u003cthen:\n   - bob\n```\nresolves to\n\n```yaml\nlist:\n- alice\n- - peter\n- bob\n```\n\n### `\u003c\u003cif:`\n\nThe condition structure is defined by the syntax field `\u003c\u003cif`. It additionally\naccepts the fields `\u003c\u003cthen` and `\u003c\u003celse`.\n\nThe condition field must provide a boolean value.\nIf it is `true` the optional `\u003c\u003cthen` field is used to substitute the control\nstructure, otherwise the optional `\u003c\u003celse` field is used.\n\nIf the appropriate case is not specified, the result is the undefined `(( ~~ ))`\nvalue. The containing field is therefore completely omitted from the output.\n\n.e.g.:\n\n```yaml\nx: test1\ncond:\n  field:\n    \u003c\u003cif: (( x == \"test\" ))\n    \u003c\u003cthen: alice\n    \u003c\u003celse: bob\n```\n\nevaluates `cond.field` to `bob`\n\nIf the *else* case is omitted, the `cond` field would be an empty map (`field`\nis omitted, because the contained control structure evaluates to *undefined*)\n\nA comparable way to do this with regular *dynaml* could look like this:\n\n```yaml\ncond: (( x == \"test\" ? \"alice\" :\"bob\" ))\n```\n\nA better way more suitable for complex cases would be:\n\n```yaml\nlocal:\n  \u003c\u003c: (( \u0026local))\n  then: alice\n  else: bob\n\ncond: (( x == \"test\" ? local.then :local.else ))\n```\n\n### `\u003c\u003cswitch:`\n\nThe `switch` control structure evaluates the switch value of the `\u003c\u003cswitch`\nfield to a string and uses it to select an appropriate regular field\nin the control map.\n\nIf it is not found the value of the optional field `\u003c\u003cdefault` is used. If no default\nis specified, the control structure evaluates to an error, if no appropriate\nregular field is available.\n\nThe *nil* value matches the `default` case. If the switch value is undefined\nthe control evaluates to the undefined value `(( ~~ ))`.\n\ne.g.:\n\n```yaml\nx: alice\n\nvalue:\n  \u003c\u003cswitch: (( x ))\n  alice: 25\n  bob: 26 \n  \u003c\u003cdefault: other\n```\n\nevaluates `value` to `25`.\n\nA comparable way to do this with regular *dynaml* could look like this:\n\n```yaml\nlocal:\n  \u003c\u003c: (( \u0026local))\n  cases:\n    alice: 25\n    bob: 26\n  default: other\n\nvalue: (( local.cases[x] || local.default ))\n```\n\n### `\u003c\u003ctype:`\n\nThe `type` control structure evaluates the type of the value of the `\u003c\u003ctype`\nfield and uses it to select an appropriate regular field\nin the control map.\n\nIf it is not found the value of the optional field `\u003c\u003cdefault` is used. If no default\nis specified, the control structure evaluates to an error, if no appropriate\nregular field is available.\n\n\ne.g.:\n\n```yaml\nx: alice\n\nvalue:\n  \u003c\u003ctype: (( x ))\n  string: alice\n  \u003c\u003cdefault: unknown\n```\n\nevaluates `value` to `alice`.\n\nA comparable way to do this with regular *dynaml* could look like this:\n\n```yaml\nlocal:\n  \u003c\u003c: (( \u0026local))\n  cases:\n    string: alice\n  default: unknown\n\nvalue: (( local.cases[type(x)] || local.default ))\n```\n\nFor more complex scenarios not only switching on strings a second syntax \ncan be used. Instead of using fields in the control map as cases, a dedicated\nfield `\u003c\u003ccases` may contain a list of cases, that are checked sequentially\n(In this flavor regular fields are not allowed anymore).\n\nEvery case is described again by a map containing the fields:\n\n- `case`: the expected value to match the switch value\n- `match`: a lambda function taking one argument and yielding a boolean value\nused to match the given switch value\n- `value`: (optional) the resulting value in case of a match. If not defined \n  the result will be the undefined value.\n\nOne of `case` or `match` must be present.\n\ne.g.:\n\n```yaml\nx: 5\nselected:\n  \u003c\u003cswitch: (( x ))\n  \u003c\u003ccases:\n    - case:\n        alice: 25\n      value: alice\n    - match: (( |v|-\u003ev == 5 ))\n      value: bob\n  \u003c\u003cdefault: unknown\n```\n\nresolves to\n\n```yaml\nx: 5\nselected: bob\n```\n\nIf `x` would be set to the complex value\n\n```yaml\nx:\n  alice: 25\n```\n\nit would resolve to\n \n```yaml\nx:\n  alice: 25\nselected: alice\n```\n\n### `\u003c\u003cfor:`\n\nThe loop control is able to execute a multi-dimensional loop and \nproduce a list or map based on the value combinations.\n\nThe loop ranges are specified by the value of the `\u003c\u003cfor` field.\nIt is possible ol loop over lists or maps.\nThe range specification can be given by either a map or list:\n- *map*: the keys of the map are the names of the control variables and the values\n       must be lists or maps specifying the ranges.\n       \n  The map key might optionally be a comma-separated pair (for example `key,value`) of\n  variable names. In this case the first name is the name for the index\n  variable and the second one for the value variable.\n  \n  If multiple ranges are specified iterations are alphabetically\n  ordered by value variable name (first) and index variable name (second) to\n  determine the traversing order.\n              \n- *list*: if the control variables are defined by a list, each list element\n   must contain two mandatory and one optional field(s):\n    - `name`: the name of the (list or map entry value) control variable\n    - `values`: a list to define the value range.\n    - `index` : (optional) the name of the variable providing the list index\n      or map key of the loop range (defaulted to `index-\u003cname\u003e`)\n  \n  Here the order in the list determine the traversal order.\n  \nTraversal is done by recursively iterating follow up ranges for every entry\nin the actual range. This means the last range is completely iterated\nfor the first values of the first ranges first.\n\nIf no index variable is specified for a loop range there is an additional\nimplicit binding for every control variable describing the actual list index\nor map key of the processed value for this dimension. It is denoted by\n`index-\u003ccontrol variable\u003e`\n\nIf multiple loop ranges are specified, the ranges may mix iterations over\nmaps and lists.\n\nThe iteration result value is determined by the value of the `\u003c\u003cdo` field.\nIt is implicitly handled as template and is evaluated for every\nset of iteration values. \n\n#### Lists as Iteration Result\n\nThe result of the evaluation using only the `\u003c\u003cdo` value field is a list.\n\ne.g.:\n\n```yaml\nalice:\n - a\n - b\nbob:\n - 1\n - 2\n - 3\nlist:\n  \u003c\u003cfor: \n     key,alice: (( .alice )) # sorted by using alice as primary sort key\n     bob: (( .bob ))\n  \u003c\u003cdo:\n    value: (( alice \"-\" key \"-\" bob \"-\" index-bob ))\n```\n\nevaluates `list` to\n\n```yaml\nlist:\n- value: a-0-1-0\n- value: a-0-2-1\n- value: a-0-3-2\n- value: b-1-1-0\n- value: b-1-2-1\n- value: b-1-3-2\n```\n\nIt first iterates over the values for `alice`. For each such value it then\niterates over the values of `bob`.\n\nA comparable way to do this with regular *dynaml* could look like this:\n\n```yaml\nlist: (( sum[alice|[]|s,key,alice|-\u003e s sum[bob|[]|s,index_bob,bob|-\u003es (alice \"-\" key \"-\" bob \"-\" index_bob)]] ))\n```\n\nA result list may omit entries if the value expression evaluates to the undefined\nvalue (`~~`). The nil value  (`~`) is kept. This way a  `for` control can be used\nto filter lists.\n\ne.g.: \n\n```yaml\nbob:\n- 1\n- 2\n- 3\nfiltered:\n  \u003c\u003cfor: \n     bob: (( .bob ))\n  \u003c\u003cdo: (( bob == 2 ? ~~ :bob ))\n```\n\nresolves to\n\n```yaml\nbob:\n- 1\n- 2\n- 3\nfiltered:\n- 1\n- 3 \n```\n\n#### Maps as Iteration Result\n\nIf the result should be a map it is required to additionally specify a\nkey value for every iteration. This is specified by the optional `\u003c\u003cmapkey`\nfield. Like the `\u003c\u003cdo` field it is implicitly handled as template and re-evaluated\nfor every iteration.\n\ne.g.:\n\n```yaml\nx: suffix\n\nalice:\n  - a\n  - b\nbob:\n  - 1\n  - 2\n  - 3\n\nmap: \n  \u003c\u003cfor:\n     - name: alice\n       values: (( .alice ))\n     - name: bob\n       values:  (( .bob ))\n  \u003c\u003cmapkey: (( alice bob ))\n  \u003c\u003cdo:\n    value: (( alice bob x )) \n```\n\nevaluates the field `map` to\n\n```yaml\nmap:\n  a1:\n    value: a1suffix\n  a2:\n    value: a2suffix\n  a3:\n    value: a3suffix\n  b1:\n    value: b1suffix\n  b2:\n    value: b2suffix\n  b3:\n    value: b3suffix\n```\n\nHere the traversal order is irrelevant as long as the generated key values\nare unique. If several evaluations of the key expression yield the same \nvalue the last one will win.\n\nA comparable way to do this with regular *dynaml* could look like this:\n\n```yaml\nmap: (( sum[alice|{}|s,index_alice,alice|-\u003e s sum[bob|{}|s,index_bob,bob|-\u003es {(alice bob)=alice bob x}]] ))\n```\n\nAn iteration value is ignored if the key or the value evaluate to the undefined\nvalue `(( ~~ ))`. Additionally the key may evaluate to the nil value `(( ~ ))`, also.\n\ne.g.:\n\n```yaml\nbob:\n  b1: 1\n  b2: 2\n  b3: 3\nfiltered:\n  \u003c\u003cfor: \n     key,bob: (( .bob ))\n  \u003c\u003cmapkey: (( key ))\n  \u003c\u003cdo: (( bob == 2 ? ~~ :bob ))\n```\n\nor\n\n```yaml\nbob:\n  b1: 1\n  b2: 2\n  b3: 3\nfiltered:\n  \u003c\u003cfor: \n     key,bob: (( .bob ))\n  \u003c\u003cmapkey: (( bob == 2 ? ~~ :key ))\n  \u003c\u003cdo: (( bob ))\n```\n\nresolve to\n\n```yaml\nbob:\n  b1: 1\n  b2: 2\n  b3: 3\nfiltered:\n  b1: 1\n  b3: 3\n```\n\n\n### `\u003c\u003cmerge:`\n\nWith `merge` it is possible to merge maps given as list value of the\n`\u003c\u003cmerge` field with regular map fields from the control structure\nto determine the final map value.\n\nThe value for `\u003c\u003cmerge:` may be a single map or a list of maps to join\nwith the directly given fields.\n\ne.g.:\n\n```yaml\nmap:\n  \u003c\u003cmerge: \n    - bob: 26\n      charlie: 1\n    - charlie: 27\n  alice: 25\n  charlie: 2\n```\n\nresolves to\n\n```yaml\nmap:\n  alice: 25\n  bob: 26\n  charlie: 27\n```\n\nIf multiple maps contain the same key, the last value (in order of list)\nwill win.\n\nThis might be combined with other control structures, for example to conditionally\nmerge multiple maps:\n\ne.g.:\n\n```yaml\nx: charlie\nmap:\n  \u003c\u003cmerge: \n    - \u003c\u003cif: (( x == \"charlie\" ))\n      \u003c\u003cthen:\n        charlie: 27\n    - \u003c\u003cif: (( x == \"alice\" ))\n      \u003c\u003cthen:\n        alice: 20\n  alice: 25\n  charlie: 2\n```\n\nresolves to\n\n```yaml\nx: charlie\nmap:\n  alice: 25\n  charlie: 27\n```\n\n# Structural Auto-Merge\n\nBy default `spiff` performs a deep structural merge of its first argument, the template file, with the given stub files. The merge is processed from right to left, providing an intermediate merged stub for every step. This means, that for every step all expressions must be locally resolvable.\n\nStructural merge means, that besides explicit dynaml `merge` expressions, values will be overridden by values of equivalent nodes found in right-most stub files. In general, flat value lists are not merged. Only lists of maps can be merged by entries in a stub with a matching index.\n\nThere is a special support for the auto-merge of lists containing maps, if the\nmaps contain a `name` field. Hereby the list is handled like a map with\nentries according to the value of the list entries' `name` field. If another\nkey field than `name` should be used, the key field of one list entry can be\ntagged with the prefix `key:` to indicate the indended key name. Such tags\nwill be removed for the processed output.\n\nIn general the resolution of matching nodes in stubs is done using the same rules that apply for the reference expressions [(( foo.bar.[1].baz ))](#-foobar1baz-).\n\nFor example, given the file **template.yml**:\n\n```yaml\nfoo:\n  - name: alice\n    bar: template\n  - name: bob\n    bar: template\n\nplip:\n  - id: 1\n    plop: template\n  - id: 2\n    plop: template\n\nbar:\n  - foo: template\n\nlist:\n  - a\n  - b\n```\n\nand file **stub.yml**:\n\n```yaml\nfoo:\n  - name: bob\n    bar: stub\n\nplip:\n  - key:id: 1\n    plop: stub\n\nbar:\n  - foo: stub\n\nlist:\n  - c\n  - d\n```\n\n```\nspiff merge template.yml stub.yml\n```\n\nreturns\n\n\n```yaml\nfoo:\n- bar: template\n  name: alice\n- bar: stub\n  name: bob\n\nplip:\n- id: 1\n  plop: stub\n- id: 2\n  plop: template\n\nbar:\n- foo: stub\n\nlist:\n- a\n- b\n```\n\nBe careful that any `name:` key in the template for the first element of the\n`plip` list will defeat the `key:id: 1` selector from the stub. When a `name`\nfield exist in a list element, then this element can only be targeted by this\nname. When the selector is defeated, the resulting value is the one provided\nby the template.\n\n## Bringing it all together\n\nMerging the following files in the given order\n\n**deployment.yml**\n```yaml\nnetworks: (( merge ))\n```\n\n**cf.yml**\n```yaml\nutils: (( merge ))\nnetwork: (( merge ))\nmeta: (( merge ))\n\nnetworks:\n  - name: cf1\n    \u003c\u003c: (( utils.defNet(network.base.z1,meta.deployment_no,30) ))\n  - name: cf2\n    \u003c\u003c: (( utils.defNet(network.base.z2,meta.deployment_no,30) ))\n```\n\n**infrastructure.yml**\n```yaml\nnetwork:\n  size: 16\n  block_size: 256\n  base:\n    z1: 10.0.0.0\n    z2: 10.1.0.0\n```\n\n**rules.yml**\n```yaml\nutils:\n  defNet: (( |b,n,s|-\u003e(*.utils.network).net ))\n  network:\n    \u003c\u003c: (( \u0026template ))\n    start: (( b + n * .network.block_size ))\n    first: (( start + ( n == 0 ? 2 :0 ) ))\n    lower: (( n == 0 ? [] :b \" - \" start - 1 ))\n    upper: (( start + .network.block_size \" - \" max_ip(net.subnets.[0].range) ))\n    net:\n      subnets:\n      - range: (( b \"/\" .network.size ))\n        reserved: (( [] lower upper ))\n        static:\n          - (( first \" - \" first + s - 1 ))\n```\n\n**instance.yml**\n```yaml\nmeta:\n  deployment_no: 1\n\n```\n\nwill yield a network setting for a dedicated deployment\n\n```yaml\nnetworks:\n- name: cf1\n  subnets:\n  - range: 10.0.0.0/16\n    reserved:\n    - 10.0.0.0 - 10.0.0.255\n    - 10.0.2.0 - 10.0.255.255\n    static:\n    - 10.0.1.0 - 10.0.1.29\n- name: cf2\n  subnets:\n  - range: 10.1.0.0/16\n    reserved:\n    - 10.1.0.0 - 10.1.0.255\n    - 10.1.2.0 - 10.1.255.255\n    static:\n    - 10.1.1.0 - 10.1.1.29\n```\n\nUsing the same config for another deployment of the same type just requires the replacement of the `instance.yml`.\nUsing a different `instance.yml`\n\n```yaml\nmeta:\n  deployment_no: 0\n\n```\n\nwill yield a network setting for a second deployment providing the appropriate settings for a unique other IP block.\n\n```yaml\nnetworks:\n- name: cf1\n  subnets:\n  - range: 10.0.0.0/16\n    reserved:\n    - 10.0.1.0 - 10.0.255.255\n    static:\n    - 10.0.0.2 - 10.0.0.31\n- name: cf2\n  subnets:\n  - range: 10.1.0.0/16\n    reserved:\n    - 10.1.1.0 - 10.1.255.255\n    static:\n    - 10.1.0.2 - 10.1.0.31\n```\n\nIf you move to another infrastructure you might want to change the basic IP layout. You can do it just by adapting the `infrastructure.yml`\n\n```yaml\nnetwork:\n  size: 17\n  block_size: 128\n  base:\n    z1: 10.0.0.0\n    z2: 10.0.128.0\n```\n\nWithout any change to your other settings you'll get\n\n```yaml\nnetworks:\n- name: cf1\n  subnets:\n  - range: 10.0.0.0/17\n    reserved:\n    - 10.0.0.128 - 10.0.127.255\n    static:\n    - 10.0.0.2 - 10.0.0.31\n- name: cf2\n  subnets:\n  - range: 10.0.128.0/17\n    reserved:\n    - 10.0.128.128 - 10.0.255.255\n    static:\n    - 10.0.128.2 - 10.0.128.31\n```\n\n## Useful to Know\n\n  There are several scenarios yielding results that do not seem to be obvious. Here are some typical pitfalls.\n\n- _The auto merge never adds nodes to existing structures_\n\n  For example, merging\n\n  **template.yml**\n  ```yaml\n  foo:\n    alice: 25\n  ```\n  with\n\n  **stub.yml**\n  ```yaml\n  foo:\n    alice: 24\n    bob: 26\n  ```\n\n   yields\n\n  ```yaml\n  foo:\n    alice: 24\n  ```\n\n  Use [\u003c\u003c: (( merge ))](#--merge-) to change this behaviour, or explicitly add desired nodes to be merged:\n\n   **template.yml**\n  ```yaml\n  foo:\n    alice: 25\n\tbob: (( merge ))\n  ```\n\n\n- _Simple node values are replaced by values or complete structures coming from stubs, structures are deep_ merged.\n\n  For example, merging\n\n  **template.yml**\n  ```yaml\n  foo: (( [\"alice\"] ))\n  ```\n  with\n\n  **stub.yml**\n  ```yaml\n  foo:\n    - peter\n    - paul\n  ```\n\n  yields\n\n  ```yaml\n  foo:\n    - peter\n    - paul\n  ```\n\n  But the template\n\n  ```yaml\n   foo: [ (( \"alice\" )) ]\n  ```\n\n  is merged without any change.\n\n- _Expressions are subject to be overridden as a whole_\n\n  A consequence of the behaviour described above is that nodes described by an expession are basically overridden by a complete merged structure, instead of doing a deep merge with the structues resulting from the expression evaluation.\n\n  For example, merging\n\n  **template.yml**\n  ```yaml\n  men:\n    - bob: 24\n  women:\n    - alice: 25\n\n  people: (( women men ))\n  ```\n  with\n\n  **stub.yml**\n  ```yaml\n  people:\n    - alice: 13\n  ```\n   yields\n\n  ```yaml\n  men:\n    - bob: 24\n  women:\n    - alice: 25\n\n  people:\n    - alice: 24\n  ```\n\n  To request an auto-merge of the structure resulting from the expression evaluation, the expression has to be preceeded with the modifier `prefer` (`(( prefer women men ))`). This would yield the desired result:\n\n  ```yaml\n  men:\n    - bob: 24\n  women:\n    - alice: 25\n\n  people:\n    - alice: 24\n    - bob: 24\n  ```\n\n- _Nested merge expressions use implied redirections_\n\n  `merge` expressions implicity use a redirection implied by an outer redirecting merge. In the following\n  example\n\n  ```yaml\n  meta:\n    \u003c\u003c: (( merge deployments.cf ))\n    properties:\n      \u003c\u003c: (( merge ))\n      alice: 42\n  ```\n  the merge expression in `meta.properties` is implicity redirected to the path `deployments.cf.properties`\n  implied by the outer redirecting `merge`. Therefore merging with\n\n  ```yaml\n  deployments:\n    cf:\n      properties:\n        alice: 24\n        bob: 42\n  ```\n\n  yields\n\n  ```yaml\n  meta:\n    properties:\n      alice: 24\n      bob: 42\n  ```\n\n- _Functions and mappings can freely be nested_\n\n  e.g.:\n\n  ```yaml\n  pot: (( lambda |x,y|-\u003e y == 0 ? 1 :(|m|-\u003em * m)(_(x, y / 2)) * ( 1 + ( y % 2 ) * ( x - 1 ) ) ))\n  seq: (( lambda |b,l|-\u003emap[l|x|-\u003e .pot(b,x)] ))\n  values: (( .seq(2,[ 0..4 ]) ))\n  ```\n\n  yields the list `[ 1,2,4,8,16 ]` for the property `values`.\n\n- _Functions can be used to parameterize templates_\n\n  The combination of functions with templates can be use to provide functions yielding complex structures.\n  The parameters of a function are part of the scope used to resolve reference expressions in a template used in the function body.\n\n  e.g.:\n\n  ```yaml\n  relation:\n    template:\n      \u003c\u003c: (( \u0026template ))\n      bob: (( x \" \" y ))\n    relate: (( |x,y|-\u003e*relation.template ))\n\n  banda: (( relation.relate(\"loves\",\"alice\") ))\n  ```\n\n  evaluates to\n\n  ```yaml\n  relation:\n    relate: lambda|x,y|-\u003e*(relation.template)\n    template:\n      \u003c\u003c: (( \u0026template ))\n      bob: (( x \" \" y ))\n\n\tbanda:\n      bob: loves alice\n  ```\n\n- _Scopes can be used to parameterize templates_\n\n  Scope literals are also considered when instantiating templates. Therefore\n  they can be used to set explicit values for relative reference expressions\n  used in templates.\n\n  e.g.:\n\n  ```yaml\n  alice: 1\n  template:\n    \u003c\u003c: (( \u0026template ))\n    sum: (( alice + bob ))\n  scoped: (( ( $alice = 25, \"bob\" = 26 ) *template ))\n  ```\n\n  evaluates to\n\n  ```yaml\n  alice: 1\n  template:\n    \u003c\u003c: (( \u0026template ))\n    sum: (( alice + bob ))\n  scoped:\n    sum: 51\n  ```\n\n\n- _Aggregations may yield complex values by using templates_\n\n  The expression of an aggregation may return complex values by returning inline lists or instantiated templates. The binding of the function will be available (as usual) for the evaluation of the template. In the example below the aggregation provides a map with both the sum and the product of the list entries containing the integers from 1 to 4.\n\n  e.g.:\n\n  ```yaml\n  sum: (( sum[[1..4]|init|s,e|-\u003e*temp] ))\n\n  temp:\n    \u003c\u003c: (( \u0026template ))\n    sum: (( s.sum + e ))\n    prd: (( s.prd * e ))\n  init:\n    sum: 0\n    prd: 1\n\t```\n\n  yields for `sum` the value\n  ```\n  sum:\n    prd: 24\n    sum: 10\n  ```\n\n- _Taking advantage of the *undefined* value_\n\n  At first glance it might look strange to introduce a value for *undefined*. But it can be really\n  useful as will become apparent with the following examples.\n\n  - Whenever a stub syntactically defines a field it overwrites the default in the template during\n    merging. Therefore it would not be possible to define some expression for that field that eventually\n\tkeeps the default value. Here the *undefined* value can help:\n\n    e.g.: merging\n\n    **template.yml**\n    ```yaml\n    alice: 24\n    bob: 25\n    ```\n\n    with\n\n    **stub.yml**\n    ```yaml\n\n    alice: (( config.alice * 2 || ~ ))\n    bob: (( config.bob * 3 || ~~ ))\n    ```\n\n    yields\n\n    ```yaml\n    alice: ~\n    bob: 25\n    ```\n\n  * There is a problem accessing upstream values. This is only possible if the local stub contains\n    the definition of the field to use. But then there will always be a value for this field, even\n\tif the upstream does not overwrite it.\n\n    Here the *undefined* value can help by providing optional access to upstream values.\n\tOptional means, that the field is only defined, if there is an upstream value. Otherwise it is\n\tundefined for the expressions in the local stub and potential downstream templates. This is\n\tpossible because the field is formally defined, and will therefore be merged, only after evaluating\n\tthe expression if it is not merged it will be removed again.\n\n    e.g.: merging\n\n    **template.yml**\n    ```yaml\n    alice: 24\n    bob: 25\n    peter: 26\n    ```\n\n    with\n\n    **mapping.yml**\n    ```yaml\n    config:\n      alice: (( ~~ ))\n\t  bob: (( ~~ ))\n\n    alice: (( config.alice || ~~ ))\n    bob: (( config.bob || ~~ ))\n    peter: (( config.peter || ~~ ))\n    ```\n\n    and\n\n    **config.yml**\n    ```yaml\n    config:\n      alice: 4711\n\t  peter: 0815\n    ```\n    yields\n\n    ```yaml\n    alice: 4711  # transferred from config's config value\n    bob: 25      # kept default value, because not set in config.yml\n    peter: 26    # kept, because mapping source not available in mapping.yml\n    ```\n\n  This can be used to add an intermediate stub, that offers a dedicated\n  configuration interface and contains logic to map this interface to a manifest\n  structure already defining default values.\n\n- _Templates versus map literals_\n\n  As described earlier templates can be used inside functions and mappings to\n  easily describe complex data structures based on expressions refering to\n  parameters. Before the introduction of map literals this was the only way\n  to achieve such behaviour. The advantage is the possibility to describe\n  the complex structure as regular part of a yaml document, which allows using\n  the regular yaml formatting  facilitating readability.\n\n  e.g.:\n\n  ```yaml\n  scaling:\n    runner_z1: 10\n    router_z1: 4\n\n    jobs: (( sum[scaling|[]|s,k,v|-\u003es [ *templates.job ] ] ))\n\n  templates:\n    job:\n      \u003c\u003c: (( \u0026template ))\n      name: (( k ))\n      instances: (( v ))\n  ```\n\n  evaluates to\n\n  ```yaml\n  scaling:\n    runner_z1: 10\n    router_z1: 4\n\n  jobs:\n    - instances: 4\n      name: router_z1\n    - instances: 10\n      name: runner_z1\n    ...\n  ```\n\n  With map literals this construct can significantly be simplified\n\n  ```yaml\n  scaling:\n    runner_z1: 10\n    router_z1: 4\n\n  jobs:  (( sum[scaling|[]|s,k,v|-\u003es [ {\"name\"=k, \"value\"=v} ] ] ))\n  ```\n\n  Nevertheless the first, template based version might still be useful, if\n  the data structures are more complex, deeper or with complex value expressions.\n  For such a scenario the description of the data structure as template should be\n  preferred. It provides a much better readability, because every field, list\n  entry and value expression can be put into dedicated lines.\n\n  But there is still a qualitative difference. While map literals are part of a\n  single expression always evaluated as a whole before map fields are available\n  for referencing, templates are evaluated as regular yaml documents that might\n  contain multiple fields with separate expressions referencing each other.\n\n  e.g.:\n\n  ```yaml\n  range: (( (|cidr,first,size|-\u003e(*templates.addr).range)(\"10.0.0.0/16\",10,255) ))\n\n  templates:\n    addr:\n      \u003c\u003c: (( \u0026template ))\n      base: (( min_ip(cidr) ))\n      start: (( base + first ))\n\t  end: (( start + size - 1 ))\n\t  range: (( start \" - \" end ))\n  ```\n\n  evaluates `range` to\n\n  ```yaml\n  range: 10.0.0.10 - 10.0.1.8\n  ...\n  ```\n\n- Defaulting and Requiring Fields\n\n  Traditionally defaulting in _spiff_ is done by a downstream template where the\n  playload data file is used as stub.\n  \n  Fields with simple values can just be specified with their values.\n  They will be overwritten by stubs using the regular _spiff_ document merging\n  mechanisms.\n  \n  It is more difficult for maps or lists. If a map is specified in the\n  template only its fields will be merged (see above), but it is never\n  replaced as a whole by settings in the playload definition files.\n  And Lists are never merged.\n  \n  Therefore maps and lists that should be defaulted as a whole must be specified\n  as initial expressions (referential or inline) in the template file.\n  \n  e.g.: merging of\n  \n  **template.yaml**\n  ```yaml\n  defaults:\n    \u003c\u003c: (( \u0026temporary ))\n    person:\n      name: alice\n      age: bob\n  config:\n    value1: defaultvalue\n    value2: defaultvalue\n    person: (( defaults.person ))\n  ```  \n  \n  and\n  \n  **payload.yaml**\n  ```yaml\n   config:\n     value2: configured\n     othervalue: I want this but don't get it\n  ```  \n    \n  evaluates to \n  \n  ```yaml\n  config:\n    person:\n      age: bob\n      name: alice\n    value1: defaultvalue\n      value2: configured\n   ```\n\n  In such a scenario the structure of the resulting document is defined by the template.\n  All kinds of variable fields or sub-structures must be forseen by the template\n  by using `\u003c\u003c: (( merge ))` expressions in maps.\n  \n  e.g.: changing template to\n  \n  **template.yaml**\n  ```yaml\n  defaults:\n    \u003c\u003c: (( \u0026temporary ))\n    person:\n      name: alice\n      age: bob\n  config:\n    \u003c\u003c: (( merge ))\n    value1: defaultvalue\n    value2: defaultvalue\n    person: (( defaults.person ))\n  ```  \n  \n  Known _optional_ fields can be described using the *undefined* (`~~`) expression:\n\n  **template.yaml**\n  ```yaml\n  config:\n    optional: (( ~~ ))\n  ```    \n  \n  Such fields will only be part of the final document if they are defined in\n  an upstream stub, otherwise they will be completely removed.\n \n  _Required_ fields can be defined with the expression `(( merge ))`. If\n  no stub contains a value for this field, the merge cannot be fullfilled and\n  an error is reported. If a dedicated message should be shown instead, the\n  merge expression can be defaulted with an error function call.\n  \n  e.g.:\n  \n  **template.yaml**\n  ```yaml\n  config:\n    password: (( merge || error(\"the field password is required\") ))\n  ```\n\n   will produce the following error if no stub contains a value:\n   ```\n   error generating manifest: unresolved nodes:\n   \t(( merge || error(\"the field password is required\") ))\tin c.yaml\tconfig.password\t()\t*the field password is required \n   ```\n   \n   This can be simplified by reducing the expression to the sole `error`\n   expression.\n\n   Besides this template based defaulting it is also possible to\n   provide defaults by upstream stubs using the [`\u0026default` marker](#-default-).\n   Here the payload can be a downstream file.\n   \n- _X509_ and providing State\n\n  When generating keys or certificates with the [X509 Functions](#x509-functions)\n  there will be new keys or certificates for every execution of _spiff_. But \n  it is also possible to use _spiff_ to maintain key state. A very simple script\n  could look like this:\n  \n  ```bash\n  #!/bin/bash\n  DIR=\"$(dirname \"$0\")/state\"\n  if [ ! -f \"$DIR/state.yaml\" ]; then\n    echo \"state:\" \u003e \"$DIR/state.yaml\"\n  fi\n  spiff merge \"$DIR/template.yaml\" \"$DIR/state.yaml\" \u003e \"$DIR/.$$\" \u0026\u0026 mv \"$DIR/.$$\" \"$DIR/state.yaml\"\n  ```\n  \n  It uses a template file (containing the rules) and a state file with the\n  actual state as stub. The first time it is executed there is an empty state\n  and the rules are not overridden, therefore the keys and certificates are\n  generated. Later on, only additional new fields are calculated, the state\n  fields already containing values just overrule the _dynaml_ expressions\n  for those fields in the template.\n  \n  If a re-generation is required, the state file can just be deleted.\n  \n  A template may look like this:\n  \n  **state/template.yaml**\n  ```yaml\n  spec:\n    \u003c\u003c: (( \u0026local ))\n    ca:\n      organization: Mandelsoft\n      commonName: rootca\n      privateKey: (( state.cakey ))\n      isCA: true\n      usage:\n        - Signature\n        - KeyEncipherment\n    peer:\n      organization: Mandelsoft\n      commonName: etcd\n      publicKey: (( state.pub ))\n      caCert: (( state.cacert ))\n      caPrivateKey: (( state.cakey ))\n      validity: 100\n      usage:\n        - ServerAuth\n        - ClientAuth\n        - KeyEncipherment\n      hosts:\n        - etcd.mandelsoft.org\n  \n  state:\n    cakey: (( x509genkey(2048) ))\n    capub: (( x509publickey(cakey) ))\n  \n    cacert: (( x509cert(spec.ca) ))\n  \n    key: (( x509genkey(2048) ))\n    pub: (( x509publickey(key) ))\n    peer: (( x509cert(spec.peer) ))\n\n  ```\n  \n  The merge then generates a rootca and some TLS certificate signed with\n  this CA.\n  \n- Generating, Deploying and Accessing Status for Kubernetes Resources\n\n  The [`sync`](#-syncexpr-condition-value-10-) function offers the possibility\n  to synchronize the template processing with external content. This can also\n  be the output of a command execution. Therefore the template processing\n  can not only be used to generate a deployment manifest, but also for\n  applying this to a target system and retrieving deployment status values\n  for the further processing.\n  \n  A typical scenario of this kind could be a kubernetes setup including\n  a service of type _LoadBalancer_. Once deployed it gets assigned\n  status information about the IP address or hostname of the assigned\n  load balancer. This information might be required for some other deployment\n  manifest.\n  \n  A simple template for such a deployment could like this:\n  \n  ```yaml\n  service:\n    apiVersion: v1\n    kind: Service\n    metadata:\n      annotations:\n        dns.mandelsoft.org/dnsnames: echo.test.garden.mandelsoft.org\n        dns.mandelsoft.org/ttl: \"500\"\n      name: test-service\n      namespace: default\n    spec:\n      ports:\n      - name: http\n        port: 80\n        protocol: TCP\n        targetPort: 8080\n      sessionAffinity: None\n      type: LoadBalancer\n  \n  deployment:\n     testservice: (( sync[pipe_uncached(service, \"kubectl\", \"apply\", \"-f\", \"-\", \"-o\", \"yaml\")|value|-\u003edefined(value.status.loadBalancer.ingress)] ))\n  \n  \n  otherconfig:\n     lb: (( deployment.testservice.status.loadBalancer.ingress ))\n  \n  ```\n- Crazy Shit: Graph Analaysis with _spiff_\n\n  It is easy to describe a simple graph with knots and edges (for example \n  for a set of components and their dependencies) just by using a map of lists.\n  \n  \u003cdetails\u003e\u003csummary\u003e\u003cb\u003egraph.yaml\u003c/b\u003e\u003c/summary\u003e\n  \n  ```yaml\n  graph:\n    a:\n    - b\n    - c\n    b: []\n    c:\n    - b\n    - a\n    d:\n    - b\n    e:\n    - d\n    - b\n  ```\n  \u003c/details\u003e\n  \n  Now it would be useful to figure out whether there are dependency cycles or\n  to determine ordered transitive dependencies for a component.\n  \n  Let's say something like this:\n  \n  \u003cdetails\u003e\u003csummary\u003e\u003cb\u003eclosures.yaml\u003c/b\u003e\u003c/summary\u003e\n\n  ```yaml\n  graph:\n  utilities:\n\n  closures: (( utilities.graph.evaluate(graph) ))\n  cycles: (( utilities.graph.cycles(closures) ))\n  ```\n  \u003c/details\u003e\n  \n  Indeed, this can be done with spiff. The only thing required is\n  a _\"small utilities stub\"_.\n  \n  \u003cdetails\u003e\u003csummary\u003e\u003cb\u003eutilities.yaml\u003c/b\u003e\u003c/summary\u003e\n  \n  ```yaml\n  utilities:\n    \u003c\u003c: (( \u0026temporary ))\n    graph:\n      _dep: (( |model,comp,closure|-\u003econtains(closure,comp) ? { $deps=[], $err=closure [comp]} :($deps=_._deps(model,comp,closure [comp]))($err=sum[deps|[]|s,e|-\u003e length(s) \u003e= length(e.err) ? s :e.err]) { $deps=_.join(map[deps|e|-\u003ee.deps]), $err=err} ))\n      _deps: (( |model,comp,closure|-\u003emap[model.[comp]|dep|-\u003e($deps=_._dep(model,dep,closure)) { $deps=[dep] deps.deps, $err=deps.err }] ))\n      join: (( |lists|-\u003esum[lists|[]|s,e|-\u003e s e] ))\n      min: (( |list|-\u003esum[list|~|s,e|-\u003e s ? e \u003c s ? e :s :e] ))\n  \n      normcycle: (( |cycle|-\u003e($min=_.min(cycle)) min ? sum[cycle|cycle|s,e|-\u003es.[0] == min ? s :(s.[1..] [s.[1]])] :cycle  ))\n      cycle: (( |list|-\u003elist ? ($elem=list.[length(list) - 1]) _.normcycle(sum[list|[]|s,e|-\u003es ? s [e] :e == elem ? [e] :s]) :list ))\n      norm: (( |deps|-\u003e{ $deps=_.reverse(uniq(_.reverse(deps.deps))), $err=_.cycle(deps.err) } ))\n      reverse: (( |list|-\u003esum[list|[]|s,e|-\u003e[e] s] ))\n  \n      evaluate: (( |model|-\u003esum[model|{}|s,k,v|-\u003es { k=_.norm(_._dep(model,k,[]))}] ))\n      cycles: (( |result|-\u003euniq(sum[result|[]|s,k,v|-\u003e v.err ? s [v.err] :s]) ))\n  ```\n  \u003c/details\u003e\n  \n  And magically _spiff_ does the work just by calling\n  ```bash\n  spiff merge closure.yaml graph.yaml utilities.yaml\n  ```\n  \n  \u003cdetails\u003e\u003csummary\u003eAnd the result is\u003c/summary\u003e\n  \n  ```yaml\n     closures:\n       a:\n         deps:\n         - c\n         - b\n         - a\n         err:\n         - a\n         - c\n         - a\n       b:\n         deps: []\n         err: []\n       c:\n         deps:\n         - a\n         - b\n         - c\n         err:\n         - a\n         - c\n         - a\n       d:\n         deps:\n         - b\n         err: []\n       e:\n         deps:\n         - d\n         - b\n         err: []\n     cycles:\n     - - a\n       - c\n       - a\n     graph:\n       a:\n       - b\n       - c\n       b: []\n       c:\n       - b\n       - a\n       d:\n       - b\n       e:\n       - d\n       - b\n  ```\n  \u003c/details\u003e\n  \n# Error Reporting\n\nThe evaluation of dynaml expressions may fail because of several reasons:\n- it is not parseable\n- involved references cannot be satisfied\n- arguments to operations are of the wrong type\n- operations fail\n- there are cyclic dependencies among expressions\n\nIf a dynaml expression cannot be resolved to a value, it is reported by the\n`spiff merge` operation using the following layout:\n\n```\n\t(( \u003cfailed expression\u003e ))\tin \u003cfile\u003e\t\u003cpath to node\u003e\t(\u003creferred path\u003e)\t\u003ctag\u003e\u003cissue\u003e\n```\n\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eExample\u003c/b\u003e\u003c/summary\u003e\n\n```\n\t(( min_ip(\"10\") ))\tin source.yml\tnode.a.[0]\t()\t*CIDR argument required\n```\n\u003c/details\u003e\n\nCyclic dependencies are detected by iterative evaluation until the document is unchanged after a step.\nNodes involved in a cycle are therefore typically reported just as unresolved node without a specific issue.\n\nThe order of the reported unresolved nodes depends on a classification of the problem, denoted by a dedicated\ntag. The following tags are used (in reporting order):\n\n| Tag | Meaning |\n| --- | ------- |\n| `*` | error in local dynaml expression |\n| `@` | dependent or involved in cyclic dependencies |\n| `-` | subsequent error because of refering to a yaml node with an error |\n\nProblems occuring during inline template processing are reported as nested problems. The classification is\npropagated to the outer node.\n\nIf a problem occurs in nested lamba calls the call stack together with the lamba function and is \nlocal binding is listed.\n\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eExample\u003c/b\u003e\u003c/summary\u003e\n\n```text\n\t(( 2 + .func(2) ))\tin local/err.yaml\tvalue\t()\t*evaluation of lambda expression failed: lambda|x|-\u003ex \u003e 0 ? _(x - 1) : *(template): {x: 2}\n\t\t... evaluation of lambda expression failed: lambda|x|-\u003ex \u003e 0 ? _(x - 1) : *(template): {x: 1}\n\t\t... evaluation of lambda expression failed: lambda|x|-\u003ex \u003e 0 ? _(x - 1) : *(template): {x: 0}\n\t\t... resolution of template 'template' failed\n\t\t\t(( z ))\tin local/err.yaml\tval\t()*'z' not found \n\n```\n\u003c/details\u003e\n\nIn case of parsing errors in dynaml expressions, the error location is shown now.\nIf it is a multi line expression the line a character/symbol number in that line\nis show, otherwise the line numer is omitted.\n\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eExample\u003c/b\u003e\u003c/summary\u003e\n\n```text\n\t((\n\t  2 ++ .func(2)\n\t))\tin local/err.yaml\tfaulty\t()\t*parse error near line 2 symbol 2 - line 2 symbol 3: \" \" \n\n```\n\u003c/details\u003e\n\n# Using _spiff_ as Go Library\n\n_Spiff_ provides a Go package (`spiffing`) that can be used to include _spiff_ templates in Go programs.\n\nAn example program could look like this:\n\n```go\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"os\"\n\n\t\"github.com/mandelsoft/spiff/dynaml\"\n\t\"github.com/mandelsoft/spiff/spiffing\"\n)\n\nfunc func_pow(arguments []interface{}, binding dynaml.Binding) (interface{}, dynaml.EvaluationInfo, bool) {\n\tinfo := dynaml.DefaultInfo()\n\n\tif len(arguments) != 2 {\n\t\treturn info.Error(\"pow takes 2 arguments\")\n\t}\n\n\ta, b, err := dynaml.NumberOperands(arguments[0], arguments[1])\n\n\tif err != nil {\n\t\treturn info.Error(\"%s\", err)\n\t}\n\t_, i := a.(int64)\n\tif i {\n\t\tr := math.Pow(float64(a.(int64)), float64(b.(int64)))\n\t\tif float64(int64(r)) == r {\n\t\t\treturn int64(r), info, true\n\t\t}\n\t\treturn r, info, true\n\t} else {\n\t\treturn math.Pow(a.(float64), b.(float64)), info, true\n\t}\n}\n\nvar state = `\nstate: {}\n`\nvar stub = `\nunused: (( input ))\nages:\n  alice: (( pow(2,5) ))\n  bob: (( alice + 1 ))\n`\n\nvar template = `\nstate:\n  \u003c\u003c\u003c: (( \u0026state ))\n  random: (( rand(\"[:alnum:]\", 10) )) \nages: (( \u0026temporary ))\n\nexample:\n  name: (( input ))  # direct reference to additional values \n  sum: (( sum[ages|0|s,k,v|-\u003es + v] ))\n  int: (( pow(2,4) ))\n  float: 2.1\n  pow: (( pow(1.1e1,2.1) ))\n`\n\nfunc Error(err error) {\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"Error: %s\\n\", err)\n\t\tos.Exit(1)\n\t}\n}\n\nfunc main() {\n\tvalues := map[string]interface{}{}\n\tvalues[\"input\"] = \"this is an input\"\n\n\tfunctions := spiffing.NewFunctions()\n\tfunctions.RegisterFunction(\"pow\", func_pow)\n\n\tspiff, err := spiffing.New().WithFunctions(functions).WithValues(values)\n\tError(err)\n\tpstate, err := spiff.Unmarshal(\"state\", []byte(state))\n\tError(err)\n\tpstub, err := spiff.Unmarshal(\"stub\", []byte(stub))\n\tError(err)\n\tptempl, err := spiff.Unmarshal(\"template\", []byte(template))\n\tError(err)\n\tresult, err := spiff.Cascade(ptempl, []spiffing.Node{pstub}, pstate)\n\tError(err)\n\tb, err := spiff.Marshal(result)\n\tError(err)\n\tnewstate, err := spiff.Marshal(spiff.DetermineState(result))\n\tError(err)\n\tfmt.Printf(\"==== new state ===\\n\")\n\tfmt.Printf(\"%s\\n\", string(newstate))\n\tfmt.Printf(\"==== result ===\\n\")\n\tfmt.Printf(\"%s\\n\", string(b))\n}\n```\n\nIt supports\n - transforming file data to and from spiffs internal node representation\n - the processing of stubs and templates with or without state handling\n - defining an outer binding for injected path names\n - defining additional spiff functions\n - enabling/disabling command execution and/or filesystem operations\n - using a [virtual filesystem](http://github.com/mandelsoft/vfs) for\n   file system operations\n","funding_links":[],"categories":["Projects"],"sub_categories":["YAML is a **supserset of JSON**"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmandelsoft%2Fspiff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmandelsoft%2Fspiff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmandelsoft%2Fspiff/lists"}