Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jtyr/ansible-config_encoder_filters

Ansible role used to deliver the Config Encoder Filters.
https://github.com/jtyr/ansible-config_encoder_filters

ansible ansible-filters apache erlang haproxy ini json logstash nginx pam toml xml yaml

Last synced: 3 months ago
JSON representation

Ansible role used to deliver the Config Encoder Filters.

Host: GitHub
URL: https://github.com/jtyr/ansible-config_encoder_filters
Owner: jtyr
License: gpl-3.0
Created: 2016-05-24T11:16:34.000Z (over 8 years ago)
Default Branch: master
Last Pushed: 2023-08-25T13:56:03.000Z (over 1 year ago)
Last Synced: 2024-10-10T12:05:52.763Z (4 months ago)
Topics: ansible, ansible-filters, apache, erlang, haproxy, ini, json, logstash, nginx, pam, toml, xml, yaml
Language: Python
Homepage:
Size: 110 KB
Stars: 61
Watchers: 7
Forks: 16
Open Issues: 1
Metadata Files:
- Readme: README.rst
- License: LICENSE

Awesome Lists containing this project

README

Config Encoder Filters
======================

This is an Ansible role used to deliver the Config Encoder Filters as
a dependency of another Ansible role.

.. image:: https://travis-ci.org/jtyr/ansible-config_encoder_filters.svg?branch=master
:target: https://travis-ci.org/jtyr/ansible-config_encoder_filters

Table of Contents
-----------------

- Motivation_
- Example_
- Usage_
- Installation_
- `Supported encoders`_
- encode_apache_
- encode_erlang_
- encode_haproxy_
- encode_ini_
- encode_json_
- encode_logstash_
- encode_lua_
- encode_nginx_
- encode_pam_
- encode_toml_
- encode_xml_
- encode_yaml_
- Utilities_
- template_replace_
- Testing_
- License_
- Author_

----

.. _Motivation:

Motivation
----------

Ansible Galaxy contains a lot of useful roles. Some of them exist in
many variations which differ only by their parameterization. The
parameterization is often used mainly in templates which generate the
configuration file. A good example such issues are roles for Nginx of
which you can find almost 200 in the Ansible Galaxy.

Nginx is possible to configure in infinite number of ways and therefore
is almost impossible to create an Ansible template file which would
capture all possible variations of the configuration. Even if a suitable
roles is found, users often want to customize even more. This is where
people normally clone the role and add parameters they are missing. Some
people try to get the change back into the original role by creating a
pull request (PR) but sometimes such change is not accepted by the
maintainer of the original role and the user ends up maintaining his/her
own clone forever.

This is why the Config Encoder filters were developed to facilitate the
creation of Ansible roles with universal configuration. The structure of
the configuration file is described as a YAML data structure stored in a
variable. The variable together with he Config Encoder filter is then
used in the template file which is used to generate the final
configuration file. This approach allows to shift the paradigm of
thinking about configuration files as templates to thinking about them as
data structures. The data structure can be dynamically generated which
allows to create truly universal configuration.

.. _Example:

Example
-------

Imagine the following INI file:

.. code:: ini

[section1]
option11=value11
option12=value12

Such configuration file can be described as a YAML data structure:

.. code:: yaml

myapp_config:
section1:
option11: value11
option12: value12

The variable is then used together with the ``encode_ini`` Config Encoder
filter in the template file ``myapp.cfg.j2`` like this:

.. code:: jinja2

And finally, the template file is used in a task like this:

.. code:: yaml

- name: Create config file
template:
src: myapp.cfg.j2
dest: /etc/myapp/myapp.cfg

When the task is executed, it creates exactly the same file as the
original INI file.

So we can describe the configuration as a data structure which is then
converted into the final configuration file format with the Config
Encoder filter.

In order to change the above configuration, we would have to overwrite
the ``myapp_config`` which is not very practical. Therefore we break the
monolithic variable into a set of variables which will allow us to change
any part of the configuration without the need to overwrite the whole
data structure:

.. code:: yaml

myapp_config_section1_option11: value11
myapp_config_section1_option12: value12

myapp_config_section1__default:
option11: "{{ myapp_config_section1_option11 }}"
option12: "{{ myapp_config_section1_option12 }}"

myapp_config_section1__custom: {}

myapp_config_default:
section1: "{{
myapp_config_section1__default | combine(
myapp_config_section1__custom) }}"

myapp_config__custom: {}

myapp_config: "{{
myapp_config__default | combine(
myapp_config__custom) }}"

Like this, if we want to change the value of the ``option11``, we only
override the variable ``myapp_config_section1_option11``:

.. code:: yaml

myapp_config_section1_option11: My new value

If we want to add a new option into the ``section1``, we add it into the
variable ``myapp_config_section1__custom`` which is then merged with the
default list of options:

.. code:: yaml

myapp_config_section1__custom:
section13: value13

And if we want to add a new section, we add it into the variable
``myapp_config__custom`` which is then merged with the default list of
sections:

.. code:: yaml

myapp_config__custom:
section2:
option21: value21

The above is showing an example for INI configuration files only but the
same principle is possible to use for all the supported Config Encoders
listed bellow.

.. _Usage:

Usage
-----

Config Encoder filters can be used in any Ansible role by adding the
``config_encoder_filters`` role into the list of dependencies in the
``meta/main.yml`` file:

.. code:: yaml

dependencies:
- config_encoder_filters

The usage directy from a Playbook requires to add the
``config_encoder_filters`` into the list of roles:

.. code:: yaml

- name: My test Play
hosts: all
roles:
- config_encoder_filters
tasks:
- name: Create config file
template:
src: my.conf.j2
dest: /tmp/my.conf

.. _Installation:

Installation
------------

The role can be downloaded either via Ansible Galaxy command:

.. code:: shell

$ ansible-galaxy install jtyr.config_encoder_filters,master,config_encoder_filters

or via Ansible Gallaxy requirements file:

.. code:: shell

$ cat ./requirements.yaml
---

- src: https://github.com/jtyr/ansible-config_encoder_filters.git
name: config_encoder_filters
$ ansible-galaxy -r ./requirements.yaml

or via Git:

.. code:: shell

$ git clone https://github.com/jtyr/ansible-config_encoder_filters.git config_encoder_filters

.. _`Supported encoders`:

Supported encoders
------------------

The following is the list of supported Config Encoder filters. Each
filter requires special data structure as its input. Each filter also has
a set of parameters which can modify the behaviour of the filter.

.. _encode-apache:

encode_apache
^^^^^^^^^^^^^

This filter helps to create configuration in the format used by Apache
web server. The expected data structure is the following:

.. code:: yaml

my_apache_vhost:
content:
- sections:
- name: VirtualHost
param: "*:80"
content:
- options:
- DocumentRoot: /www/example1
- ServerName: www.example.com
- ErrorLog: /var/log/httpd/www.example.com-error_log
- CustomLog:
- /var/log/httpd/www.example.com-access_log
- common
- "#": Other directives here ...

The variable starts with ``content`` which can contain list of
``sections`` or ``options``. ``sections`` then contain list of individual
sections which has the ``name``, ``param`` and ``content`` parameter. The
``content`` can again contain a list of `sections`` or ``options``.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: apache

DocumentRoot /www/example1
ServerName www.example.com
ErrorLog /var/log/httpd/www.example.com-error_log
CustomLog /var/log/httpd/www.example.com-access_log common
# "Other directives here ..."

The filter can have the following parameters:

- ``convert_bools=false``

Indicates whether Boolean values presented as a string should be
converted to a real Boolean value. For example ``var1: 'True'`` would
be represented as a string but by using the ``convert_bools=true`` it
will be converted into Boolean like it would be defined like ``var1:
true``.

- ``convert_nums=false``

Indicates whether number presented as a string should be converted to
number. For example ``var1: '123'`` would be represented as a string
but by using the ``convert_nums=true`` it will be converted it to a
number like it would be defined like ``var1: 123``. It's also possible
to use the YAML type casting to convert string to number (e.g. ``!!int
"1234"``, ``!!float "3.14"``).

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

- ``quote_all_nums=false``

Number values are not quoted by default. This parameter will force to
quote all numbers.

- ``quote_all_strings=false``

String values are quoted only if they contain a space. This parameter
will force to quote all strings regardless if the they contain the
space or not.

.. _encode-erlang:

encode_erlang
^^^^^^^^^^^^^

This filter helps to create configuration in the Erlang format. The
expected data structure is the following:

.. code:: yaml

my_rabbitmq_config:
- rabbit:
- tcp_listeners:
- '"127.0.0.1"': 5672
- ssl_listeners:
- 5671
- ssl_options:
- cacertfile: /path/to/testca/cacert.pem
- certfile: /path/to/server/cert.pem
- keyfile: /path/to/server/key.pem
- verify: :verify_peer
- fail_if_no_peer_cert: true
- cluster_nodes:
::
-
- :"'rabbit@node1'"
- :"'rabbit@node2'"
- :disc

The variable consists of a lists of dictionaries. The value of the key-value
pair can be another list or simple value like a string or a number. Erlang
tuples can be enforced by prepending the value with the special character
specified in the ``atom_value_indicator``. Order in tuple can be achieved by
using special construction as shown for the ``cluste_nodes`` tuple from the
above example. The indicator starting this special construction can be set with
the parameter ``ordered_tuple_indicator``.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: erlang

[
{rabbit, [
{tcp_listeners, [
{"127.0.0.1", 5672}
]},
{ssl_listeners, [
5671
]},
{ssl_options, [
{cacertfile, "/path/to/testca/cacert.pem"},
{certfile, "/path/to/server/cert.pem"},
{keyfile, "/path/to/server/key.pem"},
{verify, verify_peer},
{fail_if_no_peer_cert, true}
]},
{cluster_nodes,
{[
'rabbit@node1',
'rabbit@node2'
], disc}}
]}
].

The filter can have the following parameters:

- ``atom_value_indicator=":"``

The value of this parameter indicates the string which must be
prepended to a string value to treat it as an atom value.

- ``convert_bools=false``

- ``convert_nums=false``

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

- ``ordered_tuple_indicator=":"``

Indicator used to start the special construction with ordered tuple.

.. _encode-haproxy:

encode_haproxy
^^^^^^^^^^^^^^

This filter helps to create configuration in the format used in Haproxy.
The expected data structure is the following:

.. code:: yaml

my_haproxy_config:
- global:
- daemon
- maxconn 256
- "# This is the default section"
- defaults:
- mode http
- timeout:
- connect 5000ms
- client 50000ms
- server 50000ms
- frontend http-in:
- "# This is the bind address/port"
- bind *:80
- default_backend servers
- backend servers
- server server1 127.0.0.1:8000 maxconn 32

The variable is a list which can contain a simple string value or a dictionary
which indicates a section.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: haproxy

global
daemon
maxconn 256

# This is the default section
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms

frontend http-in
# This is the bind address/port
bind *:80
default_backend servers
backend servers
server server1 127.0.0.1:8000 maxconn 32

The filter can have the following parameters:

- ``indent=" "``

Defines the indentation unit.

.. _encode-ini:

encode_ini
^^^^^^^^^^

This filter helps to create configuration in the INI format. The expected
data structure is the following:

.. code:: yaml

my_rsyncd_config:
uid: nobody
gid: nobody
use chroot: no
max connections: 4
syslog facility: local5
pid file: /run/rsyncd.pid
ftp:
path: /srv/ftp
comment: ftp area

The variable consist of dictionaries which can be nested. If the value of the
key-value pair on the first level is of a simple type (string, number, boolean),
such pair is considered to be global and gets processed first. If the value of
the key-value pair on the first level is another dictionary, the key is
considered to be the name of the section and the inner dictionary as properties
of the section.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: ini

gid=nobody
max connections=4
pid file=/run/rsyncd.pid
syslog facility=local5
uid=nobody
use chroot=False

[ftp]
comment=ftp area
path=/srv/ftp

The special value :code:`!!!null` can be used, to create a key without any value.
This must be wrapped in quotes, to avoid being evaluated as a YAML tag.

Take the following data structure:

.. code:: yaml

config:
myconf:
key: value
keyWithoutValue: '!!!null'

This would become:

.. code:: ini

[myconf]
key=value
keyWithoutValue

The filter can have the following parameters:

- ``comment="#"``

Sign used to comment out lines when `section_is_comment=true`.

- ``delimiter="="``

Sign separating the *property* and the *value*. By default it's set to
``'='`` but it can also be set for example to ``' = '``.

- ``indent=""``

Indent the keys with the specified string. E.g. ``indent="\t"``.

- ``quote=""``

Sets the quoting of the value. Use ``quote="'"`` or ``quote='"'``.

- ``section_is_comment=false``

If this parameter is set to ``true``, the section value will be used as
a comment for the following properties of the section.

- ``ucase_prop=false``

Indicates whether the *property* should be made upper case.

.. _encode-json:

encode_json
^^^^^^^^^^^

This filter helps to create configuration in the JSON format. The
expected data structure is the following:

.. code:: yaml

my_sensu_client_config:
client:
name: localhost
address: 127.0.0.1
subscriptions:
- test

Because JSON is very similar to YAML, the variable consists of
dictionaries of which value can be either an simple type (number, string,
boolean), list or another dictionary. All can be nested in any number of
levels.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: json

{
"client": {
"address": "127.0.0.1",
"name": "localhost",
"subscriptions": [
"test"
]
}
}

The filter can have the following parameters:

- ``convert_bools=false``

- ``convert_nums=false``

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

.. _encode-logstash:

encode_logstash
^^^^^^^^^^^^^^^

This filter helps to create configuration in the format used by Logstash.
The expected data structure is the following:

.. code:: yaml

my_logstash_config:
- :input:
- :file:
path: /var/log/httpd/access_log
start_position: beginning
- :filter:
- ':if [path] =~ "access"':
- :mutate:
replace:
type: apache_access
- :grok:
match:
message: "%{COMBINEDAPACHELOG}"
- :date:
match:
- timestamp
- dd/MMM/yyyy:HH:mm:ss Z
- ':else if [path] =~ "error"':
- :mutate:
replace:
type: "apache_error"
- :else:
- :mutate:
replace:
type: "random_logs"
- :output:
- :elasticsearch:
hosts:
- localhost:9200
- :stdout:
codec: rubydebug

The variable consists of a list of sections where each section is
prefixed by a special character specified by the ``section_prefix``
(``:`` by default). The value of the top level sections can be either
another section or a dictionary. The value of the dictionary can be a
simple value, list or another dictionary.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: logstash

input {
file {
path => "/var/log/httpd/access_log"
start_position => "beginning"
}
}
filter {
if [path] =~ "access" {
mutate {
replace => {
"type" => "apache_access"
}
}
grok {
match => {
"message" => "%{COMBINEDAPACHELOG}"
}
}
date {
match => [
"timestamp",
"dd/MMM/yyyy:HH:mm:ss Z"
]
}
}
else if [path] =~ "error" {
mutate {
replace => {
"type" => "apache_error"
}
}
}
else {
mutate {
replace => {
"type" => "random_logs"
}
}
}
}
output {
elasticsearch {
hosts => [
"localhost:9200"
]
}
stdout {
codec => "rubydebug"
}
}

The filter can have the following parameters:

- ``backslash_ignore_prefix='@@@'``

This parameter defines a sets of characters than can be prepended to a string
to prevent backslahes from being escaped in the resulting configuration (e.g.
``"@@@sshd(?:\[%{POSINT:[system][auth][pid]}\])?:"`` will turn to
``"sshd(?:\[%{POSINT:[system][auth][pid]}\])?:"`` instead of to
``"sshd(?:\\[%{POSINT:[system][auth][pid]}\\])?:"``).

- ``convert_bools=false``

- ``convert_nums=false``

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

- ``section_prefix=":"``

This parameter specifies which character will be used to identify the
Logstash section.

.. _encode-lua:

encode_lua
^^^^^^^^^^

This filter helps to create configuration in a Lua friendly format.
The expected data structure is the following:

.. code:: yaml

my_lua_config:
fork: false
external_addresses:
- 1.2.3.4
- 5.6.7.8
admins:
- [email protected]
contact_info:
abuse: [email protected]
admin: [email protected]

Lua is a small scripting language, often embedded into C/C++ applications.
This encoder does a best effort to match configuration files seen in the wild,
while allowing the user to further customize how the final output is rendered.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such a template would be:

.. code:: lua

fork = false;
external_addresses = {
"1.2.3.4";
"5.6.7.8";
}
admins = {
"[email protected]";
}
contact_info = {
abuse = "[email protected]";
admin = "[email protected]";
}

The filter can have the following parameters:

- ``convert_bools=false``

- ``convert_nums=false``

- ``sort_keys=false``

Indicates whether the keys should be sorted when the output is rendered,
or left to python's implicit handling of dict ordering.

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

.. _encode-nginx:

encode_nginx
^^^^^^^^^^^^

This filter helps to create configuration in the format used by Nginx
web server. The expected data structure is the following:

.. code:: yaml

my_nginx_vhost_config:
- server:
- listen 80
- server_name $hostname
- "location /":
- root /srv/www/myapp
- index index.html

As Nginx configuration is order sensitive, the all configuration is
defined as a nested list. As it would be difficult to recognize how many
elements each configuration definition has, the list item value is no
further separated into key/value dictionary. Every line of the
configuration is treated either as a key indicating another nested list
or simply as a string.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: nginx

server {
listen 80;
server_name $hostname;

location / {
root /srv/www/myapp;
index index.html;
}
}

The filter can have the following parameters:

- ``block_semicolon=false``

Allows to add a semicolon to the end of each block.

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

- ``semicolon=';'``

Semicolon character. Set this to empty string to ignore all semicolons.

- ``semicolon_ignore_postfix='!;'``

If the line ends with ``!;`` then don't add the final semicolon.

.. _encode-pam:

encode_pam
^^^^^^^^^^

This filter helps to create configuration in the format user by Linux
Pluggable Authentication Modules (PAM). The expected data structure is
the following:

.. code:: yaml

my_system_auth_config:
aa:
type: auth
control: required
path: pam_unix.so
args:
- try_first_pass
- nullok
bb:
type: auth
control: optional
path: pam_permit.so
cc:
type: auth
control: required
path: pam_env.so
dd:
type: account
control: required
path: pam_unix.so
ee:
type: account
control: optional
path: pam_permit.so
ff:
type: account
control: required
path: pam_time.so
gg:
type: password
control: required
path: pam_unix.so
args:
- try_first_pass
- nullok
- sha512
- shadow
hh:
type: password
control: optional
path: pam_permit.so
args:
ii:
type: session
control: required
path: pam_limits.so
jj:
type: session
control: required
path: pam_unix.so
kk:
type: session
control: optional
path: pam_permit.so

The variable is a dictionary of which the key is a labels and the value
is the PAM rule. The label is used to order the PAM rules. Using labels
with even number of characters allows to insert another rule in between
of any two rules.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: pam

auth required pam_unix.so try_first_pass nullok
auth optional pam_permit.so
auth required pam_env.so

account required pam_unix.so
account optional pam_permit.so
account required pam_time.so

password required pam_unix.so try_first_pass nullok sha512 shadow
password optional pam_permit.so

session required pam_limits.so
session required pam_unix.so
session optional pam_permit.so

The filter can have the following parameters:

- ``print_label=false``

Print labels as a comment in the output.

- ``separate_types=true``

Add a newline between the groups of types.

- ``separator=" "``

Separator between the collection of tokens.

.. _encode-toml:

encode_toml
^^^^^^^^^^^

This filter helps to create configuration in the TOML format. The
expected data structure is the following:

.. code:: yaml

my_grafana_ldap_config:
verbose_logging: false
servers:
- host: 127.0.0.1
port: 389
use_ssl: no
start_tls: no
ssl_skip_verify: no
bind_dn: cn=admin,dc=grafana,dc=org
bind_password: grafana
search_filter: (cn=%s)
search_base_dns:
- dc=grafana,dc=org
- attributes:
name: givenName
surname: sn
username: cn
member_of: memberOf
email: email
- group_mappings:
- group_dn: cn=admins,dc=grafana,dc=org
org_role: Admin
- group_dn: cn=users,dc=grafana,dc=org
org_role: Editor
- group_dn: "*"
org_role: Viewer

The variable is a dictionary of which value can be either a simple type
(number, string, boolean), list or another dictionary. The dictionaries
and lists can be nested.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: toml

verbose_logging = false

[[servers]]
host = "127.0.0.1"
port = 389
use_ssl = false
start_tls = false
ssl_skip_verify = false
bind_dn = "cn=admin,dc=grafana,dc=org"
bind_password = 'grafana'
search_filter = "(cn=%s)"
search_base_dns = ["dc=grafana,dc=org"]

[servers.attributes]
name = "givenName"
surname = "sn"
username = "cn"
member_of = "memberOf"
email = "email"

[[servers.group_mappings]]
group_dn = "cn=admins,dc=grafana,dc=org"
org_role = "Admin"

[[servers.group_mappings]]
group_dn = "cn=users,dc=grafana,dc=org"
org_role = "Editor"

[[servers.group_mappings]]
group_dn = "*"
org_role = "Viewer"

The filter can have the following parameters:

- ``convert_bools=false``

- ``convert_nums=false``

- ``quote='"'``

Sets the quoting of the value. Use ``quote="'"`` or ``quote='"'``.

.. _encode-xml:

encode_xml
^^^^^^^^^^

This filter helps to create configuration in the XML format. The expected
data structure is the following:

.. code:: yaml

my_oddjob_config:
- oddjobconfig:
- service:
- ^name: com.redhat.oddjob
- object:
- ^name: /com/redhat/oddjob
- interface:
- ^name: com.redhat.oddjob
- method:
- ^name: listall
- allow:
- ^min_uid: 0
- ^max_uid: 0
- method:
- ^name: list
- allow: ''
- method:
- ^name: quit
- allow:
- ^user: root
- method:
- ^name: reload
- allow:
- ^user: root
- include:
- ^ignore_missing: "yes"
- /etc/oddjobd.conf.d/*.conf
- include:
- ^ignore_missing: "yes"
- /etc/oddjobd-local.conf

The variable can be a list of dictionaries, lists or strings. This config
encoder does not handle mixed content very well so the safest way how to
include mixed content is to define it as a string and use the parameter
``escape_xml=false``. This config encoder also produces no XML declaration.
Any XML declaration or DOCTYPE must be a part of the template file.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: xml

/etc/oddjobd.conf.d/*.conf
/etc/oddjobd-local.conf

The filter can have the following parameters:

- ``attribute_sign="^"``

XML attribute indicator.

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

.. _encode-yaml:

encode_yaml
^^^^^^^^^^^

This filter helps to create configuration in the YAML format. The
expected data structure is the following:

.. code:: yaml

my_mongodb_config:
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod.pid
net:
port: 27017
bindIp: 127.0.0.1

The variable is ordinary YAML. The only purpose of this encoder filter is
to be able to convert YAML data structure into the string in a template
file in unified way compatible with the other config encoders.

The above variable can be used in the template file like this:

.. code:: jinja2

The output of such template would be:

.. code:: yaml

net:
bindIp: "127.0.0.1"
port: 27017
processManagement:
fork: true
pidFilePath: "/var/run/mongodb/mongod.pid"
storage:
dbPath: "/var/lib/mongo"
journal:
enabled: true
systemLog:
destination: "file"
logAppend: true
path: "/var/log/mongodb/mongod.log"

The filter can have the following parameters:

- ``block_prefix=';;;'``

Allows to maintain block signs in the output.

.. code:: yaml

aaa: |-
;;;|-
bbb = ccc
ddd = eee

- ``convert_bools=false``

- ``convert_nums=false``

- ``indent=" "``

Defines the indentation unit.

- ``level=0``

Indicates the initial level of the indentation. Value ``0`` starts
indenting from the beginning of the line. Setting the value to higher
than ``0`` indents the content by ``indent * level``.

- ``quote='"'``

Sets the quoting of the value. Use ``quote="'"`` or ``quote='"'``.

.. _Utilities:

Utilities
---------

The followng is a list of utilities that can be used in conjunction with the
Config Encoder filters.

.. _template-replace:

template_replace
^^^^^^^^^^^^^^^^

This filter allows to use extra templating layer which gets processed during
the template file processing. That can be useful if it's necessary to create
repetitive but slightly different definitions inside the template file.

The extra templating layer is represented by a templating variable which
contains specially decorated variables which get replaced by its real value at
the time of template file processing. The template variable can be composed
dynamically which provides extra flexibility that would otherwise have to be
hardcoded in the template file.

The filter expects the template variable containing the specially decorated
variables as its input. The filter has one parameter which is used to replaced
the specially decorated variables in the template variable.

Let's have a look at an example of such usage:

.. code:: yaml

# The variable used as the replacement in the template variable
my_clients:
- host: myclient01
jobdefs: Default
password: Passw0rd1
file_retention: 30 days
- host: myclient02
jobdefs: HomeOnly
password: Passw0rd2
file_retention: 90 days

# The actual template variable used in the template file
bacula_director_config_job_client:
# First template variable containing the specially decorated variables
- template:
- Job:
- Name = Job-{[{ item['jobdefs'] }]}-{[{ item['host'] }]}
- Client = {[{ item['host'] }]}-fd
- JobDefs = {[{ item['jobdefs'] }]}
# Variable used to replace the specially decorated variables
items: "{{ my_clients }}"
# Second template and its items
- template:
- Client:
- Name = {[{ item['host'] }]}-fd
- Address = {[{ item['host'] }]}
- FD Port = 9102
- Catalog = Default
- Password = {[{ item['password'] }]}
- File Retention = {[{ item['file_retention'] }]}
- Job Retention = 3 months
- AutoPrune = yes
items: "{{ my_clients }}"

The above variable can be used together with the `template_replace` filter in
the template file (``bacula-dir.conf.j2``) like this:

.. code:: jinja2

{% for record in bacula_director_config_job_client %}
{%- for item in record['items'] -%}
{{ record['template'] | template_replace(item) | encode_nginx }}{{ "\n" }}
{%- endfor -%}
{% endfor %}

The template file can be called from the playbook/role like this:

.. code:: yaml

- name: Configure Bacula Director
template:
src: bacula-dir.conf.j2
dest: /etc/bacula/bacula-dir.conf

And the result of such usage is the following:

.. code:: nginx

Job {
Name = Job-Default-myclient01;
Client = myclient01-fd;
JobDefs = Default;
}

Job {
Name = Job-HomeOnly-myclient02;
Client = myclient02-fd;
JobDefs = HomeOnly;
}

Client {
Name = myclient01-fd;
Address = myclient01;
FD Port = 9102;
Catalog = Default;
Password = Passw0rd1;
File Retention = 30 days;
Job Retention = 3 months;
AutoPrune = yes;
}

Client {
Name = myclient02-fd;
Address = myclient02;
FD Port = 9102;
Catalog = Default;
Password = Passw0rd2;
File Retention = 90 days;
Job Retention = 3 months;
AutoPrune = yes;
}

.. _Testing:

Testing
-------

All encoders have a set of unit tests automated through
`tox `_. Full test can be executed like this:

.. code:: shell

$ tox

Individual encoder can be tested like this:

.. code:: shell

$ tox -- python -m unittest tests.test_config_encoders.TestYaml

Individual tests can be executed like this:

.. code:: shell

$ tox -- python -m unittest tests.test_config_encoders.TestYaml.test_string

Tests are great source of advanced examples of how to use each of the encoders.
Explore them in the directory ``tests/files``. The content of the ``.in`` files
must be assigned to a variable when using in Ansible. The output in the
``.out`` files might depend on additional parameters used in the
``tests/test_config_encoders.py`` file. Testing via Ansible can be executed
like this:

.. code:: shell

$ ansible-playbook -i localhost, tests/test_config_encoders.yaml

.. _License:

License
-------

GPLv3

.. _Author:

Author
------

Jiri Tyr