https://github.com/moremoban/moban
General purpose static text generator using Jinja2 and other python template engines
https://github.com/moremoban/moban
cli jinja2
Last synced: 8 months ago
JSON representation
General purpose static text generator using Jinja2 and other python template engines
- Host: GitHub
- URL: https://github.com/moremoban/moban
- Owner: moremoban
- License: mit
- Created: 2016-01-26T23:36:07.000Z (almost 10 years ago)
- Default Branch: dev
- Last Pushed: 2025-04-09T07:41:18.000Z (9 months ago)
- Last Synced: 2025-04-09T08:36:10.239Z (9 months ago)
- Topics: cli, jinja2
- Language: Python
- Homepage: http://moban.readthedocs.io
- Size: 2.34 MB
- Stars: 35
- Watchers: 3
- Forks: 13
- Open Issues: 27
-
Metadata Files:
- Readme: README.rst
- Changelog: CHANGELOG.rst
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- awesome-tools - moban - General purpose static text generator using Jinja2 and other python template engines. `Jinja2` (General Purpose Preprocessor / Snippets Manager)
README
================================================================================
mó bǎn - 模板 General purpose static text generator
================================================================================
.. image:: https://raw.githubusercontent.com/pyexcel/pyexcel.github.io/master/images/patreon.png
:target: https://www.patreon.com/chfw
.. image:: https://api.travis-ci.org/moremoban/moban.svg?branch=master
:target: http://travis-ci.org/moremoban/moban
.. image:: https://dev.azure.com/moremoban/moban/_apis/build/status/moremoban.moban
:target: https://dev.azure.com/moremoban/moban/_build?definitionId=1&_a=summary
.. image:: https://codecov.io/gh/moremoban/moban/branch/master/graph/badge.svg
:target: https://codecov.io/gh/moremoban/moban
.. image:: https://badge.fury.io/py/moban.svg
:target: https://pypi.org/project/moban
.. image:: https://pepy.tech/badge/moban
:target: https://pepy.tech/project/moban
.. image:: https://readthedocs.org/projects/moban/badge/?version=latest
:target: http://moban.readthedocs.org/en/latest/
.. image:: https://img.shields.io/gitter/room/gitterHQ/gitter.svg
:target: https://gitter.im/chfw_moban/Lobby
:Author: C.W. and its contributors (See contributors.rst)
:Issues: http://github.com/moremoban/moban/issues
:License: MIT
Announcement
================================================================================
In version 0.8.0, `moban.plugins.jinja2.tests.files` is moved to moban-ansible
package. `moban.plugins.jinja2.filters.github` is moved to moban-jinja2-github
package Please install them for backward compatibility.
Quick start
================================================================================
.. code-block:: bash
$ export HELLO="world"
$ moban "{{HELLO}}"
world
Or
.. code-block:: bash
$ export HELLO="world"
$ echo "{{HELLO}}" | moban
Or simply
.. code-block:: bash
$ HELLO="world" moban "{{HELLO}}"
A bit formal example:
.. code-block:: bash
$ moban -c data.yml -t my.template
world
Given data.yml as:
.. code-block:: bash
hello: world
and my.template as:
.. code-block:: bash
{{hello}}
Please note that data.yml will take precedence over environment variables.
Template inheritance and custom template directories
-------------------------------------------------------
Suppose there exists `shared/base.jj2`, and two templates `child1.jj2` and
`child2.jj2` derives from it. You can do:
.. code-block:: bash
$ moban -t child1.jj2 -td shared -o child1
$ moban -t child2.jj2 -td shared -o child2
Data overload and custom data directories
---------------------------------------------
Effectively each data file you give to moban, it overrides environment variables.
Still you can have different layers of data. For example, you can have
`shared/company_info.yml`, use `project1.yml` for project 1 and
`project2.yml` for project 2. In each of the derived data file, simply mention:
.. code-block:: bash
overrides: company_info.yml
...
Here is the command line to use your data:
.. code-block:: bash
$ moban -cd shared -c project1.yaml -t README.jj2
Custom jinja2 extension
---------------------------
moban allows the injection of user preferred jinja2 extensions:
.. code-block:: bash
$ moban -e jj2=jinja2_time.TimeExtension ...
Well, can I nick some existing functions as filters, tests? Or create a global from another library?
-----------------------------------------------------------------------------------------------------
Sure, you can use the same '-e' syntax:
.. code-block:: bash
$ moban -e jinja2=filter:module.path.filter_function \
jinja2=test:module.path.test_function \
jinja2=global:identifier=module.path.variable
In this case, you would have to include the external library in your own requirements.txt
Here is an example:
.. code-block:: bash
$ moban -e jinja2=filter:moban.externals.file_system.url_join \
jinja2=test:moban.externals.file_system.exists \
jinja2=global:description=moban.constants.PROGRAM_DESCRIPTION \
-t "{{ 'a'|url_join('b')}} {{'b' is exists}}"
Can I write my own jinja2 test, filter and/or globals?
-----------------------------------------------------------
moban allows the freedom of craftsmanship. Please refer to the docs for more
details. Here is an example:
.. code-block:: python
import sys
import base64
from moban.plugins.jinja2.extensions import JinjaFilter
@JinjaFilter()
def base64encode(string):
if sys.version_info[0] > 2:
content = base64.b64encode(string.encode("utf-8"))
content = content.decode("utf-8")
else:
content = base64.b64encode(string)
return content
And you can use it within your jinja2 template, `mytest.jj2`:
.. code-block:: python
{{ 'abc' | base64encode }}
Assume that the custom example was saved in `custom-jj2-plugin`
.. code-block:: bash
$ moban -pd custom-jj2-plugin -t mytest.jj2 ...
Moban will then load your custom jinja2 functions
Slim template syntax for jinja2
---------------------------------
with `moban-slim `_ installed,
Given a data.json file with the following content
.. code-block::
{
"person": {
"firstname": "Smith",
"lastname": "Jones",
},
}
.. code-block:: bash
$ moban --template-type slim -c data.json "{{person.firstname}} {{person.lastname}}"
Smith Jones
Handlebars.js template
----------------------------
With `moban-handlebars `_
installed,
Given a data.json file with the following content
.. code-block::
{
"person": {
"firstname": "Yehuda",
"lastname": "Katz",
},
}
.. code-block:: bash
$ moban --template-type handlebars -c data.json "{{person.firstname}} {{person.lastname}}"
Yehuda Katz
For `handlebars.js` users, yes, the example was copied from handlebarjs.com. The
aim is to show off what we can do.
Let's continue with a bit more fancy feature:
.. code-block:: bash
$ moban --template-type handlebars -c data.json "{{#with person}}{{firstname}} {{lastname}} {{/with}}"
Moban's way of `pybar3 usage `_:
Let's save the following file a `script.py` under `helper_and_partial` folder:
.. code-block:: python
from moban_handlebars.api import Helper, register_partial
register_partial('header', '
People
')
@Helper('list')
def _list(this, options, items):
result = [u'
- ']
- ')
result.extend(options['fn'](thing))
result.append(u' ')
for thing in items:
result.append(u'
result.append(u'
return result
And given `data.json` reads as the following:
.. code-block::
{
"people":[
{"name": "Bill", "age": 100},
{"name": "Bob", "age": 90},
{"name": "Mark", "age": 25}
]
}
Let's invoke handlebar template:
.. code-block:: bash
$ moban --template-type hbs -pd helper_and_partial -c data.json "{{>header}}{{#list people}}{{name}} {{age}}{{/list}}"
Handlebars-ing {{>header}... to moban.output
Handlebarsed 1 file.
$ cat moban.output
People
- Bill 100
- Bob 90
- Mark 25
Velocity template
----------------------------
With `moban-velocity `_
installed,
Given the following data.json:
.. code-block::
{"people":
[
{"name": "Bill", "age": 100},
{"name": "Bob", "age": 90},
{"name": "Mark", "age": 25}
]
}
And given the following velocity.template:
.. code-block::
Old people:
#foreach ($person in $people)
#if($person.age > 70)
$person.name
#end
#end
Third person is $people[2].name
**moban** can do the template:
.. code-block:: bash
$ moban --template-type velocity -c data.json -t velocity.template
Old people:
Bill
Bob
Third person is Mark
Can I write my own template engine?
--------------------------------------
Yes and please check for `more details `_.
Given the following template type function, and saved in custom-plugin dir:
.. code-block:: python
from moban.core.content_processor import ContentProcessor
@ContentProcessor("de-duplicate", "De-duplicating", "De-duplicated")
def de_duplicate(content: str, options: dict) -> str:
lines = content.split(b'\n')
new_lines = []
for line in lines:
if line not in new_lines:
new_lines.append(line)
return b'\n'.join(new_lines)
You can start using it like this:
.. code-block:: bash
$ moban --template-type de-duplicate -pd custom-plugin -t duplicated_content.txt
TOML data format
----------------------
`moban-anyconfig `_ should be installed first.
Given the following toml file, sample.toml:
.. code-block::
title = "TOML Example"
[owner]
name = "Tom Preston-Werner"
You can do:
.. code-block:: bash
$ moban -c sample.toml "{{owner.name}} made {{title}}"
Tom Preston-Werner made TOML Example
Not limited to toml, you can supply moban with the following data formats:
.. csv-table:: Always supported formats, quoting from python-anyconfig
:header: "Format", "Type", "Requirement"
:widths: 15, 10, 40
JSON, json, ``json`` (standard lib) or ``simplejson``
Ini-like, ini, ``configparser`` (standard lib)
Pickle, pickle, ``pickle`` (standard lib)
XML, xml, ``ElementTree`` (standard lib)
Java properties, properties, None (native implementation with standard lib)
B-sh, shellvars, None (native implementation with standard lib)
For any of the following data formats, you elect to install by yourself.
.. csv-table:: Supported formats by pluggable backend modules
:header: "Format", "Type", "Required backend"
:widths: 15, 10, 40
Amazon Ion, ion, ``anyconfig-ion-backend``
BSON, bson, ``anyconfig-bson-backend``
CBOR, cbor, ``anyconfig-cbor-backend`` or ``anyconfig-cbor2-backend``
ConifgObj, configobj, ``anyconfig-configobj-backend``
MessagePack, msgpack, ``anyconfig-msgpack-backend``
Or you could choose to install all:
.. code-block:: bash
$ pip install moban-anyconfig[all-backends]
**Why not to use python-anyconfig itself, but yet another package?**
moban gives you a promise of any location which `python-anyconfig` does not support.
**Why do it mean 'any location'?**
Thanks to `pyfilesystem 2 `_,
moban is able to read data back from `git repo `_, `pypi `_ package, `http(s) `_, zip,
tar, ftp, `s3 `_ or `you name it `_.
Templates and configuration files over HTTP(S)
================================================================================
`httpfs `_ should be installed first.
With httpfs, `moban`_ can access any files over http(s) as its
template or data file:
.. code-block:: bash
$ moban -t 'https://raw.githubusercontent.com/moremoban/pypi-mobans/dev/templates/_version.py.jj2'\
-c 'https://raw.githubusercontent.com/moremoban/pypi-mobans/dev/config/data.yml'\
-o _version.py
.. _moban: https://github.com/moremoban/moban
In an edge case, if github repo's public url is given,
this github repo shall not have sub repos. This library will fail to
translate sub-repo as url. No magic.
Templates and configuration files in a git repo
================================================================================
`gitfs2 `_ is optional since v0.7.0 but was
installed by default since v0.6.1
You can do the following with moban:
.. code-block:: bash
$ moban -t 'git://github.com/moremoban/pypi-mobans.git!/templates/_version.py.jj2' \
-c 'git://github.com/moremoban/pypi-mobans.git!/config/data.yml' \
-o _version.py
Info: Found repo in /Users/jaska/Library/Caches/gitfs2/repos/pypi-mobans
Templating git://github.com/moremoban/pypi-mobans.git!/templates/_version.py.jj2 to _version.py
Templated 1 file.
$ cat _version.py
__version__ = "0.1.1rc3"
__author__ = "C.W."
Templates and configuration files in a python package
================================================================================
`pypifs `_ is optional since v0.7.0 but
was installed by default since v0.6.1
You can do the following with moban:
.. code-block:: bash
$ moban -t 'pypi://pypi-mobans-pkg/resources/templates/_version.py.jj2' \
-c 'pypi://pypi-mobans-pkg/resources/config/data.yml' \
-o _version.py
Collecting pypi-mobans-pkg
....
Installing collected packages: pypi-mobans-pkg
Successfully installed pypi-mobans-pkg-0.0.7
Templating pypi://pypi-mobans-pkg/resources/templates/_version.py.jj2 to _version.py
Templated 1 file.
$ cat _version.py
__version__ = "0.1.1rc3"
__author__ = "C.W."
Work with S3 and other cloud based file systems
================================================================================
Please install `fs-s3fs `_::
$ pip install fs-s3fs
Then you can access your files in s3 bucket:
.. code-block:: bash
$ moban -c s3://${client_id}:${client_secrect}@moremoban/s3data.yml \
-o 'zip://my.zip!/moban.output' {{hello}}
$ unzip my.zip
$ cat moban.output
world
Where the configuration sits in a s3 bucket, the output is a file in a zip. The content of s3data.yaml is:
.. code-block:
hello: world
So what can I do with it
============================
Here is a list of other usages:
#. `Django Mobans `_, templates for django, docker etc.
#. `Math Sheets `_, generate custom math sheets in pdf
At scale, continous templating for open source projects
================================================================================
.. image:: https://github.com/moremoban/moban/raw/dev/docs/images/moban-in-pyexcel-demo.gif
**moban** enabled **continuous templating** in `pyexcel `_ and
`coala `_ project to keep
documentation consistent across the documentations of individual libraries in the same
organisation. Here is the primary use case of moban, as of now:
.. image:: https://github.com/moremoban/yehua/raw/dev/docs/source/_static/yehua-story.png
:width: 600px
Usage beyond command line
=============================
All use cases are `documented `_
Support
================================================================================
If you like moban, please support me on github,
`patreon `_
or `bounty source `_ to maintain
the project and develop it further.
With your financial support, I will be able to invest
a little bit more time in coding, documentation and writing interesting extensions.
Vision
================================================================================
Any template, any data in any location
**moban** started with bringing the high performance template engine (JINJA2) for web
into static text generation.
**moban** can use other python template engine: mako, handlebars, velocity,
haml, slim and tornado, can read other data format: json and yaml, and can access both
template file and configuration file in
any location: zip, git, pypi package, s3, etc.
Credit
================================================================================
`jinja2-fsloader `_ is the key component to enable PyFilesystem2 support in moban
v0.6x. Please show your stars there too!
Installation
================================================================================
You can install it via pip:
.. code-block:: bash
$ pip install moban
or clone it and install it:
.. code-block:: bash
$ git clone http://github.com/moremoban/moban.git
$ cd moban
$ python setup.py install
CLI documentation
================================================================================
.. code-block:: bash
usage: moban [-h] [-c CONFIGURATION] [-t TEMPLATE] [-o OUTPUT]
[-td [TEMPLATE_DIR [TEMPLATE_DIR ...]]]
[-pd [PLUGIN_DIR [PLUGIN_DIR ...]]] [-cd CONFIGURATION_DIR]
[-m MOBANFILE] [-g GROUP] [--template-type TEMPLATE_TYPE]
[-d DEFINE [DEFINE ...]] [-e EXTENSION [EXTENSION ...]] [-f]
[--exit-code] [-V] [-v]
[template]
Static text generator using any template, any data and any location.
positional arguments:
template string templates
optional arguments:
-h, --help show this help message and exit
-c CONFIGURATION, --configuration CONFIGURATION
the data file
-t TEMPLATE, --template TEMPLATE
the template file
-o OUTPUT, --output OUTPUT
the output file
Advanced options:
For better control
-td [TEMPLATE_DIR [TEMPLATE_DIR ...]], --template_dir [TEMPLATE_DIR [TEMPLATE_DIR ...]]
add more directories for template file lookup
-cd CONFIGURATION_DIR, --configuration_dir CONFIGURATION_DIR
the directory for configuration file lookup
-pd [PLUGIN_DIR [PLUGIN_DIR ...]], --plugin_dir [PLUGIN_DIR [PLUGIN_DIR ...]]
add more directories for plugin lookup
-m MOBANFILE, --mobanfile MOBANFILE
custom moban file
-g GROUP, --group GROUP
a subset of targets
--template-type TEMPLATE_TYPE
the template type, default is jinja2
-d DEFINE [DEFINE ...], --define DEFINE [DEFINE ...]
to supply additional or override predefined variables,
format: VAR=VALUEs
-e EXTENSION [EXTENSION ...], --extension EXTENSION [EXTENSION ...]
to to TEMPLATE_TYPE=EXTENSION_NAME
-f force moban to template all files despite of
.moban.hashes
Developer options:
For debugging and development
--exit-code by default, exist code 0 means no error, 1 means error
occured. It tells moban to change 1 for changes, 2 for
error occured
-V, --version show program's version number and exit
-v show verbose, try -v, -vv, -vvv