https://github.com/informaticsmatters/squonk2-data-manager-job-decoder
Provides decoding capabilities for Job definitions
https://github.com/informaticsmatters/squonk2-data-manager-job-decoder
squonk2
Last synced: about 1 year ago
JSON representation
Provides decoding capabilities for Job definitions
- Host: GitHub
- URL: https://github.com/informaticsmatters/squonk2-data-manager-job-decoder
- Owner: InformaticsMatters
- License: mit
- Created: 2022-01-11T11:17:18.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2025-03-14T18:08:39.000Z (about 1 year ago)
- Last Synced: 2025-04-23T23:08:45.718Z (about 1 year ago)
- Topics: squonk2
- Language: Python
- Homepage:
- Size: 84 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
Awesome Lists containing this project
README
Informatics Matters Data Manager Job Decoder
============================================
.. image:: https://badge.fury.io/py/im-data-manager-job-decoder.svg
:target: https://badge.fury.io/py/im-data-manager-job-decoder
:alt: PyPI package (latest)
.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/build.yaml/badge.svg
:target: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/build.yaml
:alt: Build
.. image:: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/publish.yaml/badge.svg
:target: https://github.com/InformaticsMatters/squonk2-data-manager-job-decoder/actions/workflows/publish.yaml
:alt: Publish
A package that simplifies the decoding of encoded text strings.
Given an encoded string the ``decode()`` method
returns the decoded value or an error.
For example, given the following `jinja2`_ encoded string
``'{{ foo }}, bar={{ bar }}, baz={{ baz }}'`` and variable map
``{'foo': 1, 'bar': 2, 'baz': 3}`` the decoder returns
the string ``'foo=1, bar=2, baz=3'``.
The following encoding/decoding formats are supported: -
- jinja2 (3.0)
The package also provides ``validate_job_schema()`` and
``validate_manifest_schema()`` functions, which can (should) be used to
validate the Manifests and Job definitions against the
built-in schemas.
.. _jinja2: https://jinja.palletsprojects.com/en/3.0.x/
Installation (Python)
=====================
The Job decoder is published on `PyPI`_ and can be installed from
there::
pip install im-data-manager-job-decoder
Once installed you can validate the definition (expected to be a dictionary
formed from the definition YAML file) with::
>>> from decoder import decoder
>>> error: Optional[str] = decoder.validate_manifest_schema(manifest)
>>> error: Optional[str] = decoder.validate_job_schema(job_definition)
And you can decode encoded fields within the job definition.
Here we're decoding the job's 'command' using a map of variables and their
values::
>>> decoded, success = decoder.decode(definition['command'],
variables,
'command',
decoder.TextEncoding.JINJA2_3_0)
A method also exists to generate a Data Manger Job documentation URL
for the supported repository types. For example, to get the
auto-generated documentation URL for a job definition hosted in a GitHub
repository you can do this::
>>> doc_url: str = decoder.get_job_doc_url("github",
collection
job_id,
job_definition,
manifest_url)
We support ``github`` and ``gitlab`` as repository types.
.. _PyPI: https://pypi.org/project/im-data-manager-job-decoder
Get in touch
============
- Report bugs, suggest features or view the source code `on GitHub`_.
.. _on GitHub: https://github.com/informaticsmatters/squonk2-data-manager-job-decoder