https://github.com/silvanmelchior/cme_parser
A tiny parser for more flexible conda environment files
https://github.com/silvanmelchior/cme_parser
cme-parser conda conda-environment data-science meta-environment parser python
Last synced: 7 months ago
JSON representation
A tiny parser for more flexible conda environment files
- Host: GitHub
- URL: https://github.com/silvanmelchior/cme_parser
- Owner: silvanmelchior
- Created: 2019-09-25T11:26:16.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-11-19T06:07:44.000Z (about 4 years ago)
- Last Synced: 2025-04-03T07:51:23.671Z (10 months ago)
- Topics: cme-parser, conda, conda-environment, data-science, meta-environment, parser, python
- Language: Python
- Homepage:
- Size: 22.5 KB
- Stars: 15
- Watchers: 3
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Conda Meta Environment Parser
This tiny parser takes as input a meta-environment file, transforms it into an environment file and invokes conda on it.
The input, usually called **environment.yml.meta**, is a standard conda environment file which can be enriched with conditions for certain lines/blocks, e.g. to only include them on specific platforms or if certain flags are given.
## Example
The following meta-file
```
name: demo_env
dependencies:
- numpy=1.16.4
- tensorflow=1.14.0
- tensorflow-gpu=1.14.0 [use_gpu]
- winreg-helpers [platform == win32]
```
can be used to set up a conda environment with
`python create_env.py` (A)
or
`python create_env.py --flag use_gpu` (B).
In (A), the conda package *tensorflow-gpu* is not installed. In both cases, *winreg-helpers* is only installed on windows platforms.
## Quickstart
To use CME Parser in your own project, create an **environment.yml.meta** file in your project folder, e.g. like this:
```
name: demo_env2
channels:
- conda-forge
- defaults
dependencies:
- numpy=1.16.4
- matplotlib=3.1.0 [allow_plot and plot_tool == matplotlib]
- ggplot=0.11.5 [allow_plot and plot_tool == ggplot]
- pip=19.1.1
- some-linux-package=1.2.3 [platform startswith linux]
- pip:
- selenium=3.141.0
```
If **create_env.py** gets called without an `--input` argument, it will try to find **environment.yml.meta** either in the same directory or its parent.
Thus, either directly add **create_env.py** to your project folder next to the meta-env file or add this repo as git submodule:
```
$ cd your_project_folder
$ git rm environment.yml # if exists
$ echo "environment.yml" >> .gitignore
$ git add environment.yml.meta
$ git submodule add https://github.com/silvanmelchior/cme_parser
$ git commit -m "cme parser added"
$ git push
```
If you now execute (in the case of using a submodule)
`python cme_parser/create_env.py --flag allow_plot --variable plot_tool matplotlib`
in your project folder, CME Parser will find the meta-env file and create **environment.yml**.
In general, if not specified differently with a `--output` argument, the parser will write the output into the same location as the input, removing *.meta* from the filename.
If *.meta* is not present, it will append *_out* to the filename instead.
In all cases, the parser will in the end invoke conda to create the environment specified by the output file.
## Requirements
CME Parser is written in pure Python and does only need standard library packages.
It can be executed e.g. in the conda default environment (called *base*) and was tested with Python 2.7 and 3.6 / 3.7.
## Syntax
If a line contains `[condition]`, the parser only outputs it if the condition is met (removing the condition itself).
If a line contains `[[condition` and a later line `]]`, the parser outputs everything in between only if the condition is met (removing the two lines marking the block borders).
Nested blocks are allowed.
A condition can either be an arbitrary name, which is then interpreted as a flag, or a triplet of an arbitrary name, some operator and a string.
In the latter, the name is interpreted as variable and the operator can be one of `==`, `!=`, `startswith`, `endswith`, `contains`.
The string does not need any special terminators (e.g. no `""`).
Conditions can be combined by adding `and` and `or` in between them.
Furthermore, `not` can be added in front of a condition. The precedence order is `or` (highest), `and`, `not` (lowest), so e.g. `a and not b` is evaluated as `a and (not b)`.
Brackets to enforce another evaluation order are currently not supported.
Flags are evaluated as *false* per default, except given to the parser with `-f flag_name`.
Variables have to be given to the parser with `-v var_name var_value`, otherwise an error is risen.
An exception are variables of the form `var_name%value`, which are interpreted as variable `var_name` with default value `value`.
Furthermore, the special variable `platform` is reserved and populated with the result of `sys.platform`.
## Command Line Options
| Short | Long | Description |
| --- | --- | --- |
| -h | --help | show usage help message |
| -i | --input | specify meta environment input file |
| -o | --output | specify environment output file |
| -f | --flag | set custom flag for parsing |
| -v | --variable | define custom variable for parsing |
| -q | --quiet | quietly overwrite output if already exists |
| -p | --parse-only | do not invoke conda afterwards |
| -c | --no-comment | do not add auto-generated comment to output |