Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/voschezang/mash
A Python-based shell and various utilities
https://github.com/voschezang/mash
ast python python3 scripting shell subshell yacc yacc-lex
Last synced: 5 days ago
JSON representation
A Python-based shell and various utilities
- Host: GitHub
- URL: https://github.com/voschezang/mash
- Owner: voschezang
- License: gpl-3.0
- Created: 2022-11-28T18:29:02.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-17T15:19:04.000Z (10 months ago)
- Last Synced: 2024-12-12T07:15:00.066Z (30 days ago)
- Topics: ast, python, python3, scripting, shell, subshell, yacc, yacc-lex
- Language: Python
- Homepage:
- Size: 1.66 MB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
**Docs**:
[MAIN](https://github.com/voschezang/mash/blob/main/README.md)
| [Shell](https://voschezang.github.io/mash-docs/pages/shell_help.html)
| [Reference](https://github.com/voschezang/mash/blob/main/SHELL_REFERENCE.md)
| [Library Docs](https://voschezang.github.io/mash-docs/)
# Mash | My Automation Shell
A *shell* that can be used to for automation and (REST) resource discovery. It can be used as a [cli](https://en.wikipedia.org/wiki/Command-line_interface), [repl](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop). It exposes a [complete](https://en.wikipedia.org/wiki/Turing_completeness) programming [language](SHELL_REFERENCE.md) with variables, functions, conditions and pipes. The language can be tailored towards [domain-specific](https://en.wikipedia.org/wiki/Domain-specific_language) applications and it has interoperability with Bash. In addition, this repository contains a library of utilities.
Features:
- A **DSL** that can interpret user-defined commands: `shell.py`.
- A **file-browser**. Query both static datastructures and REST APIs: `examples/filesystem.py`,
- A **REST client** to browse APIs with a programmatic yet intuitive interface: `examples/discoverable_api.py`.
- An **object parser** which converts JSON data to Python classes: `object_parser.py`.
- An **OAS-generator** for Python classes: `oas.py`.
- A subshell wrapper, to redirect the output of shell scripts: `subshell.py`.
- A parallelization framework for load testing: `parallel.py`.
Links
- [Mash language reference](SHELL_REFERENCE.md)
- [PyPI](https://pypi.org/project/mash-shell)
- [github](https://github.com/voschezang/mash)
- [docs](https://voschezang.github.io/mash-docs/)
## Shell
```sh
pip install mash-shell
python -m mash
```
See [`src/examples`](src/examples) for advances usage examples.
**Documentation**
| Implementation: [Shell](https://voschezang.github.io/mash-docs/pages/shell.html)
- Language: [Reference](https://github.com/voschezang/mash/blob/main/SHELL_REFERENCE.md).
**Table of Contents**
- [Usage](#usage)
- [CLI](#cli)
- [Filesystem (CRUD Operations)](#Filesystem%20(CRUD%20Operations))
- [Usage Examples](#Usage%20Examples)
- [Library & Tools](#Library%20&%20Tools)
- [Parallelization Utilities](#Parallelization%20Utilities)
- [Object Parser](#Object%20Parser)
## Usage
See [reference](SHELL_REFERENCE.md).
### CLI
```sh
▶ python -m mash -h
usage: shell.py [-hvsr][-f FILE] [--session SESSION] [cmd [cmd ...]]
If no positional arguments are given then an interactive subshell is started.
positional arguments:
cmd A comma- or newline-separated list of commands
optional arguments:
-h, --help show this help message and exit
-v, --verbose
-s, --safe Safe-mode. Ask for confirmation before executing commands.
-f FILE, --file FILE Read and run FILE as a commands
-r, --reload Reload last session
--session SESSION Use session SESSION
```
## Filesystem (CRUD Operations)
See `examples/filesystem.py` and `examples/discoverable.py`.
| Example | Description |
| ------------------- | ------------------------------------------------------------ |
| `cd [DIR]` | Change the current working directory. Alias: `use` |
| `list [DIR]` | List the items in a directory. Use the current working directory by default. Alias: `l` |
| `foreach [DIR]` | Iterate over a directory structure. E.g. a resource `users/{id}/email`. |
| `get NAME` | Retrieve a file. |
| `set NAME VALUE` | Modify a file. |
| `new NAME [NAME..]` | Create new directories. |
| `show [NAME]` | Display detailed information about a directory. |
| `cp`, `mv`, `rm` | Modify files. I.e. copy, move, rename or remove files. |
## Usage Examples
For real-world examples, see [lib](https://github.com/voschezang/mash/blob/main/src/lib/math.sh).
### Example 1
See `src/examples/shell_example.py`. It shows how to use a user-definnable mapping of custom functions.
It uses the library `quo` to create a user-friendly subshell with autocompletion prompts.
```sh
# python3 src/examples/shell_example.py echo 'hello world'
hello world
```
### Example 2: REST Client
Browse through a REST API as if it is mounted as a filesystem.
```sh
# python3 src/examples/rest_client_implicit.py
$ cd users # browse some.api.com/v1/users
$ get 2
| | values |
|:------|:-------------------|
| email | [email protected] |
| name | name_2 |
$ flatten 1 2 >>= get users $ email
[email protected]
[email protected]
$ cd -
$ tree
{ 'document': 'https://dummy-api.com/v1',
'users': { 0: {'email': '[email protected]', 'name': 'name_0'},
1: {'email': '[email protected]', 'name': 'name_1'},
2: {'email': '[email protected]', 'name': 'name_2'},
...
```
### Example 3: Commands
Run commands from a file with `python src/shell -f FILE` or `python -m src.shell -f FILE`.
```sh
# write to file
print A sentence. > out.txt
!cat out.txt |> export x # save text from file
print $x
y <- shell expr 2 + 2 # store result in variable $y
print "result:" $y # prints "result: 4
shell expr 2 + 2 -> z # store result in variable $z
print "result:" $z # prints "result: 4
```
### Example 4: File System Simulation
Support both static and dynamic data.
#### With Static Data
See `src/examples/filesystem_example.py`. This simulates a REST resources with a directory hierarchy.
In addition, it provides fuzzy name completion.
```sh
$ python src/examples/shell_example.py tree
# example data with dicts and lists
repository = {'worlds': [
{'name': 'earth',
'animals': [
{'name': 'terrestrial',
'snakes': [{'name': 'python'},
{'name': 'cobra'}]},
{'name': 'aquatic',
'penquins': [{'name': 'tux'}]}
]}]}
```
```sh
# note the autocompletion
$ python src/examples/filesystem.py 'cd world; cd a; cd t; cd snakes; ll'
python
cobra
```
#### With Dynamic Data
See `examples/discoverable.py`.
```sh
# list remote/auto-generated data
$ python src/examples/discoverable.py 'ls'
department_805, department_399
# refresh data, then save
$ py src/examples/discoverable.py 'ls ; save'
department_750, department_14
# reload data
$ python src/examples/discoverable.py 'ls'
department_750, department_14
```
## Library & Tools
Repository structure
```sh
src
├── examples
├── lib # mash shell scripts
└── mash
├── filesystem # Directory-based datastructures: FileSystem, View
├── object_parser # Parse JSON objects
└── shell # Shell, ShellWithFileSystem
test
```
## Setup
Using a `Makefile` for convenience.
```sh
make install
make test
```
## Parallelization Utilities
Some experiments with parallelization, concurrency and `asyncio` in Python.
### Test
Start a dummy server.
```sh
python3 src/server.py
```
Do a simple load test
```sh
python3 src/parallel.py -v
```
## Object Parser
- [src/object_parser/object_parser.py](object_parser.py) parses JSON data and instantiate Python objects.
- [src/object_parser/oas.py](oas.py) converts domain-models to OAS.
### Example
```sh
python src/object_parser_example.py
```
### REST API
Server
```sh
python src/object_parser_server.py
```
Client
```sh
curl -X 'POST' 'http://localhost:5000/v1/organizations' \
-H 'Content-Type: application/json' \
-d '{ "board": [ "string" ], "ceo": "string", "departments": [ { "manager": "string", "teams": [ { "manager": "string", "members": [ "string" ], "team_type": "A", "active": true, "capacity": 0, "value": 0 } ] } ] }'
```