https://github.com/phil65/yamling
Enhanced YAML loading and dumping
https://github.com/phil65/yamling
configuration configuration-files configuration-management python python3 serialization yaml yaml-configuration yaml-parser
Last synced: 2 months ago
JSON representation
Enhanced YAML loading and dumping
- Host: GitHub
- URL: https://github.com/phil65/yamling
- Owner: phil65
- License: mit
- Created: 2024-11-04T04:14:17.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-03-16T11:03:09.000Z (3 months ago)
- Last Synced: 2025-03-29T01:34:11.370Z (3 months ago)
- Topics: configuration, configuration-files, configuration-management, python, python3, serialization, yaml, yaml-configuration, yaml-parser
- Language: Python
- Homepage: https://phil65.github.io/yamling/
- Size: 251 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- jimsghstars - phil65/yamling - Enhanced YAML loading and dumping (Python)
README
# yamling
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://pypi.org/project/yamling/)
[](https://github.com/phil65/yamling/releases)
[](https://github.com/phil65/yamling/graphs/contributors)
[](https://github.com/phil65/yamling/discussions)
[](https://github.com/phil65/yamling/forks)
[](https://github.com/phil65/yamling/issues)
[](https://github.com/phil65/yamling/pulls)
[](https://github.com/phil65/yamling/watchers)
[](https://github.com/phil65/yamling/stars)
[](https://github.com/phil65/yamling)
[](https://github.com/phil65/yamling/commits)
[](https://github.com/phil65/yamling/releases)
[](https://github.com/phil65/yamling)
[](https://github.com/phil65/yamling)
[](https://github.com/phil65/yamling)
[](https://github.com/phil65/yamling)
[](https://codecov.io/gh/phil65/yamling/)
[](https://github.com/psf/black)
[](https://pyup.io/repos/github/phil65/yamling/)
[Read the documentation!](https://phil65.github.io/yamling/)
Yamling is a YAML handling library that provides enhanced loading and dumping capabilities for YAML files. It builds upon [PyYAML](https://pyyaml.org/) to offer additional features like environment variable support, file inclusion, and Jinja2 template resolution.
Special mentions also to `pyyaml_env_tag` as well as `pyyaml-include`, this library exposes these YAML extensions with a unified interface.
## Loading YAML
### Basic Loading
To load YAML content from a string:
```python
from yamling import load_yaml
# Simple YAML loading
data = load_yaml("""
name: John
age: 30
""")
```
To load from a file:
```python
from yamling import load_yaml_file
# Load from local file
config = load_yaml_file("config.yaml")
# Load from remote file (S3, HTTP, etc.)
remote_config = load_yaml_file("s3://bucket/config.yaml")
```
### Safety Modes
Yamling supports three safety modes when loading YAML:
```python
# Safe mode - most restrictive, recommended for untrusted input
data = load_yaml(content, mode="safe")
# Full mode - allows some additional types but restricts dangerous ones
data = load_yaml(content, mode="full")
# Unsafe mode - allows all YAML features (default)
data = load_yaml(content, mode="unsafe")
```
> **Warning**
> Always use "safe" mode when loading untrusted YAML content to prevent code execution vulnerabilities.
### File Inclusion
Yamling supports including other YAML files using the `!include` tag:
```yaml
# main.yaml
database:
!include db_config.yaml
logging:
!include logging_config.yaml
```
When loading, specify the base path for includes:
```python
config = load_yaml_file("main.yaml", include_base_path="configs/")
```
### Environment Variables
Use the `!ENV` tag to reference environment variables:
```yaml
database:
password: !ENV DB_PASSWORD
host: !ENV ${DB_HOST:localhost} # with default value
```
### Template Resolution
Yamling can resolve [Jinja2](https://jinja.palletsprojects.com/) templates in YAML:
```python
from jinja2 import Environment
import yamling
env = Environment()
yaml_content = """
message: "Hello {{ name }}!"
"""
data = load_yaml(
yaml_content,
resolve_strings=True,
jinja_env=env
)
```
### Inheritance
YAML files can inherit from other files using the `INHERIT` key. You can specify either a single file or a list of files to inherit from:
```yaml
# Single inheritance
# prod.yaml
INHERIT: base.yaml
database:
host: prod.example.com
# Multiple inheritance
# prod_with_logging.yaml
INHERIT:
- base.yaml
- logging.yaml
- monitoring.yaml
database:
host: prod.example.com
```
When using multiple inheritance, files are processed in order, with later files taking precedence over earlier ones. The current file's values take precedence over all inherited values.
For example:
```yaml
# base.yaml
database:
host: localhost
port: 5432
timeout: 30
# logging.yaml
database:
timeout: 60
logging:
level: INFO
# prod.yaml
INHERIT:
- base.yaml
- logging.yaml
database:
host: prod.example.com
```
When loading `prod.yaml`, the final configuration will be:
```yaml
database:
host: prod.example.com # from prod.yaml
port: 5432 # from base.yaml
timeout: 60 # from logging.yaml
logging:
level: INFO # from logging.yaml
```
Load with inheritance enabled:
```python
config = load_yaml_file("prod.yaml", resolve_inherit=True)
```
## Dumping YAML
To serialize Python objects to YAML:
```python
from yamling import dump_yaml
data = {
"name": "John",
"scores": [1, 2, 3],
"active": True
}
yaml_string = dump_yaml(data)
```
Dataclasses and Pydantic v2 models can also get dumped.
### Custom Class Mapping
Map custom classes to built-in types for YAML representation:
```python
from collections import OrderedDict
data = OrderedDict([("b", 2), ("a", 1)])
yaml_string = dump_yaml(data, class_mappings={OrderedDict: dict})
```
## Custom Loader Configuration
For advanced use cases, you can create a custom loader:
```python
from yamling import get_loader
import yaml
# Create custom loader with specific features
loader_cls = get_loader(
yaml.SafeLoader,
include_base_path="configs/",
enable_include=True,
enable_env=True,
resolve_strings=True,
jinja_env=jinja_env,
type_converters={int: str}
)
# Use custom loader
data = yaml.load(content, Loader=loader_cls)
```
## Custom Tag Handling
Yamling provides a `YAMLParser` class for handling custom YAML tags. This allows you to define how specific tagged values should be processed during YAML loading.
### Basic Tag Registration
You can register tag handlers using either a decorator or explicit registration:
```python
from yamling import YAMLParser
# Create parser instance
yaml_parser = YAMLParser()
# register handler
def handle_uppercase(data: str) -> str:
return data.upper()
yaml_parser.register_handler("uppercase", handle_uppercase)
```
### Using Custom Tags
Once registered, you can use the custom tags in your YAML:
```yaml
# config.yaml
message: !uppercase "hello world"
```
Load the YAML using the parser:
```python
# Load from string
data = yaml_parser.load_yaml("""
message: !uppercase "hello world"
""")
# Or load from file
data = yaml_parser.load_yaml_file("config.yaml")
print(data["message"]) # "HELLO WORLD"
```
### Class Registration
For simple cases where you just want to convert tagged dictionary data into class instances,
you can use the `register_class` method. The method will automatically use the lowercase class name
as the tag name (following YAML tag conventions), but you can override this with a custom tag name:
```python
from yamling import YAMLParser
from dataclasses import dataclass
@dataclass
class Person:
name: str
age: int
@dataclass
class HomeAddress:
street: str
city: str
yaml_parser = YAMLParser()
# Register using class name as tag (will use "person" as tag)
yaml_parser.register_class(Person)
# Register with custom tag name instead of "homeaddress"
yaml_parser.register_class(HomeAddress, "address")
# Now you can use it in YAML:
data = yaml_parser.load_yaml("""
user: !person
name: John Doe
age: 30
home: !address
street: 123 Main St
city: New York
""")
print(data["user"]) # Person(name='John Doe', age=30)
print(data["home"]) # HomeAddress(street='123 Main St', city='New York')
```
### Complex Structures
Custom tags can be used in nested structures and lists:
```yaml
team:
manager: !person
name: Alice Smith
age: 45
members:
- !person
name: Bob Johnson
age: 30
- !person
name: Carol White
age: 28
messages:
- !uppercase "welcome"
- !uppercase "goodbye"
```
### Combining with Other Features
The `YAMLParser` class supports all of Yamling's standard features:
```python
data = yaml_parser.load_yaml_file(
"config.yaml",
mode="safe", # Safety mode
include_base_path="configs/", # For !include directives
resolve_strings=True, # Enable Jinja2 template resolution
resolve_inherit=True, # Enable inheritance
jinja_env=jinja_env # Custom Jinja2 environment
)
```
### Available Tags
You can list all registered tags:
```python
tags = yaml_parser.list_tags()
print(tags) # ['!person', '!uppercase']
```
## Universal load / dump interface
Yamling provides a universal load function that can handle YAML, JSON, TOML, and INI files.
Apart from yaml, only stdlib modules are used, so no additional dependencies are required.
Here's a simple example:
```python
import yamling
# Load files based on their extension
config = yamling.load_file("config.yaml") # YAML
settings = yamling.load_file("settings.json") # JSON
params = yamling.load_file("params.toml") # TOML
# Or explicitly specify the format
data = yamling.load_file("config.txt", mode="yaml")
# Load directly from strings
yaml_text = """
name: John
age: 30
"""
data = yamling.load(yaml_text, mode="yaml")
# same in other direction
json_text = yamling.dump(data, mode="json")
yaml.dump_file("config.yaml", data)
```
> **Note**
> If [orjson](https://github.com/ijl/orjson) is installed, the loader will automatically use it for JSON parsing / dumping, offering significantly better performance compared to the standard `json` module.