Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hukkin/tomli
A lil' TOML parser
https://github.com/hukkin/tomli
config parser python toml toml-parser tomli
Last synced: about 1 month ago
JSON representation
A lil' TOML parser
- Host: GitHub
- URL: https://github.com/hukkin/tomli
- Owner: hukkin
- License: mit
- Created: 2021-05-25T08:17:13.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-07-22T17:45:34.000Z (about 2 months ago)
- Last Synced: 2024-07-22T21:23:54.242Z (about 2 months ago)
- Topics: config, parser, python, toml, toml-parser, tomli
- Language: Python
- Homepage:
- Size: 322 KB
- Stars: 461
- Watchers: 9
- Forks: 42
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/hukkin/tomli/actions?query=workflow%3ATests+branch%3Amaster+event%3Apush)
[](https://codecov.io/gh/hukkin/tomli)
[](https://pypi.org/project/tomli)
# Tomli
> A lil' TOML parser
**Table of Contents** *generated with [mdformat-toc](https://github.com/hukkin/mdformat-toc)*
- [Intro](#intro)
- [Installation](#installation)
- [Usage](#usage)
- [Parse a TOML string](#parse-a-toml-string)
- [Parse a TOML file](#parse-a-toml-file)
- [Handle invalid TOML](#handle-invalid-toml)
- [Construct `decimal.Decimal`s from TOML floats](#construct-decimaldecimals-from-toml-floats)
- [Building a `tomli`/`tomllib` compatibility layer](#building-a-tomlitomllib-compatibility-layer)
- [FAQ](#faq)
- [Why this parser?](#why-this-parser)
- [Is comment preserving round-trip parsing supported?](#is-comment-preserving-round-trip-parsing-supported)
- [Is there a `dumps`, `write` or `encode` function?](#is-there-a-dumps-write-or-encode-function)
- [How do TOML types map into Python types?](#how-do-toml-types-map-into-python-types)
- [Performance](#performance)
Tomli is a Python library for parsing [TOML](https://toml.io).
It is fully compatible with [TOML v1.0.0](https://toml.io/en/v1.0.0).
A version of Tomli, the `tomllib` module,
was added to the standard library in Python 3.11
via [PEP 680](https://www.python.org/dev/peps/pep-0680/).
Tomli continues to provide a backport on PyPI for Python versions
where the standard library module is not available
and that have not yet reached their end-of-life.
```bash
pip install tomli
```
```python
import tomli
toml_str = """
[[players]]
name = "Lehtinen"
number = 26
[[players]]
name = "Numminen"
number = 27
"""
toml_dict = tomli.loads(toml_str)
assert toml_dict == {
"players": [{"name": "Lehtinen", "number": 26}, {"name": "Numminen", "number": 27}]
}
```
```python
import tomli
with open("path_to_file/conf.toml", "rb") as f:
toml_dict = tomli.load(f)
```
The file must be opened in binary mode (with the `"rb"` flag).
Binary mode will enforce decoding the file as UTF-8 with universal newlines disabled,
both of which are required to correctly parse TOML.
```python
import tomli
try:
toml_dict = tomli.loads("]] this is invalid TOML [[")
except tomli.TOMLDecodeError:
print("Yep, definitely not valid.")
```
Note that error messages are considered informational only.
They should not be assumed to stay constant across Tomli versions.
### Construct `decimal.Decimal`s from TOML floats
```python
from decimal import Decimal
import tomli
toml_dict = tomli.loads("precision-matters = 0.982492", parse_float=Decimal)
assert isinstance(toml_dict["precision-matters"], Decimal)
assert toml_dict["precision-matters"] == Decimal("0.982492")
```
Note that `decimal.Decimal` can be replaced with another callable that converts a TOML float from string to a Python type.
The `decimal.Decimal` is, however, a practical choice for use cases where float inaccuracies can not be tolerated.
Illegal types are `dict` and `list`, and their subtypes.
A `ValueError` will be raised if `parse_float` produces illegal types.
### Building a `tomli`/`tomllib` compatibility layer
Python versions 3.11+ ship with a version of Tomli:
the `tomllib` standard library module.
To build code that uses the standard library if available,
but still works seamlessly with Python 3.6+,
do the following.
Instead of a hard Tomli dependency, use the following
[dependency specifier](https://packaging.python.org/en/latest/specifications/dependency-specifiers/)
to only require Tomli when the standard library module is not available:
```
tomli >= 1.1.0 ; python_version < "3.11"
```
Then, in your code, import a TOML parser using the following fallback mechanism:
```python
import sys
if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib
tomllib.loads("['This parses fine with Python 3.6+']")
```
- it's lil'
- pure Python with zero dependencies
- the fastest pure Python parser [\*](#performance):
16x as fast as [tomlkit](https://pypi.org/project/tomlkit/),
2.3x as fast as [toml](https://pypi.org/project/toml/)
- outputs [basic data types](#how-do-toml-types-map-into-python-types) only
- 100% spec compliant: passes all tests in
[BurntSushi/toml-test](https://github.com/BurntSushi/toml-test)
test suite
- thoroughly tested: 100% branch coverage
### Is comment preserving round-trip parsing supported?
No.
The `tomli.loads` function returns a plain `dict` that is populated with builtin types and types from the standard library only.
Preserving comments requires a custom type to be returned so will not be supported,
at least not by the `tomli.loads` and `tomli.load` functions.
Look into [TOML Kit](https://github.com/sdispater/tomlkit) if preservation of style is what you need.
### Is there a `dumps`, `write` or `encode` function?
[Tomli-W](https://github.com/hukkin/tomli-w) is the write-only counterpart of Tomli, providing `dump` and `dumps` functions.
The core library does not include write capability, as most TOML use cases are read-only, and Tomli intends to be minimal.
### How do TOML types map into Python types?
| TOML type | Python type | Details |
| ---------------- | ------------------- | ------------------------------------------------------------ |
| Document Root | `dict` | |
| Key | `str` | |
| String | `str` | |
| Integer | `int` | |
| Float | `float` | |
| Boolean | `bool` | |
| Offset Date-Time | `datetime.datetime` | `tzinfo` attribute set to an instance of `datetime.timezone` |
| Local Date-Time | `datetime.datetime` | `tzinfo` attribute set to `None` |
| Local Date | `datetime.date` | |
| Local Time | `datetime.time` | |
| Array | `list` | |
| Table | `dict` | |
| Inline Table | `dict` | |
The `benchmark/` folder in this repository contains a performance benchmark for comparing the various Python TOML parsers.
The benchmark can be run with `tox -e benchmark-pypi`.
Running the benchmark on my personal computer output the following:
```console
foo@bar:~/dev/tomli$ tox -e benchmark-pypi
benchmark-pypi installed: attrs==21.4.0,click==8.0.3,pytomlpp==1.0.10,qtoml==0.3.1,rtoml==0.7.1,toml==0.10.2,tomli==2.0.1,tomlkit==0.9.2
benchmark-pypi run-test-pre: PYTHONHASHSEED='3088452573'
benchmark-pypi run-test: commands[0] | python -c 'import datetime; print(datetime.date.today())'
2022-02-09
benchmark-pypi run-test: commands[1] | python --version
Python 3.8.10
benchmark-pypi run-test: commands[2] | python benchmark/run.py
Parsing data.toml 5000 times:
------------------------------------------------------
parser | exec time | performance (more is better)
-----------+------------+-----------------------------
rtoml | 0.891 s | baseline (100%)
pytomlpp | 0.969 s | 91.90%
tomli | 4 s | 22.25%
toml | 9.01 s | 9.88%
qtoml | 11.1 s | 8.05%
tomlkit | 63 s | 1.41%
```
The parsers are ordered from fastest to slowest, using the fastest parser as baseline.
Tomli performed the best out of all pure Python TOML parsers,
losing only to pytomlpp (wraps C++) and rtoml (wraps Rust).