https://github.com/joaomg/teed
Telco engineering data library
https://github.com/joaomg/teed
bulkcm parsing telco xml
Last synced: 3 months ago
JSON representation
Telco engineering data library
- Host: GitHub
- URL: https://github.com/joaomg/teed
- Owner: joaomg
- License: agpl-3.0
- Created: 2021-03-03T14:08:58.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2024-12-16T21:07:23.000Z (over 1 year ago)
- Last Synced: 2025-12-16T03:18:41.679Z (7 months ago)
- Topics: bulkcm, parsing, telco, xml
- Language: Python
- Homepage:
- Size: 165 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# teed
[](https://pypi.python.org/pypi/teed)
[](https://github.com/joaomg/teed)
Telco engineering data library
Probe and transform raw telco files into CSV.
## A simple BulkCm file parsing
```shell
pip install teed
python -m teed bulkcm parse data/bulkcm.xml data
Parsing data/bulkcm.xml
Created data/ManagedElement.csv
Created data/ManagementNode.csv
Created data/SubNetwork.csv
Time: 0:00:00.000856
```
## Install from source
```shell
git clone https://github.com/joaomg/teed.git
cd teed
pip install -e .
```
## Probe a file
```shell
python -m teed bulkcm probe data/bulkcm_with_vsdatacontainer.xml
```
## Parse a file output it's content to CSV files
```shell
python -m teed bulkcm parse data/bulkcm_empty.xml data
python -m teed bulkcm parse data/bulkcm_with_header_footer.xml data
python -m teed bulkcm parse data/bulkcm_with_vsdatacontainer.xml data
```
## Usage
Beside command-line teed can be used as a library:
```python
from teed import bulkcm, meas
## bulkcm
stream = bulkcm.BulkCmParser.stream_to_csv("data")
bulkcm.parse("data/bulkcm.xml", "data", stream)
## meas
meas.parse("data/mdc*xml", "data")
```
The bulkcm parser extracts content from a single file.
While the meas parser, in a single run, can process any number of XML files using wildcards and directory recursion.
The bulkcm and meas parsers also differ on CSV file creation:
- bulkcm deletes previously existing CSV files
- meas appends to existing CSV files
Take notice of these differences when calling the parsers from shell.
Or using them in data pipelines.
## Documentation
Probe, split and extract configuration content from [bulkcm](https://github.com/joaomg/teed/blob/main/teed/BULKCM.md) XML files.
Extract performance data from [meas](https://github.com/joaomg/teed/blob/main/teed/MEAS.md) XML files.
How to [build](https://github.com/joaomg/teed/blob/main/BUILD.md) teed.
## License
The teed library is licensed under:
GNU Affero General Public License v3.0
## On ETSI references and usage rights
https://www.etsi.org/intellectual-property-rights
https://www.etsi.org/images/files/IPR/etsi-ipr-policy.pdf
## Background
The teed library aims to be a comprehensive parser toolkit for telecommunications engineering data.
It's inspired by [frictionless-py](https://github.com/frictionlessdata/frictionless-py).
In fact, a production ready data pipeline can naturally glue together teed and frictionless-py.
The former extracting content from telco raw files to CSV.
And the later validating, cleaning and transforming data into a query ready system (parquet, RDBMS).
+---------------+
|Telco raw files|
| |
| .xml .asn1 |
+---------------+
|
| teed (extract)
V
+---------------+
| Tabular files |
| |
| CSV |
| Parquet |
+---------------+
|
| frictionless-py and others (clean, validate, transform, publish)
V
+---------------+
| Dataset |
| RDBMs |
| Iceberg |
| Trino |
+---------------+
Much alike to the work done by PUDL, with [Frictionless Data](https://frictionlessdata.io/) itself, in [Frictionless Public Utility Data - A Pilot Study](https://frictionlessdata.io/blog/2020/03/18/frictionless-data-pilot-study).
Take a look at [PUDL](https://github.com/catalyst-cooperative/pudl) code.