{"id":16651396,"url":"https://github.com/twardoch/yaplon","last_synced_at":"2025-09-03T16:31:32.223Z","repository":{"id":57478012,"uuid":"199175925","full_name":"twardoch/yaplon","owner":"twardoch","description":"Python 3-based commandline converter YAML ↔ JSON ↔ PLIST","archived":false,"fork":false,"pushed_at":"2022-06-10T05:54:41.000Z","size":84,"stargazers_count":12,"open_issues_count":2,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-24T21:00:44.702Z","etag":null,"topics":["cli","convert","json","plist","python","yaml"],"latest_commit_sha":null,"homepage":"https://twardoch.github.io/yaplon/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/twardoch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"custom":["https://paypal.me/adamtwar"]}},"created_at":"2019-07-27T14:36:06.000Z","updated_at":"2023-10-20T10:06:39.000Z","dependencies_parsed_at":"2022-09-10T08:22:57.805Z","dependency_job_id":null,"html_url":"https://github.com/twardoch/yaplon","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twardoch%2Fyaplon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twardoch%2Fyaplon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twardoch%2Fyaplon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twardoch%2Fyaplon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/twardoch","download_url":"https://codeload.github.com/twardoch/yaplon/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231893694,"owners_count":18442043,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","convert","json","plist","python","yaml"],"created_at":"2024-10-12T09:25:12.015Z","updated_at":"2025-09-03T16:31:32.209Z","avatar_url":"https://github.com/twardoch.png","language":"Python","funding_links":["https://paypal.me/adamtwar","https://www.paypal.me/adamtwar","https://github.com/sponsors/twardoch"],"categories":[],"sub_categories":[],"readme":"# Yaplon: Yet Another Plist/JSON/YAML/XML/CSV Converter\n\nYaplon is a versatile command-line tool for converting between various common structured data formats: JSON, YAML, Property Lists (PLIST, both binary and XML), XML, and CSV (read-only for CSV input). It's designed for developers, data analysts, system administrators, or anyone who needs to quickly and easily transform data from one format to another.\n\n## Why Yaplon?\n\nIn a world of diverse applications and services, data often needs to be exchanged and processed in different formats. Yaplon simplifies this by providing:\n\n*   **Unified Interface:** A single tool to handle multiple conversion types, reducing the need to learn and install separate utilities for each pair of formats.\n*   **Command-Line Power:** Easily integrate Yaplon into your scripts and workflows using standard input/output piping.\n*   **Flexibility:** Options to sort data (e.g., dictionary keys) before conversion for consistent output, and to minify the output for space efficiency.\n*   **Format-Specific Features:** Handles nuances like JSON with comments, binary data in Plists and YAML, different XML structuring approaches, and various CSV dialects.\n*   **Python 3.9+:** Modern Python codebase.\n\n## Installation\n\n### From PyPI (Recommended)\n\nInstall the latest stable release from the Python Package Index (PyPI):\n\n```bash\npip3 install --user --upgrade yaplon\n```\n\n### Binary Downloads\n\nDownload pre-built binaries for your platform from the [releases page](https://github.com/twardoch/yaplon/releases):\n\n- **Linux**: `yaplon-{version}-linux-x64`\n- **Windows**: `yaplon-{version}-windows-x64.exe`\n- **macOS**: `yaplon-{version}-macos-x64`\n\nMake the binary executable and add it to your PATH.\n\n### From GitHub (Development Version)\n\nTo install the latest development version directly from GitHub:\n\n```bash\npip3 install --user --upgrade git+https://github.com/twardoch/yaplon\n```\n\n### For Development\n\nIf you want to contribute to Yaplon or modify it:\n\n1.  **Clone the repository:**\n    ```bash\n    git clone https://github.com/twardoch/yaplon.git\n    cd yaplon\n    ```\n2.  **Install development dependencies:**\n    ```bash\n    make install-dev\n    ```\n    This installs the package in editable mode with all development dependencies.\n\nFor detailed build and development instructions, see [BUILD.md](./BUILD.md).\nFor GitHub Actions setup, see [GITHUB_SETUP.md](./GITHUB_SETUP.md).\n\n## Usage\n\nYaplon can be invoked using the main `yaplon` command followed by a conversion subcommand (e.g., `j2y` for JSON to YAML), or by using dedicated shortcut scripts (e.g., `json22yaml`).\n\n### Main Command Structure\n\n```bash\nyaplon \u003ccommand\u003e -i \u003cinput_file\u003e -o \u003coutput_file\u003e [options]\n```\n\n*   `\u003ccommand\u003e`: Specifies the conversion type (see \"Conversion Commands\" below).\n*   `-i \u003cinput_file\u003e`: Input file. If omitted or specified as `-`, Yaplon reads from standard input (stdin).\n*   `-o \u003coutput_file\u003e`: Output file. If omitted or specified as `-`, Yaplon writes to standard output (stdout). This allows for easy piping.\n\n### Conversion Commands\n\nThe following commands define the input and output formats:\n\n*   `c2j`: CSV to JSON\n*   `c2p`: CSV to PLIST\n*   `c2x`: CSV to XML\n*   `c2y`: CSV to YAML\n*   `j2p`: JSON to PLIST\n*   `j2x`: JSON to XML\n*   `j2y`: JSON to YAML\n*   `p2j`: PLIST to JSON\n*   `p2x`: PLIST to XML\n*   `p2y`: PLIST to YAML\n*   `x2j`: XML to JSON\n*   `x2p`: XML to PLIST\n*   `x2y`: XML to YAML\n*   `y2j`: YAML to JSON\n*   `y2p`: YAML to PLIST\n*   `y2x`: YAML to XML\n\nYaplon supports JSON input with C-style comments (`// ...` and `/* ... */`) and trailing commas in objects/arrays (similar to JSON5).\n\n### General Options\n\nThese options are available for most conversion commands:\n\n*   `-i \u003cinput_file\u003e, --in \u003cinput_file\u003e`: Specify the input file. Defaults to stdin.\n*   `-o \u003coutput_file\u003e, --out \u003coutput_file\u003e`: Specify the output file. Defaults to stdout.\n*   `-s, --sort`: Sort data before conversion. For dictionaries/maps, keys are sorted alphabetically. This helps in producing consistent output, especially for version control.\n*   `-m, --mini`: Minify output. The specific effect depends on the format (e.g., no indents or newlines in JSON/XML, flow style in YAML).\n\n### Format-Specific Options\n\n*   **For `*2p` (to Plist):**\n    *   `-b, --bin`: Output a binary Plist file. By default, an XML Plist is generated.\n*   **For `p2j`, `y2j` (from Plist/YAML to JSON):**\n    *   `-b, --bin`: Preserve binary data (e.g., from Plist `\u003cdata\u003e` or YAML `!!binary`) as a simple base64 encoded string in JSON. The default behavior is to represent binary data as a special dictionary: `{\"__bytes__\": true, \"base64\": \"...\"}`.\n*   **For `*2x` (to XML):**\n    *   `-R \u003cname\u003e, --root \u003cname\u003e`: Specify the root tag name for the XML document. This is particularly useful if the input data is a list or if you want to override the default root tag (which is often 'root' or derived from a single top-level key). This option uses the `xmltodict` backend for XML generation.\n    *   `-t \u003cname\u003e, --tag \u003cname\u003e`: Wrap the entire output in the specified tag. This option uses the `dict2xml` backend, which may produce a simpler XML structure, especially for lists. If `-t` is used, the `-R` option is ignored.\n*   **For `x2*` (from XML):**\n    *   `-N, --namespaces`: Process XML namespaces. If enabled, tag names in the resulting data structure will include namespace prefixes.\n*   **For `c2*` (from CSV):**\n    *   `-H, --header`: Treat the first row of the CSV as a header row. The CSV data will be read as a list of dictionaries (where keys are header names) instead of a list of lists.\n    *   `-d \u003cdialect\u003e, --dialect \u003cdialect\u003e`: Specify the CSV dialect (e.g., 'excel', 'excel-tab', 'unix'). If not specified, Yaplon attempts to sniff the dialect.\n    *   `-k \u003ckey_index\u003e, --key \u003ckey_index\u003e`: (Requires `-H` or implies it) Use the values from the column specified by `key_index` (a 1-based integer) as keys for a top-level dictionary. The values of this dictionary will be the row dictionaries (with the key column itself removed from the row dictionary).\n\n### Dedicated CLI Tools\n\nFor convenience, Yaplon also installs shortcut scripts that correspond directly to the conversion commands:\n\n*   `csv22json`, `csv22plist`, `csv22xml`, `csv22yaml`\n*   `json22plist`, `json22xml`, `json22yaml`\n*   `plist22json`, `plist22xml`, `plist22yaml`\n*   `xml22json`, `xml22plist`, `xml22yaml`\n*   `yaml22json`, `yaml22plist`, `yaml22xml`\n\nNote: These scripts use `22` (e.g., `json22yaml`) instead of `2` to avoid potential conflicts with other similarly named conversion tools you might have installed.\n\n### Examples\n\n**JSON to YAML (file to file using dedicated tool):**\n\n```bash\njson22yaml -i input.json -o output.yaml\n```\n\n**JSON to YAML (using pipes and main `yaplon` command):**\n\n```bash\ncat input.json | yaplon j2y \u003e output.yaml\n```\n\n**PLIST to JSON (minified, preserving binary as base64 strings):**\n\n```bash\nyaplon p2j -m -b -i input.plist -o output.json\n```\n\n**CSV with header to XML (using a specific column as key and custom root tag):**\n\n```bash\nyaplon c2x -H -k 1 --root \"Entries\" -i data.csv \u003e data.xml\n```\n\n## Technical Details\n\nThis section describes the internal workings of Yaplon.\n\n### Core Architecture\n\nYaplon's conversion process generally follows these steps:\n\n1.  **Input:** Data is read from an input file or stdin.\n2.  **Parsing (`reader.py`):** The input data is parsed into a standardized Python data structure, primarily using `collections.OrderedDict`. This preserves key order from the source format where applicable.\n    *   The `reader.sort_ordereddict` function can be invoked via the `-s`/`--sort` CLI option to recursively sort these `OrderedDict`s by key.\n3.  **Serialization (`writer.py`):** The Python `OrderedDict` object is then serialized into the target output format.\n4.  **Output:** The resulting data is written to an output file or stdout.\n\nThe Command Line Interface (CLI) is built using the [Click](https://click.palletsprojects.com/) library.\n\n### Format-Specific Handling\n\n*   **JSON:**\n    *   **Reading:** Handled by `yaplon.ojson.read_json`. This module supports JSON5-like features such as C-style comments (`//`, `/* */`) and trailing commas. These are stripped by `yaplon.file_strip.json.sanitize_json` before parsing with Python's standard `json` module (using `object_pairs_hook=OrderedDict`).\n    *   **Writing:** Handled by `yaplon.ojson.json_dump`. Can produce regular or minified JSON. Binary data is handled based on the `-b` flag (see \"Binary Data Representation\" below).\n*   **PLIST (Property List):**\n    *   **Reading:** Handled by `yaplon.oplist.read_plist`, which wraps Python's standard `plistlib`. It parses both XML and binary Plists. Plist `\u003cdata\u003e` elements are converted to Python `bytes` objects, and `\u003cdate\u003e` elements become `datetime.datetime` objects.\n    *   **Writing:** Handled by `yaplon.oplist.plist_dumps` (for XML Plist) and `yaplon.oplist.plist_binary_dumps` (for binary Plist, via `-b` flag). These also use `plistlib`.\n*   **YAML:**\n    *   **Reading:** Handled by `yaplon.oyaml.read_yaml`, which uses the [PyYAML](https://pyyaml.org/) library. It employs custom constructors to load YAML `!!binary` tags into Python `bytes` objects and `!!timestamp` tags into `datetime.datetime` objects. Data is loaded into `OrderedDict` to preserve order.\n    *   **Writing:** Handled by `yaplon.oyaml.yaml_dumps`. Uses a custom dumper for `OrderedDict` to maintain key order and for `bytes` to serialize them as `!!binary` tags. Supports minified (flow style) output with the `-m` flag.\n*   **XML:**\n    *   **Reading:** Handled by `yaplon.reader.xml`, which uses the [xmltodict](https://github.com/martinblech/xmltodict) library to parse XML into an `OrderedDict`. Namespace processing can be enabled with the `-N` flag.\n    *   **Writing:** Handled by `yaplon.writer.xml`. This function is noted as potentially \"primitive and buggy\" for complex cases. It can use two different backends:\n        *   **`xmltodict.unparse`:** Used by default or when the `-R`/`--root` option is specified. This backend is generally more feature-rich for representing complex data structures.\n        *   **`dict2xml.Converter`:** Used when the `-t`/`--tag` option is specified. This backend might produce simpler XML, especially for lists, and wraps the entire structure in a single user-defined tag.\n        *   Before writing, Python `bytes` are converted to base64 encoded strings, and `datetime.datetime` objects are converted to ISO 8601 strings.\n*   **CSV:**\n    *   **Reading:** Handled by `yaplon.reader.csv` using Python's standard `csv` module.\n        *   Can read as a list of lists or, if `-H`/`--header` is used, a list of `OrderedDict`s.\n        *   Supports dialect sniffing or explicit dialect specification (`-d`).\n        *   The `-k`/`--key` option allows restructuring the data into a top-level `OrderedDict` keyed by values from a specified column.\n    *   **Writing:** CSV writing is not currently supported. Yaplon is read-only for CSV input.\n\n### Data Representation\n\n*   **Internal Structure:** `collections.OrderedDict` is used as the primary internal data structure after parsing to ensure that key order from formats like YAML or sorted JSON is preserved and can be maintained in the output.\n*   **Binary Data:**\n    *   **In Python:** Represented as `bytes` objects.\n    *   **PLIST:** Native `\u003cdata\u003e` tags.\n    *   **YAML:** `!!binary` tag.\n    *   **JSON Output:**\n        *   Default: `{\"__bytes__\": true, \"base64\": \"...\"}` (a dictionary indicating binary content).\n        *   With `-b` flag (for `p2j`, `y2j`): A simple base64 encoded string.\n    *   **XML Output:** Base64 encoded string content within tags.\n\n### Extensibility (Conceptual)\n\nWhile not a formalized plugin system, adding support for a new format would conceptually involve:\n\n1.  Creating new reader and writer functions in `yaplon/reader.py` and `yaplon/writer.py` respectively.\n2.  The reader should parse the new format into an `OrderedDict` (or list of `OrderedDict`s, etc.).\n3.  The writer should take an `OrderedDict` and serialize it to the new format.\n4.  Adding corresponding CLI commands and options in `yaplon/__main__.py`.\n5.  Writing comprehensive tests for the new conversions.\n\n## Contributing\n\nContributions to Yaplon are welcome! Whether it's bug reports, feature suggestions, documentation improvements, or code contributions, your help is appreciated.\n\n### Development Setup\n\nPlease refer to the \"Installation \u003e For Development\" section above for instructions on how to clone the repository and set up a development environment.\n\n### Workflow\n\n1.  **Fork the repository** on GitHub.\n2.  **Create a new branch** for your feature or bug fix: `git checkout -b my-feature-branch`.\n3.  **Set up development environment:**\n    ```bash\n    make install-dev\n    ```\n4.  **Make your changes.**\n5.  **Write tests** for your changes in the `tests/` directory.\n6.  **Run development workflow:**\n    ```bash\n    make dev  # format + lint + test-fast\n    ```\n7.  **Run full test suite:**\n    ```bash\n    make test\n    ```\n8.  **Commit your changes** with a clear and descriptive commit message.\n9.  **Push your branch** to your fork: `git push origin my-feature-branch`.\n10. **Open a Pull Request** against the `master` branch of the `twardoch/yaplon` repository.\n\n### Build System\n\nThe project uses a modern build system with automated testing and releases:\n\n- **Local Development**: `make dev` for quick development workflow\n- **Testing**: `make test` for full test suite, `make test-fast` for quick tests\n- **Building**: `make build` for packages, `make build-binary` for binaries\n- **Releasing**: `python scripts/release.py --bump patch --message \"msg\"`\n\nFor detailed information, see [BUILD.md](./BUILD.md) and [USAGE.md](./USAGE.md).\n\n### Issue Tracking\n\n*   Please report bugs or suggest features by opening an issue on the [GitHub Issues page](https://github.com/twardoch/yaplon/issues).\n*   Check the `TODO.md` file in the repository for a list of known tasks and planned enhancements.\n\n### Changelog\n\nFor significant changes, please add an entry to the `CHANGELOG.md` file, describing the change and referencing the relevant issue or PR if applicable.\n\n### Code Style\n\nYaplon uses [Black](https://github.com/psf/black) for code formatting and [Flake8](https://flake8.pycqa.org/en/latest/) for linting. Please ensure your contributions adhere to these standards by running `make format` and `make lint`.\n\n### Testing\n\nThe project uses [pytest](https://docs.pytest.org/) for testing. New features should include corresponding tests, and bug fixes should ideally include a test that demonstrates the bug and verifies the fix.\n\n## Changelog\n\nSee [CHANGELOG.md](./CHANGELOG.md) for a history of changes to the project.\n\n## License\n\nYaplon is distributed under the [MIT License](./LICENSE).\n\n- Copyright (c) 2021-2024 Adam Twardoch \u003cadam+github@twardoch.com\u003e \u0026 Jules (AI Agent)\n- Copyright (c) 2012-2015 Isaac Muse \u003cisaacmuse@gmail.com\u003e\n- Based on [Serialized Data Converter for Sublime Text](https://github.com/facelessuser/SerializedDataConverter)\n\n## Links\n\n*   **Project Homepage:** [https://twardoch.github.io/yaplon/](https://twardoch.github.io/yaplon/)\n*   **Python Package on PyPI:** [https://pypi.org/project/yaplon/](https://pypi.org/project/yaplon/)\n*   **Source on GitHub:** [https://github.com/twardoch/yaplon](https://github.com/twardoch/yaplon)\n*   **Donate:** Support the project via [PayPal](https://www.paypal.me/adamtwar) or [GitHub Sponsors](https://github.com/sponsors/twardoch). (Check `.github/FUNDING.yml`)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwardoch%2Fyaplon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftwardoch%2Fyaplon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwardoch%2Fyaplon/lists"}