{"id":15493822,"url":"https://github.com/peter554/pipedantic","last_synced_at":"2025-06-20T23:35:56.144Z","repository":{"id":202974987,"uuid":"571284174","full_name":"Peter554/pipedantic","owner":"Peter554","description":"Parse \u0026 validate hierarchical pipe delimited text into Pydantic models.","archived":false,"fork":false,"pushed_at":"2023-10-23T09:16:20.000Z","size":36,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-19T11:28:45.579Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Peter554.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-27T18:36:14.000Z","updated_at":"2023-10-22T19:24:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"dab3a68b-6bfe-47fb-8b51-5f0181abee14","html_url":"https://github.com/Peter554/pipedantic","commit_stats":{"total_commits":14,"total_committers":2,"mean_commits":7.0,"dds":0.1428571428571429,"last_synced_commit":"ab7bc010576fbe3ea05027188d9bdcdedff51925"},"previous_names":["peter554/pipedantic"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Peter554/pipedantic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter554%2Fpipedantic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter554%2Fpipedantic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter554%2Fpipedantic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter554%2Fpipedantic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Peter554","download_url":"https://codeload.github.com/Peter554/pipedantic/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter554%2Fpipedantic/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261037141,"owners_count":23100933,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T08:09:32.210Z","updated_at":"2025-06-20T23:35:51.133Z","avatar_url":"https://github.com/Peter554.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pipedantic\n\n[![CI](https://github.com/Peter554/pipedantic/actions/workflows/ci.yml/badge.svg)](https://github.com/Peter554/pipedantic/actions/workflows/ci.yml)\n\nParse \u0026 validate hierarchical pipe delimited text into [Pydantic](https://github.com/pydantic/pydantic) models.\n\nA bit rough and hacky at the moment, mostly here just for fun and interest. Use with caution!\n\n## What the heck is hierarchical pipe delimited text?\n\nIt's easiest to explain with an example. For example, suppose we have `User`s, each with a `name` (string) and an `age` (int). \nAnd suppose those `User`s have `Comment`s. The comments have a `posted_at` (date) and a `text` (string).\n\nWe could then represent those `User`s and their `Comment`s in a hierarchical pipe delimited text file as so:\n\n```txt\n01|Holly|16|\n02|2022-01-01|Awesome!|\n01|Andy|24|\n02|2022-02-02|Wicked!|\n02|2022-03-03|Sweet!|\n```\n\nHere the lines starting with \"01\" represent a `User`, and the lines beneath starting with \"02\" represent that `User`s `Comment`s.\nIn JSON, this would correspond to the hierarchical data:\n\n```json\n{\n  \"users\": [\n    {\n      \"name\": \"Holly\",\n      \"age\": 16,\n      \"comments\": [\n        {\n          \"posted_at\": \"2022-01-01\",\n          \"text\": \"Awesome!\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Andy\",\n      \"age\": 24,\n      \"comments\": [\n        {\n          \"posted_at\": \"2022-02-02\",\n          \"text\": \"Wicked!\"\n        },\n        {\n          \"posted_at\": \"2022-03-03\",\n          \"text\": \"Sweet!\"\n        }\n      ]\n    }\n  ]\n}\n```\n\nWe could codify this spec and parse the pipe delimited text into Pydantic models using pipedantic!\nA spec might look like this:\n\n```py\nclass Comment(pydantic.BaseModel):\n    posted_at: datetime.date\n    text: str\n\nclass User(pydantic.BaseModel):\n    name: str\n    age: int\n    comments: list[Comment]\n\nclass Root(pydantic.BaseModel):\n    users: list[User]\n```\n\nAt which point we're ready to parse the file:\n\n```py\nparser = PipeDelimitedFileParser[Root](\n    root_model=Root,\n    line_models={\n        \"01\": User,\n        \"02\": Comment,\n    },\n)\n\nwith open(\"data\") as f:\n    data = parser.parse(file=f)\n```\n\nA `FileParseError` will be raised if the file is invalid, which contains error details and the line number of the error.\nIf the file is valid the return type will be `Root`. \n\nSee [example.py](/example.py) and also the tests for more.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeter554%2Fpipedantic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeter554%2Fpipedantic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeter554%2Fpipedantic/lists"}