{"id":14064994,"url":"https://github.com/ServerlessSam/data-file-merge","last_synced_at":"2025-07-29T19:33:52.979Z","repository":{"id":58406167,"uuid":"511648641","full_name":"ServerlessSam/data-file-merge","owner":"ServerlessSam","description":"Data-file-merge (dfm) allows you to merge JSON files into a destination file using a set of rules defined in a configuration file","archived":false,"fork":false,"pushed_at":"2024-02-25T20:08:23.000Z","size":187,"stargazers_count":14,"open_issues_count":10,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-30T16:41:23.951Z","etag":null,"topics":["json","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ServerlessSam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-07-07T19:20:56.000Z","updated_at":"2024-11-04T22:16:08.000Z","dependencies_parsed_at":"2024-02-24T21:25:39.813Z","dependency_job_id":"af99b950-1037-4a3b-b6b9-da6515d2d524","html_url":"https://github.com/ServerlessSam/data-file-merge","commit_stats":{"total_commits":153,"total_committers":5,"mean_commits":30.6,"dds":"0.13725490196078427","last_synced_commit":"b43ab58905aaf2393ee2a6bf4717b90768aeb52c"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServerlessSam%2Fdata-file-merge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServerlessSam%2Fdata-file-merge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServerlessSam%2Fdata-file-merge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ServerlessSam%2Fdata-file-merge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ServerlessSam","download_url":"https://codeload.github.com/ServerlessSam/data-file-merge/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228045987,"owners_count":17861073,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","python"],"created_at":"2024-08-13T07:04:13.556Z","updated_at":"2024-12-04T04:30:34.775Z","avatar_url":"https://github.com/ServerlessSam.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# data-file-merge\n\nThis python module/CLI allows you to merge multiple JSON files into a single destination JSON file using a set of rules defined in the [dfm config file](https://github.com/ServerlessSam/data-file-merge/wiki/Configuration)\n\nPlease refer to the [project wiki](https://github.com/ServerlessSam/data-file-merge/wiki) for complete documentation including [usage](https://github.com/ServerlessSam/data-file-merge/wiki/Usage) and [configuration file syntax](https://github.com/ServerlessSam/data-file-merge/wiki/Configuration).\n\n# Hello World Example\n\nYou can install the CLI for your operating system from [GitHub here](https://github.com/ServerlessSam/data-file-merge/releases).\n\nUsing the following config file:\n```json\n{\n    \"SourceFiles\" : [\n        {\n            \"SourceFileLocation\" : {\n                \"Path\" : \"foo/bar.json\"\n            },\n            \"SourceFileNode\" : \"$\",\n            \"DestinationFileNode\" : \"$\"\n        },\n        {\n            \"SourceFileLocation\" : {\n                \"Path\" : \"foo/baz.json\"\n            },\n            \"SourceFileNode\" : \"$\",\n            \"DestinationFileNode\" : \"$\"\n        },\n    ],\n    \"DestinationFile\" : {\n        \"DestinationFileLocation\" : {\n            \"Path\" : \"hello/world.json\"\n        }\n    }\n}\n```\n...you are able to merge the content of `foo/bar.json` and `foo/baz.json` into `hello/world.json` (this can either be an existing or new file). The `$`s indicate that we are merging from the root node of both `foo/*.json` files to the root of `hello/world.json`.\nThis merge can be achieved with the CLI command `dfm merge --root-path \u003cpath to 'foo' directory\u003e \u003cpath to this config file\u003e`.\nThe specific rules followed by the merging logic can be found in [the wiki](https://github.com/ServerlessSam/data-file-merge/wiki/Merging-Logic).\n\nMore examples can be found in the [dfm example repo](https://github.com/ServerlessSam/dfm-examples).\n\n## Considerations\n\n* Currently only JSON files are supported but YAML files could be supported with ease in future.\n* Doing the reverse operation will soon be supported (splitting a single sourcefile into multiple destination files).\n* You should be aware of:\n  * [json-path's dollar-notation syntax](https://pypi.org/project/jsonpath-ng/)\n  * [path-lib's path syntax](https://docs.python.org/3/library/pathlib.html)\n  * [regex syntax](https://www.rexegg.com/regex-quickstart.html)\n\n## Development\n\n* install pre-commit hooks: ``poetry run pre-commit install``\n* check and fix linting issues: ``make format``\n\n## Contributing to the project\n\nYou are welcome to help us rid the world of monolithic data files! Reach out to [ServerlessSam](https://github.com/ServerlessSam) to either find some tickets to complete or bring up any ideas you have.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FServerlessSam%2Fdata-file-merge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FServerlessSam%2Fdata-file-merge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FServerlessSam%2Fdata-file-merge/lists"}