{"id":18698860,"url":"https://github.com/qri-io/deepdiff","last_synced_at":"2025-04-12T07:33:14.503Z","repository":{"id":52219377,"uuid":"170623228","full_name":"qri-io/deepdiff","owner":"qri-io","description":"structured data differ with near-linear time complexity","archived":false,"fork":false,"pushed_at":"2021-05-04T20:00:33.000Z","size":1920,"stargazers_count":16,"open_issues_count":2,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-26T03:11:18.254Z","etag":null,"topics":["diff","golang","json"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qri-io.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-14T03:48:55.000Z","updated_at":"2024-03-17T11:28:19.000Z","dependencies_parsed_at":"2022-08-28T17:40:53.212Z","dependency_job_id":null,"html_url":"https://github.com/qri-io/deepdiff","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qri-io%2Fdeepdiff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qri-io%2Fdeepdiff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qri-io%2Fdeepdiff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qri-io%2Fdeepdiff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qri-io","download_url":"https://codeload.github.com/qri-io/deepdiff/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248534411,"owners_count":21120322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diff","golang","json"],"created_at":"2024-11-07T11:30:01.140Z","updated_at":"2025-04-12T07:33:13.348Z","avatar_url":"https://github.com/qri-io.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# deepdiff\n[![Qri](https://img.shields.io/badge/made%20by-qri-magenta.svg?style=flat-square)](https://qri.io)\n[![GoDoc](https://godoc.org/github.com/qri-io/deepdiff?status.svg)](http://godoc.org/github.com/qri-io/deepdiff)\n[![License](https://img.shields.io/github/license/qri-io/deepdiff.svg?style=flat-square)](./LICENSE)\n[![Codecov](https://img.shields.io/codecov/c/github/qri-io/deepdiff.svg?style=flat-square)](https://codecov.io/gh/qri-io/deepdiff)\n[![CI](https://img.shields.io/circleci/project/github/qri-io/deepdiff.svg?style=flat-square)](https://circleci.com/gh/qri-io/deepdiff)\n[![Go Report Card](https://goreportcard.com/badge/github.com/qri-io/deepdiff)](https://goreportcard.com/report/github.com/qri-io/deepdiff)\n\ndeepdiff is a structured data differ that aims for near-linear time complexity. It's intended to calculate differences \u0026 apply patches to structured data ranging from  0-500MBish of encoded JSON.\n\nDiffing structured data carries additional complexity when compared to the standard unix diff utility, which operates on lines of text. By using the structure of data itself, deepdiff is able to provide a rich description of changes that maps onto the structure of the data itself. deepdiff ignores semantically irrelevant changes like whitespace, and can isolate changes like column changes to tabular data to only the relevant switches\n\nMost algorithms in this space have quadratic time complexity, which from our testing makes them very slow on 3MB JSON documents and unable to complete on 5MB or more. deepdiff currently hovers around the 0.9Sec/MB range on 4 core processors\n\nInstead of operating on JSON directly, deepdiff operates on document trees consisting of the go types created by unmarshaling from JSON, which are two complex types:\n```\n  map[string]interface{}\n  []interface{}\n```\nand five scalar types:\n```\n  string, int, float64, bool, nil\n```\n\nBy operating on native go types deepdiff can compare documents encoded in different formats, for example decoded CSV or CBOR.\n\ndeepdiff is based off an algorithm designed for diffing XML documents outlined in\n[_Detecting Changes in XML Documents by Grégory Cobéna \u0026 Amélie Marian_](https://ieeexplore.ieee.org/document/994696)\n\nIt's been adapted to fit purposes of diffing for Qri: https://github.com/qri-io/qri, folding in parallelism primitives afforded by the go language\n\ndeepdiff also includes a tool for applying patches, see documentation for details.\n\n## Project Status:\n\n:construction_worker_woman: :construction_worker_man: This is a very new project that hasn't been properly vetted in testing enviornments. Issues/PRs welcome \u0026 appriciated. :construction_worker_woman: :construction_worker_man:\n\n## Benchmarks\n\nRun on a 4 core MacBook Pro:\n\n```\n$ go test -bench . --run XXX -v --benchmem\ngoos: darwin\ngoarch: amd64\npkg: github.com/qri-io/deepdiff\nBenchmarkDiff1-4          \t   20000\t     88167 ns/op\t   13324 B/op\t     496 allocs/op\nBenchmarkDiffDatasets-4   \t    5000\t    241119 ns/op\t   53367 B/op\t    1614 allocs/op\nBenchmarkDiff5MB-4        \t       1\t4357009141 ns/op\t783217944 B/op\t29952860 allocs/op\nPASS\nok  \tgithub.com/qri-io/deepdiff\t8.369s\n```\n\n\n### Getting Involved\n\nWe would love involvement from more people! If you notice any errors or would like to submit changes, please see our\n[Contributing Guidelines](./.github/CONTRIBUTING.md).\n\n\n## Basic Usage\n\nHere's a quick example pulled from the [godoc](https://godoc.org/github.com/qri-io/deepdiff):\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n  \"fmt\"\n  \n  \"github.com/qri-io/deepdiff\"\n)\n\n// start with two slightly different json documents\nvar aJSON = []byte(`{\n  \"a\": 100,\n  \"foo\": [1,2,3],\n  \"bar\": false,\n  \"baz\": {\n    \"a\": {\n      \"b\": 4,\n      \"c\": false,\n      \"d\": \"apples-and-oranges\"\n    },\n    \"e\": null,\n    \"g\": \"apples-and-oranges\"\n  }\n}`)\n\nvar bJSON = []byte(`{\n  \"a\": 99,\n  \"foo\": [1,2,3],\n  \"bar\": false,\n  \"baz\": {\n    \"a\": {\n      \"b\": 5,\n      \"c\": false,\n      \"d\": \"apples-and-oranges\"\n    },\n    \"e\": \"thirty-thousand-something-dogecoin\",\n    \"f\": false\n  }\n}`)\n\nfunc main() {\n\t// unmarshal the data into generic interfaces\n\tvar a, b interface{}\n\tif err := json.Unmarshal(aJSON, \u0026a); err != nil {\n\t\tpanic(err)\n\t}\n\tif err := json.Unmarshal(bJSON, \u0026b); err != nil {\n\t\tpanic(err)\n\t}\n\n\t// Diff will use default configuration to produce a slice of Changes\n\t// by default Diff will not generate update change only inserts \u0026 deletes\n\tdiffs, err := deepdiff.Diff(a, b)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\t// Format the changes for terminal output\n\tchange, err := FormatPretty(diffs)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tfmt.Println(change)\n\t// Output: baz:\n\t//   + f: false\n\t//   - g: \"apples-and-oranges\"\n\t//   a:\n\t//     ~ b: 5\n\t//   ~ e: \"thirty-thousand-something-dogecoin\"\n\t// ~ a: 99\n}\n```\n\n## License\n\nThe deepdiff library is licensed under the [GNU Lesser General Public License v3.0](https://www.gnu.org/licenses/lgpl-3.0.en.html)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqri-io%2Fdeepdiff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqri-io%2Fdeepdiff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqri-io%2Fdeepdiff/lists"}