{"id":21502233,"url":"https://github.com/mrtcode/treedb","last_synced_at":"2025-10-28T23:02:54.567Z","repository":{"id":76796168,"uuid":"83219211","full_name":"mrtcode/treedb","owner":"mrtcode","description":"A database for keeping, navigating and manipulating big trees","archived":false,"fork":false,"pushed_at":"2017-02-26T15:42:45.000Z","size":54,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-17T13:46:37.642Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mrtcode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-02-26T15:30:02.000Z","updated_at":"2024-03-15T03:18:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"8f379ecd-1fa0-4191-8072-c04c606e0836","html_url":"https://github.com/mrtcode/treedb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mrtcode/treedb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrtcode%2Ftreedb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrtcode%2Ftreedb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrtcode%2Ftreedb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrtcode%2Ftreedb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mrtcode","download_url":"https://codeload.github.com/mrtcode/treedb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrtcode%2Ftreedb/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269356322,"owners_count":24403505,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-08T02:00:09.200Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T18:14:09.164Z","updated_at":"2025-10-28T23:02:54.258Z","avatar_url":"https://github.com/mrtcode.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"\r\n\r\nA database for keeping, navigating and manipulating big trees.\r\n\r\nThe initial idea was to make an 'intelligent' in memory living tree that constantly analyzes itself\r\nand gives user, for example, auto suggestions, or\r\nautomatically relates data from different tree parts, or calculates something. \r\n\r\nThe OctopusNote concept changed a lot, was simplified and currently\r\nis implemented purely in JavaScript on Node.js.\r\nIt uses a totally different approach.\r\nLimited in size, but still very efficient because the most heavy work is done by JSON.parse and JSON.stringify.\r\nAlso much easier to scale.\r\nGoing to opensource it too in the near future.\r\n\r\nNow this database isn't very useful in its current condition,\r\nbut can be useful as a base for new projects that:\r\n\r\n* use asynchronous event loops based on Libevent\r\n* implement redis subscribe in an event loop\r\n* use Jansson for JSON stringifying and parsing\r\n* use Sqlite full text search and content-less tables\r\n* use red-black trees to index data\r\n* keep incoming operations in queues, distribute to corresponding threads\r\n* import xml files\r\n* use very large tree structures\r\n\r\nThere are trees and tree views. In RAM only the tree structure is stored.\r\nThe actual data is stored on a hard drive.\r\nView is just another tree structure that 'overlays' some area of the actual tree.\r\n\r\nEvery connected client 'sees' the tree through its view.\r\nWhen a client requests for a specific part of the tree, \r\nthe view overlays it and loads branch data from a hard drive.\r\n\r\nThis way a user can navigate in any size tree, remotely, from a limited speed connection, limited memory resources (i.e. web browser).\r\n\r\nBut in practice, for a normal user its not the case.\r\nAn average tree size for a normal user is less than 1Mb.\r\nBut this concept could be very useful if working with very big trees.\r\n\r\nAlso the idea of OctopusNote was to keep only 300 bytes per branch.\r\nIn this case storing raw data in a file with fixed length blocks would allow to\r\neasily read and write to any block just by calculating its offset.\r\n\r\nIt would be a trade-off between CPU overhead (using a traditional db engine) and storage overhead (in this case).\r\n\r\nBut still economically viable because storage is cheap, and CPU resources aren't.\r\n\r\nBut after removing the limit, a traditional database engine was used.\r\n\r\nTwo databases were tried:\r\n* Sqlite\r\n* [ForestDB](https://github.com/couchbase/forestdb)\r\n\r\nAnother important feature is synchronization.\r\nWhile each client (i.e. each browser tab) creates a new view and each view can insert,\r\nmove or delete branches, it's important to keep them synchronized with the main tree.\r\nThis is quite complex. For example a 'move' operation can trigger changes in many views.\r\nAnd then each view have to pass the changes to its browser tab.\r\n\r\nWhat a user sees in a browser is an identical copy of a view in a server.\r\nEach change that happened in a browser is synchronized with its view in a server.\r\nThe recently updated view is synchronized with the main tree and with all other views.\r\n\r\n```            \r\n                            -------------------------------------- \r\n                           |                                      |\r\n-------------+  --------\u003e  +-----------+                          |\r\nbrowser1 tree|             |server view|                          |\r\n-------------+  \u003c--------  +-----------+                          |\r\n                           |                                      |\r\n-------------+  --------\u003e  +-----------+                          |\r\nbrowser2 tree|             |server view|          TREE            |\r\n-------------+  \u003c--------  +-----------+                          |\r\n                           |                                      |\r\n-------------+  --------\u003e  +-----------+                          |\r\nbrowser3 tree|             |server view|                          |\r\n-------------+  \u003c--------  +-----------+                          |\r\n                           |                                      |\r\n                           ----------------------------------------\r\n```\r\n\r\nThe prototype of this database was successfuly finished and used for a while, but then replaced with much more simpler, practical and scalable system made in JavaScript.\r\n\r\nIf you need some kind of exotic database designed for a specific purpose, contact me.\r\n\r\n## Build\r\n\r\n```\r\napt-get install libevent-dev libsqlite3-dev libhiredis-dev libjansson-dev libxml2-dev\r\n\r\ngit clone https://github.com/mrtcode/treedb\r\ncd treedb\r\ncmake .\r\nmake\r\n```\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrtcode%2Ftreedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrtcode%2Ftreedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrtcode%2Ftreedb/lists"}