{"id":16096389,"url":"https://github.com/sio/hods","last_synced_at":"2025-04-05T20:29:40.064Z","repository":{"id":101312760,"uuid":"152886680","full_name":"sio/hods","owner":"sio","description":"Human oriented data storage (library and tools)","archived":false,"fork":false,"pushed_at":"2019-04-07T14:18:08.000Z","size":114,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-11T20:50:02.265Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://hods.ml","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-13T15:37:31.000Z","updated_at":"2019-05-21T14:00:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"40ab0269-8e89-4a50-a305-006532125c92","html_url":"https://github.com/sio/hods","commit_stats":{"total_commits":119,"total_committers":1,"mean_commits":119.0,"dds":0.0,"last_synced_commit":"cd0e8e73fe32dda80ae2060b1bc0ffe0dfd39f52"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fhods","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fhods/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fhods/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fhods/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sio","download_url":"https://codeload.github.com/sio/hods/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247398171,"owners_count":20932678,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T17:13:55.288Z","updated_at":"2025-04-05T20:29:40.039Z","avatar_url":"https://github.com/sio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Human Oriented Data Storage (HODS)\n\nDo not depend on any specific piece of software to own your data!\n\n\n## Project status\n\nWork in progress, early stages\n\n\n## Overview\n\nWorld around us generates data at crazy speeds. Thanks to advances in\ninformation technology each day more of that data becomes available to end\nusers in some machine-readable form. We rely on complex software (database\nmanagement systems) or on third parties (Facebook, Gmail) to manage and store\nthat data. When (and if) we want to *own* our data we are faced with multiple\nobstacles:\n\n- It is difficult to setup and configure an instance of database management\n  system\n- It is difficult to implement (and enforce) a reliable backup strategy\n- We become dependent on several very specific pieces of software:\n    - The database engine\n    - The application that interacts with that engine to perform basic data\n      manipulations (create-read-update-delete)\n\nFaced with that many obstacles most of us do not even bother to store\ndata of relatively low importance. And those who do are risking to lose that\ndata, which is not unexpected given such a long list of points of failure.\n\nFor the reasons stated above most of structured data floating around is\n*company*-oriented or *application*-oriented or *machine*-oriented. Not\nintended for immediate consumption by *humans*.\n\n*Making structured data more accessible to humans* is the reason this\nproject exists.\n\n## How it works\n\nTo make the idea of Human Oriented Data Storage time proof and extremely\nportable we intend to impose as little restrictions as possible. In fact, the\nwhole [specification][spec] boils down to less than a dozen data points.\nEverything else is up to the implementation and can be replaced at any moment.\n\nThe main ideas are:\n\n- **Structured data is to be stored in plain text files** serialized with some\n  markup that is readable both by humans and by machines. YAML is a very good\n  example, JSON is mostly OK too. Any other markup language is acceptable.\n- **Each data structure has to be described by a schema** to ensure data validity\n  throughout its lifecycle. Jsonschema is a good example, XSD might work for\n  some people. As with markup language, selecting the specific schema engine\n  is up to implementation.\n- **Hash sums are to be used for ensuring data integrity**. To make checksum\n  calculation implementation-independent we define the process as follows:\n    - Hash sum of the arbitrary data point is the hash sum of that data point\n      serialized into a JSON string without any formatting whitespace and\n      with all keys ordered alphabetically.\n    - Selecting the specific hash algorithm is up to the implementation.\n\nAnd that's it! To enable compatibility between different implementations the\nrequired information has to be stored in the following way (the corresponding\nJSON [schema][schemas] is available in reference implementation):\n\n```javascript\n{\n    \"info\": {\n        \"version\": string, // schema identifier for the whole document (usually URL)\n        \"schema\": {\n            \"data\": string,  // schema identifier for 'data' payload: url, filepath or name\n            \"extra\": string  // schema identifier for 'extra' payload\n        },\n        \"hashes\": {\n            \"data\": {\n                \"timestamp\": string, // ISO-8601 datetime with timezone (human-readable)\n                \"md5\": string,\n                \"sha1\": string\n                // any number of pairs \u003calgorithm name: hash value\u003e\n            },\n            \"extra\": { ... }\n        }\n    },\n    \"data\": { ... },  // 'data' payload, must correspond to the schema referenced above\n    \"extra\": { ... }  // 'extra' payload, must correspond to the schema referenced above\n    // any number of payload sections\n    // section names 'data' and 'extra' are just used in example and are not required\n}\n```\n\n\n## Reference implementation\n\nThis repository contains the reference implementation of HODS ideas in Python\nlanguage: a [library] and a [command-line application][cli].\n\nReference implementation supports:\n\n- Storing data in JSON or YAML. StrictYAML is being worked on.\n- Validating the data against JSON schema.\n- Calculating the most common hash sums: md5, sha1, sha2 family.\n\nThe list of supported technologies does not impose any restrictions on authors\nof alternative implementations and may be extended in future.\n\n\n## Installation and usage\n\nInstall latest version with pip:\n\n```\n$ pip install https://github.com/sio/hods/archive/master.zip\n```\n\nUse command line interface:\n\n```\n$ hods --help\nUsage: hods SUBCOMMAND [ARGUMENTS]\n       hods help SUBCOMMAND\n\nManage structured data stored in plain text files with YAML or JSON formatting.\n\nAvailable subcommands:\n    check\n    edit\n    new\n    rehash\n\nTo view help message for a specific subcommand use:\n    hods help SUBCOMMAND\n```\n\nIncorporate HODS into your Python projects:\n\n```python\nfrom hods import Metadata\nmeta = Metadata({'hello': 'world'})\n```\n\n\n## Support and contributing\n\nIf you need help using HODS from command line or including it into your Python\nproject, please create **[an issue](https://github.com/sio/hods/issues)**.\nIssues are also the primary venue for reporting bugs and posting feature\nrequests. General discussion related to this project is also acceptable and\nvery welcome!\n\nIn case you wish to contribute code or documentation, feel free to open **[a\npull request](https://github.com/sio/hods/pulls)**. That would certainly make\nmy day!\n\nI'm open to dialog and I promise to behave responsibly and treat all\ncontributors with respect. Please try to do the same, and treat others the way\nyou want to be treated.\n\nIf for some reason you'd rather not use the issue tracker, contacting me via\nemail is OK too. Please use a descriptive subject line to enhance visibility\nof your message. Also please keep in mind that public discussion channels are\npreferable because that way many other people may benefit from reading past\nconversations.  My email is visible under the GitHub profile and in the commit\nlog.\n\n\n## License and copyright\n\nCopyright 2018 Vitaly Potyarkin\n\n    Licensed under the Apache License, Version 2.0 (the \"License\");\n    you may not use this file except in compliance with the License.\n    You may obtain a copy of the License at\n\n        http://www.apache.org/licenses/LICENSE-2.0\n\n    Unless required by applicable law or agreed to in writing, software\n    distributed under the License is distributed on an \"AS IS\" BASIS,\n    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n    See the License for the specific language governing permissions and\n    limitations under the License.\n\n\n[cli]:     docs/commandline.md\n[library]: docs/public-api.md\n[schemas]: docs/schemas.md\n[spec]:    docs/specification.md\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsio%2Fhods","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsio%2Fhods","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsio%2Fhods/lists"}