{"id":13628940,"url":"https://github.com/Softmotions/iowow","last_synced_at":"2025-04-17T04:32:36.465Z","repository":{"id":36313480,"uuid":"40618114","full_name":"Softmotions/iowow","owner":"Softmotions","description":"A C utility library and persistent key/value storage engine","archived":false,"fork":false,"pushed_at":"2024-09-17T15:52:18.000Z","size":3891,"stargazers_count":295,"open_issues_count":3,"forks_count":21,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-09-18T08:19:35.105Z","etag":null,"topics":["android-database","c","c11","database","database-engine","ejdb","embedded-c","freebsd","key-value","key-value-database","key-value-store","kv-store","kvstore","nosql","osx","persistent-data-structure","persistent-storage","skip-list","skiplist"],"latest_commit_sha":null,"homepage":"https://iowow.softmotions.com","language":"C","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Softmotions.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":".github/SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-08-12T18:44:57.000Z","updated_at":"2024-09-17T15:52:22.000Z","dependencies_parsed_at":"2023-11-11T18:23:44.802Z","dependency_job_id":"f0ec289d-49ae-4e92-b267-16eb2926ae8b","html_url":"https://github.com/Softmotions/iowow","commit_stats":null,"previous_names":[],"tags_count":78,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Softmotions%2Fiowow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Softmotions%2Fiowow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Softmotions%2Fiowow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Softmotions%2Fiowow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Softmotions","download_url":"https://codeload.github.com/Softmotions/iowow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249316005,"owners_count":21249873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android-database","c","c11","database","database-engine","ejdb","embedded-c","freebsd","key-value","key-value-database","key-value-store","kv-store","kvstore","nosql","osx","persistent-data-structure","persistent-storage","skip-list","skiplist"],"created_at":"2024-08-01T22:00:59.927Z","updated_at":"2025-04-17T04:32:34.845Z","avatar_url":"https://github.com/Softmotions.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"IOWOW — `C11` utility library and persistent key/value storage engine\n==================================================================================================================\n\n**NOTE: Issues tracker is disabled. You are welcome to contribute, pull requests accepted.**\n\n[![license](https://img.shields.io/github/license/Softmotions/ejdb.svg)](https://github.com/Softmotions/iowow/blob/master/LICENSE)\n![Maintained](https://img.shields.io/maintenance/yes/2024.svg)\n\nWebsite https://iowow.softmotions.com\n\n# Key components\n\n* [utils](https://github.com/Softmotions/iowow/tree/master/src/utils) Useful data structures and various commonly used routines.\n* [json](https://github.com/Softmotions/iowow/tree/master/src/json) JSON parsing and manipulation routines incluing JSON Patch/Pointers support.\n* [iwkv.h](https://github.com/Softmotions/iowow/blob/master/src/kv/iwkv.h) Persistent key/value database engine.\n* [iwfsmfile.h](https://github.com/Softmotions/iowow/blob/master/src/fs/iwfsmfile.h) File blocks allocation manager like `malloc()` on files.\n\n## Used by\n\n* EJDB2 — Embeddable JSON database engine. http://ejdb.org\n* [Wirow video conferencing platform](https://github.com/wirow-io/wirow-server/)\n* [Pure C Asynchronous HTTP/IO library with websockets, SSL, routing. ](https://github.com/Softmotions/iwnet)\n\n## IWKV Features\n\n* Support of multiple key-value databases within a single file\n* Online database backups\n* Native support of integer keys\n* [Write Ahead Logging (WAL) support](http://iowow.io/wal)\n* Ultra-fast traversal of database records\n* Compound keys support\n* Good performance comparing its main competitors: `lmdb`, `leveldb`, `kyoto cabinet`\n* Tiny C11 library (200Kb) can be easily embedded into any software\n\n[![IWKV Presentation](https://iowow.softmotions.com/articles/iowow-presentation-cover-small.png)](https://iowow.softmotions.com/articles/intro/)\n\n## IWKV Limitations\n\n* Maximum iwkv storage file size: `512 GB (0x7fffffff80)`\n* Total size of a single key+value record must be not greater than `255Mb (0xfffffff)`\n\n# Supported platforms\n\n## Linux\n\n#### Building debian packages\n\n```sh\nmkdir build \u0026\u0026 cd build\ncmake .. -DCMAKE_BUILD_TYPE=Release -DPACKAGE_DEB=ON\nmake package\n```\n\n### RPM based Linux distributions\n```sh\nmkdir build \u0026\u0026 cd build\ncmake .. -DCMAKE_BUILD_TYPE=Release -DPACKAGE_RPM=ON\nmake package\n```\n\n## FreeBSD\n\nSuccessfully tested on FreeBSD 10/11\n\n## OSX\n\nSuccessfully tested on OSX 10.12/10.13\n\n## Windows\n\n[Cross-compilation for windows](http://iowow.io/iw/win)\n\n\n## MIPS based systems (+big-endian)\n\nSuccessfully tested on Debian 9.4, MIPS 32, gcc 6.x compiler.\n\n# Examples\n\n[src/kv/examples](https://github.com/Softmotions/iowow/tree/master/src/kv/examples)\n\n## Store and retrieve records\n\n```c\n#include \u003ciowow/iwkv.h\u003e\n#include \u003cstring.h\u003e\n#include \u003cstdlib.h\u003e\n\nint main() {\n  IWKV_OPTS opts = {\n    .path = \"example1.db\",\n    .oflags = IWKV_TRUNC // Cleanup database before open\n  };\n  IWKV iwkv;\n  IWDB mydb;\n  iwrc rc = iwkv_open(\u0026opts, \u0026iwkv);\n  if (rc) {\n    iwlog_ecode_error3(rc);\n    return 1;\n  }\n  // Now open mydb\n  // - Database id: 1\n  rc = iwkv_db(iwkv, 1, 0, \u0026mydb);\n  if (rc) {\n    iwlog_ecode_error2(rc, \"Failed to open mydb\");\n    return 1;\n  }\n  // Work with db: put/get value\n  IWKV_val key, val;\n  key.data = \"foo\";\n  key.size = strlen(key.data);\n  val.data = \"bar\";\n  val.size = strlen(val.data);\n\n  fprintf(stdout, \"put: %.*s =\u003e %.*s\\n\",\n          (int) key.size, (char *) key.data,\n          (int) val.size, (char *) val.data);\n\n  rc = iwkv_put(mydb, \u0026key, \u0026val, 0);\n  if (rc) {\n    iwlog_ecode_error3(rc);\n    return rc;\n  }\n  // Retrieve value associated with `foo` key\n  val.data = 0;\n  val.size = 0;\n  rc = iwkv_get(mydb, \u0026key, \u0026val);\n  if (rc) {\n    iwlog_ecode_error3(rc);\n    return rc;\n  }\n\n  fprintf(stdout, \"get: %.*s =\u003e %.*s\\n\",\n          (int) key.size, (char *) key.data,\n          (int) val.size, (char *) val.data);\n\n  iwkv_val_dispose(\u0026val);\n  iwkv_close(\u0026iwkv);\n  return 0;\n}\n```\n**Compile and run:**\n\n```sh\ngcc -std=gnu11 -Wall -pedantic -c -o example1.o example1.c\ngcc -o example1 example1.o -liowow\n\n./example1\n  put: foo =\u003e bar\n  get: foo =\u003e bar\n```\n\n## Cursor iteration example\n\n```c\n///\n/// Fills database with a set of football table records\n/// then traverse records according to club name in ascending and descending orders.\n///\n\n#include \"iwkv.h\"\n#include \u003cstring.h\u003e\n#include \u003cstdlib.h\u003e\n#include \u003cstdint.h\u003e\n\nstatic struct data_s {\n  const char *club;\n  uint8_t     points;\n} _points[] = {\n\n  { \"Aston Villa\",              25  },\n  { \"Manchester City\",          57  },\n  { \"Arsenal\",                  40  },\n  { \"Everton\",                  37  },\n  { \"West Ham United\",          27  },\n  { \"Tottenham Hotspur\",        41  },\n  { \"Wolverhampton Wanderers\",  43  },\n  { \"Norwich City\",             21  },\n  { \"Leicester City\",           53  },\n  { \"Manchester United\",        45  },\n  { \"Newcastle United\",         35  },\n  { \"Brighton \u0026 Hove Albion\",   29  },\n  { \"AFC Bournemouth\",          27  },\n  { \"Crystal Palace\",           39  },\n  { \"Sheffield United\",         43  },\n  { \"Burnley\",                  39  },\n  { \"Southampton\",              34  },\n  { \"Watford\",                  27  },\n  { \"Chelsea\",                  48  },\n  { \"Liverpool\",                82  },\n};\n\nstatic iwrc run(void) {\n  IWKV_OPTS opts = {\n    .path = \"cursor1.db\",\n    .oflags = IWKV_TRUNC // Cleanup database before open\n  };\n  IWKV iwkv;\n  IWDB db;\n  IWKV_cursor cur = 0;\n  iwrc rc = iwkv_open(\u0026opts, \u0026iwkv);\n  RCRET(rc);\n\n  rc = iwkv_db(iwkv, 1, 0, \u0026db);\n  RCGO(rc, finish);\n\n  for (int i = 0; i \u003c sizeof(_points) / sizeof(_points[0]); ++i) {\n    struct data_s *n = \u0026_points[i];\n    IWKV_val key = { .data = (void *) n-\u003eclub, .size = strlen(n-\u003eclub) };\n    IWKV_val val = { .data = \u0026n-\u003epoints, .size = sizeof(n-\u003epoints) };\n    RCC(rc, finish, iwkv_put(db, \u0026key, \u0026val, 0));\n  }\n\n  fprintf(stdout, \"\u003e\u003e\u003e\u003e Traverse in descending order\\n\");\n  RCC(rc, finish, iwkv_cursor_open(db, \u0026cur, IWKV_CURSOR_BEFORE_FIRST, 0));\n  while ((rc = iwkv_cursor_to(cur, IWKV_CURSOR_NEXT)) == 0) {\n    IWKV_val key, val;\n    RCC(rc, finish, iwkv_cursor_get(cur, \u0026key, \u0026val));\n    fprintf(stdout, \"%.*s: %u\\n\",\n            (int) key.size, (char *) key.data,\n            *(uint8_t *) val.data);\n    iwkv_kv_dispose(\u0026key, \u0026val);\n  }\n  rc = 0;\n  iwkv_cursor_close(\u0026cur);\n\n  fprintf(stdout, \"\\n\u003e\u003e\u003e\u003e Traverse in ascending order\\n\");\n  RCC(rc, finish, iwkv_cursor_open(db, \u0026cur, IWKV_CURSOR_AFTER_LAST, 0));\n  while ((rc = iwkv_cursor_to(cur, IWKV_CURSOR_PREV)) == 0) {\n    IWKV_val key, val;\n    RCC(rc, finish, iwkv_cursor_get(cur, \u0026key, \u0026val));\n    fprintf(stdout, \"%.*s: %u\\n\",\n            (int) key.size, (char *) key.data,\n            *(uint8_t *) val.data);\n    iwkv_kv_dispose(\u0026key, \u0026val);\n  }\n  rc = 0;\n  iwkv_cursor_close(\u0026cur);\n\n  // Select all keys greater or equal than: Manchester United\n  {\n    fprintf(stdout, \"\\n\u003e\u003e\u003e\u003e Records GE: %s\\n\", _points[9].club);\n    IWKV_val key = { .data = (void *) _points[9].club, .size = strlen(_points[9].club) }, val;\n    RCC(rc, finish, iwkv_cursor_open(db, \u0026cur, IWKV_CURSOR_GE, \u0026key));\n    do {\n      RCC(rc, finish, iwkv_cursor_get(cur, \u0026key, \u0026val));\n      fprintf(stdout, \"%.*s: %u\\n\",\n              (int) key.size, (char *) key.data,\n              *(uint8_t *) val.data);\n      iwkv_kv_dispose(\u0026key, \u0026val);\n    } while ((rc = iwkv_cursor_to(cur, IWKV_CURSOR_NEXT)) == 0);\n    rc = 0;\n  }\n  iwkv_cursor_close(\u0026cur);\n\nfinish:\n  if (cur) {\n    iwkv_cursor_close(\u0026cur);\n  }\n  iwkv_close(\u0026iwkv);\n  return rc;\n}\n\nint main() {\n  iwrc rc = run();\n  if (rc) {\n    iwlog_ecode_error3(rc);\n    return 1;\n  }\n  return 0;\n}\n```\n\nOutput:\n```\n\u003e\u003e\u003e\u003e Traverse in descending order\nWolverhampton Wanderers: 43\nWest Ham United: 27\nWatford: 27\nTottenham Hotspur: 41\nSouthampton: 34\nSheffield United: 43\nNorwich City: 21\nNewcastle United: 35\nManchester United: 45\nManchester City: 57\nLiverpool: 82\nLeicester City: 53\nEverton: 37\nCrystal Palace: 39\nChelsea: 48\nBurnley: 39\nBrighton \u0026 Hove Albion: 29\nAston Villa: 25\nArsenal: 40\nAFC Bournemouth: 27\n\n\u003e\u003e\u003e\u003e Traverse in ascending order\nAFC Bournemouth: 27\nArsenal: 40\nAston Villa: 25\nBrighton \u0026 Hove Albion: 29\nBurnley: 39\nChelsea: 48\nCrystal Palace: 39\nEverton: 37\nLeicester City: 53\nLiverpool: 82\nManchester City: 57\nManchester United: 45\nNewcastle United: 35\nNorwich City: 21\nSheffield United: 43\nSouthampton: 34\nTottenham Hotspur: 41\nWatford: 27\nWest Ham United: 27\nWolverhampton Wanderers: 43\n\n\u003e\u003e\u003e\u003e Records GE: Manchester United\nManchester United: 45\nManchester City: 57\nLiverpool: 82\nLeicester City: 53\nEverton: 37\nCrystal Palace: 39\nChelsea: 48\nBurnley: 39\nBrighton \u0026 Hove Albion: 29\nAston Villa: 25\nArsenal: 40\nAFC Bournemouth: 27\n```\n\n## IWSTART\n\n[IWSTART](https://github.com/Softmotions/iwstart) is an automatic CMake initial project generator for C projects based on iowow / [iwnet](https://github.com/Softmotions/iwnet) / [ejdb2](https://github.com/Softmotions/ejdb) libs.\n\nhttps://github.com/Softmotions/iwstart\n\n# License\n```\n\nMIT License\n\nCopyright (c) 2012-2024 Softmotions Ltd \u003cinfo@softmotions.com\u003e\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSoftmotions%2Fiowow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSoftmotions%2Fiowow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSoftmotions%2Fiowow/lists"}