{"id":18694476,"url":"https://github.com/luickk/tempcachedb","last_synced_at":"2025-06-11T07:09:23.798Z","repository":{"id":122825306,"uuid":"437512197","full_name":"luickk/tempCacheDb","owner":"luickk","description":"key/val database with focus on temporary ultra fast data storage","archived":false,"fork":false,"pushed_at":"2022-05-25T10:02:13.000Z","size":827,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-15T21:13:42.446Z","etag":null,"topics":["c","c-library","key-value","key-value-database","key-value-store"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luickk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-12T10:19:33.000Z","updated_at":"2022-06-21T08:17:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"85e68f88-0fcb-46a2-b6f7-d6e70fbea370","html_url":"https://github.com/luickk/tempCacheDb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luickk%2FtempCacheDb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luickk%2FtempCacheDb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luickk%2FtempCacheDb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luickk%2FtempCacheDb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luickk","download_url":"https://codeload.github.com/luickk/tempCacheDb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239552507,"owners_count":19657924,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","c-library","key-value","key-value-database","key-value-store"],"created_at":"2024-11-07T11:10:57.077Z","updated_at":"2025-02-18T21:26:03.472Z","avatar_url":"https://github.com/luickk.png","language":"C","readme":"# tempCacheDb (C17)\n\nKey/Val database with focus on temporary ultra fast data storage. Originally the project was intended to be a header only library, I dropped the idea after breaching a certain complexity and after I came up with plans to extend the project.\n\nThe idea is to provide a simple db which is able to store and manage key/val data at minimum latency in a temporary manner. Since the db is only meant for temporary data storage it's stored in memory (heap).\n\nThis type of db is suited for all kinds of data distribution in a local network or wireless type manner if latency and flexibility matters.\n\nUp to now there is still room for massive improvements in memory efficiency and speed.\n\n## Benchmark\n\nAverage key/val pull on localhost: 55 μs(micross.)\n\n### Optimization Record\n\n- Changed from spinning wait loop for pull request reply, to pthread condvar\n\n## Features\n\n- local key/val database\n- remote push/pull of data\n- full concurrency support (exmples can be found in test/)\n\nTo come:\n- set function (currently implemented by \"overwriting\" existing key)\n- synchronisation between two caches\n\n## Setup\n\nTo build the project, no dependencies have to be installed apart from cmake and standard gnu tools.\nThe lib is built to a static(just change the cmake script to compile to a dynamic lib) library and can be found in the build dir.\n\n## Usage\n\n### General\n\nSince the db is meant to be generic a basic cacheDB object consists of a void pointer and size for the key and val.\n\n```C\ntypedef struct {\n  void *key;\n  uint16_t keySize;\n\n  void *val;\n  uint16_t valSize;\n\n  int deleteMarker;\n} cacheObject;\n```\n\nIn order to manage those rather generic cacheObjects(cO), certain functions(cO free and key compare) have to be defined first and then passed to the cacheDb init.\n\n```C\n// here an example for \"strings\"\nint strKeyCmp(void *key1, void *key2, int size) {\n  char *ckey1 = (char*)key1;\n  char *ckey2 = (char*)key2;\n  for (int i = 0; i \u003c= size; i++) {\n    if (ckey1[i]!=ckey2[i]) {\n      return 0;\n    }\n  }\n  return 1;\n}\n\n// in this example we only need to free the cacheObject struct because the key/val are string literals and cannot be freed\nvoid freeCoFn(cacheObject *cO) {\n  free(cO);\n}\n```\n\nthose are then passed to the cacheDB init.\nThere are two types of cache available in this library.\nThe simpleCache and the tempCache. Therefor the simple cache is really the simplest abstraction and provides only local usage. The tempCache extends the simpleCache and provides remote cache manipulation.\nInternally the remote net functions make heavy use of the simpleCache but it can also be used externally.\n\n### Local usage\n\nTo setup the cache call `initTempCache`.\n```C\ntempCache *cacheDB;\nint err = initTempCache(\u0026cacheDB, strKeyCmp, freeCoFn);\nif (err != 0) {\n  printf(\"err code %d \\n\", err);\n  return 1;\n}\n```\n\nTo pull values from cache use `getCacheObject`. The results are then written to the queryCo (memory (for val) must not be allocated before). The queryCo must be freed be the user.\n\n```C\nif(getCacheObject(cache1-\u003elocalCache, queryCo-\u003ekey, queryCo-\u003ekeySize, queryCo)) {\n  printf(\"found val: %s \\n\", (char*)queryCo-\u003eval);\n} else {\n  printf(\"not found \\n\");\n}\n```\n\nTo push cacheObject to the cache use `pushCacheObject`. pushedCo is not freed.\nThe third argument returns the pointer to the actual pointer of the (new) cacheObject in the keyValStore of the cacheDB.\n\n```C\nerr = pushCacheObject(cache1-\u003elocalCache, pushedCo, NULL);\nif (err != 0) {\n  printf(\"err code %d \\n\", err);\n  return 1;\n}\n```\n\n### Remote usage\n\nTo manipulate a cache remotely the tempCacheClient is required.\n\n```C\ntempCacheClient *cacheClient;\nerr = initCacheClient(\u0026cacheClient);\nif (err != 0) {\n  printf(\"cClientInit err code %d \\n\", err);\n  return 1;\n}\n\nerr = cacheClientConnect(cacheClient, \"127.0.0.1\", 8080);\nif (err != 0) {\n  printf(\"cClientConnect err code %d \\n\", err);\n  return 1;\n}\n```\n\nTo push data to the remote cache use `cacheClientPushCacheObject`. pushedCo is not freed.\n```C\nerr = cacheClientPushCacheObject(cacheClient, pushedCo);\nif (err != 0) {\n  printf(\"cacheClientPushO err code %d \\n\", err);\n  return 1;\n}\n```\n\nTo pull data from the remote cache use `cacheClientPullCacheObject`. pulledCo val must not contain a pointer to allocated memory because the pulled val is written there.\n```C\nerr = cacheClientPullCacheObject(cacheClient, pulledCo-\u003ekey, pulledCo-\u003ekeySize, \u0026pulledCo);\nif (err != 0) {\n  printf(\"cacheClientPushO err code %d \\n\", err);\n  return 1;\n}\n```\n\n(additional details can be found in the header)\n\n## Details\n\n### Capacity\n\nThe size of the cacheDB can be defined in the header (`MAX_CACHE_SIZE`). If the max cache size is reached no new keys can be added to the db but existing key/vals can still be manipulated.\n\n### Testing\n\nA multitude of tests can be found in `test/`. In order to test \"every possible\" usecase all of the tests(grouped by their client/server nature) have to be used with each other. Additionally reandom key genartion must be uncommented in order to be used.\n\n### Memory management\n\nTo ensure a predictable memory behaviour and prevent potential leaks the code has been analyzed with multiple static analyzing tools and tested/ surveilled in a multitude of scenarios. Threads ensure memory release by utilizing the pthread cleanup feature.\n\n### Stability/ Err management\n\nStability and prevention of undefined behaviours was approached in a similar manner as the memory management but although thorough testing has been conducted, it cannot be guaranteed.\nSimilar to Go(lang) errors are defined in the header as an enum and returned by every function. Through that errors can almost always be recovered. Since I didn't have any real life use case error recovery specifically is not implemented (yet).\n\n### Parallelisation\n\nEvery cache (simpleCache, tempCache) is fully atomic and every function can be used in a parallel manner and are ensured not to conflict by proper mutex utilisation.\nThreads are only made use of if a new client connects (remotely) and for cache surveillance\n\n### Networking/ Comm Protocol\n\nThe remote data exchange is made possible by tcp sockets. Therefor, if required, the cache which is manipulated can setup a tcp server which can then handles all incoming manipulation requests of the cache. \u003cbr\u003e\n*The implementation also compensates for bad network connections and buffers/ merges network data if required*.\n\n- sizes are encoded in net byte order - the key/ val buffer are not checked for correct endianness\n\n- Protocol structure \u003cbr\u003e\n`uint8_t opCode(pull=1, push=2, pullReply=3) - uint16_t (query)keySize - char[] (query)key - uint16_t(val) valSize - char[] val \u003cbr\u003e``\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluickk%2Ftempcachedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluickk%2Ftempcachedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluickk%2Ftempcachedb/lists"}