{"id":27619525,"url":"https://github.com/bw-hro/sqlitemap","last_synced_at":"2025-08-03T03:10:10.338Z","repository":{"id":289122866,"uuid":"963692519","full_name":"bw-hro/sqlitemap","owner":"bw-hro","description":"sqlitemap is a lightweight C++ wrapper around SQLite that provides a simple, map-like interface.","archived":false,"fork":false,"pushed_at":"2025-04-21T16:59:25.000Z","size":112,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-21T17:35:07.524Z","etag":null,"topics":["cpp17","data-store","key-value-store","sqlite"],"latest_commit_sha":null,"homepage":"https://bw-hro.github.io/sqlitemap","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bw-hro.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-10T04:24:46.000Z","updated_at":"2025-04-21T16:56:45.000Z","dependencies_parsed_at":"2025-04-21T17:47:36.045Z","dependency_job_id":null,"html_url":"https://github.com/bw-hro/sqlitemap","commit_stats":null,"previous_names":["bw-hro/sqlitemap"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bw-hro%2Fsqlitemap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bw-hro%2Fsqlitemap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bw-hro%2Fsqlitemap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bw-hro%2Fsqlitemap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bw-hro","download_url":"https://codeload.github.com/bw-hro/sqlitemap/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250374304,"owners_count":21419943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp17","data-store","key-value-store","sqlite"],"created_at":"2025-04-23T05:21:13.137Z","updated_at":"2025-08-03T03:10:10.322Z","avatar_url":"https://github.com/bw-hro.png","language":"C++","readme":"[project_name]: sqlitemap\n\n# sqlitemap — Persistent Map Backed by SQLite\n\n![sqlitemap](sqlitemap.png)\n\n[![CI Ubuntu](https://github.com/bw-hro/sqlitemap/actions/workflows/ubuntu.yml/badge.svg?branch=main)](https://github.com/bw-hro/sqlitemap/actions/workflows/ubuntu.yml)\n[![CI Windows](https://github.com/bw-hro/sqlitemap/actions/workflows/windows.yml/badge.svg?branch=main)](https://github.com/bw-hro/sqlitemap/actions/workflows/windows.yml)\n[![CI macOS](https://github.com/bw-hro/sqlitemap/actions/workflows/macos.yml/badge.svg?branch=main)](https://github.com/bw-hro/sqlitemap/actions/workflows/macos.yml)\n[![code coverage](https://bw-hro.github.io/sqlitemap/coverage-report/badge.svg)](https://bw-hro.github.io/sqlitemap/coverage-report)\n\n[![C++17](https://img.shields.io/badge/C%2B%2B-17-blue)]()\n[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/bw-hro/sqlitemap/main/LICENSE.txt)\n[![GitHub Releases](https://img.shields.io/github/release/bw-hro/sqlitemap.svg)](https://github.com/bw-hro/sqlitemap/releases)\n[![Vcpkg Version](https://img.shields.io/vcpkg/v/bw-sqlitemap)](https://vcpkg.link/ports/bw-sqlitemap)\n[![Conan Version](https://img.shields.io/conan/v/sqlitemap)](https://conan.io/center/recipes/sqlitemap)\n\n**sqlitemap** is a lightweight C++ wrapper around SQLite that provides a simple, map-like interface.  \nIt’s designed to make working with key-value storage easy and convenient — similar to how [sqlitedict](https://github.com/piskvorky/sqlitedict) works in the Python world.\nThe library is implemented as a **single-header** file, making integration easy and is licensed under the **MIT License**.\n\n## Features\n\n- Persistent key-value storage using SQLite\n- Easy-to-use map-like interface in C++\n- Transactions\n- Custom encoding/decoding\n- Minimal dependencies (just requires [SQLite](https://sqlite.org)) \n- Please refer to the [Usage](#usage) section below for more details\n\n## Installation\n\n- Just add the [sqlitemap.hpp](https://github.com/bw-hro/sqlitemap/releases/latest/download/sqlitemap.hpp) header to your project include files:\n- Make sure that [SQLite](https://sqlite.org) is available\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n```\n\n### CMake\n\nExample of a minimal CMake setup using **FetchContent** to obtain **sqlitemap**\n\n```cmake\ncmake_minimum_required(VERSION 3.15)\nproject(sqlitemap-consumer)\nset(CMAKE_CXX_STANDARD 17)\n\n# Declare and fetch sqlitemap library\ninclude(FetchContent)\nFetchContent_Declare(\n        sqlitemap\n        GIT_REPOSITORY https://github.com/bw-hro/sqlitemap.git\n        GIT_TAG v1.1.0 # replace with desired version / branch\n        SOURCE_SUBDIR \"include\" # sqlitemap is single header only\n)\nFetchContent_MakeAvailable(sqlitemap)\n\n# First install SQLite3 on your system\n# e.g. 'sudo apt install libsqlite3-dev'\nfind_package(SQLite3 REQUIRED)\n\nadd_executable(sqlitemap-consumer main.cpp)\ntarget_include_directories(sqlitemap-consumer PRIVATE \"${sqlitemap_SOURCE_DIR}/include\")\ntarget_link_libraries(sqlitemap-consumer PRIVATE SQLite::SQLite3)\n```\nExample of a minimal CMake setup using **CPM.cmake** to obtain **sqlitemap**\n\n```cmake\ncmake_minimum_required(VERSION 3.15)\nproject(sqlitemap-consumer)\nset(CMAKE_CXX_STANDARD 17)\n\n# make CPM.cmake available\nset(CPM_DIR \"${CMAKE_CURRENT_BINARY_DIR}/cmake/CPM.cmake\")\nfile(DOWNLOAD https://github.com/cpm-cmake/CPM.cmake/releases/latest/download/get_cpm.cmake ${CPM_DIR})\ninclude(${CPM_DIR})\n\n# add sqlitemap and SQLite dependencies\nCPMAddPackage(URI \"gh:bw-hro/sqlitemap@1.1.0\" DOWNLOAD_ONLY YES)\nCPMAddPackage(\"gh:sjinks/sqlite3-cmake@3.49.1\")\n\nadd_executable(sqlitemap-consumer main.cpp)\ntarget_include_directories(sqlitemap-consumer PRIVATE \"${sqlitemap_SOURCE_DIR}/include\")\ntarget_link_libraries(sqlitemap-consumer PRIVATE SQLite::SQLite3)\n```\n\n### vcpkg\n\nYou can also use the *vcpkg* port `bw-sqlitemap` which handles all dependencies for you\n\n```sh\nvcpkg install bw-sqlitemap\n```\n\nor add dependency to *vcpkg.json* manifest file\n\n```json\n{\n  \"name\": \"your-project\",\n  \"version-string\": \"1.0.1\",\n  \"dependencies\": [\n    \"bw-sqlitemap\"\n  ]\n}\n```\n### conan\n\nYou can also use the *conan* recipe `sqlitemap` which handles all dependencies for you.\nAssuming you use cmake, just add the following to *conanfile.txt*\n\n```\n[requires]\nsqlitemap/[\u003e=1.1.0]\n\n[generators]\nCMakeDeps\nCMakeToolchain\n```\n\nand to *CMakeLists.txt*\n \n```cmake\ncmake_minimum_required(VERSION 3.15)\nproject(sqlitemap-consumer)\n\nfind_package(sqlitemap REQUIRED)\n\nadd_executable(${PROJECT_NAME} src/main.cpp)\ntarget_link_libraries(${PROJECT_NAME} PRIVATE sqlitemap::sqlitemap)\n```\n\n## Usage\n\n### Write\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    bw::sqlitemap::sqlitemap db(\"example.sqlite\");\n\n    db[\"a\"] = \"first-item\";\n    db[\"b\"] = \"second-item\";\n    db[\"c\"] = \"third-item\";\n\n    // Commit to save items\n    db.commit();\n\n    db[\"d\"] = \"yet-another-item\";\n    // Forgot to commit here, that item will never be saved.\n    db.close();\n\n    // Always remember to commit, or enable autocommit with\n    //\n    //     bw::sqlitemap::sqlitemap db(bw::sqlitemap::config()\n    //        .file(\"example.sqlite\")\n    //        .auto_commit(true));\n    //\n    // Autocommit is off by default for performance.\n\n\n    db.connect(); // reconnect, after close\n    // some additional helpful write methods, among others:\n    db.set(\"x\", \"draft\");     // set or update value\n    db.del(\"x\");              // delete value\n    db.emplace(\"y\", \"draft\"); // insert value if not exists\n    db.erase(\"y\");            // delete value returns number 0 if not exists, 1 otherwise\n    db.insert(std::pair{\"z\", \"draft\"}); // insert value, returns iterator to the inserted item\n                                        // attention: iterator can not be advanced,\n                                        // it can only be used to access the value\n    db.clear(); // clear all items\n}\n```\n\n### Read\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    bw::sqlitemap::sqlitemap db(\"example.sqlite\");\n    std::cout \u003c\u003c \"There are \" \u003c\u003c db.size() \u003c\u003c \" items in the database\" \u003c\u003c std::endl;\n    // There are 3 items in the database\n\n    // Standard map like interface. operator[], iterator, ...\n\n    for (auto\u0026 [key, value] : db)\n    {\n        std::cout \u003c\u003c key \u003c\u003c \" = \" \u003c\u003c value \u003c\u003c std::endl;\n    }\n    // a = first-item\n    // b = second-item\n    // c = third-item\n\n    std::string value_a = db[\"a\"];\n    std::cout \u003c\u003c \"value of a: \" \u003c\u003c value_a \u003c\u003c \", value of b: \" \u003c\u003c db[\"b\"] \u003c\u003c std::endl;\n    // value of a: first-item, value of b: second-item\n\n    // some additional helpful access methods, among others:\n    db.get(\"a\");      // returns value or throws sqlitemap_error if not found\n    db.try_get(\"b\");  // returns std::optional containing value or empty if not found\n    db.find(\"c\");     // returns iterator to the found item or end() if not found, attention: iterator\n                      // can not be advanced, it can only be used to access the value\n    db.contains(\"d\"); // returns true if key is found, false otherwise\n    db.count(\"e\");    // returns number of items with the given key, 0 or 1\n}\n```\n\n### Database Connection Lifecycle\n\nThe **sqlitemap** object manages the lifecycle of the SQLite database connection. When the object is created, it automatically connects to the database. When the object goes out of scope and is destroyed, it ensures that the database connection is properly closed.\n\n```c++\n{\n    bw::sqlitemap::sqlitemap db(\"example.sqlite\");\n    // The database connection is opened and ready for work\n    db[\"key\"] = \"value\";\n    db.commit();\n    // The database connection is automatically closed when `db` goes out of scope.\n    // db.connect() and db.close() can be used for manual connection management.\n}\n```\n\n### Transactions\n\n**sqlitemap** provides transaction management to ensure data consistency and performance. By default, **sqlitemap** does not auto-commit changes for performance reasons, allowing you to group multiple operations into a single transaction. When autocommit is used, the beginning of a multiple command spanning transaction must be stated explicitly.\n\n#### Explicit Transactions\n\nYou can explicitly control transactions using the `begin_transaction()`, `commit()` and `rollback()` methods:\n\n- **`begin_transaction()`**: Starts a new transaction explicitly. (required  when autocommit is used)\n- **`commit()`**: Saves all changes made during the current transaction to the database.\n- **`rollback()`**: Discards all changes made during the current transaction.\n\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    bw::sqlitemap::sqlitemap db(\"example.sqlite\"); // no autocommit active\n\n    try\n    {\n        // Write operations implicitly begins a new transaction\n        // db.begin_transaction() can be used to  make it explicit\n        db[\"key1\"] = \"value1\";\n        db[\"key2\"] = \"value2\";\n\n        // Commit the transaction to save changes\n        db.commit();\n\n        db[\"key3\"] = \"value3\";\n        throw std::runtime_error(\"An error occurred\");\n\n        // This line will not be reached, and the changes to \"key3\" will not be saved\n        db.commit();\n    }\n    catch (const std::exception\u0026 e)\n    {\n        std::cerr \u003c\u003c \"Exception: \" \u003c\u003c e.what() \u003c\u003c std::endl;\n\n        // Rollback the transaction to discard uncommitted changes\n        db.rollback();\n\n        // Use alternative value in error case\n        db[\"key3\"] = \"alt_value3\";\n        db.commit();\n    }\n}\n```\n\nIf you prefer not to manage transactions manually, you can enable auto-commit mode. In this mode, every write operation is immediately committed to the database until you explicitly state a new transaction.\n\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    bw::sqlitemap::sqlitemap db(bw::sqlitemap::config()\n        .file(\"example.sqlite\")\n        .auto_commit(true)); // Enable auto-commit mode\n\n    db[\"key1\"] = \"value1\"; // Automatically committed\n    db[\"key2\"] = \"value2\"; // Automatically committed\n\n    // begin new transaction explicitly\n    db.begin_transaction();\n    db[\"key3\"] = \"value3\";\n    db[\"key4\"] = \"value4\";\n\n    // Rollback the transaction to discard uncommitted changes\n    db.rollback();\n}\n```\n\n### Tables\n\nA database file can store multiple tables. The default table \"unnamed\" is used when no table name is specified.\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    bw::sqlitemap::sqlitemap db(bw::sqlitemap::config()\n        .file(\"example.sqlite\")\n        .auto_commit(true));\n    \n    db[\"k1\"] = \"v1\";\n    db[\"k2\"] = \"v2\";\n\n    bw::sqlitemap::sqlitemap cities(bw::sqlitemap::config()\n        .file(\"example.sqlite\")\n        .table(\"cities\")\n        .auto_commit(true));\n\n    cities[\"rostock\"] = \"https://en.wikipedia.org/wiki/Rostock\";\n    cities[\"raleigh\"] = \"https://en.wikipedia.org/wiki/Raleigh\";\n    cities[\"dalian\"] = \"https://en.wikipedia.org/wiki/Dalian\";\n\n    bw::sqlitemap::sqlitemap countries(bw::sqlitemap::config()\n        .file(\"example.sqlite\")\n        .table(\"countries\")\n        .auto_commit(true));\n\n    countries[\"germany\"] = \"https://en.wikipedia.org/wiki/Germany\";\n    countries[\"usa\"] = \"https://en.wikipedia.org/wiki/United_States\";\n    countries[\"china\"] = \"https://en.wikipedia.org/wiki/China\";\n\n    auto tables = bw::sqlitemap::get_tablenames(\"example.sqlite\");\n    // tables contains {\"cities\", \"countries\", \"unnamed\"}\n}\n```\n\n### Operation modes\n\n**sqlitemap** supports several operation modes that define how the database and its tables are handled. These modes can be configured using the `operation_mode` enum.\n\n#### Available Modes\n\n- **`operation_mode::c` (Create/Default)**: Opens the database in read-write mode. If the database or table does not exist, it will be created.\n- **`operation_mode::r` (Read-Only)**: Opens the database in read-only mode. No modifications are allowed.\n- **`operation_mode::w` (Write/Drop)**: Opens the database in read-write mode but drops the contents of the specified table before use.\n- **`operation_mode::n` (New)**: Creates a new database, erasing all existing tables.\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    using namespace bw::sqlitemap;\n\n    // Open database in read-only mode\n    sqlitemap db_readonly(config()\n        .file(\"example.sqlite\")\n        .mode(operation_mode::r));\n}\n```\nIn addition to the application-level `operation_mode`, the behavior of underlying SQLite database can be configured using [PRAGMA statements](https://www.sqlite.org/pragma.html). **sqlitemap** provides a convenient API to set these pragmas, which are executed when the connection to the database is established.\n\nAn example of configuring **sqlitemap** to optimize for high-concurrency workloads.\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\nint main()\n{\n    using namespace bw::sqlitemap;\n\n    sqlitemap sm(config()\n        .filename(\"example.sqlite\")\n        .pragma(\"journal_mode\", \"WAL\") // DELETE | TRUNCATE | PERSIST | MEMORY | WAL | OFF\n        .pragma(\"cache_size\", -64000)   // -64000 = 64000KiB, 64000 = number of pages\n        .pragma(\"temp_store = 2\")      // 0 = DEFAULT, 1 = FILE, 2 = MEMORY\n        .pragma(\"PRAGMA synchronous = NORMAL\")); // 0 = OFF, 1 = NORMAL, 2 = FULL\n\n        // configuration.pragma(...) accepts a whole statement, or a flag-value pair\n        // it prepends statements with \"PRAGMA \" when this prefix is missing\n}\n```\n\n### Encoding/Decoding\n\n**sqlitemap** supports custom encoding and decoding mechanisms for both keys and values to handle complex data types. By default, **sqlitemap** works with simple key-value pairs of `std::string`. However, you can define custom codecs to serialize and deserialize more complex types, such as structs or user-defined objects.\n\n\n```c++\n#include \u003cbw/sqlitemap/sqlitemap.hpp\u003e\n\n// point, feature definded in test/catch2/unit_tests/custom.hpp\n\nint main()\n{\n    using namespace bw::testhelper;\n\n    auto kc = key_codec([](point p) { return point::to_string(p); },\n                        [](std::string s) { return point::from_string(s); });\n\n    auto vc = value_codec([](feature f) { return feature::to_string(f); },\n                          [](std::string s) { return feature::from_string(s); });\n\n    // configure custom key and value codecs\n    sqlitemap db(config(kc, vc));\n\n    // working with point/feature directly encoding/decoding will be applied automatically\n    db.set({0, 0, 0}, {\"origin\", 5});\n    db[{0, 0, 0}] == feature{\"origin\", 5}; // true\n\n    db[{1, 0, 0}] = {\"x-direction\", 1};\n    db.get({1, 0, 0}) == feature{\"x-direction\", 1}; // true\n}\n```\n- [sqlitemap_cereal.cpp](examples/sqlitemap_cereal.cpp) demonstrates how to use **sqlitemap** with custom data types stored as blob using [cereal](https://github.com/USCiLab/cereal).\n- [sqlitemap_json.cpp](examples/sqlitemap_json.cpp) demonstrates how to use **sqlitemap** with custom data types stored as json string using [nlohmann::json](https://github.com/nlohmann/json).\n- [sqlitemap_tiles.cpp](examples/sqlitemap_tiles.cpp) demonstrates how to use **sqlitemap** with custom data types stored as blobs.\n- [sqlitemap_zlib.cpp](examples/sqlitemap_zlib.cpp) demonstrates how to use **sqlitemap** to store compressed values using [zlib](https://github.com/madler/zlib).\n- Please make sure to also inspect [sqlitemap_codecs_tests.cpp](test/catch2/unit_tests/sqlitemap_codecs_tests.cpp) were further details regarding encoding/decoding using codecs are covered.\n\n## Tests / Examples / Additional Documentation\n\n- **sqlitemap** is extensively covered by [unit tests](test), which also serve as documentation and usage examples.\n- Additionally, [sqlitemap_client](examples/sqlitemap_client.cpp) is a command-line wrapper around **sqlitemap** that demonstrates and covers its most important features and how to embed it into your own project. An executable can be found in the release section.\n","funding_links":[],"categories":["Containers"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbw-hro%2Fsqlitemap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbw-hro%2Fsqlitemap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbw-hro%2Fsqlitemap/lists"}