{"id":21629653,"url":"https://github.com/sfu-dis/mosaicdb","last_synced_at":"2025-08-19T23:09:16.576Z","repository":{"id":160290136,"uuid":"635173528","full_name":"sfu-dis/mosaicdb","owner":"sfu-dis","description":"The Art of Latency Hiding in Modern Database Engines (VLDB 2024)","archived":false,"fork":false,"pushed_at":"2025-04-19T20:51:37.000Z","size":1843,"stargazers_count":56,"open_issues_count":0,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-19T22:01:32.918Z","etag":null,"topics":["asyncio","corobase","coroutine","database","io-uring","memory-prefetch","mosaicdb","nvme-ssd","scheduling","transaction-processing"],"latest_commit_sha":null,"homepage":"https://www.vldb.org/pvldb/vol17/p577-huang.pdf","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sfu-dis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-05-02T05:57:02.000Z","updated_at":"2025-04-19T20:51:40.000Z","dependencies_parsed_at":"2025-04-19T21:43:21.341Z","dependency_job_id":null,"html_url":"https://github.com/sfu-dis/mosaicdb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sfu-dis/mosaicdb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fmosaicdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fmosaicdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fmosaicdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fmosaicdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sfu-dis","download_url":"https://codeload.github.com/sfu-dis/mosaicdb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fmosaicdb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271236280,"owners_count":24723978,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-19T02:00:09.176Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","corobase","coroutine","database","io-uring","memory-prefetch","mosaicdb","nvme-ssd","scheduling","transaction-processing"],"created_at":"2024-11-25T02:08:18.318Z","updated_at":"2025-08-19T23:09:16.537Z","avatar_url":"https://github.com/sfu-dis.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"## The Art of Latency Hiding in Modern Database Engines (VLDB '24)\n\nMosaicDB is a research storage engine that systematically hides various types of latency in modern memory-optimized database systems.\n\nSee details in our [VLDB 2024 paper](https://www.vldb.org/pvldb/vol17/p577-huang.pdf) below. If you use our work, please cite:\n\n```\nThe Art of Latency Hiding in Modern Database Engines.\nKaisong Huang, Tianzheng Wang, Qingqing Zhou and Qingzhong Meng.\nPVLDB 17(3) (VLDB 2024)\n```\n\n### Environment configurations\nStep 1: Software dependencies\n* cmake\n* python2\n* gcc-10\n* libnuma\n* libibverbs\n* libgflags\n* libgoogle-glog\n* liburing\n\nExample for Ubuntu\n```\n$ sudo apt-get install cmake gcc-10 g++-10 libc++-dev libc++abi-dev libnuma-dev libibverbs-dev libgflags-dev libgoogle-glog-dev liburing-dev\n```\n\nStep 2: Make sure you have enough huge pages\n\nMosaicDB uses `mmap` with `MAP_HUGETLB` (available after Linux 2.6.32) to allocate huge pages. Almost all memory allocations come from the space carved out here. Assuming the default huge page size is 2MB, the command below will allocate 2x MB of memory:\n```\n$ sudo sh -c 'echo [x pages] \u003e /proc/sys/vm/nr_hugepages'\n```\n\nStep 3: Set mlock limits. Add the following to `/etc/security/limits.conf` (replace \"[user]\" with your username):\n```\n[user] soft memlock unlimited\n[user] hard memlock unlimited\n```\nStep 4: Re-login to apply the changes\n\n--------\n### Compile\nWe do not allow building in the source directory:\n\n```\n$ mkdir build\n$ cd build\n$ cmake ../ -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo]\n$ make\n```\n--------\n### Run\nRequired options:\n* -log_data_dir=[path] (path to the directory where the log files will be stored)\n* -node_memory_gb=[#] (the amount of memory in GB that the node has)\n* -seconds=[#]\n* -threads=[#]\n\nMosaicDB-specific options:\n* -ycsb_read_tx_type=[hybrid-coro|nested-coro|flat-coro] (choose according to the executable)\n* -coro_scheduler=2\n* -coro_batch_size=[#] (8, by default)\n* -coro_cold_queue_size=[#] (16, by default)\n* -coro_check_cold_tx_interval=[#] (8, by default)\n\nPipeline-specific options:\n* -ycsb_read_tx_type=[hybrid-coro|nested-coro|flat-coro] (choose according to the executable)\n* -coro_scheduler=1\n* -coro_batch_size=[#] (16, by default)\n* -coro_cold_tx_threshold=[#] (8, by default)\n\nBatch-specific options:\n* -ycsb_read_tx_type=[hybrid-coro|nested-coro|flat-coro] (choose according to the executable)\n* -coro_scheduler=0\n* -coro_batch_size=[#] (16, by default)\n* -coro_cold_tx_threshold=[#] (8, by default)\n\nSequential-specific options:\n* -ycsb_read_tx_type=sequential\n\nFor more options, please refer to `sm-config.h`.\n\nExample for MosaicDB:\n```\n$ ./ycsb_SI_hybrid_coro \\\n-log_data_dir=/mnt/nvme0n1/mosaicdb-log \\\n-node_memory_gb=50 \\\n-ycsb_workload=C -ycsb_read_tx_type=hybrid-coro \\\n-ycsb_hot_table_size=300000000 \\\n-ycsb_cold_table_size=3000000 \\\n-ycsb_ops_per_tx=10 \\\n-ycsb_cold_ops_per_tx=2 \\\n-ycsb_ops_per_hot_tx=10 \\\n-ycsb_hot_tx_percent=0.9 \\\n-coro_scheduler=2 \\\n-coro_batch_size=8 \\\n-coro_cold_queue_size=16 \\\n-coro_check_cold_tx_interval=8 \\\n-seconds=10 \\\n-threads=10 \n```\n\nExample for sequential:\n```\n$ ./ycsb_SI_sequential \\\n-log_data_dir=/mnt/nvme0n1/mosaicdb-log \\\n-node_memory_gb=50 \\\n-ycsb_workload=C \\\n-ycsb_read_tx_type=sequential \\\n-ycsb_hot_table_size=300000000 \\\n-ycsb_cold_table_size=3000000 \\\n-ycsb_ops_per_tx=10 \\\n-ycsb_cold_ops_per_tx=2 \\\n-ycsb_ops_per_hot_tx=10 \\\n-ycsb_hot_tx_percent=0.9 \\\n-seconds=10 \\\n-threads=10\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsfu-dis%2Fmosaicdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsfu-dis%2Fmosaicdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsfu-dis%2Fmosaicdb/lists"}