{"id":21629681,"url":"https://github.com/sfu-dis/ermia","last_synced_at":"2025-04-11T13:51:19.271Z","repository":{"id":49139077,"uuid":"47548254","full_name":"sfu-dis/ermia","owner":"sfu-dis","description":"ERMIA: Memory-Optimized OLTP engine for Heterogeneous Workloads (SIGMOD 2016)","archived":false,"fork":false,"pushed_at":"2022-12-11T05:39:19.000Z","size":7018,"stargazers_count":88,"open_issues_count":0,"forks_count":26,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-25T10:04:38.087Z","etag":null,"topics":["ermia","oltp","serial-safety-net","serializable","serializable-snapshot-isolation","ssi","ssn","transactions"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sfu-dis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-12-07T11:31:48.000Z","updated_at":"2025-01-24T06:44:53.000Z","dependencies_parsed_at":"2023-01-26T16:31:12.054Z","dependency_job_id":null,"html_url":"https://github.com/sfu-dis/ermia","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fermia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fermia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fermia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfu-dis%2Fermia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sfu-dis","download_url":"https://codeload.github.com/sfu-dis/ermia/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248411945,"owners_count":21099031,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ermia","oltp","serial-safety-net","serializable","serializable-snapshot-isolation","ssi","ssn","transactions"],"created_at":"2024-11-25T02:08:26.703Z","updated_at":"2025-04-11T13:51:19.241Z","avatar_url":"https://github.com/sfu-dis.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"## ERMIA\n\nFast and Robust OLTP using Epoch-based Resource Management and Indirection Array\n\nSee our SIGMOD'16 paper [1] for a description of the system, our VLDBJ paper [2] for details in concurrency control, and our VLDB paper for replication.\n\n\\[1\\] Kangnyeon Kim, Tianzheng Wang, Ryan Johnson and Ippokratis Pandis. [ERMIA: Fast Memory-Optimized Database System for Heterogeneous Workloads](https://www2.cs.sfu.ca/~tzwang/ermia.pdf). SIGMOD 2016.\n\n\\[2\\] Tianzheng Wang, Ryan Johnson, Alan Fekete and Ippokratis Pandis. [Efficiently making (almost) any concurrency control mechanism serializable](https://link.springer.com/article/10.1007/s00778-017-0463-8). The VLDB Journal, Volume 26, Issue 4. 2017. [preprint](https://arxiv.org/pdf/1605.04292.pdf).\n\n\\[3\\] Tianzheng Wang, Ryan Johnson and Ippokratis Pandis. [Query Fresh: Log Shipping on Steroids](http://www.vldb.org/pvldb/vol11/p406-wang.pdf). VLDB 2018.\n\n#### Environment configurations\n\n* Software dependencies: `libnuma`. Install from your favorite package manager. ERMIA uses `mmap` with `MAP_HUGETLB` to allocate huge pages. `MAP_HUGETLB` is available after Linux 2.6.32.\n* Make sure you have enough huge pages. Almost all memory allocations come from the space carved out here. Assuming 2MB pages, the command below will allocate 40GB of memory:\n```\nsudo sh -c 'echo [x pages] \u003e /proc/sys/vm/nr_hugepages'\n```\nThis limits the maximum for --node-memory-gb to 10 for a 4-socket machine (see below).\n\n* `mlock` limits. Add the following to `/etc/security/limits.conf` (replace \"[user]\" with your login):\n```\n[user] soft memlock unlimited\n[user] hard memlock unlimited\n```\n*Re-login to apply.*\n\n#### Adjust maximum concurrent workers\n\nBy default we support up to 256 cores. The limit can be adjusted by setting `MAX_THREADS` defined under `config` in `dbcore/sm-config.h.` `MAX_THREADS` must be a multiple of 64.\n\n#### Build it\n--------\n\nCurrently the code only compiles with clang. We do not allow building in the source directory. Suppose we build in a separate directory:\n\n```\n$ mkdir build\n$ cd build\n$ CC=clang CXX=clang++ cmake ../ -DCMAKE_BUILD_TYPE=[Debug/Release/RelWithDebInfo]\n$ make -jN\n```\n\nAfter `make` there will be three executables under `build`: \n`ermia_SI` that runs snapshot isolation (not serializable);\n`ermia_SI_SSN` that runs snapshot isolation + Serial Safety Net (serializable)\n`ermia_SSI` that runs serializable snapshot isolation *\n\n* Serializable Isolation for Snapshot Databases, M. Cahill, U. Rohm, A. Fekete, SIGMOD 2008.\n\n\n#### Run it\n```\n$run.sh \\\n       [executable] \\\n       [benchmark] \\\n       [scale-factor] \\\n       [num-threads] \\\n       [duration (seconds)] \\\n       \"[other system-wide runtime options]\" \\\n       \"[other benchmark-specific runtime options]\"`\n```\n\n#### System-wide runtime options\n\n`-node_memory_gb`: how many GBs of memory to allocate per socket.\n\n`-null_log_device`: flush log buffer to `/dev/null`. With more than 30 threads, log flush (even to tmpfs) can easily become a bottleneck because of a mutex in the kernel held during the flush. This option does *not* disable logging, but it voids the ability to recover.\n\n`-tmpfs_dir`: location of the log buffer's mmap file. Default: `/tmpfs/`.\n\n`-enable_gc`: turn on garbage collection. Currently there is only one GC thread.\n\n`-enable_chkpt`: enable checkpointing.\n\n`-phantom_prot`: enable phantom protection.\n\n`-warm-up`: strategy to load versions upon recovery. Candidates are:\n- `eager`: load all latest versions during recovery, so the database is fully in-memory when it starts to process new transactions;\n- `lazy`: start a thread to load versions in the background after recovery, so the database is partially in-memory when it starts to process new transactions.\n- `none`: load versions on-demand upon access.\n\n*SSI and SSN specific:*\n\n`--safesnap`: enable safe snapshot for read-only transactions.\n\n*SSN-specific:*\n\n`--ssn-read-opt-threshold`: versions that are read by a read-mostly transaction and older than this value are considered \"old\" and will not be tracked; setting it to 0 will skip all read tracking for read-mostly transactions (`TXN_FLAG_READ_MOSTLY`).\n\n*SSI-specific:*\n`--ssi-read-only-opt`: enable P\u0026G style read-only optimization for SSI.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsfu-dis%2Fermia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsfu-dis%2Fermia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsfu-dis%2Fermia/lists"}