{"id":18583101,"url":"https://github.com/ryogrid/samehadadb","last_synced_at":"2025-05-09T02:50:01.087Z","repository":{"id":65234171,"uuid":"561806257","full_name":"ryogrid/SamehadaDB","owner":"ryogrid","description":"Simple Relational Database Management System Implementation in Golang","archived":false,"fork":false,"pushed_at":"2025-05-03T23:47:46.000Z","size":2828,"stargazers_count":64,"open_issues_count":0,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-04T00:27:19.113Z","etag":null,"topics":["bustub","database","dbms","diy","go","golang","rdb","rdbms","relational-databases","skiplist","skiplist-index","sql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryogrid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["ryogrid"]}},"created_at":"2022-11-04T14:32:33.000Z","updated_at":"2025-05-03T23:47:50.000Z","dependencies_parsed_at":"2023-09-23T14:47:43.699Z","dependency_job_id":"351568de-8c51-41e2-bbc9-5a9509b6701f","html_url":"https://github.com/ryogrid/SamehadaDB","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2FSamehadaDB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2FSamehadaDB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2FSamehadaDB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryogrid%2FSamehadaDB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryogrid","download_url":"https://codeload.github.com/ryogrid/SamehadaDB/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253181404,"owners_count":21866990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bustub","database","dbms","diy","go","golang","rdb","rdbms","relational-databases","skiplist","skiplist-index","sql"],"created_at":"2024-11-07T00:20:11.018Z","updated_at":"2025-05-09T02:50:01.080Z","avatar_url":"https://github.com/ryogrid.png","language":"Go","readme":"[![test](https://github.com/ryogrid/SamehadaDB/actions/workflows/ci.yaml/badge.svg?event=push)](https://github.com/ryogrid/SamehadaDB/actions/workflows/ci.yaml)\n\n\u003cimg src=\"samehada_logo2.webp\" width=\"500px\" /\u003e\n\n# Overview\n- Compact Relational Database Management System Implementation Wrote in Golang\n- Origin of SamehadaDB is Golang ported [BusTub](https://github.com/cmu-db/bustub) RDBMS: [go-bustub](https://github.com/brunocalza/go-bustub)\n\n# What is Samehada?\n- Samehada, which literally means shark skin, is a tool used to grate wasabi, usually for sushi, but also for other Japanese cuisines\n- Samehada features its grid shape that forms air bubbles between the grated wasabi, minimizing unnecessary spiciness\n- We are proud to call SamehadaDB because the grid produces a pleasant taste and aroma with a nice harmony\n- (Texts in this section is contributed by [ujihisa](https://github.com/ujihisa). Thanks ujihisa)\n\n# Motivation\n- RDBMS implementation which is compact but has almost of essential componets and features implemted in practical techniques is needed for newbie of database system developer\n- SamehadaDB is developed to be a RDBMS like above\n  - Golang has high readability\n  - SamehadaDB is based on BusTub RDBMS which is used in educational purpose\n\n# Current Status\n- SamehadaDB can be used as simple embedded DB (library form)\n  - [Simple Single Page Application Demo (TODO App)](http://ryogrid.net:8088/todo/)))\n  - Above demo uses SamehadaDB on backend API server\n    - [Modification for migration from PostgreSQL to SamehadaDB (please see backend/api/api.go)](https://github.com/ryogrid/TODO-Fullstack-App-Go-Gin-Postgres-React/compare/273ca76aebac16bc344861ec2ddca147b2c6f86d...5ee4feb72471d16231cc4f4c5e774ab902a0b402)\n    - [All app codes](https://github.com/ryogrid/TODO-Fullstack-App-Go-Gin-Postgres-React/tree/5ee4feb72471d16231cc4f4c5e774ab902a0b402)\n      - builded frontend JS code is little bit modified directly for escape from re-compile...\n      - above code tree can be deployed to AWS Elastic Beanstalk (on not https and use 8088 port)\n- And can be used as DB server which offers REST API I/F\n  - Please see server directory and [this note](https://gist.github.com/ryogrid/6beee126af2aaebd160a0497c2c9611f)\n  - The server listen on http://0.0.0.0:19999/Query (this means localhost, 127.0.0.1, other IP addresses your machine has)\n    - Content-Type of request and response are \"application/json\"\n  - Additionaly the server listen on http://0.0.0.0:19999/QueryMsgPack\n    - Content-Type of request is \"application/json\" **but one of response is \"application/octet-stream\"**\n    - **Response is serialized binary in [MessagePack](https://github.com/msgpack/msgpack/tree/master) specification**\n    - For understanding response data schema, easy way is reading demo client code (schema is almost same with JSON response)\n      - [code (JS)](https://github.com/ryogrid/SamehadaDB/blob/9a0475b7550f80982c18fd355ca8b9d1b5d343ee/demo-client/index.html#L20-L41)\n  - There are Win binary and Linux binary at Release page\n    - if Linux one runs without error at lauch, you are lucky :)\n    - when error occurs, you need to build by myself :)\n      - **please check that value of EnableOnMemStorage constant is false and modify it if needed (/lib/common/config.go)**\n  - there is simple client which can be useed on browser in demo-client dir\n    - very simple SPA. requirement is web browser which can executes JS only :)\n      - It can access SamehadaDB server on your machine (localhost, 127.0.0.1, etc...)\n    - It is also pleced at [here](http://ryogrid.s3-website-ap-northeast-1.amazonaws.com/samehada-client/index.html) for demonstration\n      - ryogrid.net:19999 is public SamehadaDBd server for the demonstration\n      - **Do not insert private or secret information/data! the server does not do isolation**\n- **ATTENTION: SamehadaDB is not developed for productional use! There are no warranties!**\n- By the way, procedure described on next section executes almost all defined unit tests except these of taking long time\n\n## Procedure of Executing SamehadaDB (executing unit tests)\n- Please install golang environment with package system your OS has (apt, yum, brew ...etc)\n  - If you use Windows, you can select both Windows native environment and WSL Ubuntu environment\n- If you select Windows native environments or golang environment which is installed with package system can't execute SamehadaDB, you should install official binary directly\n  - Please refer [Download and Install - The Go Programming Language](https://go.dev/doc/install)\n- Executing all unit tests which test several features and components of SamehadaDB except several tests taking long time\n  - $ git clone https://github.com/ryogrid/SamehadaDB.git\n  - $ cd SamehadaDB/lib\n  - $ go clean -testcache; go test ./... -short -v\n\n## Roadmap  \n- [x] Predicates on Seq Scan\n- [x] Multiple Item on Predicate: AND, OR\n- [x] Predicates: \u003c, \u003e, \u003c=, \u003e=\n- [x] Null [^1]\n- [x] Predicates: NOT [^1]\n- [ ] Inline types (\u003cdel\u003einteger, varchar, float, boolean\u003c/del\u003e, bigint, smallint, decimal, timestamp, datetime and etc)\n- [x] Delete Tuple\n- [x] Update Tuple\n- [x] LIMIT / OFFSET [^1]\n- [x] Varchar\n- [x] Persistent Catalog\n- [ ] Updating of Table Schema \n- [ ] \u003cdel\u003eLRU replacer\u003c/del\u003e\n- [x] Latches\n- [x] Transactions\n- [x] Rollback When Abort Occurs\n- [x] Logging\n- [x] Checkpointing\n  - [x] Simple Checkpointing (all transactions are blocked until finish of checkpointing)\n  - [ ] Fuzzy Checkpointing (ARIES)\n- [x] Recovery from Logs\n- [x] Index\n  - [x] Hash Index [^1]\n    - Hash index can be used only equal(=) operator is specified to index having columns\n    - Thread safe but serialized (not supported concurrent access)\n  - [x] SkipList Index\n    - Concurrent access is supported\n  - [x] B-tree Index [^1]\n    - Using [bltree-go-for-embedding](https://github.com/ryogrid/bltree-go-for-embedding) lib\n    - Concurrent access is supported\n  - [ ] Logging And Recovery Of Index Data\n- [ ] JOIN\n  - [x] INNER JOIN (Hash Join, Index Join, Nested Loop Join)\n    - Condition specified at ON clause should be composed of single item and can use equal(==) operator only\n  - [ ] OUTER JOIN\n  - [x] CROSS JOIN\n- [x] Aggregations (COUNT, MAX, MIN, SUM on SELECT clause including Group by and Having) [^1]\n- [x] Sort (ORDER BY clause) [^1]\n- [x] Concurrent Execution of Transactions\n  - Concurrecy control protcol is Strong Strict 2-Phase Locking (SS2PL) and locking granularity is tuple level (record level)\n  - Avoidance of phantom problem is not implemented yet\n    - So, current transaction isolation level is **\"REPEATABLE READ\"**\n  - And when table has Indexes, dirty read can be occured\n    - So, when you use Indexes, isolation level is **\"DIRTY READ\"**\n      - Why dirty read occurs is lack of consistency between table record and index entry at DELETE  \n- [ ] \u003cdel\u003eExecution Planning from hard coded SQL like method call I/F (like some kind of embedded DB)\u003c/del\u003e\n- [x] Execution of Query with SQL string\n  - not supported multi queries on a request yet\n- [x] Frontend Impl as Embedded DB Library (like SQLite)\n- [ ] Deduplication of Result Records (Distinct)\n- [x] Query Optimization (Selinger) \n  - cases below are not supported now\n    - predicate including OR operation, NOT, IS NULL\n    - projection including aggregation\n    - LIMIT, ORDER BY\n- [x] Statistics Data for Optimizer\n  - Statistics data are updated continuously with full scan...\n- [ ] TRANSACTION Statement on SQL\n  - This includes adding support of multi statements (multi statements is not suported on SQL now)\n- [ ] AS clause\n- [ ] Nested Query\n- [ ] Predicates: IN\n- [ ] DB Connector (Driver) or Other Kind of Network Access Interface\n  - [ ] MySQL or PostgreSQL Compatible Protocol\n  - [x] REST\n- [x] Deallocate and Reuse Page\n- [x] Optimization of INSERT\n  - \u003cdel\u003eCurrent implementation searches a free space which is enough for insert data with sequential scan from head (this is slow on exsisting large amount of records situation) \u003c/del\u003e\n- [ ] UNION clause\n- [x] Materialization\n  - Classes which offers functionality for materialization exists\n  - Now, HashJoinExecutor only do materialization with the classes \n- [ ] Authentication\n- [ ] Making Usable from OR Mapper of One Web Framework Such as Django (Python) on Simple Application Scope\n  - implementation of DB driver/connector for Python is needed (Or supporting major RDBMS compatible NW I/F)  \n  \n[^1]: Use on frontend is not supported now  \n\n## Specification (Limitation)\n- Supported Data Types\n  - Integer (int32)\n  - Float (float32)\n  - Varchar (variable length)\n    - Max length is a little less than 4KB now\n  - Boolean (bool/1byte)\n- **to be wrote...** \n\n## More Info\n- [Wiki on this repo](https://github.com/ryogrid/SamehadaDB/wiki)\n\n## About Skip List Index SamehadaDB has\n- [Implementation of On-Disk Concurrent Skip List as an Alternative to B-Tree Index](https://ryogrid.github.io/articles/skiplist_en)\n\n## Related Project\n- [ryogrid/bltree-go-for-embedding](https://github.com/ryogrid/bltree-go-for-embedding)\n  - Customized [hmarui66/blink-tree-go](https://github.com/hmarui66/blink-tree-go) fork which is usable in other DBMS projects as library\n\n# Advisor\n- [kumagi](https://github.com/kumagi) and more!\n\n# Acknowledgement\n- 2022 May - 2025 April: SamehadaDB project is assisted by OSS community support program of JetBrains s.r.o with offering development tools such as GoLand IDE\n\n# Past work\n- [FunnelKVS: Rust implementation of autonomous distributed key-value store which has REST interfaces](https://github.com/ryogrid/rust_dkvs)\n\n# Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=ryogrid/SamehadaDB\u0026type=Date)](https://star-history.com/#ryogrid/SamehadaDB\u0026Date)\n","funding_links":["https://github.com/sponsors/ryogrid"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryogrid%2Fsamehadadb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryogrid%2Fsamehadadb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryogrid%2Fsamehadadb/lists"}