{"id":13490878,"url":"https://github.com/cursusdb/cursusdb","last_synced_at":"2026-01-12T02:41:20.281Z","repository":{"id":208635854,"uuid":"722117815","full_name":"cursusdb/cursusdb","owner":"cursusdb","description":"CursusDB is an open-source distributed in-memory yet persisted document oriented database system with real time capabilities. ","archived":false,"fork":false,"pushed_at":"2024-05-06T05:05:22.000Z","size":56958,"stargazers_count":433,"open_issues_count":0,"forks_count":12,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-10-31T04:34:58.240Z","etag":null,"topics":["database","database-system","database-systems","dbms","distributed-database","distributed-systems","document-oriented-database","golang","multi-platform","networking","nosql","nosql-database","open-source","realtime-database","schemaless","sql-like","unstructured-documents"],"latest_commit_sha":null,"homepage":"https://cursusdb.com","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cursusdb.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-22T13:26:09.000Z","updated_at":"2024-10-31T04:18:53.000Z","dependencies_parsed_at":"2023-12-09T02:34:39.489Z","dependency_job_id":"28cca81f-f235-4f54-b07f-515a37929500","html_url":"https://github.com/cursusdb/cursusdb","commit_stats":null,"previous_names":["cursusdb/cursus","cursusdb/cursusdb"],"tags_count":99,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cursusdb%2Fcursusdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cursusdb%2Fcursusdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cursusdb%2Fcursusdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cursusdb%2Fcursusdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cursusdb","download_url":"https://codeload.github.com/cursusdb/cursusdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245989126,"owners_count":20705761,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","database-system","database-systems","dbms","distributed-database","distributed-systems","document-oriented-database","golang","multi-platform","networking","nosql","nosql-database","open-source","realtime-database","schemaless","sql-like","unstructured-documents"],"created_at":"2024-07-31T19:00:51.721Z","updated_at":"2026-01-12T02:41:20.275Z","avatar_url":"https://github.com/cursusdb.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"---- \n\n\n\u003cdiv\u003e\n\u003ch1 align=\"center\"\u003e\u003cimg src=\"images/cursusdb-header.png\"\u003e\u003c/h1\u003e\n\u003c/div\u003e\n\n## Cursus Database System\nCursusDB is a fast open source in-memory document oriented database offering security, persistence, distribution, availability and an SQL like query language(CDQL).\n\n## Table of contents\n\u003cul\u003e\n\n\u003cli\u003e\u003ca href=\"#features\"\u003eFeatures\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#design-drawings\"\u003eDesign Drawings\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#docker\"\u003eDocker\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#native-clients\"\u003eNative Clients\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#native-observers\"\u003eNative Observers\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#prebuild-binaries\"\u003ePrebuilt Binaries\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#cluster-node-setup\"\u003eCluster \u0026 Node Building \u0026 Initial Setup\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#node-replicating\"\u003eNode Replicating\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#query-language\"\u003eQuery Language\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#ping-cluster\"\u003ePing the cluster\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#inserts\"\u003eInserts\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#selects\"\u003eSelects\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#updates\"\u003eUpdates\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#deletes\"\u003eDeletes\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#pattern-matching\"\u003ePattern Matching\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#like\"\u003eLIKE\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#not-like\"\u003eNOT LIKE\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#sorting\"\u003eSorting\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#counting\"\u003eCounting\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#uniqueness\"\u003eUniqueness\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#delete-key\"\u003eDeleting a key within documents in a collection\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#operators\"\u003eOperators\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#conditional-symbols\"\u003eConditional Symbols\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#actions\"\u003eActions\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#list-collections\"\u003eList collections\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#deleting-collections\"\u003eDeleting collections\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#database-users\"\u003eDatabase users\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#listing-database-users\"\u003eListing database users\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#remove-database-users\"\u003eRemoving database users\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#logging\"\u003eLogging\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#status-codes\"\u003eStatus Codes\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#other\"\u003eOther\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#authentication-authorization\"\u003eAuthentication / Authorization\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#node-cluster\"\u003eNode / Cluster\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#document-cdql\"\u003eDocument \u0026 CDQL\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#default-ports\"\u003eDefault Ports\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#reserved-doc-keys\"\u003eReserved Document Keys\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#mysql-v-cursusdb\"\u003eMySQL v CursusDB Benchmark\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#cluster-to-node-tls-and-node-to-node-tls\"\u003eCluster to Node TLS connectivity \u0026 Node to Node replica TLS connectivity\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#what-is-a-node-observer\"\u003eWhat is a Node Observer?\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#doc-expectation-relation\"\u003eDocument Expectation \u0026 Document Relation\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#live-chat-observer\"\u003eLive Chat using an Observer\u003c/a\u003e\n\u003cli\u003e\u003ca href=\"#report-issues\"\u003eReporting Issues\u003c/a\u003e\n\u003c/ul\u003e\n\n\n\n📙📙 https://cursusdb.com/documentation\n\n\n\u003e  The idea behind CursusDB was to create something exceedingly scalable whilst never really slowing down. Say you have 1 billion documents stored within 1 collection spread across 100 nodes the cluster will query 1 billion documents in the time it takes to query 10 million as the cluster initiates a non insert action on all nodes simultaneously. This is the power of parallel search. The Cursus(cluster) system is searching say in the users collection in multiple sections of the collection simultaneously.  A cluster can query thousands of nodes at the same time.  Think of main nodes as shards of many or one collection.  Each collection locks on insert, update and delete but because of CursusDB's distributed design it's like a concurrent switch board that allows for large amounts of concurrent transactions.  A cluster or many clusters take actions, these actions are relayed as requests to 1 or many nodes simultaneously.  Consistency and reliability was one of the main goals when designing CursusDB.  If you have many cluster's setup through a TCP load balancer you can imagine transactions just don't miss if the system is configured correctly.  One more bit! Say you have multiple updates to one document the node will work in order of operation received.  The database system is well-designed, heavily tested and very stable;  It was designed and developed for my own need's with other projects/companies I have going on; Over the period of design and development it's very much turned into something special. With that I hope you all enjoy CursusDB! \n\n~ Alex Gaetano Padula\n\n\u003ch6 id=\"features\" /\u003e\n\n### Features\n- Secured cluster and node(s) communication with shared key and BASIC AUTH type implementation and OR TLS\n- If configured secured node replication sync with TLS using ``tls-replication`` config within ``.curodeconfig``\n- In-memory data during runtime\n- Parallel search. Searching section of collections within multiple nodes or replicas simultaneously at the same time.\n- Auto generated $id key for all documents unique across all nodes\n- Database Users with basic (R, RW) permissions\n- Cluster node data replication and synchronization specifically for reads\n- JSON object insert\n- Unstructured collections\n- Cluster and client authentication using BASIC AUTH type implementation\n- Node(s) (insert, update, delete) relay to observers in real time\n- Node observer automatic reconnect if connection lost\n- SQL like query language (CDQL - Cursus Document Query Language)\n- Low-latency\n- Highly available\n- Unique k:v across all nodes using exclamation at end of key name ``email!``\n- Secure by default with shared key and users\n- Highly configurable\n- Lightweight core code under 6000 lines of code in total\n- File logging and automatic log truncation based on ``log-max-lines`` config\n- Automatic reconnect of any lost node or node replica\n- Automatic node backups if ``automatic-backup`` within ``.curodeconfig`` is set to true\n- Automatic node backup clean up if ``automatic-backup-cleanup`` within ``.curodeconfig`` is set to true.\n- Automatic node recovery if data is corrupt if ``automatic-backup`` configured\n- Node data(`.cdat`) and node backups (`/backups/.cdat.{unixtime}`) are created by taking what's in memory serializing it, encrypting it and compressing it block by block via serialization-encryption(chacha20poly1305)-compression(DEFLATE)  on shutdown or backup\n\n**There are no databases like MySQL let's say where you can have multiples.  A cluster is your database that spreads data across many nodes.**\n\n\u003ch6 id=\"design-drawings\" /\u003e\n\n---\n\n![drawing3.png](images/drawing3.png)\n\n![drawing5.png](images/drawing5.png)\n\n![drawing32.png](images/drawing32.png)\n\n![drawing63.png](images/drawing63.png)\n\n![drawing82.png](images/drawing82.png)\n\n![drawing102.png](images/drawing102.png)\n\n![drawing232.png](images/drawing232.png)\n\nA node keeps track of queries/txns and if something bad happens can re-trigger what hasn't been processed.  A node syncs to a .qqueue file every 70 milliseconds this is fixed and cannot be changed.\n`.qqueue` files are encrypted.\n\n\n\u003ch6 id=\"docker\" /\u003e\n\n## Docker\nhttps://hub.docker.com/repositories/cursusdb (SOON)\n\n\u003ch6 id=\"native-clients\" /\u003e\n\n## Native Clients\n- SHELL https://github.com/cursusdb/curush ``STABLE ✔️``\n- GO https://github.com/cursusdb/cursusdb-go ``STABLE ✔️``\n- NODE.JS https://github.com/cursusdb/cursusdb-node OR https://www.npmjs.com/package/cursusdb-node ``STABLE ✔️``\n- PYTHON https://github.com/cursusdb/cursusdb-py ``STABLE ✔️``\n- JAVA https://github.com/cursusdb/cursusdb-java ``STABLE ✔️``\n- CSHARP https://github.com/cursusdb/cursusdb-cs \u0026 Available on nuget! ``STABLE ✔️``\n\n\u003ch6 id=\"native-observers\" /\u003e\n\n## Native Observers\n- NODE.JS https://github.com/cursusdb/cursusdb-observer-node OR https://www.npmjs.com/package/cursusdb-observer-node ``STABLE ✔️``\n- GO https://github.com/cursusdb/cursusdb-observer-go ``IN PROGRESS 👨‍💻``\n\n.. more coming\n\n\u003ch6 id=\"prebuild-binaries\" /\u003e\n\n## Prebuilt Binaries\nYou can find the latest stable release prebuilt binaries at\nhttps://cursusdb.com/downloads\n\n\u003ch6 id=\"cluster-node-setup\" /\u003e\n\n## Cluster \u0026 Node Building \u0026 Initial Setup\nGetting started with CursusDB is extremely easy!  First you must build a cluster and node binary.  To do that clone the source and follow below:\n\nYou must make sure you have GO installed minimum version 1.21.3, once installed follow below.\n``` \ngit clone git@github.com:cursusdb/cursusdb.git\ncd cluster\ngo build .\ncd ..\ncd node \ngo build .\n```\n\nNow you should have a curode and a cursus binary.\n\n![img.png](images/img.png)\n\n![img_1.png](images/img_1.png)\n\nNow with both we first start cursus to setup a database user, .cursusconfig and a shared key which will be used for your node as well.  This key is used to authenticate your cluster and nodes also encrypt your data at rest with ChaCha!\n\n![img_2.png](images/img_2.png)\n\nSo now that we have our credentials setup we have to setup our first node!\n\nWe can run a node on the same instance as a cluster for this example.  After completion of cluster setup through the initial run you'll get a .cursusconfig which has a few configurations.\n``` \nnodes: []\nhost: 0.0.0.0\ntls-node: false\ntls-cert: \"\"\ntls-key: \"\"\ntls: false\nport: 7681\nkey: QyjlGfs+AMjvqJd/ovUUA1mBZ3yEq72y8xBQw94a96k=\nusers:\n    - YWxleA==:7V8VGHNwVTVC7EktlWS8V3kS/xkLvRg/oODmOeIukDY=\nnode-reader-size: 2097152\nlog-max-lines: 1000\njoin-responses: false\nlogging: false\ntimezone: Local\nlog-query: false\nnode-read-deadline: 2\n```\n- **nodes** - database cluster nodes.  i.e an ip/fqdn + port combination (cluster1.example.com:7682)\n- **tls-node** - whether the cluster will connect to nodes via tls\n- **tls-cert** - path to your tls cert for cluster\n- **tls-key** - path to your tls key for cluster\n- **tls** - enable or disable tls \n- **port** - cluster port\n- **key** - encoded shared key\n- **users** - array of database users serialized, and encoded.\n- **node-reader-size** - the max size of a response from a node\n- **join-responses** - join all node responses and limit based on provided n\n- **logging** - start logging to file\n- **timezone** - Default is Local but format allowed is for example America/Toronto\n- **log-query** - Logs client ip and their query to logs and std out if enabled\n- **node-read-deadline** - Amount of time in seconds to wait for a node to respond\n\nLet's put in under nodes a local node we will start shortly.\n``` \nnodes:\n- host: 0.0.0.0\n  port: 7682\n```\n\nNow with your .cursusconfig setup let's start our node for the first time.\n\n![img_92.png](images/img_92.png)\n\nYou'll see that I've added the same key as I did for the cluster and the node is now started! \n\nLet's start our cluster now.\n\n![img_4.png](images/img_4.png)\n\nLook at that!  We are all set to start inserting data.  Let's insert a user document into a users collection with a unique email key value using the curush(CursusDB Shell)\n\n![img_5.png](images/img_5.png)\n\nWe can use curush with flag ``--host`` which will use the default port for a cluster 7681.  If we wanted to specify a different port we can used the ``--port ``flag.  If your cluster is using TLS make sure when using curush to also enable tls using flag ``--tls=true``.\n\n\ncurush will ask for a database user username and password to connect to cluster.  Once authorized you can start running queries!\n``` \ninsert into users({\"name\": \"Alex\", \"lastName\": \"Padula\", \"age\": 28, \"email!\": \"apadula@cursusdb.com\"});\n```\n![img_7.png](images/img_7.png)\n\nOn inserts every document will get a unique ``$id `` key which is unique across all nodes.\n\n![img_8.png](images/img_8.png)\n\nIf we try and insert the same document we will get an error stating an existing document already exists.  This is because we set ``email`` with and ``!`` \n\n![img_9.png](images/img_9.png)\n\n\u003ch6 id=\"node-replicating\" /\u003e\n\n## Node Replicating\n.cursusconfig\n```\nnodes:\n- host: 0.0.0.0 # node host i.e an IP or FQDN\n  port: 7682 # node port \n  replicas:\n  - host: 0.0.0.0 # replica host i.e an IP or FQDN\n    port: 7683 @ replica port.  The reason we have 7683 here is because a replica is a completely seperate node from the main.\n..\n```\n\nThe cluster makes connections on start up to your node and node replicas hence configuring the .cursusconfig the way it is.  The cluster keeps those connections alive for fast reactivity.  The cluster will automatically reconnect to any lost node.\n\nNode at ``0.0.0.0:7682`` has a configured replica at ``0.0.0.0:7683``\n\nOn the nodes end you need to configure a replica so the node you're configuring knows to replicate the data over.\n\n.curodeconfig\n``` \nreplicas:\n  - host: 0.0.0.0\n    port: 7683\ntls-cert: \"\"\ntls-key: \"\"\n..\n```\n\nDefault sync time is 10 minutes and can be configured with yaml config ``replication-sync-time`` the node will sync its data to its configured replicas.\nIf original node shuts down or is not available a replica will be used for reads, if a replica is not available another available replica will be used(a node can configure multiple replicas).\n\n![replicating-cluster-nodes.png](images/replicating-cluster-nodes.png)\n\n\u003ch6 id=\"query-language\" /\u003e\n\n## Query Language\nCase-sensitive.. Keep it lowercase as the examples.\n\n\u003ch6 id=\"ping-cluster\" /\u003e\n\n### Ping the cluster\nUsing curush or native client\n\n```\n\u003e ping;\n\u003e pong;\n```\n\n\u003ch6 id=\"inserts\" /\u003e\n\n### Inserts\n```\ninsert into users({\"name\": \"Alex\", \"last\": \"Lee\", \"age\": 28});\ninsert into users({\"name\": \"John\", \"last\": \"Josh\", \"age\": 28, \"tags\": [\"tag1\", \"tag2\"]});\n```\n\n\u003ch6 id=\"selects\" /\u003e\n\n### Selects\n```\nselect {LIMIT} from {COLLECTION} where {CONDITIONS} {ORDERING}\nselect * from users;\nselect 0,2 from users;\nselect 1 from users where name == 'Alex' || name == 'John';\nselect * from users where name == 'Alex' \u0026\u0026 age == 28;\nselect * from users where tags == \"tag1\";\nselect * from users where name == 'Alex' \u0026\u0026 age == 28 \u0026\u0026 tags == 'tag1';\n```\n\n##### NOTE\nYou can use ``==`` OR ``=``\n\nFor example \n``` \nselect 1 from users where name == 'Alex' || name == 'John';\n```\n\nOR\n``` \nselect 1 from users where name = 'Alex' || name = 'John';\n```\n\n\u003ch6 id=\"updates\" /\u003e\n\n### Updates\n```\nupdate {LIMIT} in {COLLECTION} where {CONDITIONS} {SETS} {ORDERING}\nupdate 1 in users where age \u003e= 28 set name = 'Josie' order by createdAt desc;\nupdate * in users where age \u003e 24 \u0026\u0026 name == 'Alex' set name = 'Josie' set age = 52;\nupdate n, n..\nect..\n```\n\n\u003ch6 id=\"deletes\" /\u003e\n\n### Deletes\n```\ndelete {LIMIT} from {COLLECTION} where {CONDITIONS} {ORDERING}\ndelete * from users where age \u003e= 28 || age \u003c 32;\ndelete 0,5 from users where age \u003e 28 \u0026\u0026 name == 'Alex';\nect\n```\n\n\u003ch6 id=\"pattern-matching\" /\u003e\n\n### Pattern Matching\n\n\u003ch6 id=\"like\" /\u003e\n\n#### LIKE\nStarts with 'A'\n``` \nselect * from users where firstName like 'A%lex Padula'\n```\n\nEnds with 'la'\n``` \nselect * from users where firstName like 'Alex Padu%la'\n```\n\nContains Pad\n``` \nselect * from users where firstName like 'Alex %Pad%ula'\n```\n\n\u003ch6 id=\"not-like\" /\u003e\n\n#### NOT LIKE\nStarts with 'A'\n``` \nselect * from users where firstName not like 'A%lex Padula'\n```\n\nEnds with 'la'\n``` \nselect * from users where firstName not like 'Alex Padu%la'\n```\n\nContains Pad\n``` \nselect * from users where firstName not like 'Alex %Pad%ula'\n```\n\n\u003ch6 id=\"sorting\" /\u003e\n\n### Sorting\n``` \nselect * from users order by createdOn desc;\n```\n\n``` \nselect * from users order by firstName asc;\n```\n\n\u003ch6 id=\"counting\" /\u003e\n\n### Counting\nExample\n``` \nselect count from users where $id == \"099ade86-93a8-4703-abdd-d1ccc1078b1d\";\n```\n\nResponse not joined\n``` \n[{\"127.0.0.1:7682\": [{\"count\":1}]}]\n```\n\nResponse joined if each node has 1 match and there is 5 nodes\n``` \n{\"count\":5} \n```\n\n\u003ch6 id=\"delete-key\" /\u003e\n\n### Deleting a key within documents in a collection \nIt's very simple to alter a collections documents.  Say you want to remove the ``y`` key from a documents like below:\n``` \n[{\"$id\":\"fcb773f6-2d77-45fe-a860-9dd94f5e7c07\",\"x\":5,\"y\":7},{\"$id\":\"a567925e-dbb1-405e-b4ac-12522b33d07e\",\"x\":2,\"y\":4},{\"$id\":\"4fa938f6-6813-4db9-9955-f5e3c81a9c0b\",\"x\":55,\"y\":9}]}]\n```\n\nSimple using a native client:\n``` \ncurush\u003edelete key y in example;\n[{\"127.0.0.1:7682\": {\"message\":\"Document key removed from collection successfully.\",\"statusCode\":4021,\"altered\":3}}]\n```\n\n\u003ch6 id=\"uniqueness\" /\u003e\n\n### Uniqueness\nusing ``key!`` will make sure the value is unique across all nodes!\n``` \ninsert into users({\"email!\": \"test@example.com\" ...});\n```\n\n\u003ch6 id=\"operators\" /\u003e\n\n### Operators\n- ``\u003e``\n- ``\u003e=``\n- ``\u003c``\n- ``\u003e=``\n- ``==``\n- ``=``\n- ``!=``\n\n\u003ch6 id=\"conditional-symbols\" /\u003e\n\n### Conditional Symbols\n- ``\u0026\u0026``\n- ``||``\n\n\u003ch6 id=\"actions\" /\u003e\n\n### Actions\n- ``select``\n- ``update``\n- ``delete``\n\n\u003ch6 id=\"list-collections\" /\u003e\n\n### List collections\n```\ncurush\u003ecollections;\n[{\"127.0.0.1:7682\": {\"collections\":[\"losers\",\"winners\",\"users\"]}}]\n```\n\n\u003ch6 id=\"deleting-collections\" /\u003e\n\n### Deleting collections?\nWhen you remove every document from a collection the collection is removed i.e\n\n``` \ndelete * from losers;\n```\n\n``` \n...\"1 Document(s) deleted successfully.\",\"statusCode\":2000}}]\n```\n\n```\ncurush\u003ecollections;\n[{\"127.0.0.1:7682\": {\"collections\":[\"winners\",\"users\"]}}]\n```\n\n\u003ch6 id=\"database-users\" /\u003e\n\n### Database Users\nCursusDB has 2 permissions R(read) and (RW).  RW can select, insert, delete, update and add new users whereas users with just R can only read.\n\n``` \nnew user USERNAME, PASSWORD, P\n```\n\nUsing a client like ``curush`` the CursusDB Shell Program.\n\n``` \ncurush\u003e new user someusername, somepassword, RW;\n```\n\n\u003ch6 id=\"listing-database-users\" /\u003e\n\n#### Listing Database Users\n\nGetting all database users.  User with RW permission required.\n\n```\nusers;\n```\n\ncommand returns JSON array of database users.\n``` \n[\"alex\",\"daniel\"]\n```\n\n\u003ch6 id=\"remove-database-users\" /\u003e\n\n#### Removing Database Users\n``` \ndelete user USERNAME;\n```\n\n\u003ch6 id=\"status-codes\" /\u003e\n\n## Status codes\nA CursusDB status code is a numerical value assigned to a specific message.  The numerical values are used as a shorthand to the actual message.  They are grouped by \n- ``Other`` signals, shutdowns\n- ``Authentication / Authorization`` cluster and node auth\n- ``Document \u0026 CDQL`` document and query language\n\n\u003ch6 id=\"other\" /\u003e\n\n#### Other\n- ``-1`` Received signal (with signal) -1 is just for the system it doesn't mean error in CursusDB's case.\n\n\u003ch6 id=\"authentication-authorization\" /\u003e\n\n#### Authentication / Authorization\n- ``0`` Authentication successful.\n- ``1`` Unable to read authentication header.\n- ``2`` Invalid authentication value.\n- ``3`` No user exists\n- ``4`` User not authorized\n- ``5`` Failed node sync auth\n\n\u003ch6 id=\"node-cluster\" /\u003e\n\n#### Node / Cluster\n- ``100`` - Node is at peak allocation\n- ``101`` - Invalid permission\n- ``102`` - User does not exist\n- ``103`` - Database user already exists\n- ``104`` - No node was available for insert\n- ``105`` - Node unavailable\n- ``106`` - Node ready for sync\n- ``107`` - Node replica synced successfully\n- ``108`` - Could not decode serialized sync data into hashmap\n- ``109`` - No previous data to read.  Creating new .cdat file\n- ``110`` - Could not open log file (with description)\n- ``111`` - Data file corrupt (with description)\n- ``112`` - Collection mutexes created\n- ``113`` - Could not unmarshal system yaml configuration (with description)\n- ``114`` - Could not marshal system yaml configuration (with description)\n- ``115`` - Could not decode configured shared key (with description)\n- ``116`` - Reconnected to lost connection (includes host:port)\n- ``117`` - Reconnected to lost observer connection (includes host:port)\n- ``118`` - Could not open/create configuration file (with description)\n- ``119`` - Could not open/create data file (with description)\n- ``120`` - No .qqueue file found.  Possibly first run, if so the node will create the .qqueue file after run of this method (after first run you will normally see ``505 0 recovered and processed from .qqueue.`` 0 being what was left on the query queue)\n- ``200`` - New database user created successfully\n- ``201`` - Database user removed successfully\n- ``202`` - Could not decode user username\n- ``203`` - Could not marshal users list array\n- ``204`` - There must always be one database user available\n- ``205`` - Could not marshal user for creation\n- ``206`` - Could not get node working directory for automatic backup (with description)\n- ``207`` - Could not create automatic backups directory (with description)\n- ``208`` - Could not read node backups directory  (with description)\n- ``209`` - Could not remove .cdat backup {FILE NAME} (with description)\n- ``210`` - Could not get node working directory for automatic recovery (with description)\n- ``211`` - Node recovery from backup was successful\n- ``214`` - Node was unrecoverable after all attempts\n- ``215`` - Attempting automatic recovery with latest backup\n- ``216`` - Starting to sync to with master node\n- ``217`` - Synced up with master node (with addr)\n- ``218`` - Observer HOST:PORT was unavailable during relay\n- ``219`` - Could not encode data for sync (with description)\n- ``220`` - Starting to write node data to file\n- ``221`` - Starting to write node data to backup file\n- ``222`` - Node data written to file successfully\n- ``223`` - Node data written to backup file successfully\n- ``224`` - Observer connection established (with info)\n- ``225`` - Node connection established (with info)\n- ``500`` - Unknown error (with description)\n- ``502`` - Node could not recover query queue\n- ``503`` - Could not dial self to requeue queries (with description)\n- ``504`` - Could not commit to queued query/transaction\n- ``505`` - n recovered and processed from .qqueue\n- ``507`` - Error loading X509 key pair (with description)\n\n\u003ch6 id=\"document-cdql\" /\u003e\n\n#### Document \u0026 CDQL\n- ``2000`` Document inserted/updated/deleted\n- ``4000`` Unmarsharable JSON insert\n- ``4001`` Missing action\n- ``4002`` None existent action\n- ``4003`` Nested JSON objects not permitted\n- ``4004`` Document already exists\n- ``4005`` Invalid command/query\n- ``4006`` From is required\n- ``4007`` Invalid query operator\n- ``4008`` Set is missing =\n- ``4009`` Invalid insert query missing 'insert into'\n- ``4010`` Invalid insert query is missing parentheses\n- ``4011`` Invalid update query missing set\n- ``4012`` Could not marshal JSON\n- ``4013`` Unparsable boolean value\n- ``4014`` Unparsable float value\n- ``4015`` Unparsable integer value\n- ``4016`` Missing limit value\n- ``4017`` Invalid query\n- ``4018`` Unmarsharable JSON\n- ``4019`` Update sets are missing\n- ``4020`` In is required\n- ``4021`` Document key removed from collection successfully\n- ``4022`` No documents found to alter\n- ``4023`` No unique $id could be found for insert\n- ``4024`` Batch insertion is not supported\n- ``4025`` Where is missing values\n- ``4026`` Delete key missing in\n- ``4027`` Limit skip must be an integer (with description)\n- ``4028`` Could not convert limit value to integer (with description)\n- ``4029`` Invalid limiting value (with description)\n- ``4030`` Key cannot use reserved word\n- ``4031`` Key cannot use reserved symbol\n- ``4032`` Invalid set array values (with description)\n\n\u003ch6 id=\"reserved-doc-keys\" /\u003e\n\n## Reserved Document Keys\nOn insert there are a variety of RESERVED keys.\n- `count`\n- `$id`\n- `$indx`\n- `in`\n- `not like`\n- `!like`\n- `where`\n- `chan`\n- `const`\n- `continue`\n- `defer`\n- `else`\n- `fallthrough`\n- `func`\n- `go`\n- `goto`\n- `if`\n- `interface`\n- `map`\n- `select`\n- `struct`\n- `switch`\n- `var`\n- `false`\n- `true`\n- `uint8`\n- `uint16`\n- `uint32`\n- `uint64`\n- `int8`\n- `int16`\n- `int32`\n- `int64`\n- `float32`\n- `float64`\n- `complex64`\n- `complex128`\n- `byte`\n- `rune`\n- `uint`\n- `int`\n- `uintptr`\n- `string`\n- `==`\n- `\u0026\u0026`\n- `||`\n- `\u003e`\n- `\u003c`\n- `=`\n- `*`\n\n\u003ch6 id=\"default-ports\" /\u003e\n\n## Ports\nDefault cluster port: ``7681``\nDefault node port: ``7682``\n\n\u003ch6 id=\"logging\" /\u003e\n\n## Logging \nLogs for the CursusDB cluster and node are found where you launch your binaries.\nCluster: ``cursus.log``\nNode: ``curode.log``\n\nYou can enable logging on either cluster or node enabling logging.  This will log to file instead of stdout\n``` \nlogging: true\n```\n\nWithin your yaml configs you can set ``log-max-lines`` this option will tell either node or cluster when to truncate(clear up) the log file(s).\n\n### How are logs are formatted?\n[LEVEL][YOUR CONFIGURED TZ RFC822 DATE] DATA\n\nLogs can have either level:\n- \u003cspan style=\"color: red\"\u003eERROR\u003c/span\u003e\n- \u003cspan style=\"color: purple\"\u003eFATAL\u003c/span\u003e\n- \u003cspan style=\"color: yellow\"\u003eINFO\u003c/span\u003e\n- \u003cspan style=\"color: orange\"\u003eWARN\u003c/span\u003e\n\n``` \n[INFO][26 Dec 23 08:34 EST] main(): 112 Collection mutexes created.\n[INFO][26 Dec 23 08:34 EST] SignalListener(): -1 Received signal interrupt starting database shutdown.\n[INFO][26 Dec 23 08:34 EST] WriteToFile(): 220 Starting to write node data to file.\n[INFO][26 Dec 23 08:34 EST] WriteToFile(): 222 Node data written to file successfully.\n```\n\n### Example using curush querying cluster\n``` \n./curush -host 0.0.0.0\nUsername\u003e ******\nPassword\u003e *****\ncurush\u003eselect * from users;\n\n127.0.0.1:7682: [{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec90\",\"age\":28,\"last\":\"Padula\",\"name\":\"Alex\"}]\ncurush\u003eselect * from users;\n\n127.0.0.1:7682: [{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec90\",\"age\":28,\"last\":\"Padula\",\"name\":\"Alex\"}]\ncurush\u003einsert into users({\"name\": \"Alex\", \"last\": \"Lee\", \"age\": 28});\n\n{\"collection\": \"users\", \"insert\":{\"$id\":\"ecaaba0f-d130-42c9-81ad-ea6fc3461379\",\"age\":28,\"last\":\"Lee\",\"name\":\"Alex\"},\"message\":\"Document inserted\",\"statusCode\":2000}\ncurush\u003eselect * from users;\n\n127.0.0.1:7682: [{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec90\",\"age\":28,\"last\":\"Padula\",\"name\":\"Alex\"},{\"$id\":\"ecaaba0f-d130-42c9-81ad-ea6fc3461379\",\"age\":28,\"last\":\"Lee\",\"name\":\"Alex\"}]\n```\n\n^ Single node\n\nIf multiple nodes you'd see a response similar to the one below\n\n```\ncurush\u003eselect * from users;\n\n127.0.0.1:7682: [{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec90\",\"age\":28,\"last\":\"Doe\",\"name\":\"John\"},..]\n127.0.0.1:7683: [{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec91\",\"age\":32,\"last\":\"Johnson\",\"name\":\"Sarah\"},..]\n127.0.0.1:7684: [{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec92\",\"age\":42,\"last\":\"Stint\",\"name\":\"Peter\"},..]\n\n```\n\nBy default though you wont see above..\n``` \njoin-responses: false\n```\n\nis required to see results for each node.\n\n``join-responses`` joins all documents from nodes and limits based on limit.  For example..\n\n``` \nselect 3 from posts order by createdOn desc;\n```\n\nThe ``select 3`` portion the cluster will get depending on set amount of nodes say you have 5 nodes setup, you will get back 3 * 5 but the cluster will limit to 3 as that what was requested!\n\n\n\u003ch6 id=\"cluster-to-node-tls-and-node-to-node-tls\" /\u003e\n\n## Cluster to Node TLS connectivity \u0026 Node to Node replica TLS connectivity \nIf you set ``tls-node`` on the cluster to true the cluster will expect all nodes to be listening on tls.\n\nIf you set ``tls-replication`` on a cluster node to true the cluster node will expect all node replicas to be listening on tls.\n\n\u003ch6 id=\"what-is-a-node-observer\" /\u003e\n\n## What is a Node Observer?\nA node observer is a backend service using the CursusDB Observer package to listen to incoming node events such as insert, update, and delete in real time.\n\nThe observer must be configured with the same shared key as your nodes and clusters.\n\n\u003ch6 id=\"doc-expectation-relation\" /\u003e\n\n## Document Expectation \u0026 Document Relation\nCursusDB expects simple JSON objects. For example take this user object:\n\n```{\"username!\": \"alex\", \"email!\": \"alex@test.com\", \"password\": \"xxx\", \"interests\": [\"programming\", \"music\", \"botany\"]}```\n\nThis is an object CursusDB likes.\n\nimagine you insert this object into a users collection:\n\n```insert into users({\"username!\": \"alex\", \"email!\": \"alex@test.com\", \"password\": \"xxx\", \"interests\": [\"programming\", \"music\", \"botany\"]})```\n\n```{\"insert\":{\"$id\":\"17cc0a83-f78e-4cb2-924f-3a194dedec90\", \"username!\": \"alex\", \"email!\": \"alex@test.com\", \"password\": \"xxx\", \"interests\": [\"programming\", \"music\", \"botany\"]},\"message\":\"Document inserted\",\"statusCode\":2000}```\nYou can see username and email are set up to be unique using the suffixed ``!``. If CursusDB finds a user with that email or username you'll get back a 4004 error which means document already exists.\n\nNow lets say this user can have many posts.\nWe will create a posts collection with the first post containing the users $id we created.\n\n```insert into posts({\"title\": \"First Post\", \"body\": \"This is a test post\", \"userId\": \"17cc0a83-f78e-4cb2-924f-3a194dedec90\", \"createdOn\": 1703626015})```\n\nAs you can see we sorta just related data so now it's fairly easy to query the database and say hey give me all the users posts like so:\n\n```select * from posts where userId = \"17cc0a83-f78e-4cb2-924f-3a194dedec90\";```\n\nRemember how we had the createdOn as a unix timestamp on our posts documents? Awesome we can sort all the posts and paginate them!\n\nSkipping 10 and grabbing 10\n\n```select 10,10 from posts where userId = \"17cc0a83-f78e-4cb2-924f-3a194dedec90\" order by createdOn desc;```\n\nLet`s say we want to sort the posts by title alphabetically:\n\n```select * from posts where userId = \"17cc0a83-f78e-4cb2-924f-3a194dedec90\" order by title asc;```\n\nThis is how data should be related on CursusDB either a user has many posts or lets say a user has one account profile well same thing just repeat the process.\n\n\u003ch6 id=\"report-issues\" /\u003e\n\n## Report Issues \nPlease report issues, enhancements, etc at:\n- https://github.com/cursusdb/cursusdb/discussions\n- https://github.com/cursusdb/cursusdb/issues\n\n\u003ch6 id=\"mysql-v-cursusdb\" /\u003e\n\n## CursusDB v MySQL BENCHMARK\nMost basic setup.  CursusDB cluster and node hosted same instance no TLS.\nMySQL setup exact same specification of an instance no TLS.\n\n💨 \u003cstrong\u003eMind you Cursus(Cluster) was configured with one node.  If configured with multiple inserts GREATLY speed up more for concurrency but sequentially as well.\u003c/strong\u003e\n\n### CursusDB\n```Connection time: 64ms```\n\nInserting 1002 records sequentially\n\n```insert into users({\"first\": \"James\", \"last\": \"Jones\", \"age\": 22, \"active\": true});```\n\n```Insertion time: 481.190374ms```\n\nRead skipping 1000 selecting 1 where first is James\n\n```select 1000,1 from users where first == \"James\";```\n\n\n``Read time: 743.538µs``\n\n\n### MySQL\n``Connection time: 170ms``\n\nInserting 1002 records sequentially\n\n```INSERT INTO users (first, last, age, active) VALUES (\"James\", \"Jones\", 22, true);```\n\n```Insertion time: 1.928675495s```\n\nRead skipping 1000 selecting 1 where first is James\n\n```SELECT * FROM users where first = \"James\" LIMIT 1 OFFSET 1000;```\n\n\n``Read time: 1.021852ms``\n\nTable used\n\n``CREATE TABLE users (\nfirst varchar(255),\nlast varchar(255),\nage int,\nactive BOOLEAN\n);``\n\n\u003ch6 id=\"live-chat-observer\" /\u003e\n\n## Live Chat drawing using an Observer\nHow would a chat work with an Observer configured?\nLet's say you have 2 collections, a chatrooms collections and a messages collection. On insert the node will relay to an observer if configured. On the backend where the observer lives you can have a socket server sending actions from an observer to many web socket or web transport clients. Take the example, we have 2 users in a chatroom with an $id of `12718b2b-0efe-4fe6-94ec-1adea5f212c8` which is unique. Adam sends a message to Chris which is an insert. At that point the cluster will insert into a node and the node will relay to an observer at which point we can relay that through to our connected clients to a specific room let's say. Very cool stuff!\n\n![drawing182.png](images/drawing182.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcursusdb%2Fcursusdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcursusdb%2Fcursusdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcursusdb%2Fcursusdb/lists"}