{"id":29976772,"url":"https://github.com/chifisource/chidb.jl","last_synced_at":"2026-02-12T08:05:18.328Z","repository":{"id":300336520,"uuid":"738188549","full_name":"ChifiSource/ChiDB.jl","owner":"ChifiSource","description":"toolips + algebra streams = database server","archived":false,"fork":false,"pushed_at":"2025-07-26T05:35:41.000Z","size":318,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-26T11:21:16.651Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChifiSource.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["emmaccode","UnformalPenguin"]}},"created_at":"2024-01-02T16:29:33.000Z","updated_at":"2025-07-09T06:01:02.000Z","dependencies_parsed_at":"2025-06-21T05:38:28.837Z","dependency_job_id":"3530c0ed-a098-47df-ba20-9d091ee1e177","html_url":"https://github.com/ChifiSource/ChiDB.jl","commit_stats":null,"previous_names":["chifisource/chidb.jl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ChifiSource/ChiDB.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChifiSource%2FChiDB.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChifiSource%2FChiDB.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChifiSource%2FChiDB.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChifiSource%2FChiDB.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChifiSource","download_url":"https://codeload.github.com/ChifiSource/ChiDB.jl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChifiSource%2FChiDB.jl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268676020,"owners_count":24288397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-04T02:00:09.867Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-04T09:36:36.184Z","updated_at":"2026-02-12T08:05:13.296Z","avatar_url":"https://github.com/ChifiSource.png","language":"Julia","funding_links":["https://github.com/sponsors/emmaccode","https://github.com/sponsors/UnformalPenguin"],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/ChifiSource/image_dump/blob/main/laboratory/tools/chifiDB.png\" width=\"120\"\u003e\u003c/img\u003e\n\u003c/div\u003e\n\n###### a toolips-based data-base server\n`ChiDB` is a unique data-base server designed around the `AlgebraFrames` concept and the `.ff` file format. Schema is laid using directories and filenames and data is live-read into memory. This is currently in a state of relative infancy, but is primarily being developed for my own use cases and to demonstrate the various server-building capabilities of `Toolips`.\n- [get started](#get-started)\n  - [adding chidb](#adding)\n  - [documentation](#documentation)\n- [chidb setup](#setup)\n  - [schema](#schema)\n    - [feature files](#feature-files)\n    - [readable data-types](#readable-data-types)\n    - [editing schema](#editing-schema)\n- [querying](#querying)\n  - [clients](#clients)\n  - [command list](#commands)\n  - [query examples](#example-queries)\n- [creating chidb clients](#creating-clients)\n  - [existing clients](#existing-clients)\n  - [chidb headers](#headers)\n    - [header RFC](#opcodes)\n\n  \n \n## get started\n\n#### adding\n\n#### documentation\n\n## setup\nIn order to use `ChiDB`, we first need [julia](https://julialang.org). With Julia installed, the package may be added with `Pkg`:\n```julia\nusing Pkg; Pkg.add(url = \"https://github.com/ChifiSource/ChiDB.jl\")\nusing ChiDB\n```\nTo setup a `ChiDB` server directory, run `ChiDB.start` and provide an **empty** directory as `path`. This will create a new folder called `db`, which will contain the data-bases core information.\n```julia\nstart(path::String, ip::IP4 = \"127.0.0.1\":8005)\n```\nOur `admin` login will also be printed here; by querying with this new `admin` login we may create new users.\n#### schema\n`ChiDB` schema is created using two different techniques:\n- *querying*\n- or by creating a filesystem of `.ff` and `.ref` files.\n\n`ChiDB`'s internal data is primarily stored in the `.ff` (*feature file*) format. There are no sub-tables, only reference columns. Both references and `.ff` feature files are represented by files, and the tables they reside in are represented by directories. In order to create schema, we would simply add new folders with new `.ff` files for each column to our new data-base directory. Consider the following sample directory structure:\n- project directory\n  - /db\n  - /table1\n    - /col1.ff\n    - /col2.ff\n    - /table2_col1.ref\n  - /table2\n    - /col1.ff\n\n##### feature files\nEach `.ff` file's first line will be a readable data-type. For example, `col1.ff` from above could be\n```ff\nInteger\n```\n###### readable data-types\n\n### querying\n\n\n##### clients\n\n###### commands\nCommands are issued to the server using \n- `()` indicates an optional argument.\n- `(table)/column` indicates the ability to provide the column if a table is selected, *or* provide a column and table in the `table/column` format. For example:\n```julia\nwrite!(sock, \"$(curr_header)vnewt/name|!|1|!|frank\\n\")\n```\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth\u003echaracter\u003c/th\u003e\n    \u003cth\u003ename\u003c/th\u003e\n    \u003cth\u003edescription\u003c/th\u003e\n    \u003cth\u003earguments\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003el\u003c/td\u003e\n    \u003ctd align=\"center\"\u003elist\u003c/td\u003e\n    \u003ctd\u003elists the columns within a table, and their types, or lists all tables when provided with no argument\u003c/td\u003e\n    \u003ctd\u003e(table)\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003es\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eselect\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eSelects a table.\u003c/td\u003e\n    \u003ctd\u003etable\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003et\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ecreate\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ecreates a new table\u003c/td\u003e\n    \u003ctd\u003etablename\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e\n      \u003cth\u003e\n    \u003cth align=\"center\"\u003eget-store commands\u003c/th\u003e\n      \u003cth\u003e\u003c/th\u003e\n  \u003c/tr\u003e\n      \u003ctr\u003e\n    \u003ctd align=\"center\"\u003eg\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eget\u003c/td\u003e\n    \u003ctd align=\"center\"\u003egets values using vertical indexing\u003c/td\u003e\n    \u003ctd\u003e(table)/column (range)\u003c/td\u003e\n  \u003c/tr\u003e\n        \u003ctr\u003e\n    \u003ctd align=\"center\"\u003er\u003c/td\u003e\n    \u003ctd align=\"center\"\u003egetrow\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eGets a full row of data\u003c/td\u003e\n    \u003ctd\u003e(table)/column rown\u003c/td\u003e\n  \u003c/tr\u003e\n          \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ei\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eindex\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eGets the index where a certain value occurs in a given table.\u003c/td\u003e\n    \u003ctd\u003e(table)/column value\u003c/td\u003e\n  \u003c/tr\u003e\n            \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ea\u003c/td\u003e\n    \u003ctd align=\"center\"\u003estore\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eStores values, separated by `!;`, into a given table. Will return an argument error if the incorrect shape is provided.\u003c/td\u003e\n    \u003ctd\u003e(table) value!;value2\u003c/td\u003e\n  \u003c/tr\u003e\n            \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ev\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eset\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eSets a singular value in a table.\u003c/td\u003e\n    \u003ctd\u003e(table)/column row value\u003c/td\u003e\n  \u003c/tr\u003e\n              \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ew\u003c/td\u003e\n    \u003ctd align=\"center\"\u003esetrow\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eSets the values of an entire row on a table\u003c/td\u003e\n    \u003ctd\u003e(table) row value1!;value2\u003c/td\u003e\n  \u003c/tr\u003e\n            \u003ctr\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003ecolumn management\u003c/th\u003e\n    \u003cth\u003e\u003c/th\u003e\n  \u003c/tr\u003e\n              \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ej\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ejoin\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eAdds a new column to a frame, creates a reference column when used with a column path from another table.\u003c/td\u003e\n    \u003ctd\u003e(table) (reftable)/colname (Type)\u003c/td\u003e\n  \u003c/tr\u003e\n                \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ek\u003c/td\u003e\n    \u003ctd align=\"center\"\u003etype\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eAttempts to cast a given type to a provided column.\u003c/td\u003e\n    \u003ctd\u003e(table)/colname Type\u003c/td\u003e\n  \u003c/tr\u003e\n                \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ee\u003c/td\u003e\n    \u003ctd align=\"center\"\u003erename\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eRenames a given column or table\u003c/td\u003e\n    \u003ctd\u003e(table) table_or_colname\u003c/td\u003e\n  \u003c/tr\u003e\n              \u003ctr\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003edeleters\u003c/th\u003e\n    \u003cth\u003e\u003c/th\u003e\n                \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ed\u003c/td\u003e\n    \u003ctd align=\"center\"\u003edeleteat\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eDeletes a row from a given table.\u003c/td\u003e\n    \u003ctd\u003e(table) row\u003c/td\u003e\n  \u003c/tr\u003e\n                  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ez\u003c/td\u003e\n    \u003ctd align=\"center\"\u003edelete\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eDeletes a table\u003c/td\u003e\n    \u003ctd\u003etable\u003c/td\u003e\n  \u003c/tr\u003e\n                \u003ctr\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003ebuilt-in operations\u003c/th\u003e\n    \u003cth\u003e\u003c/th\u003e\n                \u003ctr\u003e\n                                  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003ep\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ecompare\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eChecks if the provided value is the same as the stored data.\u003c/td\u003e\n    \u003ctd\u003e(table)/column rown value\u003c/td\u003e\n  \u003c/tr\u003e\n                                    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003en\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ein\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eChecks if the provided value is within the column.\u003c/td\u003e\n    \u003ctd\u003e(table)/column value\u003c/td\u003e\n  \u003c/tr\u003e\n                                \u003ctr\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003e\u003c/th\u003e\n    \u003cth align=\"center\"\u003eserver\u003c/th\u003e\n    \u003cth\u003e\u003c/th\u003e\n                \u003ctr\u003e\n                                                    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003eU\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eusers\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eLists current users.\u003c/td\u003e\n    \u003ctd\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n      \u003ctr\u003e\n    \u003ctd align=\"center\"\u003eC\u003c/td\u003e\n    \u003ctd align=\"center\"\u003enewuser\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eCreates a new user. Will return the new name, password, and dbkey.\u003c/td\u003e\n    \u003ctd\u003euser (pwd)\u003c/td\u003e\n  \u003c/tr\u003e\n                                                      \u003ctr\u003e\n    \u003ctd align=\"center\"\u003eK\u003c/td\u003e\n    \u003ctd align=\"center\"\u003esetuser\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eSets any users login -- must be admin to perform.\u003c/td\u003e\n    \u003ctd\u003euser, name (pwd)\u003c/td\u003e\n  \u003c/tr\u003e\n                                                        \u003ctr\u003e\n    \u003ctd align=\"center\"\u003eD\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ermuser\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eRemoves a user by name. Must be admin to perform.\u003c/td\u003e\n    \u003ctd\u003eusername\u003c/td\u003e\n  \u003c/tr\u003e\n                                                      \u003ctr\u003e\n    \u003ctd align=\"center\"\u003eL\u003c/td\u003e\n    \u003ctd align=\"center\"\u003elogout\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eDisconnects from the server.\u003c/td\u003e\n    \u003ctd\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n#### query examples\n\n---\n## creating clients\n\n#### existing clients\n\n#### headers\nYou will likely want an *API* of some sort to query a `ChiDB` servers. Every query, including your initial query, will be sent with a two-byte header. This header includes three fields: the `opcode` (4 bits), the `transaction id` (4 bits) (composing the first byte) and the second byte (8 bits) is dedicated to the *command character* -- a single-character reference that requests a query command.\n- The `opcode` returns a success code dependent on the status of the last query. See [opcodes](#opcodes) for a full list of opcodes.\n- The `transaction id` will be the ID of the transaction that is just issued. This needs to be sent **back** to the server on each query, and will need to be wrapped into each header we send to the server.\n  \n```text\nRequest header\n1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16\n|    opcode   | transac ID    |          command byte           |\n\nResponse header\n1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |\n|    opcode   | transac ID    |\n```\nSo, from the API's perspective both the opcode and transaction ID are sent back to the server, meaning we can just send back the first character we get as our response as the header. The second char would then be our selected query command, and from there our arguments are provided directly and separated by `|!|`. For the initial connection the first character would be nothing, and for `S` we provide spaces as separators.\n```julia\nusing Toolips\n\nsock = Toolips.connect(\"127.0.0.1\":2025)\n\nwrite!(sock, \"nS dbkey admin adminpwd\\n\")\n\nresp = String(readavailable(sock))\n# (opcode)\neightbit_header = UInt8(resp[1])\nheader = bitstring(eightbit_header)\n\nif header[1:4] == \"0001\"\n    println(\"query accepted!\")\nelse\n    throw(\"not verified, the query will not work\")\nend\n# list tables:\nwrite!(sock, string(eightbit_header) * \"l\\n\")\nresp = String(readavailable(sock))\nprintln(resp)\n```\n###### opcodes\n\n\u003cdiv align=\"center\"\u003e\n\u003ctable\u003e\n  \u003ctr\u003e\n  \u003cth\u003ecode\u003c/th\u003e\n  \u003cth\u003estatus\u003c/th\u003e\n  \u003cth\u003ename\u003c/th\u003e\u003c/th\u003e\n  \u003cth\u003ehas output\u003c/th\u003e\n\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e0001\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eOK\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003equery accept\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e0011\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eOK\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003euser created\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e0101\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eOK\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003epassword set\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e1000\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eERROR\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ebad packet\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n      \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e1100\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eERROR\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003elogin denied (connection closed)\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n        \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e1001\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eERROR\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ebad dbkey (connection closed)\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e1110\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eERROR\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ecommand error\u003c/td\u003e\n    \u003ctd align=\"center\"\u003etrue\u003c/td\u003e\n  \u003c/tr\u003e\n      \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e1010\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eERROR\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003eargument error\u003c/td\u003e\n    \u003ctd align=\"center\"\u003etrue\u003c/td\u003e\n  \u003c/tr\u003e\n        \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e1111\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eERROR\u003c/b\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003ebad transaction (connection closed)\u003c/td\u003e\n    \u003ctd align=\"center\"\u003efalse\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchifisource%2Fchidb.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchifisource%2Fchidb.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchifisource%2Fchidb.jl/lists"}