{"id":18743059,"url":"https://github.com/kxsystems/kxkdb","last_synced_at":"2025-04-12T21:24:56.077Z","repository":{"id":65468529,"uuid":"591755467","full_name":"KxSystems/kxkdb","owner":"KxSystems","description":"Kdb+ interface for the Rust programming language","archived":false,"fork":false,"pushed_at":"2023-06-05T14:28:43.000Z","size":293,"stargazers_count":4,"open_issues_count":6,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-12T03:14:12.623Z","etag":null,"topics":["kdb","q","rust"],"latest_commit_sha":null,"homepage":"https://code.kx.com/q/interfaces","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KxSystems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2023-01-21T19:04:52.000Z","updated_at":"2024-11-13T15:54:21.000Z","dependencies_parsed_at":"2023-02-17T08:35:30.388Z","dependency_job_id":null,"html_url":"https://github.com/KxSystems/kxkdb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KxSystems%2Fkxkdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KxSystems%2Fkxkdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KxSystems%2Fkxkdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KxSystems%2Fkxkdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KxSystems","download_url":"https://codeload.github.com/KxSystems/kxkdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248633210,"owners_count":21136828,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kdb","q","rust"],"created_at":"2024-11-07T16:09:54.526Z","updated_at":"2025-04-12T21:24:56.053Z","avatar_url":"https://github.com/KxSystems.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kxkdb\n\nKdb+ interface for the Rust programming language.\n\nThe interface comprises two features:\n- IPC: Connecting Rust and kdb+ processes via IPC\n- API: Embedding Rust code inside kdb+ processes\n\n## Documentation\n\nDocumentation for this interface can be found at https://docs.rs/kxkdb/.\n\n## Kdbplus\n\nThe kxkdb interface is forked from the excellent [kdbplus](https://crates.io/crates/kdbplus) interface, developed by [diamondrod](https://github.com/diamondrod).\n\n## IPC\n\nThe IPC feature enables [qipc](https://code.kx.com/q/basics/ipc/) communication between Rust and kdb+.\n\nConnectivity is via TCP or Unix Domain Sockets, with support for both [compression](https://code.kx.com/q/basics/ipc/#compression) and [TLS encryption](https://code.kx.com/q/kb/ssl/) of messages.\n\nConnection and listener methods are provided, enabling development of both\n- Rust IPC clients of kdb+ server processes\n- Rust IPC servers of kdb+ client processes\n\n### Installation\n\nAdd `kxkdb` as a dependency, with feature `ipc`.\nYou may also want to add an asynchronous runtime such as [Tokio](https://tokio.rs).\n\ne.g.\n```toml\n[dependencies]\nkxkdb = { version = \"0.0\", features = [\"ipc\"] }\ntokio = { version = \"1.24\", features = [\"full\"] }\n```\n\n### Examples\n\n#### Client\n\n```rust\nuse kxkdb::ipc::*;\nuse kxkdb::qattribute;\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    let mut socket;  // socket connection to kdb+ process\n    let mut result;  // result of sync query to kdb+ process\n    let mut message; // compound list containing message\n\n    // connect via UDS to local kdb+ process listening on port 4321\n    socket = QStream::connect(ConnectionMethod::UDS, \"\", 4321_u16, \"\").await?;\n\n    // confirm connection type\n    println!(\"Connection type: {}\", socket.get_connection_type());\n\n    // synchronously query kdb+ process using string\n    result = socket.send_sync_message(\u0026\"sum 1+til 100\").await?;\n    println!(\"result1: {}\", result);\n\n    // asynchronously define function in kdb+ process\n    socket.send_async_message(\u0026\"add_one:{x+1}\").await?;\n   \n    // synchronously call function (correctly)\n    result = socket.send_sync_message(\u0026\"add_one 41\").await?;\n    println!(\"result2: {}\", result);\n   \n    // synchronously call function (incorrectly)\n    result = socket.send_sync_message(\u0026\"add_one`41\").await?;\n    println!(\"result3: {}\", result);\n\n    // synchronously query kdb+ process using compound list\n    message = K::new_compound_list(vec![K::new_symbol(String::from(\"add_one\")), K::new_long(100)]);\n    result = socket.send_sync_message(\u0026message).await?;\n    println!(\"result4: {}\", result);\n    \n    // asynchronously call show function in kdb+ process\n    message = K::new_compound_list(vec![K::new_string(String::from(\"show\"), qattribute::NONE), K::new_symbol(String::from(\"hello from rust\"))]);\n    socket.send_async_message(\u0026message).await?;\n\n    // close socket\n    socket.shutdown().await?;\n\n    Ok(())\n}\n```\n\n#### Server\n\nSetup a credentials file containing usernames and (SHA-1 encrypted) passwords.\n\ne.g.\n```\n$ cat userpass.txt\nfred:e962cde7053eed120f928cd18e58ebd31be77543\nhomer:df43ad44d44e898f8f4e6ed91e6952bfce573e12\n```\nNote: Hashed passwords can be generated in q using `.Q.sha1`.\n\nStore the path of this file in environment variable KDBPLUS_ACCOUNT_FILE.\n\ne.g.\n```\n$ export KDBPLUS_ACCOUNT_FILE=`pwd`/userpass.txt\n```\n\nThe following code will establish a Rust server process, listening on port 4321.\n```rust\nuse kxkdb::ipc::*;\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    let mut socket;   // socket connection to kdb+ process\n\n    // listen for incoming TCP connections on port 4321\n    socket = QStream::accept(ConnectionMethod::TCP, \"127.0.0.1\", 4321).await?;\n\n    // when a connection is established, synchronously send a message to the client\n    let response = socket.send_sync_message(\u0026\"0N!string `Hello\").await?;\n    println!(\"result: {}\", response);\n\n    // close socket\n    socket.shutdown().await?;\n\n    Ok(())\n}\n\n```\n\nA kdb+ client can then connect using the correct credentials.\n\ne.g.\n```q\nq)hopen`:127.0.0.1:4321:fred:flintstone;\n\"Hello\"\n```\n\n### Type Mapping\n\nThe following table displays the input types used to construct different q types (implemented as `K` objects).\n\n| q                | Rust                                      |\n|------------------|-------------------------------------------|\n| `boolean`        | `bool`                                    |\n| `guid`           | `[u8; 16]`                                |\n| `byte`           | `u8`                                      |\n| `short`          | `i16`                                     |\n| `int`            | `i32`                                     |\n| `long`           | `i64`                                     |\n| `real`           | `f32`                                     |\n| `float`          | `f64`                                     |\n| `char`           | `char`                                    |\n| `symbol`         | `String`                                  |\n| `timestamp`      | `chrono::DateTime\u003cUtc\u003e`                   |\n| `month`          | `chrono::NaiveDate`                       |\n| `date`           | `chrono::NaiveDate`                       |\n| `datetime`       | `chrono::DateTime\u003cUtc\u003e`                   |\n| `timespan`       | `chrono::Duration`                        |\n| `minute`         | `chrono::Duration`                        |\n| `second`         | `chrono::Duration`                        |\n| `time`           | `chrono::Duration`                        |\n| `list`           | `Vec\u003cT\u003e` (`T` a corresponding type above) |\n| `compound list`  | `Vec\u003cK\u003e`                                  |\n| `table`          | `Vec\u003cK\u003e`                                  |\n| `dictionary`     | `Vec\u003cK\u003e`                                  |\n| `generic null`   | `()`                                      |\n\nNote: The input type can differ from the inner type. For example, timestamp has an input type of `chrono::DateTime\u003cUtc\u003e` but the inner type is `i64`, denoting an elapsed time in nanoseconds since `2000.01.01D00:00:00`.\n\n### Environment Variables\n\n#### KDBPLUS_ACCOUNT_FILE\n\nPath to a credential file, used by a Rust server to manage access from kdb+ clients.\n\nContains a user name and SHA-1 hashed password on each line, delimited by `':'`.\n\n#### KDBPLUS_TLS_KEY_FILE\n\nThe path to a pkcs12 file used for TLS connections.\n\n#### KDBPLUS_TLS_KEY_FILE_SECRET\n\nThe password for the above pkcs12 file.\n\n#### QUDSPATH\n\nDefines the (real or abstract) path used for [Unix Domain Sockets](https://code.kx.com/q/basics/listening-port/#unix-domain-socket) to `$QUDSPATH/kx.[PORT]`.\n\nn.b. If not defined, this will default to `/tmp/kx.[PORT]`\n\n## API\n\nThe API feature enables the development of shared object libraries in Rust, which can be [dynamically loaded](https://code.kx.com/q/ref/dynamic-load/) into kdb+.\n\nIn order to avoid large `unsafe` blocks, most native C API functions are provided with a wrapper funtion and with intuitive implementation as a trait method. The exceptions are variadic functions `knk` and `k`, which are provided under `native` namespace with the other C API functions.\n\n### Installation\n\nAdd `kxkdb` as a dependency, with feature `api`.\n\n```toml\n[dependencies]\nkxkdb={version=\"0.0\", features=[\"api\"]}\n```\n\n### Examples\n\n#### C API Style\n\n```rust\nuse kxkdb::qtype;\nuse kxkdb::api::*;\nuse kxkdb::api::native::*;\n\n#[no_mangle]\npub extern \"C\" fn create_symbol_list(_: K) -\u003e K {\n    unsafe{\n        let mut list=ktn(qtype::SYMBOL_LIST as i32, 0);\n        js(\u0026mut list, ss(str_to_S!(\"Abraham\")));\n        js(\u0026mut list, ss(str_to_S!(\"Isaac\")));\n        js(\u0026mut list, ss(str_to_S!(\"Jacob\")));\n        js(\u0026mut list, sn(str_to_S!(\"Josephine\"), 6));\n        list\n    }\n}\n \n#[no_mangle]\npub extern \"C\" fn catchy(func: K, args: K) -\u003e K {\n    unsafe{\n        let result=ee(dot(func, args));\n        if (*result).qtype == qtype::ERROR{\n            println!(\"error: {}\", S_to_str((*result).value.symbol));\n            // Decrement reference count of the error object\n            r0(result);\n            KNULL\n        } else {\n            result\n        }\n    }\n}\n\n#[no_mangle]\npub extern \"C\" fn dictionary_list_to_table() -\u003e K {\n    unsafe{\n        let dicts = knk(3);\n        let dicts_slice = dicts.as_mut_slice::\u003cK\u003e();\n        for i in 0..3 {\n            let keys = ktn(qtype::SYMBOL_LIST as i32, 2);\n            let keys_slice = keys.as_mut_slice::\u003cS\u003e();\n            keys_slice[0] = ss(str_to_S!(\"a\"));\n            keys_slice[1] = ss(str_to_S!(\"b\"));\n            let values = ktn(qtype::INT_LIST as i32, 2);\n            values.as_mut_slice::\u003cI\u003e()[0..2].copy_from_slice(\u0026[i*10, i*100]);\n            dicts_slice[i as usize] = xD(keys, values);\n        }\n        // Format list of dictionary as a table.\n        // ([] a: 0 10 20i; b: 0 100 200i)\n        k(0, str_to_S!(\"{[dicts] -1 _ dicts, (::)}\"), dicts, KNULL)\n    } \n}\n```\n\nA kdb+ process can then dynamically load and call these functions as follows:\n\n```q\nq)summon:`libc_api_examples 2: (`create_symbol_list; 1)\nq)summon[]\n`Abraham`Isaac`Jacob`Joseph\nq)`Abraham`Isaac`Jacob`Joseph ~ summon[]\nq)catchy: `libc_api_examples 2: (`catchy; 2);\nq)catchy[$; (\"J\"; \"42\")]\n42\nq)catchy[+; (1; `a)]\nerror: type\nq)behold: `libc_api_examples 2: (`dictionary_list_to_table; 1);\nq)behold[]\na  b  \n------\n0  0  \n10 100\n20 200\n```\n\n#### Rust Style\n\nThe examples below are written without `unsafe` code.\n\n```rust\nuse kxkdb::qtype;\nuse kxkdb::api::*;\nuse kxkdb::api::native::*;\n\n#[no_mangle]\npub extern \"C\" fn create_symbol_list2(_: K) -\u003e K {\n    let mut list = new_list(qtype::SYMBOL_LIST, 0);\n    list.push_symbol(\"Abraham\").unwrap();\n    list.push_symbol(\"Isaac\").unwrap();\n    list.push_symbol(\"Jacob\").unwrap();\n    list.push_symbol_n(\"Josephine\", 6).unwrap();\n    list\n}\n\n#[no_mangle]\nfn no_panick(func: K, args: K) -\u003e K {\n    let result = error_to_string(apply(func, args));\n    if let Ok(error) = result.get_error_string() {\n        println!(\"FYI: {}\", error);\n        // Decrement reference count of the error object which is no longer used.\n        decrement_reference_count(result);\n        KNULL\n    }\n    else{\n        println!(\"success!\");\n        result\n    }\n}\n\n#[no_mangle]\npub extern \"C\" fn create_table2(_: K) -\u003e K {\n    // Build keys\n    let keys = new_list(qtype::SYMBOL_LIST, 2);\n    let keys_slice = keys.as_mut_slice::\u003cS\u003e();\n    keys_slice[0] = enumerate(str_to_S!(\"time\"));\n    keys_slice[1] = enumerate_n(str_to_S!(\"temperature_and_humidity\"), 11);\n\n    // Build values\n    let values = new_list(qtype::COMPOUND_LIST, 2);\n    let time = new_list(qtype::TIMESTAMP_LIST, 3);\n    // 2003.10.10D02:24:19.167018272 2006.05.24D06:16:49.419710368 2008.08.12D23:12:24.018691392\n    time.as_mut_slice::\u003cJ\u003e().copy_from_slice(\u0026[119067859167018272_i64, 201766609419710368, 271897944018691392]);\n    let temperature = new_list(qtype::FLOAT_LIST, 3);\n    temperature.as_mut_slice::\u003cF\u003e().copy_from_slice(\u0026[22.1_f64, 24.7, 30.5]);\n    values.as_mut_slice::\u003cK\u003e().copy_from_slice(\u0026[time, temperature]);\n    \n    flip(new_dictionary(keys, values))\n}\n```\n\nAnd q code is here:\n\n```q\nq)summon:`libc_api_examples 2: (`create_symbol_list2; 1)\nq)summon[]\n`Abraham`Isaac`Jacob`Joseph\nq)chill: `libc_api_examples 2: (`no_panick; 2);\nq)chill[$; (\"J\"; \"42\")]\nsuccess!\n42\nq)chill[+; (1; `a)]\nFYI: type\nq)climate_change: libc_api_examples 2: (`create_table2; 1);\nq)climate_change[]\ntime                          temperature\n-----------------------------------------\n2003.10.10D02:24:19.167018272 22.1       \n2006.05.24D06:16:49.419710368 24.7       \n2008.08.12D23:12:24.018691392 30.5  \n```\n\n## Test\n\nTesting is conducted in two ways:\n\n1. Using cargo\n2. Running a q test script\n\n### 1. Using Cargo\n\nBefore starting the test, start a kdb+ process listening on port 5000.\n\n```bash\n$ q -p 5000\nq)\n```\n\nThen run the test:\n\n```bash\nkxkdb]$ cargo test\n```\n\n**Note:** Currently 20 tests fails for `api` examples in document. This is because the examples do not have `main` function by nature of `api` but still use `#[macro_use]`.\n\n### 2. Running a q Test Script\n\nTests are conducted with `tests/test.q` by loading the example functions built in `api_examples`.\n\n```bash\nkxkdb]$ cargo build --release\nkxkdb]$ cp target/release/libapi_examples.so tests/\nkxkdb]$ cd tests\ntests]$ q test.q\nInitialized something, probably it is your mindset.\nbool: true\nbool: false\nbyte: 0xc4\nGUID: 8c6b-8b-64-68-156084\nshort: 10\nint: 42\nint: 122\nint: 7336\nint: 723\nint: 14240\nint: 2056636\nlong: -109210\nlong: 43200123456789\nlong: -325389000000021\nlong: 0\nreal: 193810.31\nfloat: -37017.09330000\nfloat: 742.41927468\nchar: \"k\"\nsymbol: `locust\nstring: \"gnat\"\nstring: \"grasshopper\"\nerror: type\nWhat do you see, son of man?: a basket of summer fruit\nWhat do you see, son of man?: boiling pot, facing away from the north\nsymbol: `rust\nsuccess!\nFYI: type\nthis is KNULL\nPlanet { name: \"earth\", population: 7500000000, water: true }\nPlanet { name: \"earth\", population: 7500000000, water: true }\nおいしい！\nおいしい！\nおいしい！\nおいしい！\nおいしい！\nおいしい！\nおいしい！\nおいしい！\nおいしい！\nおいしい！\n\"Collect the clutter of apples!\"\ntest result: ok. 147 passed; 0 failed\nq)What are the three largest elements?: `belief`love`hope\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkxsystems%2Fkxkdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkxsystems%2Fkxkdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkxsystems%2Fkxkdb/lists"}