{"id":15641881,"url":"https://github.com/pnc/basque","last_synced_at":"2025-04-30T09:16:25.095Z","repository":{"id":66309480,"uuid":"176128450","full_name":"pnc/basque","owner":"pnc","description":"execute shell commands from a SQLite function","archived":false,"fork":false,"pushed_at":"2019-03-23T18:19:15.000Z","size":21,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-30T09:16:24.286Z","etag":null,"topics":["rust-ffi","sqlite","sqlite3","sqlite3-extension"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pnc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-17T16:26:38.000Z","updated_at":"2023-03-30T18:42:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"90c1a291-b38d-4d6e-bb3e-ebdc60422a21","html_url":"https://github.com/pnc/basque","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnc%2Fbasque","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnc%2Fbasque/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnc%2Fbasque/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pnc%2Fbasque/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pnc","download_url":"https://codeload.github.com/pnc/basque/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251674588,"owners_count":21625646,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rust-ffi","sqlite","sqlite3","sqlite3-extension"],"created_at":"2024-10-03T11:52:59.637Z","updated_at":"2025-04-30T09:16:25.060Z","avatar_url":"https://github.com/pnc.png","language":"Rust","readme":"# Basque - execute shell commands from SQLite\n\n## Pitch\n\n`xargs`/`parallel` are all fun and games until you need to include or escape quotes. What if you could use the full power of SQLite to generate, query, and store the results of shell commands?\n\n## Anti-pitch\n\nAnything you could do by running shell commands from SQLite, you could do by piping the output of those commands to SQLite using the incredible [q](http://harelba.github.io/q/) tool. You probably want that instead.\n\n## Example usage\n\n```bash\n$ cargo build\n$ sqlite3\n```\n\n```sql\nConnected to a transient in-memory database.\nUse \".open FILENAME\" to reopen on a persistent database.\nsqlite\u003e .load ./target/debug/libbasque\nsqlite\u003e create table websites(url text, body text);\nsqlite\u003e insert into websites(url) values('https://www.sqlite.org/');\nsqlite\u003e insert into websites(url) values('https://www.rust-lang.org/');\nsqlite\u003e update websites set body = basque_cmd(\"curl\", \"-L\", url);\nsqlite\u003e select url, substr(body, 1, 20) from websites;\nhttps://www.sqlite.org/|\u003c!DOCTYPE HTML PUBLI\nhttps://www.rust-lang.org/|\u003c!doctype html\u003e\n\u003chtm\n```\n\n## Remaining work\n\n- [x] Prototype to confirm `no_mangle` will work and SQLite can load the Rust-built module\n- [x] Generate `sqlite3ext.h` API struct with rust-bindgen\n- [x] Switch to Cargo for building\n- [x] Fix all the places I'm cheating and using `u64` instead of pointer types\n- [ ] Make the `GLOBAL_ROUTINES` struct threadsafe (Mutex? Container crate?)\n- [ ] Handle `SQLITE_*` error results\n- [x] Actually honor `basque_cmd` function parameters\n- [ ] Handle all the panicky `unwrap()`s\n- [x] Free Command result allocations properly\n- [ ] Return `stderr` too, probably? Or log it somehow\n- [ ] Implement a virtual table instead, so each line result is a row\n- [ ] Actually learn Rust\n\n[![CircleCI](https://circleci.com/gh/pnc/basque.svg?style=svg)](https://circleci.com/gh/pnc/basque)\n\n## Development\n\nYou will need a copy of SQLite with module loading enabled. The default (`/usr/bin`) installation on macOS does not, but the Homebrew one does. To use the Homebrew one:\n\n```\nexport PATH=/usr/local/opt/sqlite3/bin:$PATH\n```\n\nIf module loading is disabled, you'll get this error when you try to run `.load ./libbasque`:\n\n```\nError: unknown command or invalid arguments:  \"load\". Enter \".help\" for help\n```\n\n### Useful references\n\n* \u003chttps://www.sqlite.org/src/file/ext/misc/series.c\u003e\n* \u003chttps://www.sqlite.org/capi3ref.html#sqlite3_auto_extension\u003e\n* \u003chttps://doc.rust-lang.org/nomicon/ffi.html\u003e\n* \u003chttps://rust-lang.github.io/rust-bindgen/print.html\u003e\n* \u003chttps://doc.rust-lang.org/1.30.0/book/second-edition/ch19-01-unsafe-rust.html\u003e\n* \u003chttps://github.com/rust-lang/rust-bindgen/blob/master/book/src/whitelisting.md\u003e\n* \u003chttps://github.com/cloudmeter/sqlite/blob/master/sqlite3ext.h\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpnc%2Fbasque","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpnc%2Fbasque","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpnc%2Fbasque/lists"}