{"id":14156610,"url":"https://github.com/asg017/sqlite-http","last_synced_at":"2025-05-15T19:04:05.798Z","repository":{"id":56066001,"uuid":"416092087","full_name":"asg017/sqlite-http","owner":"asg017","description":"A SQLite extension for making HTTP requests purely in SQL","archived":false,"fork":false,"pushed_at":"2025-02-09T16:15:00.000Z","size":2623,"stargazers_count":244,"open_issues_count":13,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-31T23:34:00.955Z","etag":null,"topics":["sqlite","sqlite-extension"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/asg017.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-11T21:27:49.000Z","updated_at":"2025-03-25T14:10:00.000Z","dependencies_parsed_at":"2024-01-16T22:21:18.012Z","dependency_job_id":"892cb75c-31ca-4c82-af04-711d22fa793f","html_url":"https://github.com/asg017/sqlite-http","commit_stats":{"total_commits":36,"total_committers":1,"mean_commits":36.0,"dds":0.0,"last_synced_commit":"029550d3792c6d7ec755e1715e58927014c62cff"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-http","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-http/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-http/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-http/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/asg017","download_url":"https://codeload.github.com/asg017/sqlite-http/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247755560,"owners_count":20990620,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["sqlite","sqlite-extension"],"created_at":"2024-08-17T08:07:05.400Z","updated_at":"2025-04-08T00:37:53.562Z","avatar_url":"https://github.com/asg017.png","language":"Go","readme":"# sqlite-http\n\nA SQLite extension for making HTTP requests purely in SQL.\n\n- Create GET, POST, and other HTTP requests, like `curl`, `wget`, and `fetch`\n- Download response bodies, header, status codes, timing info\n- Set rate limits, timeouts\n\n## Usage\n\n```sql\n.load ./http0\nselect http_get_body('https://text.npr.org/');\n/*\n\u003c!DOCTYPE html\u003e\n\u003chtml lang=\"en\"\u003e\n\u003chead\u003e\n  \u003ctitle\u003eNPR : National Public Radio\u003c/title\u003e\n  ....\n*/\n```\n\nQuery for all custom headers in an endpoint.\n\n```sql\nselect name, value\nfrom http_headers_each(\n  http_get_headers('https://api.github.com/')\n)\nwhere name like 'X-%';\n/*\n┌────────────────────────┬────────────────────────────────────┐\n│          name          │               value                │\n├────────────────────────┼────────────────────────────────────┤\n│ X-Ratelimit-Limit      │ 60                                 │\n│ X-Ratelimit-Used       │ 8                                  │\n│ X-Content-Type-Options │ nosniff                            │\n│ X-Github-Media-Type    │ github.v3; format=json             │\n│ X-Github-Request-Id    │ CCCA:5FDF:1014BC2:10965F9:62F3DE4E │\n│ X-Ratelimit-Remaining  │ 52                                 │\n│ X-Ratelimit-Resource   │ core                               │\n│ X-Frame-Options        │ deny                               │\n│ X-Ratelimit-Reset      │ 1660152798                         │\n│ X-Xss-Protection       │ 0                                  │\n└────────────────────────┴────────────────────────────────────┘\n*/\n```\n\nScrape data from a JSON endpoint.\n\n```sql\nselect http_get_body('https://api.github.com/repos/sqlite/sqlite')\n  -\u003e\u003e '$.description' as description;\n/*\n┌───────────────────────────────────────────────┐\n│                  description                  │\n├───────────────────────────────────────────────┤\n│ Official Git mirror of the SQLite source tree │\n└───────────────────────────────────────────────┘\n*/\n```\n\nPass in specific headers in a request.\n\n```sql\nselect\n  value\nfrom json_each(\n  http_get_body(\n    'https://api.github.com/issues',\n    http_headers(\n      'Authorization', 'token ghp_16C7e42F292c6912E7710c8'\n    )\n  )\n);\n\n```\n\n## Documentation\n\nSee [`docs.md`](./docs.md) for a full API reference.\n\n## Installing\n\n| Language       | Install                                                      |                                                                                                                                                                                             |\n| -------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Python         | `pip install sqlite-http`                                    | [![PyPI](https://img.shields.io/pypi/v/sqlite-http.svg?color=blue\u0026logo=python\u0026logoColor=white)](https://pypi.org/project/sqlite-http/)                                                      |\n| Datasette      | `datasette install datasette-sqlite-http`                    | [![Datasette](https://img.shields.io/pypi/v/datasette-sqlite-http.svg?color=B6B6D9\u0026label=Datasette+plugin\u0026logoColor=white\u0026logo=python)](https://datasette.io/plugins/datasette-sqlite-http) |\n| Node.js        | `npm install sqlite-http`                                    | [![npm](https://img.shields.io/npm/v/sqlite-http.svg?color=green\u0026logo=nodedotjs\u0026logoColor=white)](https://www.npmjs.com/package/sqlite-http)                                                |\n| Deno           | [`deno.land/x/sqlite_http`](https://deno.land/x/sqlite_http) | [![deno.land/x release](https://img.shields.io/github/v/release/asg017/sqlite-http?color=fef8d2\u0026include_prereleases\u0026label=deno.land%2Fx\u0026logo=deno)](https://deno.land/x/sqlite_http)        |\n| Ruby           | `gem install sqlite-http`                                    | ![Gem](https://img.shields.io/gem/v/sqlite-http?color=red\u0026logo=rubygems\u0026logoColor=white)                                                                                                    |\n| Github Release |                                                              | ![GitHub tag (latest SemVer pre-release)](https://img.shields.io/github/v/tag/asg017/sqlite-http?color=lightgrey\u0026include_prereleases\u0026label=Github+release\u0026logo=github)                      |\n\n\u003c!--\n| Elixir         | [`hex.pm/packages/sqlite_http`](https://hex.pm/packages/sqlite_http) | [![Hex.pm](https://img.shields.io/hexpm/v/sqlite_http?color=purple\u0026logo=elixir)](https://hex.pm/packages/sqlite_http)                                                                       |\n| Go             | `go get -u github.com/asg017/sqlite-http/bindings/go`               | [![Go Reference](https://pkg.go.dev/badge/github.com/asg017/sqlite-http/bindings/go.svg)](https://pkg.go.dev/github.com/asg017/sqlite-http/bindings/go)                                     |\n| Rust           | `cargo add sqlite-http`                                             | [![Crates.io](https://img.shields.io/crates/v/sqlite-http?logo=rust)](https://crates.io/crates/sqlite-http)                                                                                 |\n--\u003e\n\nThe [Releases page](https://github.com/asg017/sqlite-http/releases) contains pre-built binaries for Linux amd64, MacOS amd64 (no arm), and Windows.\n\n### As a loadable extension\n\nIf you want to use `sqlite-http` as a [Runtime-loadable extension](https://www.sqlite.org/loadext.html), Download the `http0.dylib` (for MacOS), `http0.so` (Linux), or `http0.dll` (Windows) file from a release and load it into your SQLite environment.\n\n\u003e **Note:**\n\u003e The `0` in the filename (`http0.dylib`/ `http0.so`/`http0.dll`) denotes the major version of `sqlite-http`. Currently `sqlite-http` is pre v1, so expect breaking changes in future versions.\n\nFor example, if you are using the [SQLite CLI](https://www.sqlite.org/cli.html), you can load the library like so:\n\n```sql\n.load ./http0\nselect http_version();\n-- v0.0.1\n```\n\nOr in Python, using the builtin [sqlite3 module](https://docs.python.org/3/library/sqlite3.html):\n\n```python\nimport sqlite3\n\ncon = sqlite3.connect(\":memory:\")\n\ncon.enable_load_extension(True)\ncon.load_extension(\"./http0\")\n\nprint(con.execute(\"select http_version()\").fetchone())\n# ('v0.0.1',)\n```\n\nOr in Node.js using [better-sqlite3](https://github.com/WiseLibs/better-sqlite3):\n\n```javascript\nconst Database = require(\"better-sqlite3\");\nconst db = new Database(\":memory:\");\n\ndb.loadExtension(\"./http0\");\n\nconsole.log(db.prepare(\"select http_version()\").get());\n// { 'http_version()': 'v0.0.1' }\n```\n\nOr with [Datasette](https://datasette.io/), with the \"no network\" option to limit DDoS attacks:\n\n```\ndatasette data.db --load-extension ./http0-no-net\n```\n\n## See also\n\n- [sqlite-html](https://github.com/asg017/sqlite-html), for parsing and querying HTML using CSS selectors in SQLite (pairs great with this tool)\n- [pgsql-http](https://github.com/pramsey/pgsql-http), a similar yet very different HTTP library for POstgreSQL (didn't know about this before I started this, but interestingly enough came up with a very similar API)\n- [riyaz-ali/sqlite](https://github.com/riyaz-ali/sqlite), the brilliant Go library that this library depends on\n- [nalgeon/sqlean](https://github.com/nalgeon/sqlean), several pre-compiled handy SQLite functions, in C\n","funding_links":[],"categories":["sqlite","People"],"sub_categories":["As Main Database"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasg017%2Fsqlite-http","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fasg017%2Fsqlite-http","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasg017%2Fsqlite-http/lists"}