{"id":13760657,"url":"https://github.com/asg017/sqlite-html","last_synced_at":"2025-04-12T23:40:47.253Z","repository":{"id":51737960,"uuid":"416088944","full_name":"asg017/sqlite-html","owner":"asg017","description":"A SQLite extension for querying, manipulating, and creating HTML elements.","archived":false,"fork":false,"pushed_at":"2023-08-06T01:24:57.000Z","size":2630,"stargazers_count":383,"open_issues_count":8,"forks_count":12,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-12T23:40:02.616Z","etag":null,"topics":["sqlite","sqlite-extension"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/asg017.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-10-11T21:15:19.000Z","updated_at":"2025-03-02T23:01:40.000Z","dependencies_parsed_at":"2024-01-15T03:57:24.673Z","dependency_job_id":"0a7953c9-f4d9-425e-b00a-1203f39ebc4e","html_url":"https://github.com/asg017/sqlite-html","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-html","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-html/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-html/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asg017%2Fsqlite-html/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/asg017","download_url":"https://codeload.github.com/asg017/sqlite-html/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248647254,"owners_count":21139081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["sqlite","sqlite-extension"],"created_at":"2024-08-03T13:01:15.683Z","updated_at":"2025-04-12T23:40:47.208Z","avatar_url":"https://github.com/asg017.png","language":"Go","readme":"# sqlite-html\n\nA SQLite extension for querying, manipulating, and creating HTML elements.\n\n- Extract HTML or text from HTML with CSS selectors, like `.querySelector()`, `.innerHTML`, and `.innerText`\n- Generate a table of matching elements from a CSS selector, like `.querySelectorAll()`\n- Safely create HTML elements in a query, like `.createElement()` and `.appendChild()`\n\n`sqlite-html`'s API is modeled after the official [JSON1](https://www.sqlite.org/json1.html#jmini) SQLite extension.\n\nThis extension is written in Go, thanks to [riyaz-ali/sqlite](https://github.com/riyaz-ali/sqlite). While this library aims to be fast and efficient, it is overall slower than what a pure C SQLite extension could be, but in practice you may not notice much of a difference.\n\n## Usage\n\n```sql\n.load ./html0\nselect html_extract('\u003cp\u003e Anakin \u003cb\u003eSkywalker\u003c/b\u003e \u003c/p\u003e', 'b');\n-- \"\u003cb\u003eSkywalker\u003c/b\u003e\"\n```\n\n`sqlite-html` is similar to other HTML scraping tools like [BeautifulSoup](https://beautiful-soup-4.readthedocs.io/en/latest/) (Python) or [cheerio](https://cheerio.js.org/) (Node.js) or [nokogiri](https://nokogiri.org/) (Ruby). You can use CSS selectors to extract individual elements or groups of elements to query data from HTML sources.\n\nFor example, here we find all `href` links in an `index.html` file.\n\n```sql\nselect\n  text as name,\n  html_attribute_get(anchors, 'a', 'href') as href\nfrom html_each(readfile('index.html'), 'a') as anchors\n```\n\nWe can also safely generate HTML with `html_element`, modeled after React's [`React.createElement`](https://reactjs.org/docs/react-api.html#createelement).\n\n```sql\nselect html_element('p', null,\n  'Luke, I am your',\n  html_element('b', null, 'father'),\n  '!',\n\n  html_element('img', json_object(\n    'src', 'https://images.dog.ceo/breeds/groenendael/n02105056_4600.jpg',\n    'width', 200\n  ))\n);\n\n-- \"\u003cp\u003eLuke, I am your\u003cb\u003efather\u003c/b\u003e!\u003cimg src=\"https://images.dog.ceo/breeds/groenendael/n02105056_4600.jpg\" width=\"200.000000\"/\u003e\u003c/p\u003e\"\n```\n\n## Documentation\n\nSee [`docs.md`](./docs.md) for a full API reference.\n\n## Installing\n\n| Language       | Install                                                      |                                                                                                                                                                                             |\n| -------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Python         | `pip install sqlite-html`                                    | [![PyPI](https://img.shields.io/pypi/v/sqlite-html.svg?color=blue\u0026logo=python\u0026logoColor=white)](https://pypi.org/project/sqlite-html/)                                                      |\n| Datasette      | `datasette install datasette-sqlite-html`                    | [![Datasette](https://img.shields.io/pypi/v/datasette-sqlite-html.svg?color=B6B6D9\u0026label=Datasette+plugin\u0026logoColor=white\u0026logo=python)](https://datasette.io/plugins/datasette-sqlite-html) |\n| Node.js        | `npm install sqlite-html`                                    | [![npm](https://img.shields.io/npm/v/sqlite-html.svg?color=green\u0026logo=nodedotjs\u0026logoColor=white)](https://www.npmjs.com/package/sqlite-html)                                                |\n| Deno           | [`deno.land/x/sqlite_html`](https://deno.land/x/sqlite_html) | [![deno.land/x release](https://img.shields.io/github/v/release/asg017/sqlite-html?color=fef8d2\u0026include_prereleases\u0026label=deno.land%2Fx\u0026logo=deno)](https://deno.land/x/sqlite_html)        |\n| Ruby           | `gem install sqlite-html`                                    | ![Gem](https://img.shields.io/gem/v/sqlite-html?color=red\u0026logo=rubygems\u0026logoColor=white)                                                                                                    |\n| Github Release |                                                              | ![GitHub tag (latest SemVer pre-release)](https://img.shields.io/github/v/tag/asg017/sqlite-html?color=lightgrey\u0026include_prereleases\u0026label=Github+release\u0026logo=github)                      |\n\n\u003c!--\n| Elixir         | [`hex.pm/packages/sqlite_html`](https://hex.pm/packages/sqlite_html) | [![Hex.pm](https://img.shields.io/hexpm/v/sqlite_html?color=purple\u0026logo=elixir)](https://hex.pm/packages/sqlite_html)                                                                       |\n| Go             | `go get -u github.com/asg017/sqlite-html/bindings/go`               | [![Go Reference](https://pkg.go.dev/badge/github.com/asg017/sqlite-html/bindings/go.svg)](https://pkg.go.dev/github.com/asg017/sqlite-html/bindings/go)                                     |\n| Rust           | `cargo add sqlite-html`                                             | [![Crates.io](https://img.shields.io/crates/v/sqlite-html?logo=rust)](https://crates.io/crates/sqlite-html)                                                                                 |\n--\u003e\n\nThe [Releases page](https://github.com/asg017/sqlite-html/releases) contains pre-built binaries for Linux amd64, MacOS amd64 (no arm), and Windows.\n\n### As a loadable extension\n\nIf you want to use `sqlite-html` as a [Runtime-loadable extension](https://www.sqlite.org/loadext.html), Download the `html0.dylib` (for MacOS), `html0.so` (Linux), or `html0.dll` (Windows) file from a release and load it into your SQLite environment.\n\n\u003e **Note:**\n\u003e The `0` in the filename (`html0.dylib`/ `html0.so`/`html0.dll`) denotes the major version of `sqlite-html`. Currently `sqlite-html` is pre v1, so expect breaking changes in future versions.\n\nFor example, if you are using the [SQLite CLI](https://www.sqlite.org/cli.html), you can load the library like so:\n\n```sql\n.load ./html0\nselect html_version();\n-- v0.0.1\n```\n\nOr in Python, using the builtin [sqlite3 module](https://docs.python.org/3/library/sqlite3.html):\n\n```python\nimport sqlite3\n\ncon = sqlite3.connect(\":memory:\")\n\ncon.enable_load_extension(True)\ncon.load_extension(\"./html0\")\n\nprint(con.execute(\"select html_version()\").fetchone())\n# ('v0.0.1',)\n```\n\nOr in Node.js using [better-sqlite3](https://github.com/WiseLibs/better-sqlite3):\n\n```javascript\nconst Database = require(\"better-sqlite3\");\nconst db = new Database(\":memory:\");\n\ndb.loadExtension(\"./html0\");\n\nconsole.log(db.prepare(\"select html_version()\").get());\n// { 'html_version()': 'v0.0.1' }\n```\n\nOr with [Datasette](https://datasette.io/):\n\n```\ndatasette data.db --load-extension ./html0\n```\n\n## See also\n\n- [sqlite-http](https://github.com/asg017/sqlite-http), for making HTTP requests in SQLite (pairs great with this tool)\n- [htmlq](https://github.com/mgdm/htmlq), for a similar but CLI-based HTML query tool using CSS selectors\n- [riyaz-ali/sqlite](https://github.com/riyaz-ali/sqlite), the brilliant Go library that this library depends on\n- [nalgeon/sqlean](https://github.com/nalgeon/sqlean), several pre-compiled handy SQLite functions, in C\n","funding_links":[],"categories":["Go","sqlite","People"],"sub_categories":["As Main Database"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasg017%2Fsqlite-html","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fasg017%2Fsqlite-html","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasg017%2Fsqlite-html/lists"}