{"id":18995636,"url":"https://github.com/turbot/steampipe-plugin-html","last_synced_at":"2026-02-15T15:07:52.395Z","repository":{"id":103595962,"uuid":"563615978","full_name":"turbot/steampipe-plugin-html","owner":"turbot","description":"Use SQL to instantly query HTML resources. Open source CLI. No DB required.","archived":false,"fork":false,"pushed_at":"2023-10-25T22:05:36.000Z","size":165,"stargazers_count":3,"open_issues_count":2,"forks_count":1,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-13T01:58:43.696Z","etag":null,"topics":["postgresql","postgresql-fdw","sql","steampipe","steampipe-plugin"],"latest_commit_sha":null,"homepage":"https://hub.steampipe.io/plugins/turbot/html","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/turbot.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-09T01:18:18.000Z","updated_at":"2024-12-14T06:46:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"899452b4-e487-4c4d-83d6-159f6f4ea61a","html_url":"https://github.com/turbot/steampipe-plugin-html","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/turbot/steampipe-plugin-html","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-html","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-html/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-html/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-html/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/turbot","download_url":"https://codeload.github.com/turbot/steampipe-plugin-html/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/turbot%2Fsteampipe-plugin-html/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266844487,"owners_count":23993965,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["postgresql","postgresql-fdw","sql","steampipe","steampipe-plugin"],"created_at":"2024-11-08T17:32:10.179Z","updated_at":"2026-02-15T15:07:52.364Z","avatar_url":"https://github.com/turbot.png","language":"Go","readme":"\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003ehtml Plugin for Steampipe\u003c/h1\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca aria-label=\"Steampipe logo\" href=\"https://steampipe.io\"\u003e\n    \u003cimg src=\"https://steampipe.io/images/steampipe_logo_wordmark_padding.svg\" height=\"28\"\u003e\n  \u003c/a\u003e\n  \u003ca aria-label=\"License\" href=\"LICENSE\"\u003e\n    \u003cimg alt=\"\" src=\"https://img.shields.io/static/v1?label=license\u0026message=Apache-2.0\u0026style=for-the-badge\u0026labelColor=777777\u0026color=F3F1F0\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## HTML plugin for Steampipe\n\nWeb pages often contain data in HTML tables. This plugin's `html_table` table downloads one or more tables from a web page into one or more CSV files that you can query using the [CSV](https://hub.steampipe.io/plugins/turbot/steampipe-plugin-csv).\n\nWeb pages also contain links. This plugin's `html_link` table queries for them.\n\nThe file `./config/html.spc` is used to define a local path to which downloaded HTML files will be saved.\n\n## Get started\n\nInstall go, then:\n\n```\n$ git clone https://github.com/turbot/steampipe-plugin-html\n\n$ cp ./config/html.spc ~/.steampipe/config\n\n$ make\n\n$ steampipe query\n\n\u003e select\n  base_name,\n  name,\n  path,\n  columns\nfrom\n  html_table\nwhere\n  url = 'https://simple.wikipedia.org/wiki/List_of_U.S._states_by_population'\n  and base_name = 'wiki'\n```\n\n```\n+-----------+--------+---------------+------------------------------------------------------------------------\n| base_name | name   | path          | columns\n+-----------+--------+---------------+------------------------------------------------------------------------\n| wiki      | wiki_0 | /home/jon/csv | \"Rankinstates\u0026territories,2019\",\"Rankinstates\u0026territories,2010\",\"State\"\n+-----------+--------+---------------+------------------------------------------------------------------------\n```\n\nIn this example the plugin found one table on the page, and downloaded it as `/home/jon/csv/wiki_0.csv` (the `/home/jon/csv` path is specified in `./config/html.spc`).\n\nHere is a query of that table.\n\n```\nwith data as (\n  select\n    \"State\" as state,\n    \"Rankinstates\u0026territories,2010\"::int as rank2010,\n    \"Rankinstates\u0026territories,2019\"::int as rank2019,\n    replace(\"Percentchange,20102019[note1]\",'%','')::numeric as pct_change,\n    replace(\"PercentofthetotalU.S.population,2018[note3]\",'%','')::numeric as pct_of_us_pop\n  from\n    wiki_0\n)\nselect\n\tstate,\n  rank2010,\n  rank2019,\n  - (rank2019 - rank2010) as rank_change,\n  pct_change,\n  pct_of_us_pop\nfrom\n  data\norder by\n  pct_change desc\n```\n\n```\n+---------------+----------+----------+-------------+------------+---------------+\n| state         | rank2010 | rank2019 | rank_change | pct_change | pct_of_us_pop |\n+---------------+----------+----------+-------------+------------+---------------+\n| Utah          | 35       | 30       | 5           | 16.0       | 0.96          |\n| Texas         | 2        | 2        | 0           | 15.3       | 8.68          |\n| Colorado      | 22       | 21       | 1           | 14.5       | 1.72          |\n| NewYork       | 3        | 3        | 0           | 14.2       | 6.44          |\n| Nevada        | 36       | 33       | 3           | 14.1       | 0.92          |\n| Idaho         | 40       | 39       | 1           | 14.0       | 0.53          |\n| Arizona       | 16       | 14       | 2           | 13.9       | 2.17          |\n| NorthDakota   | 49       | 48       | 1           | 13.3       | 0.23          |\n```\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturbot%2Fsteampipe-plugin-html","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fturbot%2Fsteampipe-plugin-html","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fturbot%2Fsteampipe-plugin-html/lists"}