{"id":31814440,"url":"https://github.com/timescale/text-to-sql-generator","last_synced_at":"2026-02-23T01:32:23.524Z","repository":{"id":310245947,"uuid":"1027876618","full_name":"timescale/text-to-sql-generator","owner":"timescale","description":"Generator of questions to be used for text-to-sql evaluations","archived":false,"fork":false,"pushed_at":"2025-08-28T16:32:05.000Z","size":121,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-11T08:49:56.719Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"postgresql","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timescale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-28T17:04:56.000Z","updated_at":"2025-09-18T08:03:41.000Z","dependencies_parsed_at":"2025-08-16T19:33:29.441Z","dependency_job_id":"13371b14-4c6c-417e-9757-7086b4ac6a13","html_url":"https://github.com/timescale/text-to-sql-generator","commit_stats":null,"previous_names":["timescale/text-to-sql-generator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/timescale/text-to-sql-generator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Ftext-to-sql-generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Ftext-to-sql-generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Ftext-to-sql-generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Ftext-to-sql-generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timescale","download_url":"https://codeload.github.com/timescale/text-to-sql-generator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Ftext-to-sql-generator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29734468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-22T20:09:16.275Z","status":"ssl_error","status_checked_at":"2026-02-22T20:09:13.750Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-11T08:49:30.054Z","updated_at":"2026-02-23T01:32:23.507Z","avatar_url":"https://github.com/timescale.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Question Generator for Text-to-SQL Datasets\n\nThis tool can be used to generate questions for datasets for\n[`text-to-sql-eval`](https://github.com/timescale/text-to-sql-eval).\n\n## Requirements\n\nTo use this table, you will need:\n\n* Python\n* [uv](https://github.com/astral-sh/uv)\n* A PostgreSQL database with [pgai semantic-catalog](https://github.com/timescale/pgai/blob/main/docs/semantic_catalog/quickstart.md) setup\n\nIt's expected that for your dataset, you have setup the semantic-catalog such that\nall objects have had descriptions created/generated.\n\n## Setup\n\nAfter cloning the repo, to install dependencies:\n\n```bash\nuv sync\ncp .env.sample .env\n```\n\nYou will then need to modify `DB_URL` to point at the dataset you want to generate questions for, and add\nvalue for `OPENAI_API_KEY`.\n\n## Usage\n\n### Generate system prompt\n\n```text\n$ uv run python3 -m generator generate-prompt --help\nUsage: python -m generator generate-prompt [OPTIONS]\n\n  Generate prompt for LLM\n\nOptions:\n  --output TEXT  Output file for the prompt\n  --help         Show this message and exit.\n```\n\nYou can use the `generate-prompt` command to generate a system prompt that can\nbe used in subsequent runs of the `generate` command. If you don't do this,\nthen when you call `generate`, a prompt is automatically generated.\n\n### Generate questions\n\n```text\n$ uv run python3 -m generator generate --help\nUsage: python -m generator generate [OPTIONS]\n\n  Generate questions\n\nOptions:\n  --count INTEGER  Number of questions to generate\n  --prompt TEXT    Prompt file to use for generation\n  --help           Show this message and exit.\n```\n\nUse this function to generate X number of questions. Whatever value you pass\nto `count` is split evenly to generate X easy, Y intermediate, and Z hard\nquestions. For example, if you pass `--count 5`, then you will get 2 easy,\n2 intermediate, and 1 hard questions generated.\n\nAll generated questions are stored in the `questions` table in the\n`./data.sqlite` file.\n\nThis function does the following to generate questions:\n\n1. If a prompt file was not passed in, then generate a system prompt to use.\n2. Going in order of easy, intermediate, and then hard, ask the LLM to generate\n  question for a given complexity.\n3. Use the `ai.text_to_sql` function to generate a SQL query to answer the\n  question.\n4. Run the generated SQL query and get the first 10 sample rows, or any errors\n  that occured during execution.\n5. Ask the LLM if the generated SQL adequately answers the question, returning\n  the following information:\n  a. a reason why the generated query was or was not adequate\n  b. if not adequate, a modified SQL query that properly answers the question\n  c. if we can even fix the SQL query, or if we should generate a new question\n6. If the query was inadequate and not fixable, then return to (2) and generate\n  a new question.\n7. If the query was inadequate, but we got a modified SQL back, then run that\n  modified SQL to get new results or error, and then go back to (5).\n8. If the query was deemed adequate, save the `[question, query]` tuple to the\n  `questions` table in `./data.sqlite`.\n\n### Export questions\n\n```text\n$ uv run python3 -m generator export --help\nUsage: python -m generator export [OPTIONS]\n\n  Export generated questions\n\nOptions:\n  --help  Show this message and exit.\n```\n\nThis takes all the questions that have been previously generated and exports\nthem to `evals` folder at the root of this repo. This folder can then be copied\ninto [`text-to-sql-eval`](https://github.com/timescale/text-to-sql-eval) repo\nfor usage as eval dataset.\n\n## Quickstart\n\nTo demonstrate the repo, we provide quickstart guide. For this, we will use the\n[Analyze the Bitcoin Blockchain](https://docs.tigerdata.com/tutorials/latest/blockchain-analyze/)\nguide from Tiger Cloud docs.\n\nFirst, start up a TimescaleDB docker instance:\n\n```bash\ndocker run -d --name postgres-bitcoin \\\n    -p 127.0.0.1:5432:5432 \\\n    -e POSTGRES_HOST_AUTH_METHOD=trust \\\n    -e POSTGRES_DB=bitcoin \\\n    timescale/timescaledb-ha:pg17\n```\n\nNext is to setup the DB:\n\n```bash\npsql -h localhost -U postgres -d bitcoin -c \"\nCREATE TABLE transactions (\n   time TIMESTAMPTZ NOT NULL,\n   block_id INT,\n   hash TEXT,\n   size INT,\n   weight INT,\n   is_coinbase BOOLEAN,\n   output_total BIGINT,\n   output_total_usd DOUBLE PRECISION,\n   fee BIGINT,\n   fee_usd DOUBLE PRECISION,\n   details JSONB\n) WITH (\n   tsdb.hypertable,\n   tsdb.partition_column='time',\n   tsdb.segmentby='block_id',\n   tsdb.orderby='time DESC'\n);\n\nCREATE INDEX hash_idx ON public.transactions USING HASH (hash);\nCREATE INDEX block_idx ON public.transactions (block_id);\nCREATE UNIQUE INDEX time_hash_idx ON public.transactions (time, hash);\n\"\n```\n\nNext, download the `bitcoin_sample.zip`, unzip it, and then copy it into the DB.\nAs there's over a million rows, it may take a few minutes to load the data.\n\n```bash\nwget https://assets.timescale.com/docs/downloads/bitcoin-blockchain/bitcoin_sample.zip\nunzip bitcoin_sample.zip\npsql -h localhost -U postgres -d bitcoin -c \"\\COPY transactions FROM 'tutorial_bitcoin_sample.csv' CSV HEADER;\"\n```\n\nThen create the continuous aggregates:\n\n```bash\npsql -h localhost -U postgres -d bitcoin -c \"\nCREATE MATERIALIZED VIEW one_hour_transactions\nWITH (timescaledb.continuous) AS\nSELECT time_bucket('1 hour', time) AS bucket,\n   count(*) AS tx_count,\n   sum(fee) AS total_fee_sat,\n   sum(fee_usd) AS total_fee_usd,\n   stats_agg(fee) AS stats_fee_sat,\n   avg(size) AS avg_tx_size,\n   avg(weight) AS avg_tx_weight,\n   count(\n         CASE\n            WHEN (fee \u003e output_total) THEN hash\n            ELSE NULL\n         END) AS high_fee_count\n  FROM transactions\n  WHERE (is_coinbase IS NOT TRUE)\nGROUP BY bucket;\n\"\n\npsql -h localhost -U postgres -d bitcoin -c \"\nCREATE MATERIALIZED VIEW one_hour_blocks\nWITH (timescaledb.continuous) AS\nSELECT time_bucket('1 hour', time) AS bucket,\n   block_id,\n   count(*) AS tx_count,\n   sum(fee) AS block_fee_sat,\n   sum(fee_usd) AS block_fee_usd,\n   stats_agg(fee) AS stats_tx_fee_sat,\n   avg(size) AS avg_tx_size,\n   avg(weight) AS avg_tx_weight,\n   sum(size) AS block_size,\n   sum(weight) AS block_weight,\n   max(size) AS max_tx_size,\n   max(weight) AS max_tx_weight,\n   min(size) AS min_tx_size,\n   min(weight) AS min_tx_weight\nFROM transactions\nWHERE is_coinbase IS NOT TRUE\nGROUP BY bucket, block_id;\n\"\n\npsql -h localhost -U postgres -d bitcoin -c \"\nCREATE MATERIALIZED VIEW one_hour_coinbase\nWITH (timescaledb.continuous) AS\nSELECT time_bucket('1 hour', time) AS bucket,\n   count(*) AS tx_count,\n   stats_agg(output_total, output_total_usd) AS stats_miner_revenue,\n   min(output_total) AS min_miner_revenue,\n   max(output_total) AS max_miner_revenue\nFROM transactions\nWHERE is_coinbase IS TRUE\nGROUP BY bucket;\n\"\n\npsql -h localhost -U postgres -d bitcoin -c \"\nSELECT add_continuous_aggregate_policy('one_hour_transactions',\n   start_offset =\u003e INTERVAL '3 hours',\n   end_offset =\u003e INTERVAL '1 hour',\n   schedule_interval =\u003e INTERVAL '1 hour');\n\nSELECT add_continuous_aggregate_policy('one_hour_blocks',\n   start_offset =\u003e INTERVAL '3 hours',\n   end_offset =\u003e INTERVAL '1 hour',\n   schedule_interval =\u003e INTERVAL '1 hour');\n\nSELECT add_continuous_aggregate_policy('one_hour_coinbase',\n   start_offset =\u003e INTERVAL '3 hours',\n   end_offset =\u003e INTERVAL '1 hour',\n   schedule_interval =\u003e INTERVAL '1 hour');\n\"\n```\n\nNow generate descriptions for the schema objects:\n\n```bash\nuv run pgai semantic-catalog describe -d \"postgres://postgres@localhost:5432/bitcoin\" -f description.yaml\n```\n\nYou can view the various descriptions that were generated in the `description.yaml`\nfile. Now we can create the semantic-catalog and import the descriptions:\n\n```bash\nuv run pgai semantic-catalog create -c \"postgres://postgres@localhost:5432/bitcoin\"\nuv run pgai semantic-catalog import -d \"postgres://postgres@localhost:5432/bitcoin\" -f description.yaml\n```\n\nModify your `.env` file for this repo to point at the bitcoin database:\n\n```bash\nDB_URL=postgresql://postgres:postgres@localhost:5432/bitcoin\n```\n\nNow generate 10 questions:\n\n```bash\nuv run python3 -m generator generate\n```\n\nExport to `evals` folder at top of repo:\n\n```bash\nuv run python3 -m generator export\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Ftext-to-sql-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimescale%2Ftext-to-sql-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Ftext-to-sql-generator/lists"}