{"id":30087929,"url":"https://github.com/fabiopapais/chat-your-data","last_synced_at":"2026-04-17T08:04:48.264Z","repository":{"id":307014527,"uuid":"1027827231","full_name":"fabiopapais/chat-your-data","owner":"fabiopapais","description":"Chat with your SQL database and make complex queries with natural language using LLMs","archived":false,"fork":false,"pushed_at":"2025-07-29T01:41:06.000Z","size":417,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-10T16:09:22.901Z","etag":null,"topics":["bigquery","chainlit","langchain","llm"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabiopapais.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-28T15:37:11.000Z","updated_at":"2025-07-29T01:41:10.000Z","dependencies_parsed_at":"2025-07-29T01:37:31.353Z","dependency_job_id":null,"html_url":"https://github.com/fabiopapais/chat-your-data","commit_stats":null,"previous_names":["fabiopapais/chat-your-data"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fabiopapais/chat-your-data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabiopapais%2Fchat-your-data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabiopapais%2Fchat-your-data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabiopapais%2Fchat-your-data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabiopapais%2Fchat-your-data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabiopapais","download_url":"https://codeload.github.com/fabiopapais/chat-your-data/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabiopapais%2Fchat-your-data/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31920520,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"online","status_checked_at":"2026-04-17T02:00:06.879Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","chainlit","langchain","llm"],"created_at":"2025-08-09T04:02:01.045Z","updated_at":"2026-04-17T08:04:48.179Z","avatar_url":"https://github.com/fabiopapais.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# chat-your-data\n\nChat with your SQL database and make complex queries with natural language using LLMs.\n\n![demo](/public/chat-your-data.gif)\n\nThe system enables users to interact with databases using natural language. It understands questions, generates queries, interprets results, and creates meaningful visualizations.\n\n## Tech Stack and Architecture\n\nIt was built with: **LangChain** \u0026 **LangGraph** (LLM interaction and workflow), **Chainlit** (frontend), **BigQuery** (database) and **Google Gemini**.\n\nThe code is orchestrated between LLM functions and the Chainlit frontend following this architecture:\n\n```\n├── src/\n│   ├── app.py              # Chainlit entry point\n│   ├── workflow.py         # Core AI workflow functions\n│   ├── orchestrator.py     # Workflow orchestration (chainlit \u003c-\u003e workflow) logic\n│   ├── database.py         # BigQuery connection interface\n│   └── system_prompts.py   # System prompt used to generate answers\n├── public/                 # Static assets and UI resources\n├── tests/                  # Unit tests\n└── requirements.txt        # Dependencies\n```\n\n## Workflow\n\n![Workflow Diagram](public/graph.png)\n\nThe system follows a simple 5-step workflow:\n\n1. **Query Generation**: Converts natural language to SQL\n2. **Query Execution**: Runs SQL on the BigQuery database  \n3. **Answer Generation**: Interprets results in natural language\n4. **Query Explanation**: Explains the SQL logic and approach\n5. **Visualization**: Creates relevant Matplotlib visualizations\n\n## Running\n\n### Prerequisites\n- Python 3.10+\n- GCP account\n- BigQuery dataset with the credit risk table uploaded\n- Google AI API key\n\n### Configuration\n\nThere are required environment variables. You must change the .env and have the database set up on BigQuery:\n\n```bash\nGOOGLE_API_KEY=your_gemini_api_key\nGCP_KEY_PATH=./gcpkey.json\nDATABASE_NAME=creditRisk\n```\n\n### Installation\n\n1. **Clone repository**\n   ```bash\n   git clone https://github.com/fabiopapais/chat-your-data.git\n   cd chat-your-data\n   ```\n\n2. **Install dependencies**\n   \n   (a virtual environment is recommended)\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n3. **Configure environment**\n   ```bash\n   cp .env.example .env\n   # Edit .env with your values\n   ```\n\n4. **Add GCP credentials**\n\n   Place your service account key in the root directory (`gcpkey.json`).\n\n5. **Run application**\n   ```bash\n   chainlit run src/app.py\n   ```\n\n### Running Tests\n\nThere is a simple test file for the main workflow pipeline:\n\n```bash\npython test_workflow.py\n```\n\n## Highlights\n\n- **Scalability** - SQL-based queries and multi-table support for complex datasets\n- **Context Management** - Automatically handles large datasets and queries results without token overflow\n- **Automated Visualizations** - Generates meaningful charts when sufficient data is available\n- **Handles no result queries** - Manages cases where queries return no results and prevents errors\n- **Zero Trust for LLMs** - The system is designed with a zero trust principle, ensuring read-only queries and output sanitization (by default)\n- **Standard Workflow** - Follows similar approaches to other validated text-to-SQL systems (https://arxiv.org/pdf/2410.01066, https://arxiv.org/abs/2406.08426, etc)\n\n## Challenges and future work\n\n- Question pre-analysis to prevent prompt injection and out of context queries\n- RAG integration for more complex queries\n- Automatic Schema description\n- Improve workflow with retries and better error handling\n    \n    This was attempted but could be better with an agentic approach (agent having access to tools, contrary to current static workflow and llm request)\n- More accuracy with visualization generation \n\n    Even with a lot of given context, the LLMs still make wrong assumptions about the data, fabricate data and generate wrong visualizations. Better prompt engineering or specialist models may improve this.\n- Test dedicated models such as https://huggingface.co/defog/llama-3-sqlcoder-8b\n- Better integration with Chainlit\n\n    LangGraph streams do not work well with Chainlit, so we can't use Graphs full potential. It was necessary to implement a custom orchestrator to handle the workflow and Chainlit communication.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabiopapais%2Fchat-your-data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabiopapais%2Fchat-your-data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabiopapais%2Fchat-your-data/lists"}