{"id":50638705,"url":"https://github.com/sanniheruwala/rednotebookai","last_synced_at":"2026-06-17T09:00:31.709Z","repository":{"id":362742827,"uuid":"1260596203","full_name":"sanniheruwala/RedNotebookAI","owner":"sanniheruwala","description":"RedNotebook AI : open-source AI data notebook. Notebook-first UI (Next.js + shadcn/ui) over a FastAPI backend, with AI SQL/chart suggestions, profiling, PII detection, and a NotebookLM-style knowledge layer.","archived":false,"fork":false,"pushed_at":"2026-06-12T05:48:48.000Z","size":11406,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-12T07:20:54.747Z","etag":null,"topics":["ai","anthropic","data-analytics","data-notebook","data-visualization","echarts","fastapi","llm","nextjs","notebooklm","ollama","open-source","openai","plotly","python","shadcn-ui","sql","tailwindcss","trino","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sanniheruwala.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-05T17:00:03.000Z","updated_at":"2026-06-12T05:43:45.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sanniheruwala/RedNotebookAI","commit_stats":null,"previous_names":["sanniheruwala/querycanvasai"],"tags_count":29,"template":false,"template_full_name":null,"purl":"pkg:github/sanniheruwala/RedNotebookAI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanniheruwala%2FRedNotebookAI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanniheruwala%2FRedNotebookAI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanniheruwala%2FRedNotebookAI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanniheruwala%2FRedNotebookAI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sanniheruwala","download_url":"https://codeload.github.com/sanniheruwala/RedNotebookAI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanniheruwala%2FRedNotebookAI/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34441285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anthropic","data-analytics","data-notebook","data-visualization","echarts","fastapi","llm","nextjs","notebooklm","ollama","open-source","openai","plotly","python","shadcn-ui","sql","tailwindcss","trino","typescript"],"created_at":"2026-06-07T06:01:58.061Z","updated_at":"2026-06-17T09:00:31.695Z","avatar_url":"https://github.com/sanniheruwala.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"frontend/public/logo.png\" alt=\"RedAnalytica\" width=\"72\" /\u003e\n\n# RedNotebook AI\n\n**The open-source AI data notebook for Trino, DuckDB, and 11 more SQL engines.**\nBy [RedAnalytica](https://redanalytica.in).\n\n[![Try the demo](https://img.shields.io/badge/▶_Try_the_live_demo-22c55e?style=for-the-badge\u0026labelColor=0f172a)](https://huggingface.co/spaces/heruwala/rednotebook-demo)\n[![CI](https://github.com/sanniheruwala/RedNotebookAI/actions/workflows/ci.yml/badge.svg)](https://github.com/sanniheruwala/RedNotebookAI/actions/workflows/ci.yml)\n[![Release](https://github.com/sanniheruwala/RedNotebookAI/actions/workflows/release.yml/badge.svg)](https://github.com/sanniheruwala/RedNotebookAI/actions/workflows/release.yml)\n[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-22c55e.svg)](LICENSE)\n[![Python 3.11+](https://img.shields.io/badge/python-3.11+-22c55e.svg)](https://www.python.org/)\n[![Next.js 15](https://img.shields.io/badge/Next.js-15-22c55e.svg)](https://nextjs.org/)\n\nQuery, visualize, profile, and explore data with beautiful charts, AI suggestions, and a NotebookLM-style knowledge layer.\n\n**👉 [Try it now — no signup, no install](https://huggingface.co/spaces/heruwala/rednotebook-demo)**\n\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/rednotebook-v0.7.15.gif\"\n       alt=\"10-second demo: open the Q3 demo notebook, run SQL against DuckDB, inspect per-column histograms in the Profile tab, then summarize the result with AI.\"\n       width=\"900\" /\u003e\n\u003c/p\u003e\n\n---\n\n## Why RedNotebook AI?\n\nModern data teams jump between five tools to answer one question. RedNotebook AI puts all of it in one notebook:\n\n- **A real SQL workspace** with Monaco, AG Grid, drag-to-reorder cells, and keyboard shortcuts.\n- **Premium charts** powered by Apache ECharts with brand-aware theming.\n- **AI you can trust**, pluggable across OpenAI, Anthropic, Ollama, or a deterministic offline mock. Privacy-safe by default, schema-only context, PII masking, secrets stripped.\n- **NotebookLM-style knowledge layer.** Pull SQL, schemas, results, and charts into a notebook of sources. Ask grounded questions with `[n]` citation chips. Generate infographics and a Studio briefing (overview / FAQ / study guide / suggested next questions).\n- **Drag-and-drop file uploads.** Drop a CSV, TSV, Parquet, or JSON file anywhere in the app — DuckDB attaches it instantly as a queryable table (`SELECT * FROM customers` Just Works).\n- **One-click publish.** Mint a public, no-account-needed share link from any notebook. The published page is a self-contained HTML snapshot — your live data never leaves your machine.\n- **Read-only by default.** A SQL guard backed by `sqlglot` blocks destructive statements unless you explicitly enable writes.\n- **Local-first.** Runs on your laptop with no login. Flip a single env var (`AUTH_ENABLED=true`) to enable multi-user mode with local email+password, GitHub OAuth, API tokens, per-user namespacing, and admin invites.\n\n---\n\n## Install\n\n\u003e **Just kicking the tires?** The live demo at\n\u003e [huggingface.co/spaces/heruwala/rednotebook-demo](https://huggingface.co/spaces/heruwala/rednotebook-demo)\n\u003e runs the published image with the sample notebook pre-loaded. No\n\u003e install required, no signup, your work isn't saved between sessions.\n\n### Docker (any OS)\n\n```bash\ndocker run -d --name rednotebook \\\n  -p 8000:8000 \\\n  -v rednotebook-data:/data \\\n  ghcr.io/sanniheruwala/rednotebook-ai:latest\n```\n\nThen open [http://localhost:8000](http://localhost:8000).\n\nOr with Compose:\n\n```bash\ncp .env.example .env  # edit as needed\ndocker compose up -d\n```\n\n### Python\n\n```bash\npip install rednotebook-ai          # from PyPI (when a release is tagged)\nrednotebook run                      # starts the FastAPI server on :8000\n```\n\nThen in a second terminal:\n\n```bash\ncd frontend\nnpm install\nnpm run dev                          # starts the dev UI on :3000\n```\n\n### From source\n\n```bash\ngit clone https://github.com/sanniheruwala/RedNotebookAI.git\ncd RedNotebookAI\npython -m venv .venv \u0026\u0026 source .venv/bin/activate\npip install -e \".[dev]\"\ncp .env.example .env\nrednotebook run\n\n# in another terminal\ncd frontend \u0026\u0026 npm install \u0026\u0026 npm run dev\n```\n\n---\n\n## Where can I run this safely?\n\nRedNotebook AI is local-first. Today:\n\n| Tier | Supported? |\n|------|------------|\n| 🟢 **Your laptop** (`localhost`) | ✅ Primary use case |\n| 🟢 **Single team behind VPN / private network** | ✅ With the [hardening checklist](docs/deployment.md#tier-2--single-team-behind-a-vpn--private-network) |\n| 🔴 **Public internet, multi-user SaaS** | ⚠️ Auth, rate-limiting (`slowapi`), and audit log have all landed. Full SaaS hardening (RBAC / SSO / sharing) is on the [Phase 4 roadmap](docs/roadmap.md). |\n\nSee [`docs/deployment.md`](docs/deployment.md) for the full security model.\n\n---\n\n## Pick a data source\n\nIn the UI top bar, click **Configure connection**. **13 connectors ship in\nthe box** — no extra `pip install` step, no driver setup, no ODBC dance.\n\n| Connector       | What you'll need                                                  |\n|-----------------|-------------------------------------------------------------------|\n| DuckDB          | Nothing. Pick in-memory or a `.duckdb` file path.                 |\n| Trino           | Host, port, user, password, catalog, schema, TLS settings.        |\n| PostgreSQL      | Host, port, user, password, database.                             |\n| MySQL / MariaDB | Host, port, user, password, database.                             |\n| SQLite          | Path to the `.db` / `.sqlite` file.                               |\n| MSSQL           | Host, port, user, password, database. ODBC 18 driver is bundled.  |\n| Snowflake       | Account, warehouse, role, user, password, database.               |\n| BigQuery        | Project, dataset, service-account JSON path.                      |\n| Redshift        | Host, port, user, password, database.                             |\n| Oracle          | Host, port, user, password, database or `service_name`.           |\n| ClickHouse      | Host, port (8123 HTTP), user, password, database, `secure` flag.  |\n| Databricks SQL  | Host, `http_path`, access token, optional catalog.                |\n\nSee [docs/connectors.md](docs/connectors.md) for the full per-dialect field\nreference.\n\n### Quick start: DuckDB (no server, instant)\n\nThe default. Pick \"DuckDB (no server)\" in the dialog. Two modes:\n\n- **In-memory** (`:memory:`) — ephemeral playground. Great for one-off SQL against local files: `SELECT * FROM read_csv_auto('orders.csv') WHERE …`\n- **File** (`./local.duckdb`) — persistent. Use it like a single-user warehouse: `CREATE TABLE customers (…)`, `INSERT …`, etc.\n\nOptionally set a \"Working directory\" so relative file paths in `read_csv_auto` / `read_parquet` resolve where you expect.\n\n### Trino HTTPS defaults via `.env`\n\nFor team analytics on real data warehouses, fill in the UI dialog or set\ndefaults in `.env`:\n\n```env\nTRINO_HOST=trino.example.com\nTRINO_PORT=443\nTRINO_SCHEME=https\nTRINO_USER=alice\nTRINO_PASSWORD=...\nTRINO_CATALOG=hive\nTRINO_SCHEMA=default\nTRINO_VERIFY_SSL=true\n```\n\nCustom HTTP headers, session properties, query timeouts, and result limits are all supported.\n\n---\n\n## Configure AI\n\n| Provider | Setup |\n|----------|-------|\n| **Mock** (default) | Offline, deterministic. No setup. |\n| **OpenAI** | `AI_PROVIDER=openai`, `OPENAI_API_KEY=sk-…` |\n| **Anthropic** | `AI_PROVIDER=anthropic`, `ANTHROPIC_API_KEY=sk-ant-…` |\n| **Ollama** (local) | `AI_PROVIDER=ollama`, `OLLAMA_BASE_URL=http://localhost:11434` |\n\nPrivacy defaults:\n\n- Sample rows are **not** sent to AI unless `AI_ALLOW_SAMPLE_ROWS=true`.\n- PII columns are masked when samples are shared.\n- Secrets are stripped from SQL before any provider call.\n- Credentials are never forwarded to AI.\n\nSee [`docs/ai.md`](docs/ai.md) for details.\n\n---\n\n## Enable multi-user (optional)\n\n```env\nAUTH_ENABLED=true\nSECRET_KEY=$(openssl rand -hex 32)\nCOOKIE_SECURE=true              # set true when behind HTTPS\nALLOW_SELF_SIGNUP=false         # admin-invite only by default\n```\n\nThe first registration becomes the workspace admin. Subsequent users need an invite (`POST /api/auth/invite`). GitHub OAuth and API tokens (PAT-style) are supported out of the box. See [`docs/deployment.md`](docs/deployment.md).\n\n---\n\n## Architecture\n\n| Layer | Tech |\n|-------|------|\n| Backend | Python 3.11+, FastAPI, Pydantic, Trino client, SQLAlchemy + bundled drivers (Postgres, MySQL, MSSQL/ODBC, Snowflake, BigQuery, Redshift, Oracle, ClickHouse, Databricks, ...), DuckDB, Pandas, ECharts/Plotly |\n| Frontend | Next.js 14, TypeScript, Tailwind, shadcn/ui, Monaco, AG Grid, ECharts, framer-motion, @dnd-kit |\n| State | TanStack Query (server) + Zustand (local) |\n| Auth | Local email+password (bcrypt) + JWT cookies, GitHub OAuth, API tokens |\n| AI | Provider-pluggable (mock, OpenAI, Anthropic, Ollama) |\n| Storage | Local JSON for notebooks/knowledge/users; optional Parquet result cache |\n\n```\nrednotebook/        Python backend (FastAPI + core libs)\n├── auth/           User store, JWT sessions, password hashing, OAuth, API tokens\n├── server/         FastAPI app + routers\n├── connectors/     Trino + DuckDB + 11 SQLAlchemy dialects + registry\n├── ai/             Provider abstraction (mock, openai, anthropic, ollama)\n├── notebook/       Notebook models, JSON storage, guard-aware runner\n├── knowledge/      NotebookLM-style internal knowledge layer\n├── visualization/  Recommender, chart spec, HTML infographic generator\n├── profiling/      Stats + PII detector\n├── security/       SQL guard, secret masking\n├── migrations/     One-shot data migrations\n└── cli/            Typer CLI\n\nfrontend/           Next.js + Tailwind + shadcn/ui\ndocs/               Architecture, AI, security, deployment, connectors, roadmap\ntests/              pytest test suite\n```\n\nFull [architecture write-up](docs/architecture.md).\n\n---\n\n## Documentation\n\n- [Architecture](docs/architecture.md)\n- [Deployment tiers](docs/deployment.md)\n- [Host a public demo on Hugging Face Spaces (free, no card)](docs/deployment/try-on-huggingface.md)\n- [Host a public demo on Fly.io](docs/deployment/try-rednotebook.md)\n- [Connectors](docs/connectors.md)\n- [AI providers and privacy](docs/ai.md)\n- [Security model](docs/security.md)\n- [Visualization](docs/visualization.md)\n- [Knowledge layer + NotebookLM integration](docs/notebooklm_integration.md)\n- [Roadmap](docs/roadmap.md)\n- [Contributing](docs/contributing.md)\n\n---\n\n## Development\n\n```bash\n# Backend\npytest                              # 56+ tests\nruff check .\n\n# Frontend\ncd frontend\nnpm run typecheck\nnpm run lint\nnpm run build\n```\n\nContinuous integration runs the full suite on every push and PR. See [`.github/workflows`](.github/workflows).\n\n---\n\n## Contributing\n\nWe follow the standard open-source flow. The short version:\n\n1. **Open an issue first.** Use the\n   [bug report](https://github.com/sanniheruwala/RedNotebookAI/issues/new?template=bug_report.yml)\n   or\n   [feature request](https://github.com/sanniheruwala/RedNotebookAI/issues/new?template=feature_request.yml)\n   templates. Drive-by PRs with no linked issue may be closed without review.\n2. **Fork, branch, write, run the checks locally**\n   (`pytest`, `ruff check .`, `npm run lint \u0026\u0026 npm run typecheck \u0026\u0026 npm run build`).\n3. **Open a PR** referencing the issue (`Closes #123`).\n4. **A maintainer reviews and approves before merge.**\n   `main` is a protected branch — direct pushes are blocked, every change\n   needs ✅ green CI **and** ✅ approval from a\n   [CODEOWNER](.github/CODEOWNERS). No exceptions, even for admins.\n\nSee [`docs/contributing.md`](docs/contributing.md) for the full flow,\nbranch-naming conventions, what we say \"no\" to, and the maintainer\nrights. For security vulnerabilities, use\n[private disclosure](SECURITY.md), never a public issue.\n\n---\n\n## License\n\nApache-2.0. See [LICENSE](LICENSE).\n\n\u003cdiv align=\"center\"\u003e\n  \u003csub\u003eBuilt with care by \u003ca href=\"https://redanalytica.in\"\u003eRedAnalytica\u003c/a\u003e.\u003c/sub\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsanniheruwala%2Frednotebookai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsanniheruwala%2Frednotebookai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsanniheruwala%2Frednotebookai/lists"}