{"id":50560039,"url":"https://github.com/genpat-it/cohesive-llm","last_synced_at":"2026-06-04T11:30:40.983Z","repository":{"id":349977434,"uuid":"1203894087","full_name":"genpat-it/cohesive-llm","owner":"genpat-it","description":"Self-hostable AI platform that generates Nextflow DSL2 pipelines for the cohesive-ngsmanager framework","archived":false,"fork":false,"pushed_at":"2026-04-27T10:30:31.000Z","size":927,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-27T11:24:30.744Z","etag":null,"topics":["ai-agent","authelia","bioinformatics","caddy","cohesive-ngsmanager","docker-compose","fastapi","izs","langchain","langgraph","llm","mistral","nextflow","ngs","pipeline-generation","rag","self-hosted"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/genpat-it.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-07T13:41:25.000Z","updated_at":"2026-04-10T10:44:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/genpat-it/cohesive-llm","commit_stats":null,"previous_names":["genpat-it/cohesive-llm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/genpat-it/cohesive-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genpat-it%2Fcohesive-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genpat-it%2Fcohesive-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genpat-it%2Fcohesive-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genpat-it%2Fcohesive-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/genpat-it","download_url":"https://codeload.github.com/genpat-it/cohesive-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genpat-it%2Fcohesive-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33903134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-04T02:00:06.755Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agent","authelia","bioinformatics","caddy","cohesive-ngsmanager","docker-compose","fastapi","izs","langchain","langgraph","llm","mistral","nextflow","ngs","pipeline-generation","rag","self-hosted"],"created_at":"2026-06-04T11:30:39.931Z","updated_at":"2026-06-04T11:30:40.976Z","avatar_url":"https://github.com/genpat-it.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cohesive LLM — IZS Bioinformatics AI Platform\n\n[![Status](https://img.shields.io/badge/status-under%20development-orange?style=flat-square)](#)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green?style=flat-square)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.12-blue?style=flat-square\u0026logo=python\u0026logoColor=white)](https://www.python.org/)\n[![FastAPI](https://img.shields.io/badge/FastAPI-0.115-009688?style=flat-square\u0026logo=fastapi\u0026logoColor=white)](https://fastapi.tiangolo.com/)\n[![LangGraph](https://img.shields.io/badge/LangGraph-1.0-1c3c3c?style=flat-square)](https://langchain-ai.github.io/langgraph/)\n[![Mistral](https://img.shields.io/badge/LLM-Mistral-ff7000?style=flat-square)](https://mistral.ai/)\n[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-16-4169E1?style=flat-square\u0026logo=postgresql\u0026logoColor=white)](https://www.postgresql.org/)\n[![Caddy](https://img.shields.io/badge/Caddy-2-1f88c0?style=flat-square\u0026logo=caddy\u0026logoColor=white)](https://caddyserver.com/)\n[![Docker](https://img.shields.io/badge/Docker-Compose-2496ED?style=flat-square\u0026logo=docker\u0026logoColor=white)](https://docs.docker.com/compose/)\n[![Last commit](https://img.shields.io/github/last-commit/genpat-it/cohesive-llm?style=flat-square)](https://github.com/genpat-it/cohesive-llm/commits/main)\n[![Issues](https://img.shields.io/github/issues/genpat-it/cohesive-llm?style=flat-square)](https://github.com/genpat-it/cohesive-llm/issues)\n\n\u003e ⚠️ **Status: under active development.** APIs, the database schema, the\n\u003e docker-compose layout and the configuration variables can all change between\n\u003e commits without notice. Do **not** use this in production yet — it has no\n\u003e stable release, no formal security audit and no upgrade story.\n\nSelf-hostable platform that lets bioinformaticians describe a sequencing analysis in plain English and get back a valid Nextflow DSL2 pipeline for the [cohesive-ngsmanager](https://github.com/genpat-it/cohesive-ngsmanager) framework.\n\n## Credits\n\nThis platform was started as a master thesis project by three students of the\n[**EDISS — European Master in Digital Innovation for Sustainable Society**](https://www.master-ediss.eu/),\ncarried out remotely in collaboration with the **Istituto Zooprofilattico\nSperimentale dell'Abruzzo e del Molise \"G. Caporale\" (IZS Teramo)**:\n\n- **Martinus Grady** — [@mgradyn](https://github.com/mgradyn)\n- **Ligan Cai** — [@Tsailgan](https://github.com/Tsailgan)\n- **Zeynal Mardanli** — [@Lshiroc](https://github.com/Lshiroc)\n\nUpstream repositories:\n\n- **Backend** (LangGraph + FastAPI LLM agent) — [mgradyn/izs-llm](https://github.com/mgradyn/izs-llm)\n- **Frontend** (chat UI) — [mgradyn/izs-bioinformatics-AI-demo](https://github.com/mgradyn/izs-bioinformatics-AI-demo)\n\nThe Nextflow framework being targeted is [genpat-it/cohesive-ngsmanager](https://github.com/genpat-it/cohesive-ngsmanager).\n\n## What's inside\n\n```\ncohesive-llm/\n├── backend/             FastAPI + LangGraph LLM (anti-hallucination, AST validation)\n├── frontend/            Static UI (HTML/CSS/JS) — served by Caddy\n│   ├── index.html       Chat interface\n│   ├── drawer.html      Visual drag-and-drop pipeline editor\n│   ├── login.html       Authentication page\n│   └── js/              ES modules (chat, sidebar, results, drawer, api, modal)\n├── caddy/               Caddy image (reverse proxy + static file server)\n├── scripts/             Render configs and start the stack\n├── docker-compose.yml   All services orchestrated\n└── .env.example         Configuration template\n```\n\n## Quick start (local development)\n\n```bash\ngit clone \u003cthis-repo\u003e\ncd cohesive-llm\ncp .env.example .env\n```\n\nOpen `.env` and at minimum set:\n\n```env\nMISTRAL_API_KEY=your_real_key_here\nJWT_SECRET=$(openssl rand -hex 32)\nDEMO_PASSWORD=pick_something_strong\n```\n\nThen start the stack:\n\n```bash\n./scripts/up.sh\n```\n\nThe first run will:\n1. Clone `cohesive-ngsmanager` into a Docker volume (one-time, ~1 min)\n2. Build the backend image (downloads Python deps and embeddings, ~5 min)\n3. Start Postgres, the backend, the frontend and Caddy\n\nOpen **http://localhost:9000** and log in with `demo` / the password you set in `DEMO_PASSWORD`.\n\nTo stop everything:\n\n```bash\ndocker compose down\n```\n\nTo wipe state and start fresh (re-clones ngsmanager, drops the DB, rebuilds FAISS):\n\n```bash\ndocker compose down -v\n```\n\n### Tail the logs\n\n```bash\ndocker compose logs -f backend\ndocker compose logs -f caddy\n```\n\n### Test the API directly (bypass the frontend)\n\n```bash\ncurl -c cookies.txt -X POST http://localhost:9000/api/auth/login \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"username\":\"demo\",\"password\":\"\u003cyour password\u003e\"}'\n\ncurl -b cookies.txt http://localhost:9000/api/health\ncurl -b cookies.txt http://localhost:9000/api/system-info\ncurl -b cookies.txt -X POST http://localhost:9000/api/chat \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"session_id\":\"dev\",\"message\":\"I want to trim with fastp\"}'\n\n# Validate generated Nextflow code\ncurl -b cookies.txt -X POST http://localhost:9000/api/validate \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"nextflow_code\":\"workflow { println \\\"hello\\\" }\"}'\n\n# Drawer APIs\ncurl -b cookies.txt http://localhost:9000/api/catalog/components\ncurl -b cookies.txt http://localhost:9000/api/drawings\n```\n\n## Authentication\n\nLogin is built into the backend itself: a single demo user is seeded automatically\non first start with the credentials taken from `DEMO_USER` / `DEMO_PASSWORD` in `.env`.\n\nThe session uses a stateless JWT stored in an `HttpOnly` cookie. Requests with no\nvalid cookie get a 401 from any `/api/*` route except `/api/auth/login`. The frontend\nintercepts that 401 and redirects the browser to `/login.html`.\n\nFailed login attempts are rate-limited per real client IP via `slowapi`\n(default `5/minute`, tunable with `LOGIN_RATE_LIMIT`).\n\n\u003e Multi-user / corporate SSO (LDAP, OIDC) is **not** included on purpose — drop in\n\u003e a proxy-level auth layer (Authelia, oauth2-proxy, ...) in front of Caddy if you\n\u003e need it for production.\n\n## Deployment modes\n\n### Local development (HTTP, single demo user)\n```env\nDOMAIN=localhost\nHTTPS_MODE=off\nCOOKIE_SECURE=false\n```\n\n### Behind a corporate reverse proxy (e.g. `https://intranet.example.com/llm`)\n\nWhen you can't expose ports 80/443 directly and an upstream proxy\n(nginx, F5, Caddy, IIS, …) handles TLS for a public URL like\n`https://intranet.example.com/llm`, run the platform on a custom HTTP port and\nlet the upstream proxy forward to it.\n\n```env\nDOMAIN=intranet.example.com\nHTTPS_MODE=off                # upstream proxy handles TLS\nCADDY_HOST_PORT=9000          # host port Caddy listens on (HTTP)\nTRUSTED_PROXIES=10.0.0.0/8    # CIDR of the upstream proxy network\nCORS_ORIGINS=https://intranet.example.com\nCOOKIE_SECURE=true            # cookie sent only over HTTPS\nCOOKIE_PATH=/llm              # scope the session cookie to the sub-path\nCOOKIE_SAMESITE=lax\nBASE_PATH=/llm/               # MUST end with a trailing slash\n```\n\n`BASE_PATH` is what makes the app sub-path-aware: Caddy substitutes the\n`{{env \"BASE_PATH\"}}` placeholder in `index.html` and `login.html` at\nrequest time (via the built-in Go `templates` directive), so all relative\nURLs and fetch calls resolve under `/llm/...`. Set it once in `.env`,\nrestart Caddy, done. No code changes.\n\n`PROXY_PREFIX` is **only** needed if you want to test the sub-path\nlocally without an upstream proxy: set `PROXY_PREFIX=/llm` and Caddy\nitself handles `/llm/*` so `http://localhost:9000/llm/` works\nend-to-end. In production, leave it empty — the corporate proxy strips\nthe prefix before reaching Caddy.\n\nThen ask the sysadmins to configure the upstream proxy to:\n\n1. Forward `https://intranet.example.com/llm/*` → `http://your-server:9000/*`\n2. **Strip the `/llm` path prefix** before forwarding\n3. Set the headers: `X-Forwarded-Proto: https`, `X-Forwarded-Host: intranet.example.com`, `X-Real-IP`, `X-Forwarded-For`\n4. Allow body sizes up to ~10 MB\n5. Tell you the proxy IP/CIDR so you can set `TRUSTED_PROXIES` correctly\n\n### Self-signed HTTPS (no upstream proxy)\n```env\nDOMAIN=ai.izs.intra\nHTTPS_MODE=internal\nCOOKIE_SECURE=true\n```\nBrowser will warn about the self-signed cert on first visit (one click to accept).\n\n### Public deployment (real domain, Let's Encrypt)\n```env\nDOMAIN=ai.example.com\nHTTPS_MODE=auto\nACME_EMAIL=ops@example.com\nCOOKIE_SECURE=true\n```\nRequires ports 80 and 443 reachable from the internet and DNS pointing at the server.\n\n## How it works\n\nThe container layout is intentionally minimal (3 services + 1 init container):\n\n1. `ngsmanager-init` clones [cohesive-ngsmanager](https://github.com/genpat-it/cohesive-ngsmanager) into a Docker volume on first start.\n2. `postgres` stores users, conversations and messages.\n3. `backend` (FastAPI + LangGraph) reads the framework from `/ngsmanager`, builds the RAG knowledge base on first start, and serves `/api/auth/*`, `/api/chat`, `/api/conversations/*`, `/api/validate`, `/api/catalog/*`, `/api/drawings/*`, `/api/system-info`. Includes Java + Nextflow for in-container pipeline validation.\n4. `caddy` is the single entry point: serves the static frontend (HTML/JS/CSS) from `/srv` via `file_server` + `templates`, and reverse-proxies `/api/*` to the backend.\n\n```\n                  ┌──────────┐\n   browser   ───▶ │  caddy   │  serves frontend at /llm/* (file_server)\n                  │  :9000   │  proxies /llm/api/* → backend:8080\n                  └────┬─────┘\n                       │\n              ┌────────┴────────┐\n              ▼                 ▼\n        ┌──────────┐      ┌──────────┐\n        │ backend  │      │ postgres │\n        │  :8080   │◀────▶│  :5432   │\n        └──────────┘      └──────────┘\n              │\n              │ reads (read-only)\n              ▼\n        ┌──────────────┐\n        │ /ngsmanager  │  (Docker volume, cloned by ngsmanager-init)\n        └──────────────┘\n```\n\n## Visual Pipeline Drawer\n\nThe platform includes a **drag-and-drop pipeline editor** at `/drawer` where\nusers can visually design Nextflow pipelines without writing prompts.\n\n### How it works\n\n1. **Drag** components from the categorized palette onto the canvas\n2. **Connect** outputs to inputs by dragging between port dots\n3. **Click \"Generate Pipeline\"** — the graph is sent to the architect LLM,\n   which generates the Nextflow code including data shaping and channel operations\n4. **Validate** the generated code directly against the framework with `nextflow -preview`\n\n### Features\n\n- **Named I/O ports** on each node, derived from the framework catalog\n- **Color-coded ports**: blue = data channels (reads, assembly), orange = runtime parameters (reference, schema, host)\n- **Save/load drawings** to the database, with version stamping (framework commit + LLM model)\n- **Version mismatch warning** when loading a drawing created with an older framework version\n- **Shareable links**: `/drawer?drawing=ID` — any logged-in user can open the same drawing\n- **Export**: SVG and PNG download of the canvas\n- **GitHub links**: each node links to its source `.nf` file on GitHub\n\n### Relationship with the chat\n\nThe drawer skips the consultant (conversation) phase and goes straight to the\nexecutor subgraph (hydrator → architect → repair → renderer → diagram). The\nvisual graph replaces the design plan that the consultant normally produces.\n\nDrawings generated from the drawer appear in the chat sidebar with a pipeline\npreview thumbnail and a link back to the drawer.\n\n## System stats dashboard\n\nA live status bar at the bottom of the chat shows:\n\n- **LLM model** currently configured (e.g. `labs-devstral-small-2512`)\n- **GPU**: name, VRAM usage, temperature (requires NVIDIA Container Toolkit)\n- **RAM**: usage percentage\n- **Framework commit**: clickable link to the ngsmanager commit on GitHub\n\nStats refresh every 30 seconds via `GET /api/system-info`.\n\n## Pipeline validation\n\nGenerated Nextflow code can be validated in-browser against the real framework\nusing `nextflow -preview`. The backend container includes Java and Nextflow,\nso validation is fully self-contained.\n\n- **Green** = syntactically valid\n- **Green + warning** = valid syntax, missing runtime parameters (expected for framework pipelines)\n- **Red** = syntax or structural errors, shown in a styled dialog\n\nAvailable both in the chat results panel and in the drawer.\n\n## LLM architecture\n\nThe \"AI\" of the platform is a **LangGraph state machine** with grounded RAG.\nThe whole point is to make it impossible for the LLM to hallucinate tools\nthat don't exist in the cohesive-ngsmanager framework.\n\n### The agent graph\n\n```\n         ┌──────────────┐\n         │   /api/chat  │\n         └──────┬───────┘\n                ▼\n       ┌──────────────────┐\n       │ planner subgraph │\n       │  ┌────────────┐  │\n       │  │ consultant │  │  ← natural-language conversation,\n       │  └────────────┘  │    builds the design plan with\n       │  ┌────────────┐  │    RAG-validated component IDs\n       │  │  trim msgs │  │\n       │  └────────────┘  │\n       └────────┬─────────┘\n                │ status == APPROVED?\n        ┌───────┴───────┐\n        │ no            │ yes\n        ▼               ▼\n       END    ┌─────────────────────┐\n              │ executor subgraph   │\n              │  ┌─────────────┐    │\n              │  │  hydrator   │    │  ← injects the actual .nf source\n              │  └──────┬──────┘    │    code for every selected step\n              │         ▼           │\n              │  ┌─────────────┐    │\n              │  │  architect  │◀──┐│  ← generates a strict Pydantic AST\n              │  └──────┬──────┘   ││    (NextflowPipelineAST)\n              │         │ valid?   ││\n              │      ┌──┴──┐       ││\n              │      │ no  │ yes   ││\n              │      ▼     │       ││\n              │   ┌──────┐ │       ││\n              │   │repair├─┘       ││  ← max 8 retries with the\n              │   └──┬───┘         ││    validation error injected\n              │      │             ││\n              │      └─────────────┘│\n              │         ▼           │\n              │  ┌─────────────┐    │\n              │  │  renderer   │    │  ← AST → Nextflow Groovy via Jinja2\n              │  └──────┬──────┘    │\n              │         ▼           │\n              │  ┌─────────────┐    │\n              │  │   diagram   │    │  ← AST → Mermaid (deterministic)\n              │  └─────────────┘    │\n              └─────────────────────┘\n                         │\n                         ▼\n                 nextflow_code, mermaid_code, ast_json\n```\n\nSource: `backend/app/services/graph.py` and `backend/app/services/agents.py`.\n\n### Models\n\n| Role | Model | Provider |\n|---|---|---|\n| Main LLM (consultant + architect) | `labs-devstral-small-2512` | Mistral (`langchain-mistralai`) |\n| Embeddings (RAG semantic search) | `Qwen/Qwen3-Embedding-0.6B` | local (`langchain-huggingface`) |\n| Judge LLM (eval / `test_consultant_rag.py` only) | `llama-3.3-70b-versatile` | Groq (optional, only for evaluation) |\n\nConfigured in `backend/app/services/llm.py` and `backend/app/core/config.py`.\nThe Mistral key is **required** at runtime; the Groq key is only needed for the evaluation suite.\n\n### Knowledge base\n\nLives in `backend/data/`:\n\n| File / dir | What it is | Generated by |\n|---|---|---|\n| `data/catalog/catalog_part1_components.json` | Per-step metadata: tool, domain, inputs, outputs, keywords | `sync_framework.py` |\n| `data/catalog/catalog_part2_templates.json` | Per-pipeline-template metadata + `logic_flow` | `sync_framework.py` |\n| `data/catalog/catalog_part3_resources.json` | Helper Groovy functions extracted from the framework | `sync_framework.py` |\n| `data/catalog/tool_whitelist.json` | Flat allow-list of every tool the LLM is allowed to mention | `sync_framework.py` |\n| `data/code_store_hollow.jsonl` | The **actual** Groovy source of every step / template, indexed by ID | `sync_framework.py` |\n| `data/faiss_index/` | Binary FAISS index built from the catalog text, used for semantic search | `rebuild_faiss_index.py` |\n\nThe two-stage retrieval inside `backend/app/services/tools.py` first does\n**keyword + metadata scoring** over the JSON catalogs, then falls back to a\n**FAISS semantic search** with relative-distance pruning, and finally\n**hydrates** every selected ID with the verbatim Groovy source from\n`code_store_hollow.jsonl` so the architect LLM never has to invent tool flags.\n\nA diagram of the same flow lives in [`backend/data/README.md`](backend/data/README.md).\n\n## Updating the framework and rebuilding the knowledge base\n\nThe catalog and FAISS index are derived artifacts that must be regenerated\nwhenever the upstream `cohesive-ngsmanager` framework changes (new step,\nrenamed module, updated params, …).\n\n### Update only the framework checkout\n\n```bash\ndocker compose run --rm ngsmanager-init   # git pull inside the volume\ndocker compose restart backend\n```\n\nThis refreshes the source files under `/ngsmanager` but **does not** rebuild\nthe catalog or the embeddings — the backend keeps using the previously\ngenerated `data/catalog/*.json` and `data/faiss_index/`.\n\n### Full sync (catalog + whitelist + FAISS)\n\nWhen the framework changed in a meaningful way, run the full sync. It walks\n`/ngsmanager`, regenerates all catalog files, the tool whitelist, and the\nFAISS index:\n\n```bash\n# inside the backend container, with the framework already mounted at /ngsmanager\ndocker compose exec backend python sync_framework.py\n\n# or, if you want to skip the (slower) FAISS rebuild during iteration:\ndocker compose exec backend python sync_framework.py --skip-faiss\n\n# from the host, pointing at any local checkout:\ndocker compose exec backend python sync_framework.py --ngsmanager-dir /ngsmanager\n```\n\nAfter it finishes, **restart the backend** so the new index is loaded into memory:\n\n```bash\ndocker compose restart backend\n```\n\n### Only rebuild the FAISS index\n\nIf you tweaked one of the `catalog_part*.json` files by hand and just want\nto refresh the embeddings (no framework re-scan):\n\n```bash\ndocker compose exec backend python rebuild_faiss_index.py\ndocker compose restart backend\n```\n\nThe first run downloads the `Qwen/Qwen3-Embedding-0.6B` model (~1.2 GB)\ninto `HF_HOME=/tmp/huggingface`, so it takes a couple of minutes; subsequent\nrebuilds reuse the cached model and finish in seconds.\n\n## Testing the LLM pipeline\n\nThe backend ships with several test/eval scripts under `backend/`. They are\n**not** wired into the runtime container and use `subprocess` against the\nreal Nextflow CLI for validation, so they're meant to be run manually\n(typically from a host with `nextflow` installed and the framework cloned\nside-by-side).\n\n### Quick smoke test of the LangGraph state machine\n\n`backend/test_graph.py` runs a single in-process invocation, useful to\nverify the graph compiles and can produce a Nextflow string:\n\n```bash\ndocker compose exec backend python test_graph.py\n```\n\n### Mermaid renderer unit tests\n\n`backend/test_mermaid.py` validates the deterministic AST → Mermaid renderer\nin isolation (no LLM call, no Mistral key required):\n\n```bash\ndocker compose exec backend python test_mermaid.py\n```\n\n### End-to-end pipeline validation against the real framework\n\n`backend/test_e2e.py` is the heavy one: it sends a curated list of\nuser prompts (`L1` simple → `L4` complex) to `POST /api/chat`, extracts the\ngenerated `.nf` code, drops it into the framework's `pipelines/` dir as\n`_llm_e2e_test.nf`, and runs `nextflow -preview` on it to make sure the\ngenerated workflow is syntactically and semantically valid against the\nactual cohesive-ngsmanager modules.\n\n```bash\n# all scenarios\npython backend/test_e2e.py\n\n# only the L1 + L2 levels\npython backend/test_e2e.py --levels 1 2\n\n# single ad-hoc prompt\npython backend/test_e2e.py --prompt \"Trim Illumina paired-end reads with fastp\"\n```\n\nRequires:\n- The backend reachable at `http://localhost:8080` (override with `API_URL`)\n- A working `nextflow` binary on the host\n- The `cohesive-ngsmanager` repo cloned locally and pointed at via `NGSMANAGER_DIR`\n\n### LLM-as-judge academic evaluation\n\n`backend/test_consultant_rag.py` uses **pytest** plus a Groq-hosted Llama-3.3\n\"strict academic reviewer\" to score the consultant, architect and diagram\nnodes on faithfulness, relevance, syntax, logic and mapping. Needs both\n`MISTRAL_API_KEY` and `GROQ_API_KEY` in the environment:\n\n```bash\ndocker compose exec -e GROQ_API_KEY=... backend pytest -v test_consultant_rag.py\n```\n\n### Full evaluation report\n\n`backend/evaluate_llm.py` is the most exhaustive: it walks every prompt\nscenario, validates every generated pipeline against a hard-coded allow-list\nof valid framework components, and writes a full markdown report.\n\n```bash\ndocker compose exec backend python evaluate_llm.py --output report.md\n```\n\n### Single-pipeline validator\n\n`backend/validate_pipeline.py` takes one `.nf` file (or one prompt) and runs\njust the `nextflow -preview` validation step against the framework. Handy\nwhen iterating on a single failing scenario:\n\n```bash\npython backend/validate_pipeline.py path/to/pipeline.nf\n```\n\n## Scripts\n\n### Repo-level (`scripts/`)\n- `up.sh` — render Caddyfile from `.env` and `docker compose up -d`\n- `render-caddyfile.sh` — generate `caddy/Caddyfile` from the env vars\n- `check-secrets.sh` — secret scanner, runs as a git pre-push hook\n\nInstall the pre-push hook:\n```bash\nln -sf ../../scripts/check-secrets.sh .git/hooks/pre-push\n```\n\n### Backend-level (`backend/`, run inside the container)\n- `sync_framework.py` — full sync from cohesive-ngsmanager: code store, catalog, whitelist, FAISS\n- `rebuild_faiss_index.py` — only rebuild the FAISS index from the existing catalog JSONs\n- `generate_catalog.py` — only regenerate the catalog (sub-step of `sync_framework.py`)\n- `test_graph.py` — quick LangGraph smoke test\n- `test_mermaid.py` — unit tests for the AST → Mermaid renderer\n- `test_e2e.py` — end-to-end pipeline generation + `nextflow -preview` validation\n- `test_consultant_rag.py` — pytest + Groq-judged academic evaluation\n- `evaluate_llm.py` — full markdown evaluation report\n- `validate_pipeline.py` — single-`.nf`/single-prompt validator\n- `generate_report.py` — assemble a report from previously cached results\n- `generate_catalog.py` — regenerate catalog files from the framework\n\n## Configuration reference\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `DOMAIN` | `localhost` | Hostname users access the platform with |\n| `HTTPS_MODE` | `off` | `off` / `internal` / `auto` |\n| `CADDY_HOST_PORT` | `9000` | Host port for the HTTP listener |\n| `CADDY_HOST_HTTPS_PORT` | `9443` | Host port for the HTTPS listener |\n| `TRUSTED_PROXIES` | `private_ranges` | CIDR(s) of upstream proxies |\n| `MISTRAL_API_KEY` | _required_ | API key for the LLM provider |\n| `NGSMANAGER_REPO` | github.com/genpat-it/cohesive-ngsmanager | Framework repo to clone |\n| `NGSMANAGER_BRANCH` | `main` | Framework branch |\n| `POSTGRES_USER` / `POSTGRES_PASSWORD` / `POSTGRES_DB` | `cohesive` / dev / `cohesive` | Database credentials |\n| `DATABASE_URL` | `postgresql+psycopg2://...` | Connection string for SQLAlchemy |\n| `JWT_SECRET` | _required_ | 32-byte random string for signing JWTs |\n| `DEMO_USER` / `DEMO_PASSWORD` | `demo` / placeholder | Initial seed user |\n| `CORS_ORIGINS` | `http://localhost:9000,...` | Comma-separated allowed origins |\n| `COOKIE_SECURE` | `false` | Set to `true` behind HTTPS |\n| `COOKIE_PATH` | `/` | Set to `/llm` if behind a path-prefix proxy |\n| `COOKIE_SAMESITE` | `lax` | `lax` / `strict` / `none` |\n| `LOGIN_RATE_LIMIT` | `5/minute` | Per-IP rate limit on `/auth/login` |\n| `BASE_PATH` | `/` | Sub-path the app is served under (must end with `/`) |\n| `NVIDIA_VISIBLE_DEVICES` | `all` | GPU visibility for the backend container (system stats) |\n\nFull reference with comments: [`.env.example`](.env.example)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgenpat-it%2Fcohesive-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgenpat-it%2Fcohesive-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgenpat-it%2Fcohesive-llm/lists"}