{"id":50359426,"url":"https://github.com/ria-19/semantic_router","last_synced_at":"2026-05-30T00:30:25.638Z","repository":{"id":327713903,"uuid":"1108295244","full_name":"ria-19/semantic_router","owner":"ria-19","description":"A QLoRA fine-tuned Llama-3.1-8B-Instruct AI agent for structured outputs, secure Python sandbox execution, hybrid code search, web scraping, and safe human-in-the-loop automation. Perfect for developers, researchers, and AI-assisted coding.","archived":false,"fork":false,"pushed_at":"2025-12-05T09:07:45.000Z","size":186,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-12-08T17:28:09.294Z","etag":null,"topics":["ai-agents","fine-tuning","headless-browser","hybrid-rag","llm","lora","qlora","sandbox","semantic-search","unsloth"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ria-19.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-02T09:26:49.000Z","updated_at":"2025-12-05T09:07:49.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ria-19/semantic_router","commit_stats":null,"previous_names":["ria-19/semantic_router"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/ria-19/semantic_router","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ria-19%2Fsemantic_router","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ria-19%2Fsemantic_router/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ria-19%2Fsemantic_router/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ria-19%2Fsemantic_router/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ria-19","download_url":"https://codeload.github.com/ria-19/semantic_router/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ria-19%2Fsemantic_router/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33676190,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-29T02:00:06.066Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","fine-tuning","headless-browser","hybrid-rag","llm","lora","qlora","sandbox","semantic-search","unsloth"],"created_at":"2026-05-30T00:30:19.958Z","updated_at":"2026-05-30T00:30:25.629Z","avatar_url":"https://github.com/ria-19.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Semantic Router (Fine-Tuned Agent Brain)\n\n![Python](https://img.shields.io/badge/Python-3.11-blue?logo=python\u0026logoColor=white)\n![Unsloth](https://img.shields.io/badge/Training-Unsloth-green)\n![Llama 3](https://img.shields.io/badge/Model-Llama_3.1-blueviolet)\n![Pydantic](https://img.shields.io/badge/Validation-Pydantic-e92063)\n![MIT License](https://img.shields.io/badge/License-MIT-yellow)\n\n\u003e **Phase 1: Synthetic Data Engineering (Complete)**  \n\u003e **Phase 2: Dataset Publication (Complete)**  \n\u003e **Phase 3: QLoRA Fine-Tuning (In Progress)**\n\n## Mission\n\nTo replace massive, high-latency System Prompts in Agent architectures with a **specialized, fine-tuned Adapter**. \n\nThis project builds the \"Brain\" of an Autonomous Coding Agent. Instead of relying on a generic LLM to guess how to use tools, we are fine-tuning **Llama-3.1-8B** to function as a deterministic **Intent Router** that converts natural language queries into strict, executable JSON tool calls with \u003c 200ms latency.\n\n## Dataset Release\n\nThe training dataset is now publicly available on HuggingFace:\n\n**[tai-tai-sama/semantic-router-dataset](https://huggingface.co/datasets/tai-tai-sama/semantic-router-dataset)**\n\n- **502 high-fidelity examples** (451 train / 51 test)\n- Generated by **GPT-5.2, Gemini 2.5, Llama-3.3** ensemble\n- **60-70% validation pass rate** (only production-ready examples survive)\n- Stratified by intent distribution and status types\n- MIT Licensed for commercial use\n\n## Architecture\n\nWe moved beyond simple scripting to a modular **Data Factory** approach. The pipeline generates high-diversity training examples (Domains + Personas + Edge Cases) while enforcing strict logic constraints through a 3-layer validation system.\n\n### The File Structure\n```text\n.\n├── generate_data.py       # Main entry point for the data factory\n├── src/\n│   ├── schemas.py         # Pydantic definitions for all tools\n│   ├── prompts.py         # Advanced prompt logic (CoT, Anti-Hallucination)\n│   ├── generator.py       # Multi-model generation logic\n│   ├── client.py          # Instructor client setup\n│   ├── formatting.py      # ChatML Formatter: JSON → Llama-3 Training Format\n│   ├── upload_data.py     # HuggingFace Dataset Uploader\n│   └── utils.py           # Validation \u0026 Null-Tax removal\n├── data/\n│   ├── raw/               # JSONL files (User → Thought → Tool)\n│   └── formatted/         # ChatML formatted training data\n└── notebooks/             # Colab notebooks for Unsloth training\n```\n\n## The \"Golden\" Data Strategy\n\nThis is not just random synthetic data. The pipeline implements **Logic-Aware Generation** to prevent common agent failures:\n\n### 1. Anti-Hallucination (The \"Magic Path\" Fix)\n- **Constraint:** The generator distinguishes between **Discovery** (Search) and **Action** (File Ops).\n- **Result:** The agent never attempts to read a file (`file_manager`) unless the user explicitly provides the path or context.\n\n### 2. Avoiding the \"Null Tax\"\n- **Constraint:** Output schemas use `exclude_none=True`.\n- **Result:** We save ~20% token usage per example by stripping empty fields (e.g., `content: null` during read operations), resulting in faster inference.\n\n### 3. Structured Chain-of-Thought (CoT)\n- **Constraint:** Enforced minimum 6-word reasoning and explicit tool selection logic.\n- **Result:** The model doesn't just parrot the user; it explains *why* it chose a tool (e.g., *\"User is asking for a concept, so I must use semantic search, not exact match\"*).\n\n### 4. \"Model Roulette\" Generation\n- **Strategy:** We rotate between **GPT-5.2** (High-fidelity), **Gemini 2.5 Pro** (Advanced reasoning), **Gemini 2.5 Flash** (Speed), and **Llama-3.3-70B** (Diversity) to prevent \"Model Collapse\" and avoid API rate limits.\n- **Result:** Linguistic diversity across 40+ domains, 35+ personas, and 70+ query styles.\n\n### 5. 3-Layer Validation Pipeline\n\n| Layer | Type | Purpose |\n| :--- | :--- | :--- |\n| **1. Structural** | Pydantic Schema | JSON well-formedness, required keys, type correctness |\n| **2. Quality** | Heuristic Analysis | Anti-parroting, substantive reasoning (\u003e6 words), specific outputs |\n| **3. Domain Logic** | Safety \u0026 Semantics | Unsafe code detection (e.g., `rm -rf`), content validation |\n\n**Only 60-70% of generated examples survive all gates.**\n\n## The Toolset (Schema)\n\nThe fine-tuned model is trained to route requests to these four deterministic tools:\n\n| Tool | Capability | Logic Constraint |\n| :--- | :--- | :--- |\n| **`codebase_search`** | RAG / Semantic Search | Must choose `exact` vs `semantic` vs `hybrid` mode based on query type |\n| **`file_manager`** | Read / Write / Patch / List | Requires explicit paths. No guessing. Strict validation rules |\n| **`sandbox_exec`** | Python Interpreter | For calculation, verification, or logic testing only. 30s timeout |\n| **`ask_human`** | Human-in-the-Loop | Triggered by ambiguity or high-risk actions (e.g., DB deletion) |\n\n### Output Format: Discriminated Union\n\nThe model produces a flattened discriminated union with two variants:\n\n**Type A: Tool Invocation** (`status=\"running\"`)\n```json\n{\n  \"status\": \"running\",\n  \"thought\": \"User needs to find authentication logic. Semantic search is appropriate.\",\n  \"tool_use\": {\n    \"tool_name\": \"codebase_search\",\n    \"arguments\": {\n      \"query\": \"authentication middleware JWT validation\",\n      \"mode\": \"semantic\"\n    }\n  },\n  \"final_answer\": null\n}\n```\n\n**Type B: Direct Answer** (`status=\"complete\"`)\n```json\n{\n  \"status\": \"complete\",\n  \"thought\": null,\n  \"tool_use\": null,\n  \"final_answer\": \"The server runs on port 8000 by default. Override with the PORT environment variable.\"\n}\n```\n\n## Usage\n\nThis project uses `uv` for modern, fast Python dependency management.\n\n### 1. Setup Environment\nCreate a `.env` file with your API keys:\n```bash\nGROQ_API_KEY=gsk_...\nGEMINI_API_KEY=AIza...\nHF_TOKEN=hf_...\n```\n\n### 2. Generate Synthetic Data\nRun the factory to create `data/raw/router_train.jsonl`:\n```bash\nuv run generate_data.py\n```\n*Note: Check `src/config.py` to adjust batch size, domains, and personas.*\n\n### 3. Format for Llama-3\nConvert the raw JSONL into ChatML format:\n```bash\nuv run src/formatting.py\n```\n\n### 4. Upload to HuggingFace\nVersion control your dataset:\n```bash\nuv run src/upload_data.py\n```\n\n### 5. Fine-Tune with Unsloth\nUse the provided Colab notebook in `notebooks/` or follow the training guide in the dataset card.\n\n## Dataset Statistics\n\n- **Total Examples:** 502 (451 train / 51 test)\n- **Intent Distribution:**\n  - 35% Search operations\n  - 24% Compute/execution tasks\n  - 18% File modifications\n  - 15% Direct answers\n  - 8% Human escalations\n- **Diversity Metrics:**\n  - 40+ domains (E-commerce, Healthcare, Fintech, ML, Gaming, etc.)\n  - 35+ personas (SRE, CTO, QA, Data Scientist, Junior Dev, etc.)\n  - 70+ query styles (Fragmented, narrative, urgent, code-mixed, etc.)\n\n## Key Learnings \u0026 Design Decisions\n\n### Why Multi-Model Generation?\nSingle-model synthetic data suffers from **mode collapse** and stylistic uniformity. By rotating between GPT-5.2, Gemini 2.5, and Llama-3.3, we achieved:\n- Diverse linguistic patterns\n- Robust generalization across domains\n- API rate limit distribution\n\n### Why Strict Validation?\nCommon synthetic datasets accept malformed outputs that hurt model performance. Our 3-layer validation ensures:\n- Only production-ready examples enter training\n- Unsafe operations are filtered\n- Generic or parroted responses are rejected\n\n### Why Discriminated Union Schema?\nThe `status` field acts as a **type discriminator**, allowing the model to learn:\n- When to invoke tools vs. answer directly\n- Proper null handling (avoiding the \"null tax\")\n- Clean separation between reasoning and action\n\n## Knowledge Base\n\nCheck the `docs/` folder for research notes on:\n- **LoRA / PEFT**: Why we freeze base weights and only train adapters\n- **Unsloth**: Optimization techniques for 2x faster training with lower memory\n- **Data Hygiene**: Schema validation rules and quality filtering\n- **Prompt Engineering**: Meta-prompting architecture for generation\n\n## Roadmap\n\n- [x] Phase 1: Synthetic Data Engineering\n- [x] Phase 2: Dataset Publication on HuggingFace\n- [ ] Phase 3: QLoRA Fine-Tuning with Unsloth\n- [ ] Phase 4: Evaluation Suite (Accuracy, Latency, Safety)\n- [ ] Phase 5: Production Deployment \u0026 Benchmarking\n\n## Contributing\n\nContributions are welcome! Please:\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Acknowledgments\n\n- **Unsloth** for efficient fine-tuning infrastructure\n- **Instructor** for structured output generation\n- **Anthropic, OpenAI, Google** for frontier model access\n- **HuggingFace** for dataset hosting\n\n## Contact\n\n**Riya Sangwan** - [@ria-19](https://github.com/ria-19)\n\n**Dataset:** [tai-tai-sama/semantic-router-dataset](https://huggingface.co/datasets/tai-tai-sama/semantic-router-dataset)\n\n---\n\n*\"Synthetic data is only as good as the validation pipeline that produces it.\"*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fria-19%2Fsemantic_router","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fria-19%2Fsemantic_router","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fria-19%2Fsemantic_router/lists"}