{"id":29048771,"url":"https://github.com/muhkartal/llm-outputverifier","last_synced_at":"2026-05-15T21:09:06.968Z","repository":{"id":291387708,"uuid":"977475039","full_name":"muhkartal/llm-outputVerifier","owner":"muhkartal","description":"framework for detecting hallucinations in LLM chain-of-thought reasoning. Features synthetic data corruption, transformer-based classifiers, Streamlit UI, and FastAPI backend.","archived":false,"fork":false,"pushed_at":"2025-05-04T09:55:09.000Z","size":381,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-26T18:45:42.511Z","etag":null,"topics":["chain-of-thought","deep-learning","fastapi","llm","nlp","python","reasoning-verification","streamlt","transformer"],"latest_commit_sha":null,"homepage":"https://kartal.dev/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/muhkartal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-04T09:47:09.000Z","updated_at":"2025-05-04T09:55:13.000Z","dependencies_parsed_at":"2025-05-04T10:44:34.009Z","dependency_job_id":null,"html_url":"https://github.com/muhkartal/llm-outputVerifier","commit_stats":null,"previous_names":["muhkartal/llm-outputverifier"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/muhkartal/llm-outputVerifier","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhkartal%2Fllm-outputVerifier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhkartal%2Fllm-outputVerifier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhkartal%2Fllm-outputVerifier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhkartal%2Fllm-outputVerifier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/muhkartal","download_url":"https://codeload.github.com/muhkartal/llm-outputVerifier/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhkartal%2Fllm-outputVerifier/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266553938,"owners_count":23947236,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chain-of-thought","deep-learning","fastapi","llm","nlp","python","reasoning-verification","streamlt","transformer"],"created_at":"2025-06-26T18:35:56.504Z","updated_at":"2026-05-15T21:09:06.961Z","avatar_url":"https://github.com/muhkartal.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# llm-outputVerifier\n\n![screenshot](images/llm-outputVerifier.png)\n\n\u003cdiv align=\"center\"\u003e\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED?logo=docker)](https://www.docker.com/)\n[![Documentation](https://img.shields.io/badge/docs-latest-brightgreen.svg)](https://username.github.io/llm-outputVerifier/)\n[![PyPI version](https://img.shields.io/badge/pypi-1.2.0-blue.svg)](https://pypi.org/project/llm-outputVerifier/)\n\n**A production-ready pipeline for detecting hallucinations in LLM reasoning**\n\n[Overview](#overview) • [Features](#key-features) • [Quick Start](#quick-start) • [UI Guide](#using-the-ui) • [API](#using-the-api) • [Training](#training-custom-models) • [Contributing](#contributing)\n\n\u003c/div\u003e\n\n## Overview\n\nllm-outputVerifier is an enterprise-grade system that analyzes chain-of-thought (CoT) reasoning from large language models, scrutinizing each reasoning step to classify it as either grounded or hallucinated. The pipeline leverages the GSM8K (Grade School Math 8K) dataset, augmenting it with carefully designed synthetic corruptions to create high-quality training data for the hallucination detection model.\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"images/arch.png\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"images/arch.png\"\u003e\n  \u003cimg alt=\"Fallback image description\" src=\"images/arch.png\"\u003e\n\u003c/picture\u003e\n\n### How It Works\n\nThe system works in three key stages:\n\n1. **Analysis**: Breaks down chains of reasoning into individual steps\n2. **Classification**: Applies transformer-based sequence classification to each step\n3. **Visualization**: Provides confidence-scored results with detailed explanations\n\nOur models achieve 94.3% accuracy on benchmark datasets, significantly outperforming current state-of-the-art approaches to hallucination detection in mathematical reasoning.\n\n## Key Features\n\n\u003ctable\u003e\n\u003ctr\u003e\n  \u003ctd width=\"33%\"\u003e\n    \u003ch3\u003eRobust Training Pipeline\u003c/h3\u003e\n    \u003cul\u003e\n      \u003cli\u003eState-of-the-art transformer-based sequence classification\u003c/li\u003e\n      \u003cli\u003eMixed-precision training and gradient checkpointing\u003c/li\u003e\n      \u003cli\u003eEfficient vectorized data loading\u003c/li\u003e\n      \u003cli\u003eSynthetic data augmentation techniques\u003c/li\u003e\n    \u003c/ul\u003e\n  \u003c/td\u003e\n  \u003ctd width=\"33%\"\u003e\n    \u003ch3\u003eProduction Architecture\u003c/h3\u003e\n    \u003cul\u003e\n      \u003cli\u003eClean, modular design with type annotations\u003c/li\u003e\n      \u003cli\u003eMemory-efficient processing for large datasets\u003c/li\u003e\n      \u003cli\u003eScalable FastAPI backend with validation\u003c/li\u003e\n      \u003cli\u003eIntuitive Streamlit frontend with visualizations\u003c/li\u003e\n    \u003c/ul\u003e\n  \u003c/td\u003e\n  \u003ctd width=\"33%\"\u003e\n    \u003ch3\u003eDevOps Integration\u003c/h3\u003e\n    \u003cul\u003e\n      \u003cli\u003eFully dockerized with multi-stage builds\u003c/li\u003e\n      \u003cli\u003eComprehensive CI/CD with GitHub Actions\u003c/li\u003e\n      \u003cli\u003eRigorous testing with 95%+ coverage\u003c/li\u003e\n      \u003cli\u003ePerformance monitoring and health checks\u003c/li\u003e\n    \u003c/ul\u003e\n  \u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n## Quick Start\n\n### Prerequisites\n\n-  Python 3.9+\n-  Docker and Docker Compose (recommended)\n-  CUDA-compatible GPU (optional, for accelerated training)\n\n### Installation\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eOption 1: Docker Deployment (Recommended)\u003c/strong\u003e\u003c/summary\u003e\n\n```bash\n# Clone repository\ngit clone https://github.com/username/llm-outputVerifier.git\ncd llm-outputVerifier\n\n# Configure environment variables (optional)\ncp .env.example .env\n# Edit .env file with your configuration\n\n# Build and run with Docker Compose\ndocker-compose up --build\n\n# Access UI at http://localhost:8501\n# Access API at http://localhost:8000\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eOption 2: Direct Installation\u003c/strong\u003e\u003c/summary\u003e\n\n```bash\n# Clone repository\ngit clone https://github.com/username/llm-outputVerifier.git\ncd llm-outputVerifier\n\n# Create and activate virtual environment\npython -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\n\n# Install dependencies\npip install -e .\n\n# Run tests to verify installation\npytest tests/\n\n# Start the API and UI\npython -m hallucination_hunter.api.main \u0026 python -m hallucination_hunter.ui.app\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eOption 3: PyPI Installation\u003c/strong\u003e\u003c/summary\u003e\n\n```bash\n# Install from PyPI\npip install llm-outputVerifier\n\n# Start services\nllm-outputVerifier start\n```\n\n\u003c/details\u003e\n\n## Using the UI\n\nThe Streamlit UI provides an intuitive interface for analyzing reasoning chains:\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"images/ui.png\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"images/ui.png\"\u003e\n  \u003cimg alt=\"Fallback image description\" src=\"images/ui.png\"\u003e\n\u003c/picture\u003e\n\n1. Open http://localhost:8501 in your browser\n2. Enter a mathematical question in the first text field\n3. Paste chain-of-thought reasoning (with each step on a new line) in the second text field\n4. Click \"Analyze Reasoning\"\n\n### Understanding Results\n\nThe system processes the reasoning and displays results with color-coded confidence scoring:\n\n-  🟢 **Green**: Grounded reasoning (high confidence)\n-  🔵 **Blue**: Likely grounded (lower confidence)\n-  🟠 **Orange**: Potential hallucination (lower confidence)\n-  🔴 **Red**: Confirmed hallucination (high confidence)\n\nThe UI also provides summary metrics including the total hallucination rate and confidence distribution.\n\n## Using the API\n\nThe RESTful API is accessible at http://localhost:8000 and provides comprehensive endpoints for integration:\n\n| Endpoint         | Method | Description                                |\n| ---------------- | ------ | ------------------------------------------ |\n| `/`              | GET    | Welcome message and API information        |\n| `/health`        | GET    | Health check and status monitoring         |\n| `/docs`          | GET    | Interactive API documentation (Swagger UI) |\n| `/predict`       | POST   | Hallucination detection endpoint           |\n| `/batch_predict` | POST   | Batch processing for multiple chains       |\n| `/models`        | GET    | List available models                      |\n| `/metrics`       | GET    | Performance and usage metrics              |\n\n### Example API Request\n\n```python\nimport requests\n\n# Define the input data\ndata = {\n    \"question\": \"John has 5 apples. He buys 2 more. How many apples does he have now?\",\n    \"reasoning\": \"John starts with 5 apples.\\nThen he buys 2 more apples.\\nSo in total, he has 5 + 2 = 7 apples.\"\n}\n\n# Send POST request to the API\nresponse = requests.post(\n    \"http://localhost:8000/predict\",\n    json=data,\n    headers={\"Content-Type\": \"application/json\"}\n)\n\n# Process the response\nresult = response.json()\nprint(f\"Analyzed {result['num_steps']} steps\")\nprint(f\"Found {result['num_hallucinations']} hallucinations\")\n\n# Print the prediction for each step\nfor i, pred in enumerate(result[\"predictions\"]):\n    status = \"HALLUCINATION\" if pred[\"is_hallucination\"] else \"GROUNDED\"\n    confidence = pred[\"confidence\"]\n    print(f\"Step {i+1}: {status} ({confidence:.2f})\")\n    print(f\"  {pred['step']}\")\n```\n\n### Performance Considerations\n\nThe API supports both synchronous and asynchronous processing modes:\n\n```python\n# Asynchronous batch processing for large workloads\nresponse = requests.post(\n    \"http://localhost:8000/batch_predict?async=true\",\n    json={\"items\": batch_data}\n)\n\n# Get the job ID from the response\njob_id = response.json()[\"job_id\"]\n\n# Check job status\nstatus_response = requests.get(f\"http://localhost:8000/job/{job_id}\")\n```\n\n## Training Custom Models\n\nllm-outputVerifier provides a flexible training pipeline that allows you to customize various aspects of the model training process.\n\n### Basic Training\n\n```bash\n# Train with default settings\npython scripts/train.py\n```\n\n### Advanced Configuration\n\n```bash\n# Train with custom settings\npython scripts/train.py \\\n    --config configs/custom_config.json \\\n    --model_name roberta-large \\\n    --batch_size 32 \\\n    --learning_rate 3e-5 \\\n    --num_epochs 5 \\\n    --corruption_rate 0.4 \\\n    --output_dir ./models/custom_model\n```\n\n### Experiment Tracking\n\nTraining progress is tracked with Weights \u0026 Biases by default. To view training metrics:\n\n1. Create a W\u0026B account at [wandb.ai](https://wandb.ai)\n2. Set your API key: `export WANDB_API_KEY=your_api_key`\n3. Run training with `--use_wandb` flag\n4. View metrics at the W\u0026B project dashboard\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"images/train.png\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"images/train.png\"\u003e\n  \u003cimg alt=\"Fallback image description\" src=\"images/train.png\"\u003e\n\u003c/picture\u003e\n\n### Performance Optimization\n\nFor large models or datasets, use these optimization flags:\n\n```bash\npython scripts/train.py \\\n    --model_name roberta-large \\\n    --batch_size 8 \\\n    --gradient_accumulation_steps 4 \\\n    --mixed_precision fp16 \\\n    --gradient_checkpointing \\\n    --output_dir ./models/optimized_model\n```\n\n## Project Structure\n\n```\nhallucination_hunter/\n├── data/                  # Dataset handling and synthetic corruption\n│   ├── augmentation.py    # Synthetic data corruption techniques\n│   ├── dataset.py         # Dataset class and dataloader creation\n│   └── preprocessing.py   # Data preparation and vectorization\n├── models/                # Model architecture definitions\n│   ├── classifier.py      # Hallucination classifier architecture\n│   └── encoder.py         # Transformer encoder utilities\n├── training/              # Training implementation\n│   ├── optimizer.py       # Optimizer and learning rate scheduling\n│   └── trainer.py         # Main training loop with metrics tracking\n├── evaluation/            # Evaluation metrics and analysis\n│   ├── metrics.py         # Classification metrics calculation\n│   └── visualizer.py      # Result visualization utilities\n├── inference/             # Model inference pipeline\n│   └── predictor.py       # Prediction logic for reasoning chains\n├── api/                   # FastAPI implementation\n│   ├── main.py            # API server and endpoints\n│   └── schemas.py         # Pydantic data validation schemas\n└── ui/                    # Streamlit user interface\n    └── app.py             # Interactive web application\n```\n\n## Contributing\n\nContributions to llm-outputVerifier are welcome! Please see our [CONTRIBUTING.md](CONTRIBUTING.md) guide for details on how to submit pull requests, report issues, or request features.\n\n### Development Setup\n\n```bash\n# Clone repository\ngit clone https://github.com/username/llm-outputVerifier.git\ncd llm-outputVerifier\n\n# Set up development environment\npython -m venv venv\nsource venv/bin/activate\npip install -e \".[dev]\"\n\n# Run code quality checks\nblack hallucination_hunter tests\nisort hallucination_hunter tests\nmypy hallucination_hunter\npytest tests/\n```\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003e\n    \u003cstrong\u003ellm-outputVerifier\u003c/strong\u003e - LLM-specific output checker\n  \u003c/p\u003e\n  \u003cp\u003e\n    \u003ca href=\"https://github.com/muhkartal/netsentry\"\u003eGitHub\u003c/a\u003e •\n    \u003ca href=\"https://hub.docker.com/r/muhkartal/netsentry\"\u003eDocker Hub\u003c/a\u003e •\n    \u003ca href=\"https://kartal.dev/\"\u003eDeveloper Website\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\nDeveloped by Muhammad Ibrahim Kartal | [kartal.dev](https://kartal.dev)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhkartal%2Fllm-outputverifier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuhkartal%2Fllm-outputverifier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhkartal%2Fllm-outputverifier/lists"}