{"id":34925651,"url":"https://github.com/aronno1920/capstoneexaminer","last_synced_at":"2026-05-26T14:02:59.280Z","repository":{"id":320338258,"uuid":"1079131414","full_name":"Aronno1920/CapstoneExaminer","owner":"Aronno1920","description":"The AI Examiner System you've described is clearly centered on leveraging the reasoning and structured output capabilities of Large Language Models (LLMs) for a robust academic assessment tool.","archived":false,"fork":false,"pushed_at":"2025-10-23T07:02:18.000Z","size":9582,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-11T18:02:00.279Z","etag":null,"topics":["chain-of-thought","concept-extraction","fastapi","llm-integration","prompting","scoring","semantic-analysis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aronno1920.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-19T06:57:50.000Z","updated_at":"2025-10-23T07:02:22.000Z","dependencies_parsed_at":"2025-10-23T09:06:59.833Z","dependency_job_id":null,"html_url":"https://github.com/Aronno1920/CapstoneExaminer","commit_stats":null,"previous_names":["aronno1920/capstoneexaminer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Aronno1920/CapstoneExaminer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aronno1920%2FCapstoneExaminer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aronno1920%2FCapstoneExaminer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aronno1920%2FCapstoneExaminer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aronno1920%2FCapstoneExaminer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aronno1920","download_url":"https://codeload.github.com/Aronno1920/CapstoneExaminer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aronno1920%2FCapstoneExaminer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33523669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T03:12:49.672Z","status":"ssl_error","status_checked_at":"2026-05-26T03:12:47.976Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chain-of-thought","concept-extraction","fastapi","llm-integration","prompting","scoring","semantic-analysis"],"created_at":"2025-12-26T14:29:41.862Z","updated_at":"2026-05-26T14:02:59.274Z","avatar_url":"https://github.com/Aronno1920.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Examiner System\n\n**An AI-powered narrative answer grading system using Large Language Models (LLMs) for semantic understanding and automated evaluation.**\n\n## 🌟 Overview\n\nThe AI Examiner System is a sophisticated solution for automatically grading narrative (essay-style) answers using advanced AI techniques. It employs Chain-of-Thought (CoT) reasoning and semantic analysis to understand the actual meaning of both ideal answers and student responses, providing fair, consistent, and detailed grading with comprehensive feedback.\n\n### Key Features\n\n- **🤖 Advanced AI Grading**: Uses GPT-4, Claude, or other powerful LLMs for semantic understanding\n- **🔄 Chain-of-Thought Processing**: Structured reasoning approach for consistent and explainable grading\n- **📊 Comprehensive Analysis**: Extracts key concepts, evaluates semantic similarity, and applies rubric-based scoring\n- **📝 Detailed Feedback**: Provides constructive feedback with strengths, weaknesses, and improvement suggestions\n- **⚡ REST API**: Easy integration with existing educational platforms\n- **🎯 Bias Monitoring**: Built-in mechanisms to ensure fair and unbiased grading\n- **📈 Scalable Architecture**: Supports both single and batch grading operations\n\n## 🚀 Getting Started\n\n### Prerequisites\n\n- Python 3.8 or higher\n- OpenAI API key (for GPT models) OR Anthropic API key (for Claude models)\n- pip package manager\n\n### Installation\n\n1. **Install dependencies**\n```bash\npip install -r requirements.txt\n```\n\n2. **Set up environment variables**\n```bash\n# Copy the example environment file\ncopy .env.example .env\n\n# Edit .env and add your API keys\nOPENAI_API_KEY=your_openai_api_key_here\nLLM_PROVIDER=openai\nLLM_MODEL=gpt-4\n```\n\n3. **Run the system**\n```bash\n# Start the REST API server\npython main.py\n\n# Or run the example usage\npython examples/usage_example.py\n```\n\n## 📖 Quick Usage\n\n### Python Example\n\n```python\nimport asyncio\nfrom src.models.schemas import IdealAnswer, StudentAnswer, GradingRubric, GradingCriteria\nfrom src.services.grading_service import ai_examiner\n\n# Create grading rubric\nrubric = GradingRubric(\n    subject=\"Physics\",\n    topic=\"Newton's Laws of Motion\",\n    criteria=[\n        GradingCriteria(name=\"Understanding\", description=\"Concept comprehension\", max_points=100.0)\n    ],\n    total_max_points=100.0\n)\n\n# Define ideal answer\nideal_answer = IdealAnswer(\n    question_id=\"physics_001\",\n    content=\"Newton's three laws describe forces and motion...\",\n    rubric=rubric,\n    subject=\"Physics\"\n)\n\n# Student answer\nstudent_answer = StudentAnswer(\n    student_id=\"STU001\",\n    question_id=\"physics_001\",\n    content=\"Newton has three laws about motion...\"\n)\n\n# Grade the answer\nasync def grade():\n    result = await ai_examiner.grade_answer(student_answer, ideal_answer)\n    print(f\"Score: {result.percentage:.1f}% - {result.detailed_feedback}\")\n\nasyncio.run(grade())\n```\n\n### REST API Example\n\n```bash\n# Start the server\npython main.py\n\n# Access the interactive docs\nopen http://localhost:8000/docs\n\n# Grade an answer via API\ncurl -X POST \"http://localhost:8000/grade\" -H \"Content-Type: application/json\" -d '{\n  \"student_answer\": {\"student_id\": \"STU001\", \"question_id\": \"Q1\", \"content\": \"Answer text...\"},\n  \"ideal_answer\": {\"question_id\": \"Q1\", \"content\": \"Ideal answer...\", \"subject\": \"Physics\", \"rubric\": {...}}\n}'\n```\n\n## 🏗️ System Architecture\n\nThe system implements the design principles you specified:\n\n### 1. System Design \u0026 Tool Selection\n- **Core LLM**: Supports GPT-4, Claude 3, and other powerful models\n- **Grading Rubric**: Quantifiable criteria with points and weights\n- **Prompting Framework**: Chain-of-Thought (CoT) for reasoning logic\n\n### 2. Prompt Engineering\n- **Expert Academic Examiner Role**: LLM adopts examiner persona\n- **Ideal Answer Integration**: Comprehensive reference comparison\n- **Chain-of-Thought Logic**: Step-by-step semantic analysis and scoring\n- **Structured Output**: JSON format for consistent parsing\n\n### 3. Deployment \u0026 Maintenance\n- **REST API**: Scalable FastAPI implementation\n- **Bias Monitoring**: Confidence scoring and audit trails\n- **Explainability**: Detailed justifications for all scores\n\n## 🔧 Configuration\n\n### Environment Variables\n\n| Variable | Description | Default |\n|----------|-------------|----------|\n| `OPENAI_API_KEY` | OpenAI API key | - |\n| `ANTHROPIC_API_KEY` | Anthropic API key | - |\n| `LLM_PROVIDER` | Provider (openai/anthropic) | openai |\n| `LLM_MODEL` | Model to use | gpt-4 |\n| `GRADING_TEMPERATURE` | Temperature (0.0-1.0) | 0.2 |\n| `API_PORT` | API server port | 8000 |\n\n## 📊 Grading Process\n\nThe system uses Chain-of-Thought reasoning with these steps:\n\n1. **Semantic Understanding**: Extract key concepts from ideal answer\n2. **Student Analysis**: Evaluate concept coverage and accuracy\n3. **Concept Comparison**: Compare each concept with evidence\n4. **Rubric Application**: Apply scoring criteria systematically\n5. **Final Evaluation**: Generate comprehensive feedback\n\n## 📋 API Endpoints\n\n- `POST /grade` - Grade a single answer\n- `POST /grade/batch` - Grade multiple answers\n- `POST /analyze/concepts` - Extract key concepts\n- `GET /health` - System health check\n- `GET /examples/rubric` - Example grading rubric\n- `GET /docs` - Interactive API documentation\n\n## 🧪 Testing\n\n```bash\n# Run tests\npytest tests/ -v\n\n# Run with coverage\npytest --cov=src tests/\n```\n\n## 📈 Features\n\n✅ **Core LLM Integration** (GPT-4, Claude)\n✅ **Chain-of-Thought Prompting**\n✅ **Semantic Analysis \u0026 Concept Extraction**\n✅ **Rubric-based Scoring**\n✅ **REST API with FastAPI**\n✅ **Comprehensive Feedback**\n✅ **Bias Monitoring \u0026 Confidence Scoring**\n✅ **Batch Processing**\n✅ **Interactive Documentation**\n✅ **Example Usage Scripts**\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests\n5. Submit a pull request\n\n---\n\n**Built for educators and students with AI-powered precision** 🎓\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faronno1920%2Fcapstoneexaminer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faronno1920%2Fcapstoneexaminer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faronno1920%2Fcapstoneexaminer/lists"}