{"id":51119009,"url":"https://github.com/sulthonzh/ai-prompt-validator","last_synced_at":"2026-06-25T00:30:36.935Z","repository":{"id":363417099,"uuid":"1260788922","full_name":"sulthonzh/ai-prompt-validator","owner":"sulthonzh","description":"AI prompt validation, optimization, and testing toolkit for developers","archived":false,"fork":false,"pushed_at":"2026-06-17T13:39:39.000Z","size":161,"stargazers_count":0,"open_issues_count":10,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-17T16:31:29.793Z","etag":null,"topics":["ai","claude","developer-tools","gemini","llm","openai","optimization","prompt","testing","validation"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sulthonzh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-05T21:54:00.000Z","updated_at":"2026-06-17T13:35:38.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sulthonzh/ai-prompt-validator","commit_stats":null,"previous_names":["sulthonzh/ai-prompt-validator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sulthonzh/ai-prompt-validator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sulthonzh%2Fai-prompt-validator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sulthonzh%2Fai-prompt-validator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sulthonzh%2Fai-prompt-validator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sulthonzh%2Fai-prompt-validator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sulthonzh","download_url":"https://codeload.github.com/sulthonzh/ai-prompt-validator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sulthonzh%2Fai-prompt-validator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34755061,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-24T02:00:07.484Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","claude","developer-tools","gemini","llm","openai","optimization","prompt","testing","validation"],"created_at":"2026-06-25T00:30:36.854Z","updated_at":"2026-06-25T00:30:36.926Z","avatar_url":"https://github.com/sulthonzh.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Prompt Validator\n\nA comprehensive CLI tool for validating, optimizing, and testing AI prompts to ensure they're effective, clear, and well-structured.\n\n## Features\n\n### 🔍 **Prompt Validation**\n- **Quality Scoring**: Get a score out of 100 based on multiple metrics\n- **Rule-Based Validation**: Built-in rules covering common prompt issues\n- **Detailed Analysis**: Identify specific problems and get suggestions\n- **Metrics Tracking**: Track clarity, specificity, structure, and creativity scores\n\n### 🎯 **Prompt Optimization**\n- **Intelligent Suggestions**: Get specific recommendations to improve your prompts\n- **Multiple Optimization Types**: Structure, clarity, specificity, format, and length optimizations\n- **Interactive Optimization**: Choose which optimizations to apply\n- **Automatic Application**: Apply all suggestions with one command\n\n### 🧪 **Testing \u0026 Benchmarking**\n- **Test Suites**: Create and run test suites with expected outputs\n- **Batch Testing**: Validate multiple prompts at once\n- **Performance Benchmarking**: Test prompt performance against AI models\n- **Regression Testing**: Ensure prompt quality doesn't degrade over time\n\n### 🛠️ **Flexible Configuration**\n- **Custom Rules**: Add your own validation rules\n- **Rule Management**: Enable/disable specific rules\n- **Multiple Output Formats**: JSON, YAML, or human-readable\n- **Config Profiles**: Different configurations for different use cases\n\n## Installation\n\n```bash\nnpm install -g ai-prompt-validator\n```\n\nOr use it without installation:\n\n```bash\nnpx ai-prompt-validator\n```\n\n## Quick Start\n\n### Basic Validation\n\n```bash\n# Validate a simple prompt\naiv validate \"Write a function to calculate fibonacci numbers\"\n\n# Get detailed analysis\naiv validate \"Write a function to calculate fibonacci numbers\" --verbose\n\n# Get JSON output\naiv validate \"Write a function to calculate fibonacci numbers\" --json\n```\n\n### Prompt Optimization\n\n```bash\n# Get optimization suggestions\naiv optimize \"Write good code\"\n\n# Apply optimizations automatically\naiv optimize \"Write good code\" --apply\n\n# Interactive optimization\naiv optimize \"Write good code\" --interactive\n```\n\n### Prompt Comparison\n\n```bash\n# Compare two prompt versions\naiv compare \"Write a function\" \"Write a TypeScript function that handles edge cases\"\n\n# Get comparison as JSON\naiv compare \"good code\" \"Write secure Node.js auth\" --json\n\n# Markdown output for PRs\naiv compare \"v1 prompt\" \"v2 prompt\" --markdown\n```\n\n## Usage Examples\n\n### Validation Examples\n\n```bash\n# Basic validation\n$ aiv validate \"Write a function to calculate fibonacci numbers\"\n\n🎯 Prompt Validation Results\n==================================================\nStatus: ✅ VALID\nScore: 85/100\n\n✅ No issues found!\n\n# Validation with issues\n$ aiv validate \"Write good code\"\n\n🎯 Prompt Validation Results\n==================================================\nStatus: ❌ INVALID\nScore: 45/100\n\n🚨 Issues Found:\n❌ ambiguous-terms: Prompt contains ambiguous terms\n     💡 Replace \"good\" with specific criteria\n⚠️ missing-format: Prompt doesn't specify output format\n     💡 Add output format preference\n```\n\n### Optimization Examples\n\n```bash\n# Get suggestions\n$ aiv optimize \"Write good code\"\n\n🎯 Optimization Suggestions:\n==================================================\n1. Replace ambiguous terms (high priority)\n   Type: clarity\n   💡 Replace \"good\" with specific criteria\n   📝 Before: Write good code\n   ✨ After: Write efficient code with proper error handling\n\n2. Specify output format (medium priority)\n   Type: format\n   💡 Add output format preference\n   📝 Before: Write good code\n   ✨ After: Write efficient code in JSON format\n```\n\n### Advanced Features\n\n```bash\n# Custom rule validation\naiv validate \"Write a function\" --rules empty-prompt,too-short\n\n# Create configuration\naiv config --init\n\n# Manage rules\naiv rules --list\naiv rules --enable specific-rule\naiv rules --disable specific-rule\n```\n\n## Configuration\n\nCreate a configuration file (`aiv-config.json`) to customize the behavior:\n\n```json\n{\n  \"rules\": [\"empty-prompt\", \"too-short\", \"missing-context\"],\n  \"excludedRules\": [\"uppercase-heavy\"],\n  \"outputFormat\": \"json\",\n  \"strictMode\": true,\n  \"customRules\": [\n    {\n      \"id\": \"custom-rule\",\n      \"name\": \"Custom Rule\",\n      \"description\": \"A custom validation rule\",\n      \"pattern\": \"test\",\n      \"severity\": \"warning\",\n      \"message\": \"Prompt must contain test\",\n      \"suggestion\": \"Add test to your prompt\"\n    }\n  ]\n}\n```\n\n## Built-in Rules\n\nThe tool comes with several built-in validation rules:\n\n### **Critical Rules**\n- `empty-prompt`: Prompt cannot be empty\n- `too-short`: Prompt should be at least 10 characters\n- `too-long`: Prompt should not exceed 2000 characters\n\n### **Quality Rules**\n- `missing-context`: Prompt should include relevant context\n- `missing-instruction`: Prompt should have clear instructions\n- `missing-format`: Prompt should specify output format\n- `ambiguous-terms`: Prompt should avoid ambiguous language\n- `leading-question`: Prompt should not be a yes/no question\n- `uppercase-heavy`: Prompt should not be all uppercase\n\n### **Custom Rules**\nYou can add your own rules using:\n```bash\n# Add custom rules from file\naiv rules --add custom-rules.json\n\n# Format for custom rules:\n[\n  {\n    \"id\": \"my-rule\",\n    \"name\": \"My Custom Rule\",\n    \"description\": \"Description of the rule\",\n    \"pattern\": \"regex-pattern\",\n    \"severity\": \"error|warning|info\",\n    \"message\": \"Error message\",\n    \"suggestion\": \"Suggestion text\"\n  }\n]\n```\n\n## Metrics Explained\n\n### **Clarity Score (0-100)**\nBased on sentence structure, word complexity, and readability\n- Higher score = Better clarity\n- Penalizes very long sentences\n- Rewards balanced sentence lengths\n\n### **Specificity Score (0-100)**\nBased on specific technical terms vs generic language\n- Higher score = More specific requirements\n- Rewards technical terms\n- Penalizes generic terms like \"good\", \"bad\", \"nice\"\n\n### **Structure Score (0-100)**\nBased on organization and flow\n- Higher score = Better structure\n- Rewards multiple sentences\n- Rewards imperative language\n- Good question/statement balance\n\n### **Creativity Score (0-100)**\nBased on vocabulary diversity\n- Higher score = More diverse vocabulary\n- Rewards unique word usage\n- Penalizes repetitive language\n\n## Test Suite Format\n\nCreate test suites for batch testing:\n\n```json\n{\n  \"id\": \"example-suite\",\n  \"name\": \"Example Test Suite\",\n  \"description\": \"Test suite for example prompts\",\n  \"prompts\": [\n    {\n      \"prompt\": \"Write a function to calculate fibonacci numbers\",\n      \"expectedAnswer\": \"function fibonacci(n) { }\",\n      \"expectedFormat\": \"code\",\n      \"constraints\": {\n        \"maxLength\": 100,\n        \"minLength\": 50,\n        \"requiredWords\": [\"function\", \"fibonacci\"],\n        \"bannedWords\": [\"console\"]\n      }\n    }\n  ],\n  \"expectedOutputs\": [\n    {\n      \"id\": \"output-1\",\n      \"promptId\": \"0\",\n      \"expected\": \"fibonacci function\",\n      \"description\": \"Should contain fibonacci function\",\n      \"tolerance\": 0.8\n    }\n  ]\n}\n```\n\n## API Integration\n\nThe tool can integrate with various AI models for benchmarking:\n\n```bash\n# Benchmark prompts with OpenAI\naiv benchmark \"Write a function\" --model gpt-3.5-turbo --api-key your-key\n\n# Benchmark multiple prompts\naiv benchmark \"Write a function\" \"Create a class\" --model gpt-4 --iterations 5\n\n# Save benchmark results\naiv benchmark \"Write a function\" --output benchmark-results.json\n```\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Add tests for new functionality\n4. Ensure all tests pass\n5. Submit a pull request\n\n## License\n\nMIT License - see LICENSE file for details\n\n## Support\n\nFor issues and questions:\n- GitHub Issues: [Create an issue](https://github.com/sulthonzh/ai-prompt-validator/issues)\n- Documentation: [Full documentation](https://github.com/sulthonzh/ai-prompt-validator/wiki)\n\n## Roadmap\n\n- [ ] Support for more AI models\n- [ ] Prompt template generation\n- [ ] Collaborative prompt sharing\n- [ ] Performance analytics dashboard\n- [ ] Integration with popular AI platforms\n- [ ] Advanced rule engine with machine learning\n- [ ] Real-time prompt optimization","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsulthonzh%2Fai-prompt-validator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsulthonzh%2Fai-prompt-validator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsulthonzh%2Fai-prompt-validator/lists"}