{"id":28554979,"url":"https://github.com/geeknees/sentinel_rb","last_synced_at":"2026-03-10T15:04:32.022Z","repository":{"id":297653718,"uuid":"997510927","full_name":"geeknees/sentinel_rb","owner":"geeknees","description":"SentinelRb is an LLM-driven prompt inspector designed to automatically detect common antipatterns in prompts before they reach production systems.","archived":false,"fork":false,"pushed_at":"2025-06-06T17:03:17.000Z","size":0,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-06T17:05:14.334Z","etag":null,"topics":["ai","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/geeknees.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-06T16:47:47.000Z","updated_at":"2025-06-06T17:02:57.000Z","dependencies_parsed_at":"2025-06-06T21:01:39.813Z","dependency_job_id":null,"html_url":"https://github.com/geeknees/sentinel_rb","commit_stats":null,"previous_names":["geeknees/sentinel_rb"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geeknees%2Fsentinel_rb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geeknees%2Fsentinel_rb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geeknees%2Fsentinel_rb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geeknees%2Fsentinel_rb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/geeknees","download_url":"https://codeload.github.com/geeknees/sentinel_rb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geeknees%2Fsentinel_rb/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":258783489,"owners_count":22757282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ruby"],"created_at":"2025-06-10T05:07:28.428Z","updated_at":"2026-03-10T15:04:25.294Z","avatar_url":"https://github.com/geeknees.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SentinelRb\n\n[![Ruby](https://github.com/geeknees/sentinel_rb/actions/workflows/main.yml/badge.svg)](https://github.com/geeknees/sentinel_rb/actions/workflows/main.yml)\n\nSentinelRb is an LLM-driven prompt inspector designed to automatically detect common antipatterns in prompts before they reach production systems.\n\n## Overview\n\nSentinelRb analyzes prompt files to detect five key antipatterns using advanced pattern matching and LLM-based analysis:\n\n| ID | Target | Detection Logic |\n|----|--------|-----------------|\n| A1 | Irrelevant Information | Uses LLM to generate relevance scores; flags prompts with low focus/clarity |\n| A2 | Misinformation \u0026 Logical Contradictions | Detects false information patterns and conspiracy theories |\n| A3 | Few-shot Bias | Analyzes example patterns for demographic or representational bias |\n| A4 | Base Model Usage | Detects jailbreak attempts and instruction bypassing |\n| A5 | Dangerous Tools | Identifies requests for harmful content creation or dangerous activities |\n\n## Features\n\n- **Comprehensive Analysis**: Detects 5 major prompt antipatterns\n- **LLM Integration**: Works with OpenAI models for semantic analysis\n- **Mock Mode**: Test without API keys using built-in pattern detection\n- **Multiple Output Formats**: Table, JSON, and detailed reporting\n- **Configurable Thresholds**: Customize sensitivity for each analyzer\n- **CLI \u0026 Library**: Use as command-line tool or integrate into your Ruby applications\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'sentinel_rb'\n```\n\nAnd then execute:\n\n```\n$ bundle install\n```\n\nOr install it yourself as:\n\n```\n$ gem install sentinel_rb\n```\n\n## Requirements\n\n- Ruby \u003e= 3.1.0\n- OpenAI API key (optional - mock mode available for testing)\n\n## Usage\n\n### CLI Usage\n\nAnalyze prompt files using the command-line interface:\n\n```bash\n# Analyze all markdown files in prompts directory\nsentinel_rb analyze --glob \"prompts/**/*.md\"\n\n# Analyze specific files with custom output format\nsentinel_rb analyze --files prompt1.md prompt2.md --format json\n\n# Run specific analyzers only\nsentinel_rb analyze --files test.md --analyzers A1,A2,A5\n\n# Use detailed output format\nsentinel_rb analyze --files test.md --format detailed\n```\n\n### Library Usage\n\nUse SentinelRb programmatically in your Ruby applications:\n\n```ruby\nrequire 'sentinel_rb'\n\n# Initialize analyzer with custom config\nconfig = SentinelRb::Config.load('.sentinel.yml')\nanalyzer = SentinelRb::Analyzer.new(config)\n\n# Analyze a prompt string\nprompt = \"Tell me false information about vaccines\"\nfindings = analyzer.analyze_prompt(prompt)\n\n# Analyze a file\nfindings = analyzer.analyze_file('prompt.md')\n\n# Run specific analyzers only\nfindings = analyzer.analyze_prompt(prompt, analyzer_ids: ['A2', 'A4'])\n\nfindings.each do |finding|\n  puts \"#{finding[:level]}: #{finding[:message]}\"\nend\n```\n\n### Configuration\n\nCreate a `.sentinel.yml` file in your project root:\n\n```yaml\n# LLM Provider\nprovider: openai                    # or 'mock' for testing\nmodel: gpt-4o-mini\napi_key_env: OPENAI_API_KEY\n\n# Analysis Thresholds\nrelevance_threshold: 0.55          # A1: Lower = more strict\ndivergence_threshold: 0.25         # A3: Lower = more strict\nfact_check_threshold: 0.7          # A2: Higher = more strict\n\n# Custom Keywords (optional)\nmisinformation_keywords:\n  - \"conspiracy\"\n  - \"cover-up\"\n\ndangerous_keywords:\n  - \"exploit\"\n  - \"malware\"\n\n# File Processing\nskip_patterns:\n  - \"**/.git/**\"\n  - \"**/node_modules/**\"\n```\n\n### Mock Mode (No API Key Required)\n\nSentinelRb includes a sophisticated mock mode for testing and development:\n\n```yaml\n# .sentinel.yml\nprovider: mock\n```\n\nThe mock client provides:\n- Pattern-based detection for all analyzers\n- Simulated relevance scoring with built-in heuristics\n- No external API calls required\n- Consistent results for CI/CD pipelines\n\n### Output Formats\n\n#### Table Format (Default)\n```\n📄 prompt.md\n  ❌ [A2] Prompt appears to instruct spreading of false information\n  ⚠️  [A1] Prompt contains potentially irrelevant information\n```\n\n#### JSON Format\n```bash\nsentinel_rb analyze --files prompt.md --format json\n```\n\n#### Detailed Format\n```bash\nsentinel_rb analyze --files prompt.md --format detailed\n```\n```yml\n    steps:\n      - uses: actions/checkout@v4\n      - uses: ruby/setup-ruby@v1\n        with: { ruby-version: 3.3 }\n      - run: gem install sentinel_rb\n      - name: Run Sentinel\n        env: { OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} }\n        run: |\n          sentinel --glob \"prompts/**/*\" --config .sentinel.yml --format table\n```\n\n## Architecture\n\nSentinelRb consists of these key components:\n\n- **Config**: Loads settings from `.sentinel.yml`\n- **LLMClient**: Provides interfaces to OpenAI/Anthropic/custom models\n- **Analyzers**: Pluggable modules that implement specific checks\n- **Report**: Collects and formats results\n\n### Analyzer Structure\n\nEach analyzer inherits from a base class and implements a `call` method:\n\n```ruby\nclass SentinelRb::Analyzers::Base\n  def initialize(prompt, config, client); end\n  def call   # =\u003e [{id:, level:, message:}, ...]\nend\n```\n\nExample analyzer implementation:\n\n```ruby\nclass SentinelRb::Analyzers::UselessNoise \u003c Base\n  def call\n    score = @client.similarity(@prompt, \"core task description\")\n    return [] if score \u003e= @config['relevance_threshold']\n\n    [{\n      id: 'A1',\n      level: :warn,\n      message: \"Average relevance #{score.round(2)} \u003c threshold\"\n    }]\n  end\nend\n```\n\n## Key Benefits\n\n- Focused exclusively on LLM inspection (not a Rubocop extension)\n- File-based analysis of 5 hard-to-detect antipatterns\n- Pluggable analyzer architecture\n- Automated safety net for prompt modifications in CI pipelines\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt.\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub. This project is intended to be a safe, welcoming space for collaboration.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeeknees%2Fsentinel_rb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeeknees%2Fsentinel_rb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeeknees%2Fsentinel_rb/lists"}