Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/codenamev/chaos_to_the_rescue

Safe-by-default LLM-powered method generation and Rails error rescue suggestions
https://github.com/codenamev/chaos_to_the_rescue

Last synced: 4 months ago
JSON representation

Safe-by-default LLM-powered method generation and Rails error rescue suggestions

Awesome Lists containing this project

README 

          
# ChaosToTheRescue



**Safe-by-default LLM-powered method generation and Rails error rescue suggestions**



ChaosToTheRescue is a Ruby gem that uses Large Language Models (LLMs) to:

1. Generate missing methods on-the-fly via `method_missing`

2. Suggest fixes for Rails exceptions in development



## ⚠️ Safety First



This gem is **DISABLED BY DEFAULT** and includes multiple safety features:



- ✅ **Disabled by default** - Must be explicitly enabled

- ✅ **No auto-execution** - Generated code returned as strings only (unless explicitly enabled)

- ✅ **Allowlist-only** - Methods must match patterns or be explicitly allowed

- ✅ **Secret redaction** - ENV vars, API keys, tokens filtered before LLM prompts

- ✅ **No file writes** - Suggestions only logged, no automatic code edits

- ✅ **Development-only** - Intended for development/test environments only



**DO NOT USE IN PRODUCTION** - This gem is designed for development and debugging workflows only.



## Installation



Add to your Gemfile:



```ruby

gem "chaos_to_the_rescue"

```



For LLM features, also add:



```ruby

gem "ruby_llm", "~> 0.1"  # Required for LLM integration

```



Then run:



```bash

bundle install

```



For Rails applications, generate the initializer:



```bash

rails generate chaos_to_the_rescue:install

```



### Configure API Keys



ChaosToTheRescue uses RubyLLM under the hood, which supports multiple LLM providers. You need to configure at least one API key:



**Option 1: Environment Variables (Recommended)**

```bash

export OPENAI_API_KEY="sk-..."

export ANTHROPIC_API_KEY="sk-ant-..."

```



**Option 2: Configuration Block**

```ruby

ChaosToTheRescue.configure do |config|

  config.openai_api_key = "sk-..."

  # or

  config.anthropic_api_key = "sk-ant-..."

end

```



**Supported Models:**

- OpenAI: `gpt-5.2`, `gpt-5.2-pro`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`

- Anthropic: `claude-sonnet-4-5`, `claude-opus-4-5`, `claude-sonnet-4`, `claude-opus-4`

- Google: `gemini-2.0-flash-exp`, `gemini-1.5-pro`

- And many more via RubyLLM (xAI, DeepSeek, local models, etc.)



## Features



### 1. Method Generation with `ChaosRescue`



Generate missing methods on-the-fly using LLMs. When a method is called that doesn't exist, ChaosToTheRescue can generate and define it automatically.



**Supports both instance methods and class methods:**



```ruby

require "chaos_to_the_rescue"



ChaosToTheRescue.configure do |config|

  config.enabled = true

  config.auto_define_methods = true

  config.allowed_method_name_patterns = [/^calc_/]

end



class Calculator

  include ChaosToTheRescue::ChaosRescue

  chaos_rescue_enabled!

end



# Instance method generation

calc = Calculator.new

result = calc.calc_fibonacci(10)  # Method generated via LLM



# Class method generation

result = Calculator.calc_sum(1, 2, 3)  # Class method generated via LLM

```



### 2. Method Guidance and Verification



Provide guidance to the LLM for better method generation and verify method outputs:



#### Adding Guidance for Method Generation



```ruby

class Calculator

  include ChaosToTheRescue::ChaosRescue

  chaos_rescue_enabled!



  # Provide guidance BEFORE the method is generated

  chaos_guidance :pow, <<~GUIDANCE

    This method should correctly handle negative bases:

    - Negative base with even exponent returns positive: (-5)^2 = 25

    - Negative base with odd exponent returns negative: (-5)^3 = -125

    Common mistake: Using base.abs which always returns positive.

  GUIDANCE

end



# Global guidance (applies to all classes)

ChaosToTheRescue.configure do |config|

  config.method_guidance[:divide] = "Handle division by zero gracefully"

end

```



#### Adding Guidance to Existing Methods



```ruby

class Calculator

  def pow(base, exponent)

    base ** exponent  # Method already defined

  end



  # Add guidance AFTER definition for verification purposes

  additional_chaos_guidance :pow, "Should handle negative bases correctly"

end

```



#### Verifying Method Outputs



**Verify with class method:**

```ruby

result = Calculator.verify_chaos(:pow, -5, 2)



if result.failed?

  puts "Issues found:"

  puts result.diagnosis

  puts result.suggestion

end

```



**Auto-fix incorrect implementations:**

```ruby

# Automatically applies fix if verification fails

Calculator.verify_chaos(:pow, -5, 2, auto_apply: true)

```



**Verify using block syntax:**

```ruby

# Simple verification

result = ChaosToTheRescue.verify do

  Calculator.new.pow(-5, 2)

end



# With auto-fix

result = ChaosToTheRescue.verify(auto_apply: true) do

  Calculator.new.pow(-5, 2)

end



if result.failed?

  puts "Method was incorrect:"

  puts result.diagnosis

  puts "Actual result: #{result.actual_result}"

end

```



See `examples/guidance_and_verification.rb` for comprehensive examples.



### 3. Rails Exception Rescue Suggestions



Get LLM-powered suggestions for fixing Rails exceptions:



```ruby

class ApplicationController < ActionController::Base

  include ChaosToTheRescue::RescueFrom



  rescue_from StandardError do |exception|

    suggestion = chaos_suggest_fix(

      exception,

      guidance: "This is a payment processing error"

    )



    Rails.logger.info("Fix suggestion: #{suggestion[:title]}")

    Rails.logger.info("Diagnosis: #{suggestion[:diagnosis]}")

    Rails.logger.info("Proposed changes: #{suggestion[:proposed_changes]}")

    Rails.logger.info("Files to edit: #{suggestion[:files_to_edit].join(', ')}")



    render "errors/500", status: :internal_server_error

  end

end

```



## Configuration



All configuration is done via the `ChaosToTheRescue.configure` block:



```ruby

ChaosToTheRescue.configure do |config|

  # === CORE SETTINGS ===



  # Enable/disable globally (default: false)

  config.enabled = Rails.env.development? || Rails.env.test?



  # Auto-define generated methods (default: false)

  config.auto_define_methods = false



  # === METHOD GENERATION ALLOWLIST ===



  # Regex patterns for allowed method names

  config.allowed_method_name_patterns = [/^calc_/, /^compute_/]



  # Explicit allowlist of method names

  config.allowlist = [:my_dynamic_method, "another_method"]



  # Or allow all method names (use with caution!)

  config.allow_everything!



  # === LLM SETTINGS ===



  # LLM model to use (default: "gpt-5-mini")

  config.model = "gpt-5-mini"



  # Maximum characters in prompts (default: 8000)

  config.max_prompt_chars = 8000



  # === LOGGING SETTINGS ===



  # Log level (default: :info)

  config.log_level = :info



  # Log suggestions to file (default: true)

  config.log_suggestions = true



  # Suggestion log path (default: "tmp/chaos_to_the_rescue_suggestions.log")

  config.suggestions_log_path = "tmp/chaos_to_the_rescue_suggestions.log"



  # === SECURITY SETTINGS ===



  # Add custom redaction patterns (already includes common secrets)

  config.redact_patterns << /CUSTOM_SECRET[=:]\s*['"]?([^'"\s]+)['"]?/i

end

```



### Default Redaction Patterns



The gem automatically redacts:

- Environment variable references (`ENV["KEY"]`, `ENV.fetch`)

- AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AKIA...`)

- OpenAI tokens (`sk-...`, `OPENAI_API_KEY`)

- GitHub tokens (`ghp_...`, `gho_...`, `ghu_...`, `ghs_...`, `ghr_...`, `GITHUB_TOKEN`)

- Rails credentials (`Rails.application.credentials.*`)

- Generic patterns (`API_KEY`, `SECRET`, `TOKEN`, `PASSWORD`)

- Private keys (`-----BEGIN PRIVATE KEY-----`)



## Usage Examples



### Example 1: Calculator with Dynamic Methods



```ruby

ChaosToTheRescue.configure do |config|

  config.enabled = true

  config.auto_define_methods = true

  config.allowed_method_name_patterns = [/^calc_/]

end



class Calculator

  include ChaosToTheRescue::ChaosRescue

  chaos_rescue_enabled!

end



# Instance methods

calc = Calculator.new

puts calc.calc_sum(1, 2, 3, 4, 5)        # LLM generates sum method

puts calc.calc_average(10, 20, 30)        # LLM generates average method

puts calc.calc_fibonacci(10)              # LLM generates fibonacci method



# Class methods

puts Calculator.calc_factorial(5)         # LLM generates class method

puts Calculator.calc_gcd(48, 18)          # LLM generates class method

```



### Example 2: Per-Class Enable/Disable



```ruby

class SafeClass

  include ChaosToTheRescue::ChaosRescue

  # Not enabled - will not generate methods

end



class ExperimentalClass

  include ChaosToTheRescue::ChaosRescue

  chaos_rescue_enabled!  # Explicitly enabled for this class

end

```



### Example 3: Rails Controller Error Handling



```ruby

class OrdersController < ApplicationController

  include ChaosToTheRescue::RescueFrom



  rescue_from StandardError do |exception|

    if ChaosToTheRescue.configuration.enabled

      suggestion = chaos_suggest_fix(

        exception,

        guidance: "User was trying to complete checkout"

      )



      # Log the suggestion

      Rails.logger.error("Exception: #{exception.class.name}: #{exception.message}")

      Rails.logger.info("Suggestion: #{suggestion[:title]}")

      Rails.logger.info("Diagnosis: #{suggestion[:diagnosis]}")



      # Check suggestion log file at tmp/chaos_to_the_rescue_suggestions.log

    end



    render "errors/500", status: :internal_server_error

  end

end

```



### Example 4: Allowlist Specific Methods



```ruby

ChaosToTheRescue.configure do |config|

  config.enabled = true

  config.allowlist = [:format_currency, :parse_date, :generate_slug]

end



class Helper

  include ChaosToTheRescue::ChaosRescue

  chaos_rescue_enabled!

end



helper = Helper.new

helper.format_currency(1234.56)  # Allowed - will generate

helper.dangerous_method          # Not allowed - raises NoMethodError

```



### Example 5: Allow All Methods (Quick Testing)



```ruby

# WARNING: Use with caution - allows any method to be generated

ChaosToTheRescue.configure do |config|

  config.enabled = true

  config.allow_everything!  # Convenience method to allow all method names

end



class Calculator

  include ChaosToTheRescue::ChaosRescue

  chaos_rescue_enabled!

end



calc = Calculator.new

calc.divide(10, 2)           # Will work - all methods allowed

calc.anything_you_want(1, 2) # Will work - all methods allowed

```



## How It Works



### Method Generation Flow



1. You call a missing method (instance or class method) on a class that includes `ChaosRescue`

2. ChaosToTheRescue checks if the gem is enabled (globally and per-class)

3. Checks if the method name matches allowlist or patterns

4. Builds context (class name, method name, arguments, method type)

5. Redacts any sensitive information from the context

6. Sends prompt to LLM to generate method code

7. Optionally defines the method on the object or class (if `auto_define_methods` enabled)

8. Returns the result



### Exception Suggestion Flow



1. An exception occurs in a Rails controller

2. You call `chaos_suggest_fix` in your `rescue_from` block

3. ChaosToTheRescue builds exception context (class, message, backtrace)

4. Sanitizes request data and redacts secrets

5. Sends prompt to LLM to analyze the error

6. Returns structured suggestion with diagnosis and proposed changes

7. Logs suggestion to file if enabled



## Security Considerations



### What Gets Redacted



Before sending any data to an LLM, ChaosToTheRescue redacts:

- All environment variable references

- API keys, secrets, tokens, passwords

- AWS, OpenAI, GitHub credentials

- Rails credentials

- Private keys



### What Doesn't Get Sent



- File contents (only file paths from backtraces)

- User session data (unless in exception context)

- Database credentials



### Recommendations



1. **Never enable in production** - Only use in development/test

2. **Review prompts** - Set `log_level: :debug` to see what's sent to LLM

3. **Use allowlists** - Don't allow all methods, use specific patterns

4. **Don't auto-define** - Keep `auto_define_methods: false` and review generated code

5. **Custom redaction** - Add your own patterns for domain-specific secrets



## Development



After checking out the repo:



```bash

bin/setup              # Install dependencies

bundle exec rspec      # Run tests

bin/console            # Interactive prompt

bundle exec standardrb # Lint code

```



## Testing



Run the full test suite:



```bash

bundle exec rspec

```



Run specific test files:



```bash

bundle exec rspec spec/configuration_spec.rb

bundle exec rspec spec/chaos_rescue_spec.rb

```



## Troubleshooting



### "RubyLLM is not available"



You need to add `ruby_llm` to your Gemfile:



```ruby

gem "ruby_llm", "~> 0.1"

```



Then run `bundle install`.



### Methods Not Being Generated



Check that:

1. `config.enabled = true` is set

2. Class has `chaos_rescue_enabled!` called

3. Method name matches patterns or is in allowlist

4. RubyLLM is installed and configured



### No Suggestions Logged



Check that:

1. `config.log_suggestions = true` is set

2. The `tmp/` directory exists and is writable

3. ChaosToTheRescue is enabled



## Roadmap



Potential future features:

- Multiple LLM provider support (Anthropic, Cohere, local models)

- Caching of generated methods

- Web UI for reviewing suggestions

- Type signature validation

- Integration with CI/CD for automated error analysis



## Contributing



Bug reports and pull requests are welcome on GitHub at https://github.com/codenamev/chaos_to_the_rescue.



This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/codenamev/chaos_to_the_rescue/blob/main/CODE_OF_CONDUCT.md).



## License



The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).



## Code of Conduct



Everyone interacting in the ChaosToTheRescue project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/codenamev/chaos_to_the_rescue/blob/main/CODE_OF_CONDUCT.md).



## Credits



Created by [Valentino Stoll](https://github.com/codenamev)



## Disclaimer



This gem is experimental and should only be used in development environments. The authors are not responsible for any issues arising from using generated code in production. Always review generated code before using it in any capacity.