https://github.com/bewinxed/llm-whip
File monitoring tool that detects lazy patterns and anti-cheat behaviors in code and LLM conversations
https://github.com/bewinxed/llm-whip
Last synced: about 1 month ago
JSON representation
File monitoring tool that detects lazy patterns and anti-cheat behaviors in code and LLM conversations
- Host: GitHub
- URL: https://github.com/bewinxed/llm-whip
- Owner: Bewinxed
- Created: 2025-06-12T14:20:54.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-13T17:15:24.000Z (about 1 year ago)
- Last Synced: 2025-06-13T17:36:18.972Z (about 1 year ago)
- Language: TypeScript
- Size: 4.67 MB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM Whip
[](https://www.npmjs.com/package/llm-whip)
[](https://www.npmjs.com/package/llm-whip)
[](https://github.com/bewinxed/llm-whip/actions)
[](https://www.typescriptlang.org/)
[](https://opensource.org/licenses/MIT)
A TypeScript CLI tool that monitors code for lazy patterns and anti-cheat detection when working with LLMs.
## Overview
LLM Whip detects common shortcuts and anti-patterns in code:
- TODO comments and placeholders
- Stub implementations
- "The important thing is..." statements
- Not implemented errors
- Other lazy coding patterns
## Features
- **Directory Auditing**: Scan codebases for existing patterns
- **Real-time Monitoring**: Watch files as they change
- **TypeScript Configuration**: Type-safe configuration files
- **Multiple Output Formats**: Table, JSON, CSV export
- **Baseline Tracking**: Alert only on new patterns
- **Configurable Patterns**: Define custom detection rules
## Quick Start
```bash
# Install
npm install -g llm-whip
# Monitor current directory with alerts
llm-whip
# Monitor with keyboard interrupts + sound
llm-whip --interrupt --sound
# Audit existing code
llm-whip audit ./src
```
## Installation
```bash
bun add -g llm-whip
# or
npm install -g llm-whip
```
## Usage
```bash
# Show help
llm-whip --help
# Monitor current directory
llm-whip
# Monitor specific directories
llm-whip ./src ./lib
# Monitor with keyboard interrupts (sends ESC + text + Enter to active window)
llm-whip ./src --interrupt
# Monitor with sound alerts
llm-whip ./src --sound
# Create configuration file
llm-whip init
# Audit current directory
llm-whip audit
```
## Commands
| Command | Description |
| -------------------------- | ---------------------------------------- |
| `llm-whip` | Monitor current directory |
| `llm-whip init [dir]` | Create configuration file |
| `llm-whip audit [dirs...]` | Scan directories for patterns |
| `llm-whip watch ` | Monitor directories in real-time |
## Options
| Option | Description |
| ------------------- | -------------------------------------------------------- |
| `--config=` | Custom configuration file |
| `--format=` | Audit output format (table/json/csv) |
| `--grep=` | Filter files by content patterns |
| `--interrupt` | Enable keyboard interrupts (sends ESC + text + Enter to active window) |
| `--sound` | Enable sound alerts |
## Configuration
LLM Whip supports both TypeScript (`.ts`) and JSON (`.json`) configuration files.
### TypeScript Configuration
Create `llm-whip.config.ts`:
```typescript
import type { Config } from 'llm-whip/types';
export const config: Config = {
patterns: [
{
name: 'todo',
pattern: 'TODO',
severity: 'high',
reactions: ['sound', 'alert', 'interrupt'],
message: 'TODO comment detected',
interruptMessage:
'TODO comments should be completed before submitting code. Please implement the actual functionality instead of leaving placeholder comments.',
},
{
name: 'important-thing',
pattern: 'The important thing is',
severity: 'medium',
reactions: ['alert'],
interruptMessage:
"Detected 'The important thing is...' - this often indicates avoiding detailed implementation. Please provide specific, actionable details.",
},
],
reactions: {
sound: { command: 'afplay /System/Library/Sounds/Glass.aiff' },
interrupt: {
delay: 500,
sequence: ['\\u001b', '{message}', '\\n'], // ESC + message + Enter
},
alert: { format: 'color' },
},
debounce: 2000,
fileTracking: true,
};
```
### JSON Configuration
Create `llm-whip.config.json`:
```json
{
"$schema": "https://raw.githubusercontent.com/bewinxed/llm-whip/main/schema.json",
"patterns": [
{
"name": "todo",
"pattern": "TODO",
"severity": "high",
"reactions": ["sound", "alert", "interrupt"],
"message": "TODO comment detected",
"interruptMessage": "TODO comments should be completed before submitting code."
}
],
"reactions": {
"sound": true,
"alert": { "format": "color" },
"interrupt": {
"delay": 500,
"sequence": ["\\u001b", "{message}", "\\n"]
}
}
}
```
## Default Patterns
Patterns use **JavaScript regex syntax** and are case-insensitive by default:
| Pattern | Regex | Example Match |
| ----------------- | -------------------------------------- | ------------------------------------ |
| `todo` | `TODO` | `// TODO: implement this` |
| `placeholder` | `placeholder\|stub` | `// placeholder implementation` |
| `not-implemented` | `not implemented\|NotImplementedError` | `throw new Error("not implemented")` |
| `important-thing` | `The important thing is` | `The important thing is to...` |
**Custom Pattern Examples:**
```typescript
{
name: "fixme",
pattern: "FIXME\|BUG\|HACK", // Matches FIXME, BUG, or HACK
severity: "high"
},
{
name: "console-log",
pattern: "console\\.(log\|debug)", // Matches console.log or console.debug
severity: "low"
}
```
## Keyboard Interrupts
The `--interrupt` flag enables keyboard interrupts that send detailed warnings to the active window when patterns are detected:
```bash
# Enable keyboard interrupts
llm-whip ./src --interrupt
```
When a pattern is detected, LLM Whip will:
1. Press ESC to clear any current input
2. Type a warning message to the active window
3. Press Enter to send the message
4. The message includes:
- Pattern type and custom message
- File path and line number
- Timestamp
**Note:** The `--interrupt` flag will automatically add keyboard interrupts to all patterns, even if they don't have "interrupt" in their reactions array in the config file.
Each pattern can have a custom `interruptMessage` that gets sent:
```typescript
{
name: "todo",
pattern: "TODO",
reactions: ["interrupt"],
interruptMessage: "TODO comments should be completed before submitting code. Please implement the actual functionality instead of leaving placeholder comments."
}
```
## Sound Alerts
The `--sound` flag enables cross-platform sound alerts:
```bash
# Enable sound alerts
llm-whip ./src --sound
```
Default sounds by platform:
- **macOS**: Glass.aiff (fallback: Ping.aiff)
- **Windows**: Windows Critical Stop.wav (fallback: console beep)
- **Linux**: alarm-clock-elapsed.oga (fallback: bell.oga)
## Monitoring LLM Conversations
LLM Whip can monitor LLM conversation outputs by watching log files. This helps detect anti-cheat patterns in both your code and the LLM's responses.
### Method 1: Using `tee` to clone output
When using Claude Code CLI or other LLM tools, pipe the output to a file that LLM Whip monitors:
```bash
# Terminal 1: Start LLM Whip monitoring
llm-whip ./project ./logs
# Terminal 2: Run Claude Code with output logging
claude-code ./project 2>&1 | tee logs/claude-session.log
```
### Method 2: Direct file monitoring
Create a log file and have your LLM tool write to it:
```bash
# Start monitoring the logs directory
llm-whip ./src ./logs
# Your LLM tool outputs to logs/conversation.txt
# LLM Whip will detect patterns in real-time
```
### Method 3: Script wrapper
Create a wrapper script that automatically logs and monitors:
```bash
#!/bin/bash
# llm-monitor.sh
# Create logs directory
mkdir -p ./logs
# Start LLM Whip in background
llm-whip ./src ./logs &
WHIP_PID=$!
# Run your LLM tool with logging
claude-code "$@" 2>&1 | tee logs/session-$(date +%Y%m%d-%H%M%S).log
# Clean up
kill $WHIP_PID 2>/dev/null
```
Then use it like: `./llm-monitor.sh ./my-project`
## Advanced Usage
### Baseline Tracking
Only alert on new patterns:
```bash
llm-whip audit ./src > /dev/null
llm-whip watch ./src
```
### Export Formats
```bash
# JSON export
llm-whip audit ./src --format=json > issues.json
# CSV export
llm-whip audit ./src --format=csv > issues.csv
```
### Configuration Priority
1. Local `llm-whip.config.ts` (current directory)
2. Custom path via `--config=path`
3. Built-in defaults
## Development
```bash
bun install
bun test
bun run build
```
## License
MIT