An open API service indexing awesome lists of open source software.

https://github.com/ramakay/claude-self-reflect

Enable claude to self reflect.
https://github.com/ramakay/claude-self-reflect

ai-memory claude claude-desktop conversation-memory mcp qdrant semantic-search vector-search

Last synced: about 7 hours ago
JSON representation

Enable claude to self reflect.

Awesome Lists containing this project

README

          

# Claude Self-Reflect


Repobeats analytics image

[![npm version](https://badge.fury.io/js/claude-self-reflect.svg)](https://www.npmjs.com/package/claude-self-reflect)
[![npm downloads](https://img.shields.io/npm/dm/claude-self-reflect.svg)](https://www.npmjs.com/package/claude-self-reflect)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![GitHub CI](https://github.com/ramakay/claude-self-reflect/actions/workflows/ci.yml/badge.svg)](https://github.com/ramakay/claude-self-reflect/actions/workflows/ci.yml)

[![Claude Code](https://img.shields.io/badge/Claude%20Code-Compatible-6B4FBB)](https://github.com/anthropics/claude-code)
[![MCP Protocol](https://img.shields.io/badge/MCP-Enabled-FF6B6B)](https://modelcontextprotocol.io/)
[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED?logo=docker&logoColor=white)](https://www.docker.com/)
[![Local First](https://img.shields.io/badge/Local%20First-Privacy-4A90E2)](https://github.com/ramakay/claude-self-reflect)

[![GitHub stars](https://img.shields.io/github/stars/ramakay/claude-self-reflect.svg?style=social)](https://github.com/ramakay/claude-self-reflect/stargazers)
[![GitHub issues](https://img.shields.io/github/issues/ramakay/claude-self-reflect.svg)](https://github.com/ramakay/claude-self-reflect/issues)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/ramakay/claude-self-reflect/pulls)

**Claude forgets everything. This fixes that.**

Give Claude perfect memory of all your conversations. Search past discussions instantly. Never lose context again.

**100% Local by Default** • **20x Faster** • **Zero Configuration** • **Production Ready**

## Why This Exists

Claude starts fresh every conversation. You've solved complex bugs, designed architectures, made critical decisions - all forgotten. Until now.

## Table of Contents

- [Quick Install](#quick-install)
- [Performance](#performance)
- [The Magic](#the-magic)
- [Before & After](#before--after)
- [Real Examples](#real-examples)
- [NEW: Real-time Indexing Status](#new-real-time-indexing-status-in-your-terminal)
- [Key Features](#key-features)
- [Code Quality Insights](#code-quality-insights)
- [Architecture](#architecture)
- [Requirements](#requirements)
- [Documentation](#documentation)
- [Keeping Up to Date](#keeping-up-to-date)
- [Troubleshooting](#troubleshooting)
- [Contributors](#contributors)

## Quick Install

```bash
# Install and run automatic setup (5 minutes, everything automatic)
npm install -g claude-self-reflect
claude-self-reflect setup

# That's it! The setup will:
# - Run everything in Docker (no Python issues!)
# - Configure everything automatically
# - Install the MCP in Claude Code
# - Start monitoring for new conversations
# - Keep all data local - no API keys needed
```

> [!TIP]
> **v4.0+ Auto-Migration**: Updates from v3.x automatically migrate during npm install - no manual steps needed!

Cloud Mode (Better Search Accuracy)

```bash
# Step 1: Get your free Voyage AI key
# Sign up at https://www.voyageai.com/ - it takes 30 seconds

# Step 2: Install with Voyage key
npm install -g claude-self-reflect
claude-self-reflect setup --voyage-key=YOUR_ACTUAL_KEY_HERE
```

> [!NOTE]
> Cloud mode provides 1024-dimensional embeddings (vs 384 local) for more accurate semantic search but sends conversation data to Voyage AI for processing.

## Performance

v4.0 Performance Improvements

| Metric | v3.x | v4.0 | Improvement |
|--------|------|------|-------------|
| **Status Check** | 119ms | 6ms | **20x faster** |
| **Storage Usage** | 100MB | 50MB | **50% reduction** |
| **Import Speed** | 10/sec | 100/sec | **10x faster** |
| **Memory Usage** | 500MB | 50MB | **90% reduction** |
| **Search Latency** | 15ms | 3ms | **5x faster** |

### How We Compare

| Feature | Claude Self-Reflect | MemGPT | LangChain Memory |
|---------|---------------------|---------|------------------|
| **Local-first** | Yes | No | Partial |
| **No API keys** | Yes | No | No |
| **Real-time indexing** | Yes 2-sec | Manual | No |
| **Search speed** | <3ms | ~50ms | ~100ms |
| **Setup time** | 5 min | 30+ min | 20+ min |
| **Docker required** | Yes | Python | Python |

## The Magic

![Self Reflection vs The Grind](docs/images/red-reflection.webp)

## Before & After

![Before and After Claude Self-Reflect](docs/diagrams/before-after-combined.webp)

## Real Examples

```
You: "How did we fix that 100% CPU usage bug?"
Claude: "Found it - we fixed the circular reference causing 100% CPU usage
in the server modularization. Also fixed store_reflection dimension
mismatch by creating separate reflections_local and reflections_voyage."

You: "What about that Docker memory issue?"
Claude: "The container was limited to 2GB but only using 266MB. We found
the issue only happened with MAX_QUEUE_SIZE=1000 outside Docker.
With proper Docker limits, memory stays stable at 341MB."

You: "Have we worked with JWT authentication?"
Claude: "Found conversations about JWT patterns including User.authenticate
methods, TokenHandler classes, and concepts like token rotation,
PKCE, and social login integration."
```

## NEW: Real-time Indexing Status in Your Terminal

See your conversation indexing progress directly in your statusline:

### Fully Indexed (100%)
![Statusline showing 100% indexed](docs/images/statusbar-1.png)

### Active Indexing (50% with backlog)
![Statusline showing 50% indexed with 7h backlog](docs/images/statusbar-2.png)

Works with [Claude Code Statusline](https://github.com/sirmalloc/ccstatusline) - shows progress bars, percentages, and indexing lag in real-time! The statusline also displays MCP connection status (✓ Connected) and collection counts (28/29 indexed).

## Code Quality Insights

AST-GREP Pattern Analysis (100+ Patterns)

### Real-time Quality Scoring in Statusline
Your code quality displayed live as you work:
- 🟢 **A+** (95-100): Exceptional code quality
- 🟢 **A** (90-95): Excellent, production-ready
- 🟢 **B** (80-90): Good, minor improvements possible
- 🟡 **C** (60-80): Fair, needs refactoring
- 🔴 **D** (40-60): Poor, significant issues
- 🔴 **F** (0-40): Critical problems detected

### Pattern Categories Analyzed
- **Security Patterns**: SQL injection, XSS vulnerabilities, hardcoded secrets
- **Performance Patterns**: N+1 queries, inefficient loops, memory leaks
- **Error Handling**: Bare exceptions, missing error boundaries
- **Type Safety**: Missing type hints, unsafe casts
- **Async Patterns**: Missing await, promise handling
- **Testing Patterns**: Test coverage, assertion quality

### How It Works
1. **During Import**: AST elements extracted from all code blocks
2. **Pattern Matching**: 100+ patterns from unified registry
3. **Quality Scoring**: Weighted scoring normalized by lines of code
4. **Statusline Display**: Real-time feedback as you code

> [!TIP]
> Run `python scripts/session_quality_tracker.py` to analyze your current session quality!

## Key Features

MCP Tools Available to Claude

**Search & Memory Tools:**
- `reflect_on_past` - Search past conversations using semantic similarity with time decay (supports quick/summary modes)
- `store_reflection` - Store important insights or learnings for future reference
- `get_next_results` - Paginate through additional search results
- `search_by_file` - Find conversations that analyzed specific files
- `search_by_concept` - Search for conversations about development concepts
- `get_full_conversation` - Retrieve complete JSONL conversation files (v2.8.8)

**NEW: Temporal Query Tools (v3.3.0):**
- `get_recent_work` - Answer "What did we work on last?" with session grouping
- `search_by_recency` - Time-constrained search like "docker issues last week"
- `get_timeline` - Activity timeline with statistics and patterns

**Runtime Configuration Tools (v4.0):**
- `switch_embedding_mode` - Switch between local/cloud modes without restart
- `get_embedding_mode` - Check current embedding configuration
- `reload_code` - Hot reload Python code changes
- `reload_status` - Check reload state
- `clear_module_cache` - Clear Python cache

**Status & Monitoring Tools:**
- `get_status` - Real-time import progress and system status
- `get_health` - Comprehensive system health check
- `collection_status` - Check Qdrant collection health and stats

> [!TIP]
> Use `reflect_on_past --mode quick` for instant existence checks - returns count + top match only!

All tools are automatically available when the MCP server is connected to Claude Code.

Statusline Integration

See your indexing progress right in your terminal! Works with [Claude Code Statusline](https://github.com/sirmalloc/ccstatusline):
- **Progress Bar** - Visual indicator `[████████ ] 91%`
- **Indexing Lag** - Shows backlog `• 7h behind`
- **Auto-updates** every 60 seconds
- **Zero overhead** with intelligent caching

[Learn more about statusline integration →](docs/statusline-integration.md)

Project-Scoped Search

Searches are **project-aware by default**. Claude automatically searches within your current project:

```
# In ~/projects/MyApp
You: "What authentication method did we use?"
Claude: [Searches ONLY MyApp conversations]

# To search everywhere
You: "Search all projects for WebSocket implementations"
Claude: [Searches across ALL your projects]
```

Memory Decay

Recent conversations matter more. Old ones fade. Like your brain, but reliable.
- **90-day half-life**: Recent memories stay strong
- **Graceful aging**: Old information fades naturally
- **Configurable**: Adjust decay rate to your needs

> [!NOTE]
> Memory decay ensures recent solutions are prioritized while still maintaining historical context.

Performance at Scale

- **Search**: <3ms average response time
- **Scale**: 600+ conversations across 24 projects
- **Reliability**: 100% indexing success rate
- **Memory**: 96% reduction from v2.5.15
- **Real-time**: HOT/WARM/COLD intelligent prioritization

> [!TIP]
> For best performance, keep Docker allocated 4GB+ RAM and use SSD storage.

## Architecture

View Architecture Diagram & Details

![Import Architecture](docs/diagrams/import-architecture.png)

### HOT/WARM/COLD Intelligent Prioritization

- **HOT** (< 5 minutes): 2-second intervals for near real-time import
- **WARM** (< 24 hours): Normal priority with starvation prevention
- **COLD** (> 24 hours): Batch processed to prevent blocking

Files are categorized by age and processed with priority queuing to ensure newest content gets imported quickly while preventing older files from being starved.

### Components
- **Vector Database**: Qdrant for semantic search
- **MCP Server**: Python-based using FastMCP
- **Embeddings**: Local (FastEmbed) or Cloud (Voyage AI)
- **Import Pipeline**: Docker-based with automatic monitoring

## Requirements

> [!WARNING]
> **Breaking Change in v4.0**: Collections now use prefixed naming (e.g., `csr_project_local_384d`). Run migration automatically via `npm update`.

System Requirements

### Minimum Requirements
- **Docker Desktop** (macOS/Windows) or **Docker Engine** (Linux)
- **Node.js** 16+ (for the setup wizard)
- **Claude Code** CLI
- **4GB RAM** available for Docker
- **2GB disk space** for vector database

### Recommended
- **8GB RAM** for optimal performance
- **SSD storage** for faster indexing
- **Docker Desktop 4.0+** for best compatibility

### Operating Systems
- macOS 11+ (Intel & Apple Silicon)
- Windows 10/11 with WSL2
- Linux (Ubuntu 20.04+, Debian 11+)

## Documentation

Technical Stack

- **Vector DB**: Qdrant (local, your data stays yours)
- **Embeddings**:
- Local (Default): FastEmbed with all-MiniLM-L6-v2
- Cloud (Optional): Voyage AI
- **MCP Server**: Python + FastMCP
- **Search**: Semantic similarity with time decay

Advanced Topics

- [Performance tuning](docs/performance-guide.md)
- [Security & privacy](docs/security.md)
- [Windows setup](docs/windows-setup.md)
- [Architecture details](docs/architecture-details.md)
- [Contributing](CONTRIBUTING.md)

Troubleshooting

- [Troubleshooting Guide](docs/troubleshooting.md)
- [GitHub Issues](https://github.com/ramakay/claude-self-reflect/issues)
- [Discussions](https://github.com/ramakay/claude-self-reflect/discussions)

Uninstall

For complete uninstall instructions, see [docs/UNINSTALL.md](docs/UNINSTALL.md).

Quick uninstall:
```bash
# Remove MCP server
claude mcp remove claude-self-reflect

# Stop Docker containers
docker-compose down

# Uninstall npm package
npm uninstall -g claude-self-reflect
```

## Keeping Up to Date

> [!TIP]
> **For Existing Users**: Simply run `npm update -g claude-self-reflect` to get the latest features and improvements. Updates are automatic and preserve your data.

Recent Improvements

- **20x faster performance** - Status checks, search, and imports
- **Runtime configuration** - Switch modes without restarting
- **Unified state management** - Single source of truth
- **AST-GREP integration** - Code quality analysis
- **Temporal search tools** - Find recent work and time-based queries
- **Auto-migration** - Updates handle breaking changes automatically

[Full changelog](docs/release-history.md)

## Troubleshooting

Common Issues and Solutions

### 1. "No collections created" after import
**Symptom**: Import runs but Qdrant shows no collections
**Cause**: Docker can't access Claude projects directory
**Solution**:
```bash
# Run diagnostics to identify the issue
claude-self-reflect doctor

# Fix: Re-run setup to set correct paths
claude-self-reflect setup

# Verify .env has full paths (no ~):
cat .env | grep CLAUDE_LOGS_PATH
# Should show: CLAUDE_LOGS_PATH=/Users/YOUR_NAME/.claude/projects
```

### 2. MCP server shows "ERROR" but it's actually INFO
**Symptom**: `[ERROR] MCP server "claude-self-reflect" Server stderr: INFO Starting MCP server`
**Cause**: Claude Code displays all stderr output as errors
**Solution**: This is not an actual error - the MCP is working correctly. The INFO message confirms successful startup.

### 3. "No JSONL files found"
**Symptom**: Setup can't find any conversation files
**Cause**: Claude Code hasn't been used yet or stores files elsewhere
**Solution**:
```bash
# Check if files exist
ls ~/.claude/projects/

# If empty, use Claude Code to create some conversations first
# The watcher will import them automatically
```

### 4. Docker volume mount issues
**Symptom**: Import fails with permission errors
**Cause**: Docker can't access home directory
**Solution**:
```bash
# Ensure Docker has file sharing permissions
# macOS: Docker Desktop → Settings → Resources → File Sharing
# Add: /Users/YOUR_USERNAME/.claude

# Restart Docker and re-run setup
docker compose down
claude-self-reflect setup
```

### 5. Qdrant not accessible
**Symptom**: Can't connect to localhost:6333
**Solution**:
```bash
# Start services
docker compose --profile mcp up -d

# Check if running
docker compose ps

# View logs for errors
docker compose logs qdrant
```

Diagnostic Tools

### Run Comprehensive Diagnostics
```bash
claude-self-reflect doctor
```

This checks:
- Docker installation and configuration
- Environment variables and paths
- Claude projects and JSONL files
- Import status and collections
- Service health

### Check Logs
```bash
# View all service logs
docker compose logs -f

# View specific service
docker compose logs qdrant
docker compose logs watcher
```

### Generate Diagnostic Report
```bash
# Create diagnostic file for issue reporting
claude-self-reflect doctor > diagnostic.txt
```

Getting Help

1. **Check Documentation**
- [Troubleshooting Guide](docs/troubleshooting.md)
- [FAQ](docs/faq.md)
- [Windows Setup](docs/windows-setup.md)

2. **Community Support**
- [GitHub Discussions](https://github.com/ramakay/claude-self-reflect/discussions)
- [Discord Community](https://discord.gg/claude-self-reflect)

3. **Report Issues**
- [GitHub Issues](https://github.com/ramakay/claude-self-reflect/issues)
- Include diagnostic output when reporting

## Contributors

Special thanks to our contributors:
- **[@TheGordon](https://github.com/TheGordon)** - Fixed timestamp parsing (#10)
- **[@akamalov](https://github.com/akamalov)** - Ubuntu WSL insights
- **[@kylesnowschwartz](https://github.com/kylesnowschwartz)** - Security review (#6)

---

Built with care by [ramakay](https://github.com/ramakay) for the Claude community.