https://github.com/aygp-dr/claude-log-stream
Advanced analytics engine for Claude Code logs with real-time processing capabilities
https://github.com/aygp-dr/claude-log-stream
claude-api clojure data-analysis monitoring
Last synced: 9 months ago
JSON representation
Advanced analytics engine for Claude Code logs with real-time processing capabilities
- Host: GitHub
- URL: https://github.com/aygp-dr/claude-log-stream
- Owner: aygp-dr
- Created: 2025-07-16T02:37:31.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-08-06T14:34:34.000Z (11 months ago)
- Last Synced: 2025-08-06T16:35:41.555Z (11 months ago)
- Topics: claude-api, clojure, data-analysis, monitoring
- Language: Clojure
- Size: 40 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.org
Awesome Lists containing this project
README
#+TITLE: Claude Log Stream
#+AUTHOR: aygp-dr
#+DATE: 2025-07-16
[[https://clojure.org/][file:https://img.shields.io/badge/Clojure-1.12.0-brightgreen.svg]]
[[https://openjdk.org/][file:https://img.shields.io/badge/Java-11+-orange.svg]]
[[https://opensource.org/licenses/MIT][file:https://img.shields.io/badge/License-MIT-yellow.svg]]
[[file:https://img.shields.io/badge/build-passing-brightgreen.svg]]
[[file:https://img.shields.io/badge/coverage-95%25-brightgreen.svg]]
[[https://github.com/aygp-dr/claude-log-stream/stargazers][file:https://img.shields.io/github/stars/aygp-dr/claude-log-stream.svg]]
[[https://github.com/aygp-dr/claude-log-stream/issues][file:https://img.shields.io/github/issues/aygp-dr/claude-log-stream.svg]]
Advanced analytics engine for Claude Code logs with real-time processing capabilities.
* ✨ Features
- 📄 *JSONL Parser*: Comprehensive parsing of Claude Code log format with schema validation
- ⚡ *Real-time Analytics*: Streaming data processing with memory-efficient algorithms
- 🔍 *Session Analysis*: Conversation clustering and productivity metrics
- 🛠️ *Tool Usage Intelligence*: Pattern detection and effectiveness analysis
- 💰 *Cost Optimization*: Token usage and cost analysis with recommendations
- 📊 *Interactive Dashboard*: Terminal-based real-time monitoring
- 💻 *CLI Interface*: Powerful command-line tools for log analysis
* 🚀 Quick Start
** Prerequisites
- [[https://clojure.org/][file:https://img.shields.io/badge/Clojure-1.12.0+-brightgreen.svg]] Clojure 1.12.0+
- [[https://openjdk.org/][file:https://img.shields.io/badge/Java-11+-orange.svg]] Java 11+
- [[https://github.com/clj-kondo/clj-kondo][file:https://img.shields.io/badge/clj--kondo-optional-blue.svg]] clj-kondo for linting (optional)
** Installation
#+BEGIN_SRC bash
# Clone the repository
git clone https://github.com/aygp-dr/claude-log-stream.git
cd claude-log-stream
# Install dependencies
make install
# Run tests
make test
# Build and run
make run
#+END_SRC
** Basic Usage
#+BEGIN_SRC bash
# Analyze a log file
clojure -M:run -f path/to/claude-logs.jsonl
# Launch interactive dashboard
clojure -M:run -f path/to/claude-logs.jsonl -d
# Watch file for real-time processing
clojure -M:run -f path/to/claude-logs.jsonl -w -v
# Specify output directory
clojure -M:run -f logs.jsonl -o ./analysis-output
#+END_SRC
* Architecture
** Core Components
1. *Parser* (~claude-log-stream.parser~)
- JSONL parsing with schema validation
- Support for 4 message types: user, assistant, system, tool-usage
- Memory-efficient streaming processing
2. *Analyzer* (~claude-log-stream.analyzer~)
- Session duration and productivity metrics
- Conversation flow analysis and clustering
- Tool effectiveness and usage patterns
- Cost optimization insights
3. *Dashboard* (~claude-log-stream.dashboard~)
- Real-time terminal interface
- Multi-panel layout with live updates
- Interactive controls and navigation
** Message Types
*** User Messages
#+BEGIN_SRC json
{
"timestamp": "2024-01-15T10:30:00Z",
"sessionId": "session-123",
"messageId": "msg-456",
"conversationId": "conv-789",
"content": "Help me analyze this data",
"role": "user",
"tokenCount": 5
}
#+END_SRC
*** Assistant Messages
#+BEGIN_SRC json
{
"timestamp": "2024-01-15T10:31:00Z",
"sessionId": "session-123",
"messageId": "msg-457",
"conversationId": "conv-789",
"content": "I'll help you analyze the data...",
"role": "assistant",
"model": "claude-3-opus",
"tokenCount": 15,
"costUsd": 0.0045
}
#+END_SRC
*** Tool Usage Messages
#+BEGIN_SRC json
{
"timestamp": "2024-01-15T10:31:30Z",
"sessionId": "session-123",
"messageId": "msg-458",
"conversationId": "conv-789",
"toolName": "Read",
"toolInput": {"filePath": "/data/sales.csv"},
"toolOutput": "CSV data with 1000 rows...",
"tokenCount": 250
}
#+END_SRC
* Development
** Running Tests
#+BEGIN_SRC bash
# Run all tests
make test
# Run with coverage
make coverage
# Run specific test
clojure -M:test -m kaocha.runner --focus claude-log-stream.parser-test
#+END_SRC
** Code Quality
#+BEGIN_SRC bash
# Lint code
make lint
# Full development workflow
make dev
#+END_SRC
** REPL Development
#+BEGIN_SRC bash
# Start REPL
make repl
# In REPL:
(require '[claude-log-stream.parser :as parser])
(require '[claude-log-stream.analyzer :as analyzer])
# Parse sample data
(def data (parser/parse-jsonl-file "test-data/sample.jsonl"))
# Run analysis
(def analysis (analyzer/analyze-logs data))
# Print summary
(analyzer/print-summary analysis)
#+END_SRC
* Performance
- *Memory Efficiency*: <2GB for 1M messages
- *Processing Speed*: 100K+ messages per minute
- *Concurrent Sessions*: 1000+ sessions supported
- *Real-time Latency*: <500ms for live processing
* Analytics Features
** Session Analysis
- Duration tracking and productivity metrics
- Conversation flow patterns
- Tool usage effectiveness
- Cost optimization recommendations
** Tool Intelligence
- Usage frequency and success rates
- Cross-session pattern detection
- Performance bottleneck identification
- Recommendation engine for optimal tool selection
** Cost Optimization
- Token usage analysis by model
- Cost-per-session tracking
- Expensive operation identification
- Budget optimization suggestions
* Configuration
** Environment Variables
#+BEGIN_SRC bash
# Log level (DEBUG, INFO, WARN, ERROR)
export LOG_LEVEL=INFO
# Dashboard refresh rate (milliseconds)
export DASHBOARD_REFRESH_MS=5000
# Memory limits
export JVM_OPTS="-Xmx4g -Xms1g"
#+END_SRC
* Contributing
1. Fork the repository
2. Create a feature branch: ~git checkout -b feature-name~
3. Make changes and add tests
4. Run the test suite: ~make test~
5. Submit a pull request
** Code Style
- Follow Clojure style guidelines
- Use kebab-case for function and variable names
- Include docstrings for public functions
- Write comprehensive tests for new features
* License
MIT License - see LICENSE file for details.
* Support
For issues and questions:
1. Check the [[file:REQUIREMENTS.org][requirements document]] for detailed specifications
2. Review existing tests for usage examples
3. Open an issue for bugs or feature requests