{"id":29616572,"url":"https://github.com/norandom/direct-transriberr","last_synced_at":"2026-04-09T12:35:43.361Z","repository":{"id":305292489,"uuid":"1022489097","full_name":"norandom/direct-transriberr","owner":"norandom","description":"Command-line utility for OpenAI Whisper models for locally computed transcribe jobs for audio and video content","archived":false,"fork":false,"pushed_at":"2025-07-19T07:51:09.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-19T12:17:07.882Z","etag":null,"topics":["docker","ffmpeg","openai","python","transcription","whisper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/norandom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-19T07:17:53.000Z","updated_at":"2025-07-19T07:51:22.000Z","dependencies_parsed_at":"2025-07-19T12:18:19.556Z","dependency_job_id":"318db063-e33e-4768-9cba-1087b4b88d6b","html_url":"https://github.com/norandom/direct-transriberr","commit_stats":null,"previous_names":["norandom/direct-transriberr"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/norandom/direct-transriberr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fdirect-transriberr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fdirect-transriberr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fdirect-transriberr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fdirect-transriberr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/norandom","download_url":"https://codeload.github.com/norandom/direct-transriberr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/norandom%2Fdirect-transriberr/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266222295,"owners_count":23894992,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","ffmpeg","openai","python","transcription","whisper"],"created_at":"2025-07-21T01:01:33.620Z","updated_at":"2025-12-30T22:07:01.396Z","avatar_url":"https://github.com/norandom.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Direct Transcriber\n\nA local CPU-only batch audio transcription tool optimized for RAGflow integration. Uses OpenAI Whisper models running entirely on your machine - no data leaves your system.\n\n## Features\n\n- **100% Local Processing** - No external API calls, all processing on CPU\n- **Automatic Model Selection** - Detects available RAM and suggests optimal Whisper model\n- **Batch Processing** - Process entire directories of audio files\n- **RAGflow Optimized** - Markdown output with metadata and chunking options\n- **Progress Tracking** - Rich progress bars for individual files and batches\n- **Multi-format Support** - Handles audio (MP3, WAV, M4A, FLAC, OGG, WMA, AAC) and video (MP4, AVI, MOV, MKV, WMV, FLV, WebM, M4V)\n- **Memory Safe** - Monitors system resources and prevents OOM errors\n\n## Installation\n\n### Option 1: Pre-built Docker Images (Recommended)\n\n**Using GitHub Container Registry (fastest):**\n```bash\n# Use pre-built images - no build required!\ndocker run --rm -v ./input:/input -v ./output:/output -v ./models:/models \\\n  ghcr.io/user/direct-transcriber:latest \\\n  direct-transcriber batch /input --output-dir /output --yes\n\n# Or with docker-compose\ncurl -O https://raw.githubusercontent.com/user/direct-transcriber/main/docker-compose.ghcr.yml\ndocker-compose -f docker-compose.ghcr.yml up\n```\n\n### Option 2: Local Docker Build\n\n```bash\n# Clone repository\ngit clone https://github.com/norandom/direct-transriberr\ncd direct-transcriber\n\n# Build and run with Docker Compose\ndocker-compose up --build\n```\n\n### Option 3: Python Installation\n\n**Requirements:**\n- Python 3.9+\n- FFmpeg (for audio processing)\n- uv (for dependency management)\n\n**Install with uv:**\n```bash\n# Clone repository\ngit clone https://github.com/norandom/direct-transriberr\ncd direct-transcriber\n\n# Install with uv\nuv pip install -e .\n```\n\n**System Dependencies:**\n\n**Ubuntu/Debian:**\n```bash\nsudo apt update\nsudo apt install ffmpeg\n```\n\n**macOS:**\n```bash\nbrew install ffmpeg\n```\n\n**CentOS/RHEL:**\n```bash\nsudo yum install ffmpeg\n```\n\n## Usage\n\n### Docker Usage (Recommended)\n\n**Using the helper script for external files:**\n\n```bash\n# Batch process external directory\n./scripts/transcribe-external.sh -m /path/to/your/media -o /path/to/output\n\n# Single external file\n./scripts/transcribe-external.sh -f /path/to/video.mp4 -o /path/to/output\n\n# With specific model\n./scripts/transcribe-external.sh -m /path/to/media -o /path/to/output --model large-v3\n```\n\n**Direct Docker Compose usage:**\n\n```bash\n# Copy files to input directory\ncp /path/to/your/media/* ./input/\n\n# Run transcription (models will be downloaded to ./models on first run)\ndocker-compose up --build\n\n# For external directories, set environment variables\nMEDIA_DIR=/path/to/your/media OUTPUT_DIR=/path/to/output docker-compose --profile external up --build transcriber-external\n```\n\n### Python/uv Usage\n\n**Prerequisites:**\n```bash\n# Activate the virtual environment\nsource .venv/bin/activate\n```\n\n**Batch Processing:**\n\n```bash\n# Process all audio and video files in a directory\ndirect-transcriber batch /path/to/media/files/\n\n# With custom output directory\ndirect-transcriber batch /audio/ --output-dir /transcriptions/\n\n# Force specific model\ndirect-transcriber batch /audio/ --model large-v3\n\n# Include timestamps for reference\ndirect-transcriber batch /audio/ --timestamps\n\n# Chunk output for better RAG performance\ndirect-transcriber batch /audio/ --chunk-size 500\n\n# Save both markdown and JSON\ndirect-transcriber batch /audio/ --format both\n```\n\n**Single File:**\n\n```bash\n# Transcribe single audio file\ndirect-transcriber single audio.mp3\n\n# Transcribe single video file (extracts audio automatically)\ndirect-transcriber single video.mp4\n\n# With custom output  \ndirect-transcriber single media.mp4 --output transcript.md\n\n# JSON output\ndirect-transcriber single media.mp4 --format json\n\n# RAG-optimized output with intelligent chunking\ndirect-transcriber single media.mp4 --rag-optimized --chunking-strategy semantic\n\n# Fixed-size chunking for consistent chunk sizes\ndirect-transcriber single media.mp4 --rag-optimized --chunking-strategy fixed --chunk-size 1000\n\n# Example: Process a video segment with medium model and RAG optimization\ndirect-transcriber single test_5min.mp4 --model medium --rag-optimized --chunking-strategy semantic --output test_5min_medium.md\n```\n\n### Command Options\n\n- `--model, -m`: Whisper model (tiny, base, small, medium, large-v3)\n- `--output-dir, -o`: Output directory for batch processing\n- `--format, -f`: Output format (md, json, both)\n- `--timestamps`: Include timestamps in markdown\n- `--chunk-size`: Chunk size for RAG optimization (characters)\n- `--rag-optimized`: Enable RAG-optimized output with intelligent chunking\n- `--chunking-strategy`: Chunking strategy (semantic, sentence, fixed)\n- `--workers, -w`: Number of parallel workers (auto-detected)\n- `--yes, -y`: Skip confirmation prompts\n\n## Model Selection\n\nThe tool automatically selects the best Whisper model based on available RAM:\n\n| Model | RAM Required | Quality | Speed |\n|-------|-------------|---------|--------|\n| tiny | 1 GB | Lowest | Fastest |\n| base | 1.5 GB | Good | Fast |\n| small | 2 GB | Better | Moderate |\n| medium | 4 GB | High | Slower |\n| large-v3 | 6 GB | Best | Slowest |\n\n## Output Formats\n\n### Clean Markdown (Default)\n```markdown\n# Video Transcription\n**File:** `/full/path/to/video/meeting.mp4`\n**Duration:** 15:32 | **Model:** large-v3 | **Language:** en\n**Source:** video | **Transcribed:** 2024-01-15 14:30:22\n\n---\n\nThe speaker discusses the importance of machine learning in modern applications. They explain how neural networks can be trained to recognize patterns in data.\n\nAnother topic covered is the implementation of transformer models for natural language processing tasks.\n```\n\n### With Timestamps\n```markdown\n# Audio Transcription\n**File:** `/full/path/to/audio/meeting.mp3`\n\n## [00:00] - [02:15]\nThe speaker discusses the importance of machine learning in modern applications.\n\n## [02:15] - [04:30]\nThey explain how neural networks can be trained to recognize patterns in data.\n```\n\n### Chunked for RAG\n```markdown\n# Audio Transcription\n**File:** `/full/path/to/audio/meeting.mp3`\n\n## Segment 1 (00:00-05:00)\n[Content chunk optimized for semantic search]\n\n## Segment 2 (05:00-10:00)\n[Next semantic chunk]\n```\n\n## Docker Image\n\n### Optimized Container\n\nThe Docker image uses a multi-stage build process to minimize size while maintaining full functionality:\n\n- **Base**: Python 3.11-slim for minimal footprint\n- **Size**: ~800MB (compared to 2GB+ for standard PyTorch images)\n- **Security**: Runs as non-root user\n- **Optimization**: Removes test files, caches, and unnecessary components\n\n### Available Tags\n\n- `ghcr.io/norandom/direct-transriberr:latest` - Latest stable release\n- `ghcr.io/norandom/direct-transriberr:main` - Latest development build\n- `ghcr.io/norandom/direct-transriberr:v1.0.0` - Specific version tags\n\n## Docker Configuration\n\n### Environment Variables\n\nCreate a `.env` file based on `.env.example`:\n\n```bash\n# External media directory (absolute path)\nMEDIA_DIR=/path/to/your/media/files\n\n# Output directory for transcriptions (absolute path)\nOUTPUT_DIR=/path/to/your/transcriptions\n\n# Model to use (optional, auto-detected if not specified)\nWHISPER_MODEL=large-v3\n\n# Memory limit for container (optional)\nMEMORY_LIMIT=8G\n```\n\n### Volume Mounting\n\nThe Docker setup supports several volume mounting options:\n\n1. **Default directories**: `./input`, `./output`, and `./models`\n2. **Environment variables**: Use `MEDIA_DIR` and `OUTPUT_DIR`\n3. **Single file mounting**: Mount specific files for transcription\n4. **External script**: Use `scripts/transcribe-external.sh` for easy external file processing\n5. **Persistent model storage**: Models are downloaded once to `./models` and reused\n\n## Performance\n\n- **CPU Optimization**: Uses available CPU cores minus 1 for system\n- **Memory Management**: Monitors RAM usage and prevents OOM\n- **Batch Processing**: Processes multiple files with progress tracking\n- **Format Support**: Automatic audio format conversion via FFmpeg\n- **Docker Isolation**: Containerized processing with resource limits\n- **Model Persistence**: Whisper models downloaded once and cached in `./models`\n- **Enhanced Progress**: Detailed progress tracking with processing times\n- **CPU Optimized**: FP16 warnings suppressed, CPU-specific optimizations\n\n## RAG Optimization Features\n\nDirect Transcriber now includes advanced RAG (Retrieval-Augmented Generation) optimizations:\n\n### Intelligent Chunking Strategies\n\n**Semantic Chunking (Recommended):**\n- Breaks content at natural topic boundaries\n- Detects discourse markers and transitions\n- Preserves context and meaning\n- Ideal for complex discussions and lectures\n\n**Sentence Chunking:**\n- Groups sentences into coherent chunks\n- Respects sentence boundaries\n- Good for clear, structured speech\n- Maintains readability\n\n**Fixed-Size Chunking:**\n- Consistent chunk sizes with smart overlap\n- Predictable for downstream processing\n- Good for batch processing workflows\n\n### Enhanced Metadata Extraction\n\n- **Keyword Extraction**: Automatic identification of key terms\n- **Entity Recognition**: Names, numbers, times, and proper nouns\n- **Topic Classification**: Domain-specific topic identification\n- **Quality Scoring**: Confidence-based chunk quality assessment\n- **Context Linking**: Inter-chunk relationships and context\n\n### RAG-Optimized Output\n\n```bash\n# Enable RAG optimization\ndirect-transcriber batch /audio --rag-optimized\n\n# Choose chunking strategy\ndirect-transcriber batch /audio --rag-optimized --chunking-strategy semantic\n\n# Custom chunk size\ndirect-transcriber batch /audio --rag-optimized --chunk-size 1500\n```\n\n**Output Features:**\n- Structured markdown with semantic sections\n- JSON sidecar files for programmatic access\n- Cross-chunk context preservation\n- Quality metrics and confidence scores\n- Entity and keyword extraction\n- Topic classification\n\n**Example RAG Output:**\n```markdown\n# Audio Transcription (RAG Optimized)\n**Chunks:** 15 | **Strategy:** SemanticChunking\n\n## Segment 1 (00:00 - 02:30)\n**ID:** `lecture_001` | **Quality:** 0.92 | **Topics:** technology, AI\n\nThe speaker discusses machine learning fundamentals...\n\n🏷️ **Entities:** Neural Networks, Deep Learning, PyTorch\n```\n\n### Integration Benefits\n\n- **Better Retrieval**: Semantic chunks improve search relevance\n- **Context Preservation**: Overlapping chunks maintain continuity\n- **Quality Filtering**: Low-confidence segments are flagged\n- **Structured Data**: JSON output enables programmatic processing\n- **Metadata Rich**: Enhanced information for better indexing\n\n## License\n\nMIT License - see LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorandom%2Fdirect-transriberr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnorandom%2Fdirect-transriberr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorandom%2Fdirect-transriberr/lists"}