{"id":29085183,"url":"https://github.com/open-technology-foundation/transcribe","last_synced_at":"2026-01-20T16:32:39.247Z","repository":{"id":280280080,"uuid":"941496335","full_name":"Open-Technology-Foundation/transcribe","owner":"Open-Technology-Foundation","description":"A robust Python package for high-quality audio transcription using OpenAI's Whisper and post-processing with GPT models.","archived":false,"fork":false,"pushed_at":"2025-10-23T06:18:45.000Z","size":97190,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-23T08:28:09.807Z","etag":null,"topics":["python","transcription"],"latest_commit_sha":null,"homepage":"https://yatti.id/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Open-Technology-Foundation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-02T12:41:20.000Z","updated_at":"2025-10-23T06:18:48.000Z","dependencies_parsed_at":"2025-07-31T02:15:18.937Z","dependency_job_id":"95d72a02-8535-43b8-afd1-c4bde2ee5c09","html_url":"https://github.com/Open-Technology-Foundation/transcribe","commit_stats":null,"previous_names":["open-technology-foundation/transcribe"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Open-Technology-Foundation/transcribe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Open-Technology-Foundation%2Ftranscribe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Open-Technology-Foundation%2Ftranscribe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Open-Technology-Foundation%2Ftranscribe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Open-Technology-Foundation%2Ftranscribe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Open-Technology-Foundation","download_url":"https://codeload.github.com/Open-Technology-Foundation/transcribe/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Open-Technology-Foundation%2Ftranscribe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28607248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T16:10:39.856Z","status":"ssl_error","status_checked_at":"2026-01-20T16:10:39.493Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","transcription"],"created_at":"2025-06-27T22:30:57.034Z","updated_at":"2026-01-20T16:32:39.241Z","avatar_url":"https://github.com/Open-Technology-Foundation.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Transcribe\n\nA robust Python package for high-quality audio transcription using OpenAI's Whisper API with intelligent post-processing via GPT models.\n\n## Features\n\n- **High-Quality Transcription**: Uses OpenAI's Whisper API for accurate audio-to-text conversion\n- **Intelligent Post-Processing**: GPT models clean and format transcripts for readability\n- **Timestamp Support**: Precise timing information for each speech segment\n- **Subtitle Generation**: Create SRT and VTT subtitle files with proper formatting\n- **Parallel Processing**: Efficiently process large audio files using multiple workers\n- **Content-Aware Processing**: Specialized handling for different content types (technical, medical, etc.)\n- **Multi-language Support**: Automatic language detection and processing\n- **Flexible Configuration**: Configure via command line, environment variables, or JSON files\n- **Smart Caching**: Avoid redundant API calls and reduce costs\n- **Robust Error Handling**: Automatic retry mechanisms with graceful degradation\n\n## Installation\n\n### Using the Installation Script (Recommended)\n\nThe simplest way to install is using the provided installation script:\n\n```bash\n# Clone the repository\ngit clone https://github.com/Open-Technology-Foundation/transcribe.git\ncd transcribe\n\n# Local installation (recommended - installs in current directory)\n./install.sh --local\n\n# OR System-wide installation (requires sudo)\nsudo ./install.sh --system\n```\n\nAfter installation, create a `.env` file with your OpenAI API key:\n\n```bash\necho \"OPENAI_API_KEY=your_api_key_here\" \u003e .env\n```\n\n### Manual Installation\n\nIf you prefer manual installation:\n\n#### Local Development Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/Open-Technology-Foundation/transcribe.git\ncd transcribe\n\n# Create and activate virtual environment\npython3 -m venv .venv\nsource .venv/bin/activate\n\n# Install dependencies and package in development mode\npip install -r requirements.txt\npip install -e .\n\n# Make wrapper scripts executable\nchmod +x transcribe clean-transcript create-sentences\n```\n\n#### System-wide Installation\n\n```bash\n# Clone to system location\nsudo git clone https://github.com/Open-Technology-Foundation/transcribe.git /usr/share/transcribe\ncd /usr/share/transcribe\n\n# Create virtual environment and install\nsudo python3 -m venv .venv\nsudo .venv/bin/pip install -r requirements.txt\nsudo .venv/bin/pip install -e .\n\n# Make scripts executable\nsudo chmod +x transcribe clean-transcript create-sentences\n\n# Create system-wide symlinks\nsudo ln -sf /usr/share/transcribe/transcribe /usr/local/bin/transcribe\nsudo ln -sf /usr/share/transcribe/clean-transcript /usr/local/bin/clean-transcript\nsudo ln -sf /usr/share/transcribe/create-sentences /usr/local/bin/create-sentences\n```\n\n## Quick Start\n\n### Basic Transcription\n\n```bash\n# Simple transcription\ntranscribe audio_file.mp3 -o transcript.txt\n\n# With context for better accuracy\ntranscribe audio_file.mp3 -o transcript.txt -c \"medical,technical\" -v\n\n# Generate timestamps\ntranscribe audio_file.mp3 -o transcript.txt -T\n\n# Create SRT subtitles\ntranscribe audio_file.mp3 --srt\n\n# Create VTT subtitles  \ntranscribe audio_file.mp3 --vtt\n```\n\n### Large File Processing\n\n```bash\n# Use parallel processing for large files\ntranscribe audio_file.mp3 -w 4 --parallel --max-parallel-workers 4\n```\n\n### Post-Processing Tools\n\n```bash\n# Clean existing transcript\nclean-transcript raw_transcript.txt -o clean_transcript.txt -c \"context\"\n\n# Format text into sentences\ncreate-sentences transcript.txt -o formatted.txt\n```\n\n## Command Line Options\n\n### Transcribe\n\n```\nUsage: transcribe [options] audio_file\n\nOutput Options:\n  -o, --output FILE      Output file (default: input filename with .txt extension)\n  -O, --output-to-stdout Output transcription to stdout\n\nProcessing Options:\n  -P, --no-post-processing   Disable post-processing cleanups (default: enabled)\n  -l, --chunk-length MS      Audio chunk length in milliseconds (default: 600000)\n  -L, --input-language LANG  Input audio language (default: auto-detect)\n  -c, --context TEXT         Context for post-processing (e.g., medical,legal,technical)\n  -W, --transcribe-model MODEL Whisper model for transcription (default: whisper-1)\n  -m, --model MODEL          GPT model for post-processing (default: gpt-4o)\n  -s, --max-chunk-size N     Maximum chunk size for post-processing (default: 3000)\n  -t, --temperature N        Temperature for generation (default: 0.05)\n  -p, --prompt TEXT          Prompt to guide initial transcription\n  -w, --max-workers N        Parallel workers for transcription (default: 1)\n\nAdvanced Post-Processing Options:\n  --summary-model MODEL      Model for context summarization (default: gpt-4o-mini)\n  --auto-context            Auto-determine domain context from content\n  --raw                     Save raw transcript before post-processing\n  --auto-language           Auto-detect language from transcript content\n  --parallel                Enable parallel processing for large transcripts\n  --max-parallel-workers N  Parallel workers for post-processing (default: CPU count)\n  --cache                   Enable caching for better performance\n  --content-aware           Enable content-aware specialized processing\n  --clear-cache             Clear cached results before starting\n\nTimestamp and Subtitle Options:\n  -T, --timestamps          Include timestamp information in output\n  --srt                     Generate SRT subtitle file (enables timestamps)\n  --vtt                     Generate VTT subtitle file (enables timestamps)\n\nLogging Options:\n  -v, --verbose             Enable verbose output\n  -d, --debug               Enable debug output\n\nExamples:\n  # Basic transcription\n  transcribe audio_file.mp3\n  \n  # Advanced transcription with all features\n  transcribe audio_file.mp3 -o transcript.txt --content-aware --parallel --cache\n  \n  # Transcription with context\n  transcribe audio_file.mp3 -c \"medical,technical\" -m gpt-4o\n  \n  # Generate subtitles\n  transcribe audio_file.mp3 --srt\n  \n  # Parallel processing\n  transcribe audio_file.mp3 -w 4 --parallel --max-parallel-workers 4\n```\n\n### Clean Transcript\n\n```\nUsage: clean-transcript [options] input_file\n\nOptions:\n  -L, --input-language LANG    Input language (translate to English if specified)\n  -c, --context TEXT           Domain-specific context (e.g., medical,legal)\n  -m, --model MODEL            OpenAI model to use (default: gpt-4o)\n  --summary-model MODEL        OpenAI model for summaries (default: gpt-4o-mini)\n  -M, --max-tokens N           Maximum tokens (default: 4096)\n  -s, --max-chunk-size N       Maximum chunk size for processing (default: 3000)\n  -t, --temperature N          Temperature for generation (default: 0.05)\n  -o, --output FILE            Output file (default: stdout)\n  -v, --verbose                Enable verbose output\n  -d, --debug                  Enable debug output\n```\n\n### Create Sentences\n\n```\nUsage: create-sentences [options] input_file\n\nOptions:\n  -o, --output FILE            Output file (default: stdout)\n  -p, --paragraphs             Create paragraphs instead of sentences\n  -m, --min-sentences N        Minimum sentences per paragraph (default: 2)\n  -M, --max-sentences N        Maximum sentences per paragraph (default: 8)\n  -s, --max-sentence-length N  Maximum sentence length in bytes (default: 3000)\n  -v, --verbose                Enable verbose output\n  -d, --debug                  Enable debug output\n```\n\n## Environment Variables\n\n- `OPENAI_API_KEY`: Your OpenAI API key (required)\n- `OPENAI_COMPLETION_MODEL`: Default model for post-processing (default: gpt-4o)\n- `OPENAI_SUMMARY_MODEL`: Default model for summaries (default: gpt-4o-mini)\n- `TRANSCRIBE_CHUNK_LENGTH_MS`: Audio chunk length in milliseconds (default: 600000)\n- `TRANSCRIBE_MAX_CHUNK_SIZE`: Maximum text chunk size for processing (default: 3000)\n- `TRANSCRIBE_PARALLEL`: Enable parallel processing (true/false, default: true)\n- `TRANSCRIBE_MAX_WORKERS`: Maximum number of workers (default: 4)\n- `TRANSCRIBE_TEMP_DIR`: Directory for temporary files (default: system temp)\n- `TRANSCRIBE_LOG_LEVEL`: Logging level (INFO, DEBUG, etc., default: INFO)\n\n## Configuration File\n\nYou can specify configuration options in a JSON file:\n\n```json\n{\n  \"openai\": {\n    \"models\": {\n      \"transcription\": \"whisper-1\",\n      \"completion\": \"gpt-4o\",\n      \"summary\": \"gpt-4o-mini\"\n    }\n  },\n  \"transcription\": {\n    \"chunk_length_ms\": 600000,\n    \"temperature\": 0.05,\n    \"parallel\": true,\n    \"max_workers\": 4\n  },\n  \"processing\": {\n    \"max_chunk_size\": 3000,\n    \"temperature\": 0.1,\n    \"max_tokens\": 4096,\n    \"post_processing\": true\n  },\n  \"output\": {\n    \"save_raw\": true,\n    \"create_paragraphs\": true,\n    \"min_sentences_per_paragraph\": 2,\n    \"max_sentences_per_paragraph\": 8\n  },\n  \"system\": {\n    \"temp_dir\": \"\",\n    \"log_level\": \"INFO\"\n  }\n}\n```\n\n## Advanced Usage\n\n### Parallel Processing\n\nFor large audio files (\u003e30 minutes), configure parallel processing:\n\n```bash\n# Parallel transcription workers\ntranscribe audio_file.mp3 -w 4\n\n# Parallel post-processing \ntranscribe audio_file.mp3 --parallel --max-parallel-workers 4\n\n# Combined parallel processing\ntranscribe audio_file.mp3 -w 4 --parallel --max-parallel-workers 4\n```\n\n### Advanced Features\n\n```bash\n# Content-aware processing (automatically detects content type)\ntranscribe audio_file.mp3 --content-aware\n\n# Enable caching to reduce API costs\ntranscribe audio_file.mp3 --cache\n\n# Auto-detect domain context from content\ntranscribe audio_file.mp3 --auto-context\n\n# All advanced features combined\ntranscribe audio_file.mp3 --content-aware --parallel --cache --auto-context\n\n# Save raw transcript before post-processing\ntranscribe audio_file.mp3 --raw --content-aware\n```\n\n### Using as a Python Package\n\n```python\nfrom transcribe_pkg.core.transcriber import Transcriber\nfrom transcribe_pkg.core.processor import TranscriptProcessor\nfrom transcribe_pkg.utils.subtitle_utils import save_subtitles\n\n# Initialize transcriber\ntranscriber = Transcriber(\n    model=\"whisper-1\",\n    language=\"en\",\n    temperature=0.05,\n    chunk_length_ms=600000\n)\n\n# Initialize processor with advanced features\nprocessor = TranscriptProcessor(\n    model=\"gpt-4o\",\n    summary_model=\"gpt-4o-mini\",\n    max_chunk_size=3000,\n    cache_enabled=True,\n    content_aware=True\n)\n\n# Transcribe audio file\ntranscript = transcriber.transcribe(\n    audio_path=\"lecture.mp3\",\n    prompt=\"University lecture on quantum physics\",\n    with_timestamps=True,\n    max_workers=4\n)\n\n# Generate subtitles if timestamps are available\nif isinstance(transcript, dict) and \"segments\" in transcript:\n    save_subtitles(transcript, \"lecture.srt\", format_type=\"srt\")\n\n# Process transcript with content-aware analysis\nprocessed_text = processor.process(\n    text=transcript[\"text\"] if isinstance(transcript, dict) else transcript,\n    context=\"scientific,physics,education\",\n    use_parallel=True,\n    content_analysis=True\n)\n\n# Save results\nwith open(\"lecture_processed.txt\", \"w\", encoding=\"utf-8\") as f:\n    f.write(processed_text)\n```\n\n## Development\n\n### Running Tests\n\n```bash\n# Run all tests\npython run_tests.py\n\n# Run with coverage report\npython run_tests.py -c\n\n# Run specific test module\npython run_tests.py -p tests.test_text_utils\n\n# Run specific test method\npython run_tests.py -p tests.test_text_utils.TestTextUtils.test_create_sentences_normal\n```\n\n### Benchmarking\n\nPerformance testing scripts are available in `tests/benchmark/`:\n\n```bash\n# Benchmark parallel processing\n./tests/benchmark/benchmark-parallel.sh sample_audio.mp3\n\n# Test specific parallel features\npython tests/benchmark/test-parallel-flag.py\npython tests/benchmark/test-medium-file.py\npython tests/benchmark/test-large-file.py\n```\n\nFor detailed setup and development information, see [DEVELOPMENT.md](DEVELOPMENT.md).\n\n## Contributing\n\nContributions are welcome! Please:\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes with tests\n4. Follow the coding standards in [DEVELOPMENT.md](DEVELOPMENT.md)\n5. Submit a Pull Request\n\n## License\n\nThis project is licensed under the GNU General Public License v3.0 - see the [LICENSE](LICENSE) file for details.\n\n## Authors\n\n- Gary Dean\n- AI Transcription Team\n\n---\n\n*This README was comprehensively updated to reflect the current codebase*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-technology-foundation%2Ftranscribe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopen-technology-foundation%2Ftranscribe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-technology-foundation%2Ftranscribe/lists"}