{"id":15145995,"url":"https://github.com/0xry4n/transcripter","last_synced_at":"2026-02-06T21:32:20.585Z","repository":{"id":255436277,"uuid":"847089148","full_name":"0xRy4n/transcripter","owner":"0xRy4n","description":"Search in real-time for snippets from YouTube transcripts.","archived":false,"fork":false,"pushed_at":"2024-08-30T18:44:04.000Z","size":90,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-10T00:43:40.140Z","etag":null,"topics":["podcast","redis","transcript-search","transcripts","youtube"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0xRy4n.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-24T20:18:28.000Z","updated_at":"2024-08-31T17:29:52.000Z","dependencies_parsed_at":"2024-08-30T21:52:26.979Z","dependency_job_id":null,"html_url":"https://github.com/0xRy4n/transcripter","commit_stats":null,"previous_names":["0xry4n/transcripter"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xRy4n%2Ftranscripter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xRy4n%2Ftranscripter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xRy4n%2Ftranscripter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xRy4n%2Ftranscripter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0xRy4n","download_url":"https://codeload.github.com/0xRy4n/transcripter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219867314,"owners_count":16555821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["podcast","redis","transcript-search","transcripts","youtube"],"created_at":"2024-09-26T12:00:31.543Z","updated_at":"2026-02-06T21:32:20.554Z","avatar_url":"https://github.com/0xRy4n.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Transcripter\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/0xRy4n/transcripter/actions/workflows/housekeeping.yml/badge.svg\" alt=\"Linting and Documentation\"\u003e\n  \u003ca href=\"https://app.codacy.com/gh/0xRy4n/transcripter/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade\"\u003e\n    \u003cimg src=\"https://app.codacy.com/project/badge/Grade/7233dfa7498c4801bbfc024b2675b2b0\" alt=\"Codacy Badge\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/transcripter\"\u003e\n    \u003cimg src=\"https://github.com/0xRy4n/transcripter/actions/workflows/publish.yml/badge.svg\" alt=\"Publish to PyPI\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://opensource.org/licenses/MIT\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/License-MIT-yellow.svg\" alt=\"License: MIT\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n![Transcripter](https://i.imgur.com/jg9sc7p.gif)\n\n\u003ca href=\"https://0xry4n.github.io/transcripter\"\u003e ![Static Badge](https://img.shields.io/badge/Documentation-View_Now-green?link=https%3A%2F%2F0xry4n.github.io%2Ftranscripter%2F) \u003c/a\u003e \u003ca href=\"https://x.com/0xry4n\"\u003e![X Follow](https://img.shields.io/twitter/follow/0xRy4n)\u003c/a\u003e\n\n\u003c/div\u003e\n\nTranscripter is a tool for indexing and searching YouTube video transcripts. It makes use of the YouTube Data API to fetch video details and transcripts, and collates and stores the transcript chunks in Redis for efficient full-text search.\n\nTranscripter's main purpose is to provide a simple way to add transcript-search functionality to your application. Inspired by (but unrelated to) \u003chttps://ippsec.rocks/\u003e. \n\n## Structure\n\n- `transcripter/`: Main package\n  - `core/`: Core functionality\n    - `redis_manager.py`: Manages Redis operations\n    - `youtube_manager.py`: Handles YouTube API interactions\n  - `services/`: Service layer\n    - `indexing_service.py`: Manages the indexing process\n    - `search_service.py`: Manages the search process\n  - `config.py`: Configuration management\n  - `logs.py`: Logging management\n- `examples/`: Example applications\n  - `basic/`: Basic example\n    - `app.py`: Simple application demonstrating usage\n- `index-config.yml`: Configuration file for indexing. Used by the example app.\n- `docker-compose.yml`: Docker compose file for the example app.\n\n## How It Works\n\n1. The `IndexingService` periodically fetches video details and transcripts from configured YouTube playlists, channels, or individual videos.\n2. Transcripts are processed and stored in Redis using the `RedisManager`.\n3. The `YouTubeManager` handles interactions with the YouTube API.\n4. Indexed data can be searched efficiently using Redis's full-text search capabilities.\n\n## Installation\n\nYou can install from PyPI:\n```bash\npip install transcripter\n```\n\n#### Manual Installation\n\n```bash\nmake install\n```\nor\n```bash\npip3 install git+https://github.com/0xRy4n/transcripter\n```\n\n**Note**: Transcripter _requires_ a Redis Stack server to run. It's recommended to use the official Docker image like so:\n\n```bash\ndocker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`\n```\n\nYou can run the Redis Stack either locally or remote, depending on your preference.\n\n## Configuration\n\n### Environment Variables\n\nSet the following environment variables:\n\n- `YOUTUBE_API_KEY`: Your YouTube API key\n- `REDIS_HOST`: Redis server host\n- `REDIS_PORT`: Redis server port\n- `REDIS_PASSWORD`: Redis server password (if applicable)\n- `REDIS_INDEX`: Name of the Redis index to use\n- `TRANSCRIPTER_LOG_LEVEL`: Log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)\n\n### Indexing Configuration\n\nConfigure the indexing process in your application by specifying a `indexing-config.yml` file like so:\n\n```yaml\n# Change me\nsources:\n  playlists:\n    # playlist id\n    - PLlkHl5i0GaAlBvAhi9R7S4vQ0IhR3ePCU\n  channels:\n    - # channel id\n  videos:\n    - # video id\n\nindexing:\n  interval: 3600  # Reindex every hour\n```\n\nYou can then pass the path to this file in to the `Config` object:\n\n```python\nconfig = Config(config_path='path/to/indexing-config.yml')\n```\nBy default, it will search for a file called `indexing-config.yml` in the root directory of the repository (one directory up from the `transcripter` directory).\n\n\nAlternatively, you can simply instantiate `Config` without any arguments, and set each property directly:\n```python\nfrom transcripter.config import Config\n\nconfig = Config()\nconfig.playlists = ['playlist_id1', 'playlist_id2']\nconfig.channels = ['channel_id1', 'channel_id2']\nconfig.videos = ['video_id1', 'video_id2']\nconfig.indexing_interval = 3600  # Indexing interval in seconds\n```\n\n## Starting the Indexing Service\n\n```python\nfrom transcripter.services.indexing_service import IndexingService\n\nindexing_service = IndexingService(config)\n```\nIndexing your videos is easy.\n```python\nindexing_service.index_all()\n```\nOnce a video has been indexed, it will not be indexed again, and it's transcript won't be pulled on successive runs. \n\nYou'll likely want to run the indexing service periodically as a background job. It is up to you to decide how you would like to implement this.\n\n## Searching\n\nYou can search the indexed data using the `SearchService` class.\n\n```python\nfrom transcripter.services.search_service import SearchService\n\nsearch_service = SearchService(config)\n```\n\nYou can then search for videos by title:\n\n```python\nsearch_service.search('search query')\n```\n\nIt will return a list of stored transcript chunks in the following format:\n```json\n[\n  {\n    \"snippet\": \"directory don't worry about it too much it's not going to be on the test there's not going to be a test but if there were\",\n    \"start_time\": 300.56,\n    \"timecode\": \"00:05:00\",\n    \"video_id\": \"_haJQY-Y70E\",\n    \"video_title\": \"Linux Structure and Commands - Intro to Hacking w/ HTB Academy #4\"\n  },\n  {\n    \"snippet\": \"directory don't worry about it too much it's not going to be on the test there's not going to be a test but if there were\",\n    \"start_time\": 300.56,\n    \"timecode\": \"00:05:00\",\n    \"video_id\": \"_haJQY-Y70E\",\n    \"video_title\": \"Linux Structure and Commands - Intro to Hacking w/ HTB Academy #4\"\n  }\n]\n```\n\n## Example Web App\n\nA fully functional example web app is included in the `examples/basic` directory. It is a simple web app that allows you to search for videos by title. It is built with Flask and uses Bootstrap for the frontend.\n\nYou can change the `index-config.yml` file in the root of the repository to configure the playlists, channels, and videos you want to index.\n\nAdditionally, you can opt to either create a `.env` file in the root of the repository and set the environment variables directly, or set them in your operating system's environment variables.\n\nThe example app is containerized with Docker, making it simple to run:\n```bash\ndocker compose up\n```\n\nAlternatively, you can run the app without Docker by installing the dependencies and running the `app.py` directly:\n\nFirst you'll need to run Redis Stack in a docker container:\n```bash\ndocker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest\n```\n\nThen install the dependencies and run the app:\n```bash\npip3 install -e .\npython3 examples/basic/app.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xry4n%2Ftranscripter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0xry4n%2Ftranscripter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xry4n%2Ftranscripter/lists"}