https://github.com/arthurfdlr/yt-cc
💬 YoutubeCC - Parse JSON3 Youtube Closed Captions
https://github.com/arthurfdlr/yt-cc
Last synced: about 2 months ago
JSON representation
💬 YoutubeCC - Parse JSON3 Youtube Closed Captions
- Host: GitHub
- URL: https://github.com/arthurfdlr/yt-cc
- Owner: ArthurFDLR
- License: mit
- Created: 2024-04-27T23:56:30.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-09T05:01:25.000Z (about 2 years ago)
- Last Synced: 2025-03-04T15:48:31.048Z (over 1 year ago)
- Language: Python
- Homepage:
- Size: 14.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.ipynb
- License: LICENSE
Awesome Lists containing this project
README
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# YoutubeCC (`yt-cc`) - A Youtube Closed Captions (`.json3`) parser"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`yt-cc` is a Python library that parses Youtube Closed Captions (`.json3`) files. It is a simple and easy-to-use library that can be used to query precise parts of the video transcript or iterate over the entire transcript."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"from yt_dlp import YoutubeDL\n",
"from yt_cc import YoutubeCC"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"DATA_PATH = Path.cwd() / \".tmp\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Downloading the Closed Captions with `yt-dlp`\n",
"\n",
"To download the closed captions of a Youtube video, you can use the `yt-dlp` command-line tool. You can install `yt-dlp` using `pip`:\n",
"\n",
"```bash\n",
"pip install yt-dlp\n",
"```\n",
"\n",
"To download the closed captions of a Youtube video in the `.json3` format, you can use the following command:\n",
"\n",
"```bash\n",
"yt-dlp --write-auto-sub --skip-download --sub-format json3 \n",
"```\n",
"\n",
"You can also download the closed captions with the `yt-dlp` Python API:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[youtube] Extracting URL: https://www.youtube.com/watch?v=oHWuv1Aqrzk\n",
"[youtube] oHWuv1Aqrzk: Downloading webpage\n",
"[youtube] oHWuv1Aqrzk: Downloading ios player API JSON\n",
"[youtube] oHWuv1Aqrzk: Downloading android player API JSON\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: [youtube] Skipping player responses from android clients (got player responses for video \"aQvGIIdgFDM\" instead of \"oHWuv1Aqrzk\")\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[youtube] oHWuv1Aqrzk: Downloading m3u8 information\n",
"[info] oHWuv1Aqrzk: Downloading subtitles: en\n",
"[info] oHWuv1Aqrzk: Downloading 1 format(s): 248+251\n",
"Deleting existing file /home/arthur/Documents/02.workspace/02.active/clips-analytics/yt-cc/.tmp/oHWuv1Aqrzk.en.json3\n",
"[info] Writing video subtitles to: /home/arthur/Documents/02.workspace/02.active/clips-analytics/yt-cc/.tmp/oHWuv1Aqrzk.en.json3\n",
"[download] Destination: /home/arthur/Documents/02.workspace/02.active/clips-analytics/yt-cc/.tmp/oHWuv1Aqrzk.en.json3\n",
"[download] 100% of 71.66KiB in 00:00:00 at 1.28MiB/s\n"
]
},
{
"data": {
"text/plain": [
"PosixPath('/home/arthur/Documents/02.workspace/02.active/clips-analytics/yt-cc/.tmp/oHWuv1Aqrzk.en.json3')"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"YoutubeDL(\n",
" params = dict(\n",
" paths=dict(home=str(DATA_PATH)),\n",
" skip_download=True,\n",
" outtmpl=\"%(id)s.%(ext)s\",\n",
" subtitlesformat=\"json3\",\n",
" writeautomaticsub=True,\n",
" )\n",
").download([\"https://www.youtube.com/watch?v=oHWuv1Aqrzk\"])\n",
"\n",
"yt_cc_file_path = next(DATA_PATH.glob(\"*.json3\"))\n",
"yt_cc_file_path"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Parse the Closed Captions with `yt-cc`"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"
YoutubeCC
\n",
" - \n",
- File: /home/arthur/Documents/02.workspace/02.active/clips-analytics/yt-cc/.tmp/oHWuv1Aqrzk.en.json3 \n",
- lines: 202 \n",
- Segments: 768 \n",
"
"
"
"
" "
],
"text/plain": [
"YoutubeCC(file=/home/arthur/Documents/02.workspace/02.active/clips-analytics/yt-cc/.tmp/oHWuv1Aqrzk.en.json3, lines=202, segments=768)"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"youtube_caption = YoutubeCC(yt_cc_file_path)\n",
"youtube_caption"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LineCC(event_id=1.0, start_time_ms=0.0, duration_ms=218599.0, window_id=nan, window_style_id=1.0, window_position_id=1.0, append=nan, segments=[])\n",
"LineCC(event_id=nan, start_time_ms=2820.0, duration_ms=5279.0, window_id=1.0, window_style_id=nan, window_position_id=nan, append=nan, segments=['is', ' there', ' cool', ' small', ' projects', ' like', ' uh'])\n",
"LineCC(event_id=nan, start_time_ms=5510.0, duration_ms=2589.0, window_id=1.0, window_style_id=nan, window_position_id=nan, append=1.0, segments=['\\n'])\n",
"LineCC(event_id=nan, start_time_ms=5520.0, duration_ms=6180.0, window_id=1.0, window_style_id=nan, window_position_id=nan, append=nan, segments=['archive', ' sanity', ' and', ' and', ' so', ' on', ' that', \" you're\"])\n",
"LineCC(event_id=nan, start_time_ms=8089.0, duration_ms=3611.0, window_id=1.0, window_style_id=nan, window_position_id=nan, append=1.0, segments=['\\n'])\n",
"LineCC(event_id=nan, start_time_ms=8099.0, duration_ms=5580.0, window_id=1.0, window_style_id=nan, window_position_id=nan, append=nan, segments=['thinking', ' about', ' the', ' the', ' the', ' the', ' world', ' the'])\n",
"LineCC(event_id=nan, start_time_ms=11690.0, duration_ms=1989.0, window_id=1.0, window_style_id=nan, window_position_id=nan, append=1.0, segments=['\\n'])\n"
]
}
],
"source": [
"\n",
"for i, line_cc in enumerate(youtube_caption):\n",
" print(line_cc)\n",
" if i > 5:\n",
" break"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" event_id\n",
" start_time_ms\n",
" duration_ms\n",
" window_id\n",
" window_style_id\n",
" window_position_id\n",
" append\n",
" \n",
" \n",
" \n",
" \n",
" 0\n",
" 1.0\n",
" 0\n",
" 218599.0\n",
" NaN\n",
" 1.0\n",
" 1.0\n",
" NaN\n",
" \n",
" \n",
" 1\n",
" NaN\n",
" 2820\n",
" 5279.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" NaN\n",
" \n",
" \n",
" 2\n",
" NaN\n",
" 5510\n",
" 2589.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" 1.0\n",
" \n",
" \n",
" 3\n",
" NaN\n",
" 5520\n",
" 6180.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" NaN\n",
" \n",
" \n",
" 4\n",
" NaN\n",
" 8089\n",
" 3611.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" 1.0\n",
" \n",
" \n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" \n",
" \n",
" 197\n",
" NaN\n",
" 212580\n",
" 3900.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" NaN\n",
" \n",
" \n",
" 198\n",
" NaN\n",
" 214850\n",
" 1630.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" 1.0\n",
" \n",
" \n",
" 199\n",
" NaN\n",
" 214860\n",
" 3739.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" NaN\n",
" \n",
" \n",
" 200\n",
" NaN\n",
" 216470\n",
" 2129.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" 1.0\n",
" \n",
" \n",
" 201\n",
" NaN\n",
" 216480\n",
" 2119.0\n",
" 1.0\n",
" NaN\n",
" NaN\n",
" NaN\n",
" \n",
" \n",
"\n",
"
202 rows × 7 columns
\n","
],
"text/plain": [
" event_id start_time_ms duration_ms window_id window_style_id \\\n",
"0 1.0 0 218599.0 NaN 1.0 \n",
"1 NaN 2820 5279.0 1.0 NaN \n",
"2 NaN 5510 2589.0 1.0 NaN \n",
"3 NaN 5520 6180.0 1.0 NaN \n",
"4 NaN 8089 3611.0 1.0 NaN \n",
".. ... ... ... ... ... \n",
"197 NaN 212580 3900.0 1.0 NaN \n",
"198 NaN 214850 1630.0 1.0 NaN \n",
"199 NaN 214860 3739.0 1.0 NaN \n",
"200 NaN 216470 2129.0 1.0 NaN \n",
"201 NaN 216480 2119.0 1.0 NaN \n",
"\n",
" window_position_id append \n",
"0 1.0 NaN \n",
"1 NaN NaN \n",
"2 NaN 1.0 \n",
"3 NaN NaN \n",
"4 NaN 1.0 \n",
".. ... ... \n",
"197 NaN NaN \n",
"198 NaN 1.0 \n",
"199 NaN NaN \n",
"200 NaN 1.0 \n",
"201 NaN NaN \n",
"\n",
"[202 rows x 7 columns]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"youtube_caption.lines"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"\n",
"\n",
" \n",
" \n",
" \n",
" text\n",
" asr_confidence\n",
" offset_ms\n",
" pen_id\n",
" start_time_ms\n",
" line_id\n",
" \n",
" \n",
" \n",
" \n",
" 0\n",
" is\n",
" 248.0\n",
" NaN\n",
" None\n",
" 2820\n",
" 1\n",
" \n",
" \n",
" 1\n",
" there\n",
" 248.0\n",
" 599.0\n",
" None\n",
" 3419\n",
" 1\n",
" \n",
" \n",
" 2\n",
" cool\n",
" 248.0\n",
" 839.0\n",
" None\n",
" 3659\n",
" 1\n",
" \n",
" \n",
" 3\n",
" small\n",
" 248.0\n",
" 1020.0\n",
" None\n",
" 3840\n",
" 1\n",
" \n",
" \n",
" 4\n",
" projects\n",
" 248.0\n",
" 1380.0\n",
" None\n",
" 4200\n",
" 1\n",
" \n",
" \n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" ...\n",
" \n",
" \n",
" 763\n",
" sounds\n",
" 248.0\n",
" 1260.0\n",
" None\n",
" 216120\n",
" 199\n",
" \n",
" \n",
" 764\n",
" kind\n",
" 240.0\n",
" 1320.0\n",
" None\n",
" 216180\n",
" 199\n",
" \n",
" \n",
" 765\n",
" of\n",
" 248.0\n",
" 1500.0\n",
" None\n",
" 216360\n",
" 199\n",
" \n",
" \n",
" 766\n",
" \\n\n",
" NaN\n",
" NaN\n",
" None\n",
" 216470\n",
" 200\n",
" \n",
" \n",
" 767\n",
" crazy\n",
" 248.0\n",
" NaN\n",
" None\n",
" 216480\n",
" 201\n",
" \n",
" \n",
"\n",
"
768 rows × 6 columns
\n","
],
"text/plain": [
" text asr_confidence offset_ms pen_id start_time_ms line_id\n",
"0 is 248.0 NaN None 2820 1\n",
"1 there 248.0 599.0 None 3419 1\n",
"2 cool 248.0 839.0 None 3659 1\n",
"3 small 248.0 1020.0 None 3840 1\n",
"4 projects 248.0 1380.0 None 4200 1\n",
".. ... ... ... ... ... ...\n",
"763 sounds 248.0 1260.0 None 216120 199\n",
"764 kind 240.0 1320.0 None 216180 199\n",
"765 of 248.0 1500.0 None 216360 199\n",
"766 \\n NaN NaN None 216470 200\n",
"767 crazy 248.0 NaN None 216480 201\n",
"\n",
"[768 rows x 6 columns]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"youtube_caption.segments"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"is there cool small projects like uh\n",
"archive sanity and and so on that you're\n",
"thinking about the the the\n"
]
}
],
"source": [
"print(youtube_caption.get_text(start_time_ms=0, end_time_ms=10000))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
}