{"id":15022053,"url":"https://github.com/hitomarukonpaku/twspace-crawler","last_synced_at":"2025-05-16T01:05:58.052Z","repository":{"id":40495345,"uuid":"415567427","full_name":"HitomaruKonpaku/twspace-crawler","owner":"HitomaruKonpaku","description":"Script to monitor \u0026 download Twitter Spaces 24/7","archived":false,"fork":false,"pushed_at":"2025-01-07T21:21:07.000Z","size":837,"stargazers_count":227,"open_issues_count":10,"forks_count":42,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-05-12T15:30:39.280Z","etag":null,"topics":["discord","ffmpeg","nodejs","twitter","twitter-api","twitter-spaces","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HitomaruKonpaku.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"hitomarukonpaku","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"custom":null}},"created_at":"2021-10-10T11:34:26.000Z","updated_at":"2025-05-09T16:57:43.000Z","dependencies_parsed_at":"2023-02-12T12:01:55.462Z","dependency_job_id":"1e6f457d-aacb-42b6-9117-f654c60eda4e","html_url":"https://github.com/HitomaruKonpaku/twspace-crawler","commit_stats":{"total_commits":429,"total_committers":7,"mean_commits":"61.285714285714285","dds":0.07925407925407923,"last_synced_commit":"ca3206ba12312213fc41b0df5d0b22dad5c93b4f"},"previous_names":[],"tags_count":68,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HitomaruKonpaku%2Ftwspace-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HitomaruKonpaku%2Ftwspace-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HitomaruKonpaku%2Ftwspace-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HitomaruKonpaku%2Ftwspace-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HitomaruKonpaku","download_url":"https://codeload.github.com/HitomaruKonpaku/twspace-crawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254448579,"owners_count":22072764,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["discord","ffmpeg","nodejs","twitter","twitter-api","twitter-spaces","typescript"],"created_at":"2024-09-24T19:57:23.640Z","updated_at":"2025-05-16T01:05:53.043Z","avatar_url":"https://github.com/HitomaruKonpaku.png","language":"TypeScript","funding_links":["https://ko-fi.com/hitomarukonpaku"],"categories":[],"sub_categories":[],"readme":"# twspace-crawler\n\n[![npm](https://img.shields.io/npm/v/twspace-crawler)](https://www.npmjs.com/package/twspace-crawler)\n[![npm](https://img.shields.io/npm/dt/twspace-crawler)](https://www.npmjs.com/package/twspace-crawler)\n\n\u003e **Node.js script \u0026 command-line app to automatically monitor \u0026 download [Twitter Spaces](https://help.twitter.com/en/using-twitter/spaces).**\n\n\u003e **UPDATE 2023-07-01: SINCE TWITTER CHANGED THEIR APIS, READ [INSTALLATION](INSTALLATION.md#unofficial-api) FIRST**\n\n## Contents\n\n1. [Requirements](#requirements)\n1. [Env](#env)\n1. [Docker](#docker)\n1. [Installation](#installation)\n1. [Usage](#usage)\n1. [Options](#options)\n1. [Commands](#commands)\n1. [Webhooks](#webhooks)\n1. [Changelog](CHANGELOG.md)\n\n## Requirements\n\n- [Node.js](https://nodejs.org) (\u003e=14)\n- [FFMPEG](https://www.ffmpeg.org)\n\n## Env\n\n| Name | Desc\n| - | -\n| LOG_LEVEL | -\n| TWITTER_AUTH_TOKEN | -\n| TWITTER_CSRF_TOKEN | -\n| SKIP_DOWNLOAD | -\n| SKIP_DOWNLOAD_AUDIO | -\n| SKIP_DOWNLOAD_CAPTION | -\n\n## Docker\n\n\u003e Recommended\n\n### Monitor users\n\n- See [example](./example/)\n- Download [docker-compose.yaml](./example/docker-compose.yaml) \u0026 [config.yaml](./example/config.yaml)\n- Sign in Twitter\n- `Open DevTools (F12)` \u003e `Application` \u003e `Storage` \u003e `Cookies`\n- Update docker compose environment\n  - `TWITTER_AUTH_TOKEN`: Value of `auth_token`\n  - `TWITTER_CSRF_TOKEN`: Value of `ct0`\n- Run\n\n    ```sh\n    docker compose up -d\n    ```\n\n### Download Space\n\n```sh\ndocker run --rm \\\n  -e TWITTER_AUTH_TOKEN=\u003cauth_token\u003e \\\n  -e TWITTER_CSRF_TOKEN=\u003ccsrf_token\u003e \\\n  -v ./logs:/app/logs \\\n  -v ./download:/app/download \\\n  ghcr.io/hitomarukonpaku/twspace-crawler \\\n  --id 1mnGeRRRwkrJX\n```\n\n## Installation\n\n[FULL INSTALLATION](INSTALLATION.md)\n\n### Command-line installation\n\n```bash\nnpm install --global twspace-crawler\n```\n\n### Module installation\n\n```bash\nnpm install twspace-crawler\n```\n\n## Usage\n\n### Monitor user(s) indefinitely, wait for live Space and download when Space ended\n\n```\ntwspace-crawler --user nakiriayame,LaplusDarknesss\n```\n\n```\ntwspace-crawler --env ./.env --config ./config.json\n```\n\n### Monitor \u0026 download Space by id\n\n```\ntwspace-crawler --id 1yoJMWvbybNKQ\n```\n\n### Download Space by playlist url\n\n```\ntwspace-crawler --url https://prod-fastly-ap-northeast-1.video.pscp.tv/Transcoding/v1/hls/1Nq1QFkYTQ4v1X4BTV_aJ_pFeQhYyuYXY7ykz5xB7v5NvGwFMJMKwnRBmxyi9twF4BZ90ZKks5wdGKqESVsjLw...\n```\n\n### Download Space by playlist url with additional metadata (if Space url still available)\n\n```\ntwspace-crawler --id 1yoJMWvbybNKQ --url https://prod-fastly-ap-northeast-1.video.pscp.tv/Transcoding/v1/hls/1Nq1QFkYTQ4v1X4BTV_aJ_pFeQhYyuYXY7ykz5xB7v5NvGwFMJMKwnRBmxyi9twF4BZ90ZKks5wdGKqESVsjLw...\n```\n\n## Options\n\n### General options\n\n```\n  -h, --help                        Display help\n  -d, --debug                       Show debug logs\n```\n\n### One-time download options\n\n```\n  --id \u003cSPACE_ID\u003e                   Monitor \u0026 download live Space with its id\n  -surl, --space-url \u003cSPACE_URL\u003e    Monitor \u0026 download live Space with its URL\n  --force                           Force download Space when using with --id\n  --url \u003cPLAYLIST_URL\u003e              Download Space using playlist url\n```\n\n### User monitoring options\n\n```\n  --env \u003cENV_PATH\u003e                  Path to .env file, default to current working\n                                    folder (See .env.example)\n  --config \u003cCONFIG_PATH\u003e            Path to config file (See config.example.json)\n  --user \u003cUSER\u003e                     Monitor \u0026 download live Spaces from users\n                                    indefinitely, separate by comma (,)\n```\n\n### Additional options\n\n```\n  --skip-download                   Do not download anything\n  --skip-download-audio             Do not download audio\n  --skip-download-caption           Do not download caption\n\n  --notification                    Show notification about new live Space\n  --force-open                      Force open Space in browser\n```\n\n## Commands\n\nUse to manually process audio/captions\n\n### List\n\n- cc\n\n```\n  download|d \u003cSPACE_ID\u003e \u003cENDPOINT\u003e \u003cTOKEN\u003e  Download Space captions, with\n                                            - ENDPOINT: Chat endpoint\n                                            - TOKEN: Chat access token\n\n  extract|e \u003cFILE\u003e [STARTED_AT]             Extract Space captions\n```\n\n### Example\n\n```\ntwspace-crawler cc d 1yoJMWneoZwKQ https://prod-chatman-ancillary-ap-northeast-1.pscp.tv 2Ozpcu2xxqb5wxMdkyodUCygOrbYMLv8rq...\n```\n\n```\ntwspace-crawler cc e /download/sample_cc.jsonl\n```\n\n```\ntwspace-crawler cc e /download/sample_cc.jsonl 1633612289669\n```\n\n## Webhooks\n\nCurrently support Discord Webooks\n\nCheck [config.example.yaml](config.example.yaml) ~~or [config.example.json](config.example.json)~~ for more detail\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhitomarukonpaku%2Ftwspace-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhitomarukonpaku%2Ftwspace-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhitomarukonpaku%2Ftwspace-crawler/lists"}