{"id":46269853,"url":"https://github.com/stiles/tiktools","last_synced_at":"2026-03-04T03:01:32.546Z","repository":{"id":325704189,"uuid":"1102127534","full_name":"stiles/tiktools","owner":"stiles","description":"A Python toolkit for TikTok data extraction and analysis using TikAPI.","archived":false,"fork":false,"pushed_at":"2025-11-27T21:08:33.000Z","size":179,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-26T17:51:40.222Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stiles.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-22T21:42:35.000Z","updated_at":"2026-01-22T21:28:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/stiles/tiktools","commit_stats":null,"previous_names":["stiles/tiktools"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/stiles/tiktools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stiles%2Ftiktools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stiles%2Ftiktools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stiles%2Ftiktools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stiles%2Ftiktools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stiles","download_url":"https://codeload.github.com/stiles/tiktools/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stiles%2Ftiktools/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30070479,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T01:03:42.280Z","status":"online","status_checked_at":"2026-03-04T02:00:07.464Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-04T03:01:31.941Z","updated_at":"2026-03-04T03:01:32.530Z","avatar_url":"https://github.com/stiles.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tiktools\n\n**A Python toolkit for TikTok data extraction and analysis using TikAPI.**\n\nExtract post metadata, thumbnails and get video transcription and translation with TikTok's built-in subtitles.\n\n## Features\n\n- **Fetch post metadata** - Download complete post data for any TikTok user\n- **Extract transcripts** - Get speech-to-text from videos using TikTok's ASR subtitles\n- **Download thumbnails** - Save video thumbnails in multiple sizes and formats\n- **Translate transcripts** - Translate transcripts using AWS Translate or other services\n- **Incremental updates** - Only fetch new content to save API costs\n- **Audio detection** - Flag videos with non-original audio (songs vs. speech)\n- **Pip-installable** - Easy to install and use in your projects\n- **Extensible** - Build custom analysis tools on top of the core toolkit\n\n## Example in the wild\n\n**[davis.food](https://davis.food)** - A live dashboard tracking @davis_big_dawg's viral school lunch reviews built mostly with tiktools. \n\n- Automated data collection (posts, transcripts, thumbnails)\n- Real-time engagement tracking\n- Interactive D3.js charts and statistics\n- GitHub Actions workflow for automatic updates\n\nThe complete source code is available: [github.com/stiles/davis.food](https://github.com/stiles/davis.food)\n\n## Installation\n\n```bash\npip install tiktools\n```\n\n## Quick start\n\n### 1. Set up API keys\n\nTikTools requires a [TikAPI](https://tikapi.io) key for core functionality:\n\n```bash\nexport TIKAPI_KEY=\"your_tikapi_key_here\"\n```\n\nFor testing, you can use the sandbox key: `DemoAPIKeyTokenSeHYGXDfd4SFD320Sc39Asd0Sc39Asd4s`\n\n### 2. Fetch posts, extract transcripts and download thumbnails\n\n```python\nfrom tiktools import fetch_user_posts, extract_transcripts, download_thumbnails\nfrom pathlib import Path\n\n# Fetch posts\nposts_data = fetch_user_posts(\n    username=\"davis_big_dawg\",\n    output_file=Path(\"data/davis_big_dawg/posts.json\")\n)\n\n# Extract transcripts\ntranscript_results = extract_transcripts(\n    posts_file=Path(\"data/davis_big_dawg/posts.json\"),\n    language=\"eng\"\n)\n\n# Download thumbnails\nthumbnail_results = download_thumbnails(\n    posts_file=Path(\"data/davis_big_dawg/posts.json\"),\n    thumbnail_type=\"origin\"  # High quality\n)\n\nprint(f\"Extracted {transcript_results['transcripts_downloaded']} transcripts\")\nprint(f\"Downloaded {thumbnail_results['thumbnails_downloaded']} thumbnails\")\n```\n\n### 3. Use the CLI scripts\n\n```bash\n# Fetch all posts\npython scripts/fetch_posts.py davis_big_dawg\n\n# Extract transcripts\npython scripts/extract_transcripts.py data/davis_big_dawg/davis_big_dawg_posts.json\n\n# Download thumbnails\npython scripts/download_thumbnails.py data/davis_big_dawg/davis_big_dawg_posts.json --type origin\n\n# Translate transcripts (requires AWS credentials)\npython scripts/translate_transcripts.py data/davis_big_dawg/transcripts/davis_big_dawg_transcripts.json --target en\n\n# See generic analysis template\npython scripts/analyze.py data/davis_big_dawg/transcripts/davis_big_dawg_transcripts.json\n```\n\n## Incremental updates\n\nSave API costs by only fetching new content:\n\n```bash\n# Only fetch NEW posts\npython scripts/fetch_posts.py davis_big_dawg --update\n\n# Only transcribe NEW posts\npython scripts/extract_transcripts.py data/davis_big_dawg/davis_big_dawg_posts.json --update\n\n# Only download thumbnails for NEW posts\npython scripts/download_thumbnails.py data/davis_big_dawg/davis_big_dawg_posts.json --update\n\n# Only translate NEW transcripts\npython scripts/translate_transcripts.py data/davis_big_dawg/transcripts/davis_big_dawg_transcripts.json --target en --update\n```\n\n## Output structure\n\n```\ndata/\n└── davis_big_dawg/\n    ├── davis_big_dawg_posts.json       # Post metadata\n    ├── thumbnails/\n    │   ├── 7575304937580547342.jpg     # Individual thumbnails\n    │   └── davis_big_dawg_thumbnails.json  # Thumbnail metadata\n    └── transcripts/\n        ├── 7575304937580547342.txt     # Individual transcripts\n        ├── 7575304937580547342.en.txt  # Translated transcripts\n        ├── davis_big_dawg_transcripts.json  # All transcripts\n        └── davis_big_dawg_translations.json # All translations\n```\n\n## Example: Food reviews analysis\n\n**Live example:** [davis.food](https://davis.food) - A production dashboard tracking @davis_big_dawg's viral school lunch reviews with automatic updates.\n\nSee `examples/food_reviews/` for a complete example analyzing @davis_big_dawg's school lunch reviews.\n\nThe example includes a standalone script that demonstrates the full workflow:\n\n```bash\ncd examples/food_reviews\n\n# Install tiktools and dependencies\npip install tiktools openai\n\n# Set up API keys\nexport TIKAPI_KEY=\"your_tikapi_key_here\"\nexport OPENAI_API_KEY=\"your_openai_key_here\"\n\n# Fetch posts, transcripts, thumbnails and extract review data\npython fetch_davis_archive.py --extract-reviews --download-thumbnails\n\n# Calculate statistics\npython calculate_stats.py data/davis_big_dawg/davis_big_dawg_reviews.json\n```\n\nFeatures:\n- Fetches all posts and transcripts for a TikTok user\n- Downloads video thumbnails (URLs expire quickly!)\n- Extracts structured review data using OpenAI\n- Calculates statistics by category and day\n- Update mode to only fetch new content\n\nSee the example README for full documentation and customization options.\n\n**Production deployment:** The complete [davis.food source code](https://github.com/stiles/davis.food) demonstrates how to build a production dashboard with GitHub Actions, Eleventy, D3.js charts and automated data updates.\n\n## API Reference\n\n### Core functions\n\n#### `fetch_user_posts()`\n\nFetch TikTok post metadata for a user.\n\n```python\nfrom tiktools import fetch_user_posts\nfrom pathlib import Path\n\ndata = fetch_user_posts(\n    username=\"davis_big_dawg\",\n    api_key=None,  # Uses TIKAPI_KEY env var\n    max_posts=100,  # Limit number of posts\n    output_file=Path(\"output.json\"),\n    sandbox=False,\n    update_mode=False  # Only fetch new posts\n)\n```\n\n#### `extract_transcripts()`\n\nExtract transcripts from TikTok videos using subtitle files.\n\n```python\nfrom tiktools import extract_transcripts\nfrom pathlib import Path\n\nresults = extract_transcripts(\n    posts_file=Path(\"posts.json\"),\n    output_dir=None,  # Defaults to posts_file.parent/transcripts\n    output_format=\"individual\",  # or \"combined\" or \"both\"\n    language=\"eng\",\n    update_mode=False  # Only process new posts\n)\n```\n\n#### `get_best_subtitle()`\n\nGet the best available subtitle for a post (prioritizes ASR over MT).\n\n```python\nfrom tiktools import get_best_subtitle\n\nsubtitle = get_best_subtitle(post, preferred_language=\"eng\")\nif subtitle:\n    print(f\"Found {subtitle['LanguageCodeName']} ({subtitle['Source']})\")\n```\n\n#### `download_thumbnails()`\n\nDownload video thumbnails for posts.\n\n```python\nfrom tiktools import download_thumbnails\nfrom pathlib import Path\n\nresults = download_thumbnails(\n    posts_file=Path(\"posts.json\"),\n    output_dir=None,  # Defaults to posts_file.parent/thumbnails\n    thumbnail_type=\"cover\",  # or \"origin\", \"dynamic\", \"zoom_960\"\n    update_mode=False,  # Only download for new posts\n    skip_existing=False  # Skip if file exists\n)\n```\n\n**Important:** TikTok's thumbnail URLs have time-limited signatures and expire quickly (typically within hours). For best results, download thumbnails immediately after fetching posts:\n\n```python\n# Recommended: Download thumbnails while URLs are fresh\ndata = fetch_user_posts(\n    username=\"davis_big_dawg\",\n    output_file=Path(\"data/davis_big_dawg/posts.json\"),\n    download_thumbnails=True,\n    thumbnail_type=\"origin\"\n)\n```\n\nOr use the CLI:\n```bash\npython scripts/fetch_posts.py davis_big_dawg --download-thumbnails --thumbnail-type origin\n```\n\n**Thumbnail types:**\n- `cover` - Standard cover image (default)\n- `origin` - Original quality cover (highest quality)\n- `dynamic` - Animated cover\n- `zoom_240`, `zoom_480`, `zoom_720`, `zoom_960` - Specific resolutions\n\n#### `translate_transcripts()`\n\nTranslate transcripts to multiple languages.\n\n```python\nfrom tiktools import translate_transcripts\nfrom pathlib import Path\n\nresults = translate_transcripts(\n    transcripts_file=Path(\"transcripts/user_transcripts.json\"),\n    target_languages=[\"en\", \"es\"],\n    service=\"aws\",  # AWS Translate\n    output_dir=None,  # Defaults to transcripts_file.parent\n    update_mode=False,  # Only translate new transcripts\n    estimate_only=False,  # Estimate costs without translating\n    source_language=None  # Auto-detected if None\n)\n```\n\n**Translation service setup:**\n\nFor AWS Translate, configure AWS credentials:\n```bash\n# Option 1: AWS CLI\naws configure\n\n# Option 2: Environment variables\nexport AWS_ACCESS_KEY_ID=\"your_key\"\nexport AWS_SECRET_ACCESS_KEY=\"your_secret\"\nexport AWS_DEFAULT_REGION=\"us-east-1\"\n```\n\n**Cost estimation:**\n```python\n# Estimate costs before translating\nresults = translate_transcripts(\n    transcripts_file=Path(\"transcripts/user_transcripts.json\"),\n    target_languages=[\"en\", \"es\", \"fr\"],\n    estimate_only=True\n)\nprint(f\"Estimated cost: ${results['estimated_cost']:.4f} USD\")\n```\n\n### API Client\n\n```python\nfrom tiktools import TikAPIClient\n\nclient = TikAPIClient()  # Uses TIKAPI_KEY env var\n\n# Get profile\nprofile = client.get_profile(\"davis_big_dawg\")\nprint(profile['nickname'], profile['videoCount'])\n\n# Iterate through posts\nfor post in client.get_posts(profile['secUid'], max_count=10):\n    print(post['desc'])\n```\n\n## Transcript limitations\n\nTikTok's automatic speech recognition (ASR) has some limitations:\n\n1. **Speech recognition errors**: May misinterpret words (e.g., \"Baha Blast\" → \"Brawha Blast\")\n2. **Non-speech audio**: Videos using TikTok sounds may contain song lyrics instead of speech\n\n**Recommendations**:\n- Filter by `is_original_audio: true` for speech-only content\n- Manually verify proper nouns and brand names for journalistic work\n- Check the `needs_review` flag if using AI extraction\n\n## Requirements\n\n- Python 3.8+\n- TikAPI key (get one at [tikapi.io](https://tikapi.io))\n- Optional: OpenAI API key (for AI-powered analysis examples)\n\n## Dependencies\n\nCore dependencies:\n- `tikapi` - TikTok API client\n- `requests` - HTTP requests\n- `pathlib` - File path handling\n\nOptional dependencies:\n- `boto3` - AWS Translate for translation features (install with: `pip install boto3`)\n\n## Development\n\n```bash\n# Clone the repository\ngit clone https://github.com/stiles/tiktools.git\ncd tiktools\n\n# Install in development mode\npip install -e .\n\n# Run tests\npytest tests/\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nMIT License - see LICENSE file for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstiles%2Ftiktools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstiles%2Ftiktools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstiles%2Ftiktools/lists"}