Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Evil0ctal/Douyin_TikTok_Download_API
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。
https://github.com/Evil0ctal/Douyin_TikTok_Download_API
api async crawler douyin douyin-api douyin-scraper douyin-tiktok-api douyin-tiktok-download fastapi no-watermark online-parsing python pywebio scraper spider tiktok tiktok-api tiktok-scraper tiktok-signature web-scraping
Last synced: 2 months ago
JSON representation
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。
- Host: GitHub
- URL: https://github.com/Evil0ctal/Douyin_TikTok_Download_API
- Owner: Evil0ctal
- License: apache-2.0
- Created: 2021-11-07T01:35:09.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-09-26T03:34:45.000Z (4 months ago)
- Last Synced: 2024-10-28T18:07:40.326Z (3 months ago)
- Topics: api, async, crawler, douyin, douyin-api, douyin-scraper, douyin-tiktok-api, douyin-tiktok-download, fastapi, no-watermark, online-parsing, python, pywebio, scraper, spider, tiktok, tiktok-api, tiktok-scraper, tiktok-signature, web-scraping
- Language: Python
- Homepage: https://douyin.wtf
- Size: 8.44 MB
- Stars: 9,070
- Watchers: 66
- Forks: 1,414
- Open Issues: 33
-
Metadata Files:
- Readme: README.en.md
- License: LICENSE
Awesome Lists containing this project
- awesome - Evil0ctal/Douyin_TikTok_Download_API - 🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。 (Python)
- StarryDivineSky - Evil0ctal/Douyin_TikTok_Download_API
README
Douyin_TikTok_Download_API(抖音/TikTok API)
[English](./README.en.md)\|[Simplified Chinese](./README.md)
🚀"Douyin_TikTok_Download_API" is a high-performance asynchronous API that can be used out of the box[Tik Tok](https://www.douyin.com)\|[TikTok](https://www.tiktok.com)\|[Bilibili](https://www.bilibili.com)Data crawling tool supports API calling, online batch analysis and downloading.
[](LICENSE)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/releases/latest)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/stargazers)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/network/members)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues?q=is%3Aissue+is%3Aclosed)
[](https://pypi.org/project/douyin-tiktok-scraper/)[](https://pypi.org/project/douyin-tiktok-scraper/#files)[](https://pypi.org/project/douyin-tiktok-scraper/)[](https://pypi.org/project/douyin-tiktok-scraper/)
[](https://api.douyin.wtf/docs)[](https://api.tikhub.io/docs)
[](https://afdian.net/@evil0ctal)[](https://ko-fi.com/evil0ctal)[](https://www.patreon.com/evil0ctal)
## Sponsor
These sponsors have paid to be placed here,**Douyin_TikTok_Download_API**The project will always be free and open source. If you would like to become a sponsor of this project, please check out my[GitHub Sponsor Page](https://github.com/sponsors/evil0ctal)。
TikHub.io API: is the leading API provider for scraping Douyin, Xiaohongshu, TikTok, Instagram, Youtube, and more.
Trusted by the major influencer marketing and social media listening platforms.
TikTok Captcha Solver: Bypass any TikTok captcha in just two lines of code.
Scale your TikTok automation and get unblocked with SadCaptcha.
## 👻Introduction
> 🚨If you need to use a private server to run this project, please refer to:[Deployment preparations](./README.md#%EF%B8%8F%E9%83%A8%E7%BD%B2%E5%89%8D%E7%9A%84%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C%E8%AF%B7%E4%BB%94%E7%BB%86%E9%98%85%E8%AF%BB),[Docker deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%BA%8C-docker),[One-click deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%B8%80-linux)
This project is based on[PyWebIO](https://github.com/pywebio/PyWebIO)，[FastAPI](https://fastapi.tiangolo.com/)，[HTTPX](https://www.python-httpx.org/), fast and asynchronous[Tik Tok](https://www.douyin.com/)/[TikTok](https://www.tiktok.com/)Data crawling tool, and realizes online batch parsing and downloading of videos or photo albums without watermarks, data crawling API, and iOS shortcut commands without watermark downloads through the Web. You can deploy or modify this project yourself to achieve more functions, or you can call it directly in your project[scraper.py](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/Stable/scraper.py)or install an existing[pip package](https://pypi.org/project/douyin-tiktok-scraper/)As a parsing library, it is easy to crawl data, etc.....
_Some simple application scenarios:_
_Download prohibited videos, perform data analysis, download without watermark on iOS (with[Shortcut command APP that comes with iOS](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)Cooperate with the API of this project to achieve in-app downloads or read clipboard downloads), etc....._
## 🔊 V4 version notes
- If you are interested in writing this project together, please add us on WeChat`Evil0ctal`Note: Github project reconstruction, everyone can communicate and learn from each other in the group. Advertising and illegal things are not allowed. It is purely for making friends and technical exchanges.
- This project uses`X-Bogus`Algorithms and`A_Bogus`The algorithm requests the Web API of Douyin and TikTok.
- Due to Douyin's risk control, after deploying this project, please**Obtain the cookie of Douyin website in the browser and replace it in config.yaml.**
- Please read the document below before raising an issue. Solutions to most problems will be included in the document.
- This project is completely free, but when using it, please comply with:[Apache-2.0 license](https://github.com/Evil0ctal/Douyin_TikTok_Download_API?tab=Apache-2.0-1-ov-file#readme)
## 🔖TikHub.io API
[TikHub.io](https://api.tikhub.io/)It is an API platform that provides various public data interfaces including Douyin and TikTok. If you want to support[Douyin_TikTok_Download_API](https://github.com/Evil0ctal/Douyin_TikTok_Download_API)For project development, we strongly recommend that you choose[TikHub.io](https://api.tikhub.io/)。
#### Features:
> 📦 Ready to use right out of the box
Simplify the use process and use the packaged SDK to quickly carry out development work. All API interfaces are designed based on RESTful architecture and are described and documented using OpenAPI specifications, with sample parameters included to ensure easier calling.
> 💰 Cost advantage
There are no preset package restrictions and no monthly usage thresholds. All consumption is billed immediately based on actual usage, and tiered billing is performed based on the user's daily requests. At the same time, free quota can be obtained through daily sign-in in the user backend. , and these free credits will not expire.
> ⚡️ Fast support
We have a large Discord community server, where administrators and other users will quickly reply to you and help you quickly solve current problems.
> 🎉Embrace open source
Part of TikHub's source code will be open sourced on Github, and it will sponsor authors of some open source projects.
#### Link:
- Github:[TikHub Github](https://github.com/TikHubIO)
- Discord:[Tikhub discord](https://discord.com/invite/aMEAS8Xsvz)
- Register:[TikHub signup](https://beta-web.tikhub.io/en-us/users/signup)
- API Docs:[Cheers to my father, Dex](https://api.tikhub.io/)
## 🖥Demo site: I am very vulnerable...please do not stress test (·•᷄ࡇ•᷅ )
> 😾The online download function of the demo site has been turned off, and due to cookie reasons, Douyin's parsing and API services cannot guarantee availability on the Demo site.
🍔Web APP:
🍟API Document:
🌭TikHub API Document:
💾iOS Shortcut (shortcut command):[Shortcut release](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/discussions/104?sort=top)
📦️Desktop downloader (recommended by warehouse):
- [Johnserf-Seed/TikTokDownload](https://github.com/Johnserf-Seed/TikTokDownload)
- [HFrost0/bilix](https://github.com/HFrost0/bilix)
- [Tairraos/TikDown - \[needs update\]](https://github.com/Tairraos/TikDown/)
## ⚗️Technology stack
- [/app/web](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/app/web)-[PyWebIO](https://www.pyweb.io/)
- [/app/api](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/app/api)-[FastAPI](https://fastapi.tiangolo.com/)
- [/crawlers](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/crawlers)-[HTTPX](https://www.python-httpx.org/)
> **_/crawlers_**
- Submit requests to APIs on different platforms and retrieve data. After processing, a dictionary (dict) is returned, and asynchronous support is supported.
> **_/app/api_**
- Get request parameters and use`Crawlers`The related classes process the data and return it in JSON form, download the video, and cooperate with iOS shortcut commands to achieve fast calling and support asynchronous.
> **_/app/web_**
- use`PyWebIO`A simple web program created to process the values entered on the web page and then use them`Crawlers`The related class processing interface outputs related data on the web page.
**_Most of the parameters of the above files can be found in the corresponding`config.yaml`Make changes in_**
## 💡Project file structure
./Douyin_TikTok_Download_API
├─app
│ ├─api
│ │ ├─endpoints
│ │ └─models
│ ├─download
│ └─web
│ └─views
└─crawlers
├─bilibili
│ └─web
├─douyin
│ └─web
├─hybrid
├─tiktok
│ ├─app
│ └─web
└─utils
## ✨Supported functions:
- Batch parsing on the web page (supports Douyin/TikTok mixed parsing)
- Download videos or photo albums online.
- make[pip package](https://pypi.org/project/douyin-tiktok-scraper/)Conveniently and quickly import your projects
- [iOS shortcut commands to quickly call API](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)Achieve in-app download of watermark-free videos/photo albums
- Complete API documentation ([Demo/Demonstration](https://api.douyin.wtf/docs))
- Rich API interface:
- Douyin web version API
- [x] Video data analysis
- [x] Get user homepage work data
- [x] Obtain the data of works liked by the user's homepage
- [x] Obtain the data of collected works on the user's homepage
- [x] Get user homepage information
- [x] Get user collection work data
- [x] Get user live stream data
- [x] Get the live streaming data of a specified user
- [x] Get the ranking of users who give gifts in the live broadcast room
- [x] Get single video comment data
- [x] Get the comment reply data of the specified video
- [x] Generate msToken
- [x] Generate verify_fp
- [x] Generate s_v_web_id
- [x] Generate X-Bogus parameters using interface URL
- [x] Generate A_Bogus parameters using interface URL
- [x] Extract a single user id
- [x] Extract list user id
- [x] Extract a single work id
- [x] Extract list work id
- [x] Extract live broadcast room number from list
- [x] Extract live broadcast room number from list
- TikTok web version API
- [x] Video data analysis
- [x] Get user homepage work data
- [x] Obtain the data of works liked by the user's homepage
- [x] Get user homepage information
- [x] Get user home page fan data
- [x] Get user homepage follow data
- [x] Get user homepage collection work data
- [x] Get user homepage collection data
- [x] Get user homepage playlist data
- [x] Get single video comment data
- [x] Get the comment reply data of the specified video
- [x] Generate msToken
- [x] Generate ttwid
- [x] Generate X-Bogus parameters using interface URL
- [x] Extract a single user sec_user_id
- [x] Extract list user sec_user_id
- [x] Extract a single work id
- [x] Extract list work id
- [x] Get user unique_id
- [x] Get list unique_id
- Bilibili web version API
- [x] Get individual video details
- [x] Get video stream address
- [x] Obtain user-published video work data
- [x] Get all favorites information of the user
- [x] Get video data in specified favorites
- [x] Get information about a specified user
- [x] Get comprehensive popular video information
- [x] Get comments for specified video
- [x] Get the reply to the specified comment under the video
- [x] Get the specified user's updates
- [x] Get real-time video barrages
- [x] Get specified live broadcast room information
- [x] Get live room video stream
- [x] Get the anchors who are live broadcasting in the specified partition
- [x] Get a list of all live broadcast partitions
- [x] Obtain video sub-p information through bv number
* * *
## 📦Call the parsing library (obsolete and needs to be updated):
> 💡PyPi：
Install the parsing library:`pip install douyin-tiktok-scraper`
```python
import asyncio
from douyin_tiktok_scraper.scraper import Scraper
api = Scraper()
async def hybrid_parsing(url: str) -> dict:
# Hybrid parsing(Douyin/TikTok URL)
result = await api.hybrid_parsing(url)
print(f"The hybrid parsing result:\n {result}")
return result
asyncio.run(hybrid_parsing(url=input("Paste Douyin/TikTok/Bilibili share URL here: ")))
```
## 🗺️Supported submission formats:
> 💡Tip: Including but not limited to the following examples, if you encounter link parsing failure, please open a new one[issue](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)
- Douyin sharing password (copy in APP)
```text
7.43 pda:/ 让你在几秒钟之内记住我 https://v.douyin.com/L5pbfdP/ 复制此链接，打开Dou音搜索，直接观看视频！
```
- Douyin short URL (copy within APP)
```text
https://v.douyin.com/L4FJNR3/
```
- Douyin normal URL (copy from web version)
```text
https://www.douyin.com/video/6914948781100338440
```
- Douyin discovery page URL (APP copy)
```text
https://www.douyin.com/discover?modal_id=7069543727328398622
```
- TikTok short URL (copy within APP)
```text
https://www.tiktok.com/t/ZTR9nDNWq/
```
- TikTok normal URL (copy from web version)
```text
https://www.tiktok.com/@evil0ctal/video/7156033831819037994
```
- Douyin/TikTok batch URL (no need to use matching separation)
```text
https://v.douyin.com/L4NpDJ6/
https://www.douyin.com/video/7126745726494821640
2.84 nqe:/ 骑白马的也可以是公主%%百万转场变身https://v.douyin.com/L4FJNR3/ 复制此链接，打开Dou音搜索，直接观看视频！
https://www.tiktok.com/t/ZTR9nkkmL/
https://www.tiktok.com/t/ZTR9nDNWq/
https://www.tiktok.com/@evil0ctal/video/7156033831819037994
```
## 🛰️API documentation
**_API documentation:_**
local:
Online:
**_API demo:_**
- Crawl video data (TikTok or Douyin hybrid analysis)`https://api.douyin.wtf/api/hybrid/video_data?url=[视频链接/Video URL]&minimal=false`
- Download videos/photo albums (TikTok or Douyin hybrid analysis)`https://api.douyin.wtf/api/download?url=[视频链接/Video URL]&prefix=true&with_watermark=false`
**_For more demonstrations, please view the document content..._**
## ⚠️Preparation work before deployment (please read carefully):
- You need to solve the problem of crawler cookie risk control by yourself, otherwise the interface may become unusable. After modifying the configuration file, you need to restart the service for it to take effect, and it is best to use cookies from accounts that you have already logged in to.
- Douyin web cookie (obtain and replace the cookie in the configuration file below):
-
- TikTok web-side cookies (obtain and replace the cookies in the configuration file below):
-
- I turned off the online download function of the demo site. The video someone downloaded was so huge that it crashed the server. You can right-click on the web page parsing results page to save the video...
- The cookies of the demo site are my own and are not guaranteed to be valid for a long time. They only serve as a demonstration. If you deploy it yourself, please obtain the cookies yourself.
- If you need to directly access the video link returned by TikTok Web API, an HTTP 403 error will occur. Please use the API in this project.`/api/download`The interface downloads TikTok videos. This interface has been manually closed in the demo site, and you need to deploy this project by yourself.
- here is one**Video tutorial**You can refer to:**__**
## 💻Deployment (Method 1 Linux)
> 💡Tips: It is best to deploy this project to a server in the United States, otherwise strange BUGs may occur.
Recommended for everyone to use[Digitalocean](https://www.digitalocean.com/)server, because you can have sex for free.
Use my invitation link to sign up and you can get a $200 credit, and when you spend $25 on it, I can also get a $25 reward.
My invitation link:
> Use script to deploy this project with one click
- This project provides a one-click deployment script that can quickly deploy this project on the server.
- The script was tested on Ubuntu 20.04 LTS. Other systems may have problems. If there are any problems, please solve them yourself.
- Download using wget command[install.sh](https://raw.githubusercontent.com/Evil0ctal/Douyin_TikTok_Download_API/main/bash/install.sh)to the server and run
wget -O install.sh https://raw.githubusercontent.com/Evil0ctal/Douyin_TikTok_Download_API/main/bash/install.sh && sudo bash install.sh
> Start/stop service
- Use the following commands to control running or stopping the service:
- `sudo systemctl start Douyin_TikTok_Download_API.service`
- `sudo systemctl stop Douyin_TikTok_Download_API.service`
> Turn on/off automatic operation at startup
- Use the following commands to set the service to run automatically at boot or cancel automatic run at boot:
- `sudo systemctl enable Douyin_TikTok_Download_API.service`
- `sudo systemctl disable Douyin_TikTok_Download_API.service`
> Update project
- When the project is updated, ensure that the update script is executed in the virtual environment and all dependencies are updated. Enter the project bash directory and run update.sh:
- `cd /www/wwwroot/Douyin_TikTok_Download_API/bash && sudo bash update.sh`
## 💽Deployment (Method 2 Docker)
> 💡Tip: Docker deployment is the simplest deployment method and is suitable for users who are not familiar with Linux. This method is suitable for ensuring environment consistency, isolation and quick setup.
> Please use a server that can normally access Douyin or TikTok, otherwise strange BUG may occur.
### Preparation
Before you begin, make sure Docker is installed on your system. If you haven't installed Docker yet, you can install it from[Docker official website](https://www.docker.com/products/docker-desktop/)Download and install.
### Step 1: Pull the Docker image
First, pull the latest Douyin_TikTok_Download_API image from Docker Hub.
```bash
docker pull evil0ctal/douyin_tiktok_download_api:latest
```
Can be replaced if needed`latest`Label the specific version you need to deploy.
### Step 2: Run the Docker container
After pulling the image, you can start a container from this image. Here are the commands to run the container, including basic configuration:
```bash
docker run -d --name douyin_tiktok_api -p 80:80 evil0ctal/douyin_tiktok_download_api
```
Each part of this command does the following:
- `-d`: Run the container in the background (detached mode).
- `--name douyin_tiktok_api `: Name the container`douyin_tiktok_api `。
- `-p 80:80`: Map port 80 on the host to port 80 of the container. Adjust the port number based on your configuration or port availability.
- `evil0ctal/douyin_tiktok_download_api`: The name of the Docker image to use.
### Step 3: Verify the container is running
Check if your container is running using the following command:
```bash
docker ps
```
This will list all active containers. Find`douyin_tiktok_api `to confirm that it is functioning properly.
### Step 4: Access the App
Once the container is running, you should be able to pass`http://localhost`Or API client access Douyin_TikTok_Download_API. Adjust the URL if a different port is configured or accessed from a remote location.
### Optional: Custom Docker commands
For more advanced deployments, you may wish to customize Docker commands to include environment variables, volume mounts for persistent data, or other Docker parameters. Here is an example:
```bash
docker run -d --name douyin_tiktok_api -p 80:80 \
-v /path/to/your/data:/data \
-e MY_ENV_VAR=my_value \
evil0ctal/douyin_tiktok_download_api
```
- `-v /path/to/your/data:/data`: Change the`/path/to/your/data`Directory mounted to the container`/data`Directory for persisting or sharing data.
- `-e MY_ENV_VAR=my_value`: Set environment variables within the container`MY_ENV_VAR`, whose value is`my_value`。
### Configuration file modification
Most of the configuration of the project can be found in the following directories:`config.yaml`File modification:
- `/crawlers/douyin/web/config.yaml`
- `/crawlers/tiktok/web/config.yaml`
- `/crawlers/tiktok/app/config.yaml`
### Step 5: Stop and remove the container
When you need to stop and remove a container, use the following commands:
```bash
# Stop
docker stop douyin_tiktok_api
# Remove
docker rm douyin_tiktok_api
```
## 📸Screenshot
**_API speed test (compared to official API)_**
🔎点击展开截图
Douyin official API:
API of this project:
TikTok官方API:
API of this project:
**_Project interface_**
🔎点击展开截图
Web main interface:
Web main interface:
## 📜 Star History
[](https://star-history.com/#Evil0ctal/Douyin_TikTok_Download_API&Timeline)
[Apache-2.0 license](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/Stable/LICENSE)
> Start: 2021/11/06
> GitHub:[@Evil0ctal](https://github.com/Evil0ctal)