Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sogladev/twitch-compilation-from-cluster
Create twitch clip compilations of closely related twitch streamers using clustering data, web scrape, python, ffmpeg
https://github.com/sogladev/twitch-compilation-from-cluster
Last synced: about 1 month ago
JSON representation
Create twitch clip compilations of closely related twitch streamers using clustering data, web scrape, python, ffmpeg
- Host: GitHub
- URL: https://github.com/sogladev/twitch-compilation-from-cluster
- Owner: sogladev
- Created: 2022-06-06T16:30:07.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-04-05T13:57:12.000Z (9 months ago)
- Last Synced: 2024-05-09T16:12:35.061Z (8 months ago)
- Language: Python
- Size: 117 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Create Twitch Compilations and Upload to Youtube
Table of Contents
# Twitch-Compilation-From-Cluster
Create Twitch compilations and upload to Youtube with ease.
Find clips by creators, clip ids, clip urls, game ids, category name, or a `cluster`.
A Cluster is based off Twitch Atlas (13.3MB!)
Closely related creators can be put in a cluster defined in config file and a compilation can be made
## Twitch Atlas
https://twitchatlas.com
https://twitchatlas.com/data.json
nodes
edges
jq .nodes (NA)<-- #001
--description DESCRIPTION
ex. Best twitch clips past 30 days!
--youtube_category_id YOUTUBE_CATEGORY_ID
Gaming 20, Entertainment 24, People Blogs 22
--playlist_title PLAYLIST_TITLE
Name of playlist to publish, written to JSON during title thumbnail writing
```
example
```
./main.py --project cluster_sample_30d --days 30 --duration 60 --resolution 360 --clusters cluster_sample --game_ids 509658 --description "Sample cluster clips" --youtube_category_id 24 --title Sample --dir 1
```
or use
` create_compilation_from_project.py `
```
./create_compilation_from_project.py --project sample_30d --dir 1
```
to set arguments from project configuration in `data.py`
## json
Missing "project" name from clips
```
jq 'keys' < clips.json
[
"0",
"1",
"10",
"100",
"101",
```
```
jq 'keys' < compilation.json
[
"clips",
"n",
"project",
"wd"
]
```
### GUI
See submodule or link below
https://github.com/jellemeeus/gui-compilation-from-cluster
https://github.com/jellemeeus/gui-compilation-from-cluster-vue
React
![old GUI Screenshot][gui-screenshot]
Vue
![GUI Screenshot][gui-screenshot2]
### Upload
```
./youtubeuploader-bin/youtubeuploader -filename ./proj-sample_30d-1/proj-sample_30d-1.mp4 --metaJSON ./proj-sample_30d-1/meta.json -thumbnail ./proj-sample_30d-1/thumbnail_with_icon.jpg -oAuthPort 8082
```
## Setup upload
See 'setup' at https://github.com/porjo/youtubeuploader
Requires
```./client_secrets.json```
When you create the credentials in the APIs Console, make sure you select "Desktop Application"
Add yourself to test users (email)
## Notes
Set setting vars with arguments main.py or load project settings to set arg
1. Find clips
```mermaid
flowchart LR;
twitch<-->find_and_add_clips
find_and_add_clips-->clips
twitch((Twitch API))
clips[(Clips)]
```
2. Create compilation by selecting clips
```mermaid
flowchart LR;
clips-->select_clips-->gui-->compilation
clips[(Clips)]
select_clips-->console-->compilation
compilation[(compilation.pkl)]
```
3. Download/Edit
```mermaid
flowchart LR;
download<-->select_clips
compilation-->download
compilation<-->select_clips
download-->create
compilation[(compilation.pkl)]
create([create video])
```
4. Creating the video
```mermaid
flowchart LR;
compilation-->format_download_to_input-->merge_input_to_output
disk-->format_download_to_input
merge_input_to_output-->meta
compilation[(compilation.pkl)]
disk[(.mp4 clips)]
meta([timing data])
```
5. Writing title, description, metadata
```mermaid
flowchart LR;
subgraph read data
compilation-->write_description_thumbnail
meta2-->write_description_thumbnail
twitch-->write_description_thumbnail
end
write_description_thumbnail-->meta
compilation[(compilation.pkl)]
twitch((Twitch API))
meta([thumbnail,title,description,meta.json])
meta2([timing data])
```
6. Updating publish and uploading
```mermaid
flowchart LR;
meta-->youtube_uploader-->youtube
youtube((Youtube API))
meta([thumbnail,title,description,meta.json])
compilation-->compilations--set publish flag-->clips
compilation[(compilation.pkl)]
compilations[(Compilations)]
clips[(Clips)]
```
## Config file
cfg/data.py
projects:
contains data to auto set arguments
see model/project.py
clusters:
see model/cluster.py
## Database
Clips table contains data requested by twitch API
Example table clips
|creator|url|duration|view_count|created_at|game|clipper_name|game_id|language|thumbnail_url|title|published|broken|
|-------|---|--------|----------|----------|----|------------|-------|--------|-------------|-----|---------|------|
|creator|url.tv/clip_id|24.3|5815|2021-11-08|Just Chatting|clipper|509658|en|image.jpg|title|0|0|
|creator|url.tv/clip_id2|26.0|5741|2022-01-15|Just Chatting|clipper|509658|en|image.jpg|title|0|0|
Compilations contains published clips
Example table compilations
|id|creators|urls|duration|time|project|pid|
|--|--------|----|--------|----|-------|---|
|3|c1,c2,c3,c4|url1,url2,url3,url4|633|2022-06-14|just_chatting_30d|3|
## Implementing GUI
```mermaid
flowchart LR;
selfdf-->jsonclips-->gui
Compilation--may not exist-->jsoncompilation-->gui-->compilationcsv-->Compilation*
selfdf[(self.df choices)]
jsonclips[(clips.JSON)]
jsoncompilation[(compilation.JSON)]
compilationcsv[(compilation.csv)]
```
## Download
youtube-dl
Stream for twitch clips
add `-f 720` options to download 720p (saves bandwidth)
```
360 mp4 360p 30fps
480 mp4 480p 30fps
720 mp4 720p 30fps
1080 mp4 1080p 30fps (best)
```
## Merge mp4
Faster to intermediate TransportStream files then concatenate to mp4
requires temporary TS file cleanup
```
ffmpeg -i "concat:fileIntermediate1.ts|fileIntermediate2.ts" -c copy -bsf:a aac_adtstoasc mergedVideo.mp4
```
## Add Text to top center of video
optional to add title text
```
ffmpeg -i input.mp4 -vf "drawtext=fontfile=/path/to/font.ttf:text='Stack Overflow':fontcolor=white:fontsize=48:box=1:[email protected]:boxborderw=5:x=(w-text_w)/2:y=10" -codec:a copy output.mp4
```
missing `./OpenSans-Regular.ttf` can be found here https://fontsgeek.com/fonts/Open-Sans-Regular
## Gif creation of sample video
```
ffmpeg -i input.mp4 -vf "setpts=0.2*PTS,fps=10,scale=320:-1:flags=lanczos" -c:v pam -f image2pipe - | convert -delay 10 - -loop 0 -layers optimize output.gif
```
## License
Distributed under the MIT License. See `LICENSE.txt` for more information.
[Python.org]: https://img.shields.io/badge/Python-14354C?style=for-the-badge&logo=python&logoColor=white
[Python-url]: https://www.python.org/
[React.js]: https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB
[React-url]: https://reactjs.org/
[Bootstrap.com]: https://img.shields.io/badge/Bootstrap-563D7C?style=for-the-badge&logo=bootstrap&logoColor=white
[Bootstrap-url]: https://getbootstrap.com
[Electron.js]: https://img.shields.io/badge/Electron-191970?style=for-the-badge&logo=Electron&logoColor=white
[Electron-url]: https://www.electronjs.org/
[Twitch.com]: https://img.shields.io/badge/Twitch-9146FF?style=for-the-badge&logo=twitch&logoColor=white
[Twitch-url]: https://twitch.com
[Youtube-url]: https://youtube.com
[Youtube.com]: https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge&logo=youtube&logoColor=white
[gui-screenshot]: https://github.com/jellemeeus/gui-compilation-from-cluster/raw/main/screenshot.png
[gui-screenshot2]: https://raw.githubusercontent.com/jellemeeus/gui-compilation-from-cluster-vue/main/images/gui.png