Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Kidsnd274/Av1anStaxRipWrapper
Wrapper script to use Av1an with StaxRip
https://github.com/Kidsnd274/Av1anStaxRipWrapper
aomenc av1-codec av1an ffmpeg python-script rav1e staxrip svt-av1 video
Last synced: 3 months ago
JSON representation
Wrapper script to use Av1an with StaxRip
- Host: GitHub
- URL: https://github.com/Kidsnd274/Av1anStaxRipWrapper
- Owner: Kidsnd274
- Created: 2022-12-10T17:15:46.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-06-30T17:50:14.000Z (7 months ago)
- Last Synced: 2024-08-02T20:45:43.206Z (6 months ago)
- Topics: aomenc, av1-codec, av1an, ffmpeg, python-script, rav1e, staxrip, svt-av1, video
- Language: Python
- Homepage:
- Size: 103 KB
- Stars: 16
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Av1anStaxRipWrapper
Python wrapper script to use Av1an with StaxRip
[Av1anStaxRipWrapperRav1e](README_RAV1E.md) (OUTDATED rav1e specialized script README)\
[INSTALL AND USAGE GUIDE (YouTube)](https://youtu.be/a7IPQcNVwTY)
## Contents
- [Av1anStaxRipWrapper](#av1anstaxripwrapper)
- [Contents](#contents)
- [Usage](#usage)
- [Requirements](#requirements)
- [Setup](#setup)
- [Portable Installation (Recommended)](#portable-installation-recommended)
- [Alternative Installation (install tools to system PATH)](#alternative-installation-install-tools-to-system-path)
- [Command Line Options](#command-line-options)
- [Automatic Thread Detection](#automatic-thread-detection)
- [Override Worker Count and Threads Per Local Machine](#override-worker-count-and-threads-per-local-machine)
- [Setting the override worker count](#setting-the-override-worker-count)
- [Check to see if it's working](#check-to-see-if-its-working)
- [Structure of override-workers.json](#structure-of-override-workersjson)
## Usage
This script makes use of the Command Line option in StaxRip. There are some required arguments needed in the command that allows Av1an to work with StaxRip. Namely `-i "%source_file%" -o "%encoder_out_file%" -t "%temp_dir%av1an_temp"`. `-s "%startup_dir%"` is needed if you want a portable installation. Portable installation is described in more detail at the [Setup](#setup) section.
A good starting command would be:
```
"%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\VapourSynth\python.exe" "%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\Av1anStaxRipWrapper.py" -s "%startup_dir%" -i "%source_file%" -o "%encoder_out_file%" -t "%temp_dir%av1an_temp" --pix-format yuv420p10le -e rav1e --photon-noise 2 --chroma-noise --sc-downscale-height 540 -v "--quantizer 60 --speed 6 --tiles 2 --threads 2"
```
Everything after the `-t` parameter will affect the encoding parameters (either Av1an or the selected encoder). Refer to [Command Line Options](#command-line-options) section for more information.
This starting command uses the rav1e encoder and uses the encoding parameters passed in by the argument `-v`.
## Requirements
- [ffmpeg](https://ffmpeg.org/download.html) (with shared libraries) (Av1an requirement)
- [aomenc](https://aomedia.googlesource.com/aom/) or
- [rav1e](https://github.com/xiph/rav1e/releases) or
- [SVT-AV1](https://gitlab.com/AOMediaCodec/SVT-AV1)
- [StaxRip](https://github.com/staxrip/staxrip/releases)
This script also requires the `psutil` module in Python to automatically detect CPU core counts to pass into av1an.
- You can run with the `--disable-automatic-thread-detection` flag to disable this feature and requirement
## Setup
### Portable Installation (Recommended)
Since Av1an requires `ffmpeg`, `aomenc` or `rav1e` or `SVT-AV1`, `VapourSynth` to be in PATH, this script can automatically help you add important folders (from StaxRip) temporarily to PATH when it's being run (for portable installation). In this way, your actual system PATH is not affected when running Av1an. However, this means that you need to install the tools in specific folders in StaxRip as the wrapper script looks for them there.
Portable mode is enabled when the `-s` flag is used with the StaxRip startup directory.
1. **Clone** this repository (or download the files) into `StaxRip\Apps\Encoders\Av1anStaxRipWrapper`
You can run `git clone https://github.com/Kidsnd274/Av1anStaxRipWrapper.git"` in `StaxRip\Apps\Encoders`
2. Ensure encoders, **Av1an and the wrapper script** are extracted in the right directories
- Av1an: `StaxRip\Apps\Encoders\Av1an` (IMPORTANT)
- aomenc: `StaxRip\Apps\Encoders\aomenc`
- rav1e: `StaxRip\Apps\Encoders\rav1e`
- SVT-AV1: `StaxRip\Apps\Encoders\SVT-AV1` or `StaxRip\Apps\Encoders\SVT-AV1\SvtAv1EncApp`
- FFMPEG: `StaxRip\Apps\Encoders\Av1an` (moving it to an `ffmpeg` folder would interfere with StaxRip's own ffmpeg)
- Wrapper Script: `StaxRip\Apps\Encoders\Av1anStaxRipWrapper`
3. Run the `setup_py_vp_environment.bat` script from the Av1anStaxRipWrapper folder to install required Python modules and VapourSynth plugins
4. Use the **Command Line** encoder profile to create an Av1an Encoder Profile
Good starting command (v2.40.0 and above):
```
"%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\VapourSynth\python.exe" "%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\Av1anStaxRipWrapper.py" -s "%startup_dir%" -i "%source_file%" -o "%encoder_out_file%" -t "%temp_dir%\av1an_temp" --pix-format yuv420p10le -e rav1e --photon-noise 2 --chroma-noise --sc-downscale-height 540 -v "--quantizer 60 --speed 6 --tiles 2 --threads 2"
```
Below v2.40.0
```
"%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\VapourSynth\python.exe" "%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\Av1anStaxRipWrapper.py" -s "%startup_dir%" -i "%source_file%" -o "%encoder_out_file%" -t "%temp_dir%av1an_temp" --pix-format yuv420p10le -e rav1e --photon-noise 2 --chroma-noise --sc-downscale-height 540 -v "--quantizer 60 --speed 6 --tiles 2 --threads 2"
```
**IMPORTANT**: Make sure to change **Output File Type** to `mkv`!
5. Save this Encoder Profile with the name `AV1 | av1an`, and it will appear in the AV1 drop-down menu.
### Alternative Installation (install tools to system PATH)
Basically, just make sure `Av1an`, `ffmpeg`, `aomenc`, `rav1e`, `SVT-AV1`, `Python`, `VapourSynth` are all accessible from PATH and follow the steps above from Step 2.
The command in StaxRip used is\
`python "%startup_dir%\Apps\Encoders\Av1anStaxRipWrapper\Av1anStaxRipWrapperRav1e.py" -i "%source_file%" -o "%encoder_out_file%" -t "%temp_dir%av1an_temp" --pix-format yuv420p10le -e rav1e --photon-noise 2 --chroma-noise --sc-downscale-height 540 -v "--quantizer 60 --speed 6 --tiles 2 --threads 2"`
## Command Line Options
```
options:
Av1an wrapper for StaxRip
optional arguments:
-h, --help show this help message and exit
--version Print Av1an, ffmpeg and the encoders' versions
-i INPUT Input File (for StaxRip)
-o OUTPUT Output File (for StaxRip)
-t TEMPDIR Temp Directory (for StaxRip)
-s STAXRIP_STARTUP_DIR, --staxrip-startup-dir STAXRIP_STARTUP_DIR
Specify StaxRip Startup Directory so that the wrapper script will automatically add important
folders to PATH for av1an to detect (only needed for portable installations)
# Av1an parameters
-e ENCODER, --encoder ENCODER
Video encoder to use. [default: aom][possible values: aom, rav1e, vpx, svt-av1, x264, x265]
-v ENCODER_ARGS, --video-params ENCODER_ARGS
Arguments passed to video encoder
-a OTHER_ARGS, --other-args OTHER_ARGS
Other Av1an arguments
-f FFMPEG_OPTIONS, --ffmpeg FFMPEG_OPTIONS
FFmpeg filter options
--photon-noise PHOTON_NOISE
Generates a photon noise table and applies it using grain synthesis [strength: 0-64] (disabled
by default) (Av1an parameter)
--chroma-noise Adds chroma grain synthesis to the grain table generated by `--photon-noise`. (Default: false)
(Av1an parameter)
--sc-downscale-height SC_DOWNSCALE_HEIGHT
Optional downscaling for scene detection. By default, no downscaling is performed. (Av1an
parameter)
--pix-format PIX_FORMAT
FFmpeg pixel format
# Threading parameters (Using any of these would disable the wrapper's Automatic Thread Detection feature)
--workers WORKERS Number of workers to spawn [0 = automatic] (Av1an Paramter)
--set-thread-affinity SET_THREAD_AFFINITY
Pin each worker to a specific set of threads of this size (disabled by default) (Av1an
parameter)
--disable-automatic-thread-detection
Disable the wrapper's automatic thread detection
--set-worker-override
Set the override workers count and thread affinity count for local computer
```
## Automatic Thread Detection
In my testing, I found that I get the best utilization and encoding speeds when creating the same number of workers as the number of physical cores available in your system. Along with having the number of threads per worker be 2 if your system supports hyperthreading/SMT and 1 if your system does not.
With automatic thread detection, the wrapper script will automatically detect and pass in the relevant parameters to Av1an. For example,
```
Ryzen 7 5800X
Physical Cores: 8 cores
Logical Cores: 16 cores
Hyperthreading/SMT: Enabled
Threading Parameters: --workers 8 --set-thread--affinity 2
Intel i5-6600K
Physical Cores: 4 cores
Logical Cores: 4 cores
Hyperthreading/SMT: Disabled
Threading Parameters: --workers 4 --set-thread-affinity 1
```
If your system uses an Intel 12th Gen chip and above (with the new hybrid architecture with P-cores and E-cores), this feature is disabled.
If you do not want to use this feature, you can use the `--disable-automatic-thread-detection` flag or any of the Threading parameters under [Command Line Options](#command-line-options). Using any the Threading parameters would disable this feature.
## Override Worker Count and Threads Per Local Machine
When using StaxRip in a networked environment (where multiple computers access the same StaxRip install), you might want to override the Automatic Thread Detection feature for some computers, but not affect other machines. This feature allows you to do so without modifying the command line option, by storing a configuration file in a local computer.
* Note that this will override the `--workers` and `--set-thread-affinity` flags for the script. If you include these flags in the `--other-args` flag, both of them will be sent to Av1an. Not sure what will happen there but you are welcome to find out.
### Setting the override worker count
To set the override feature, run the script with the `--set-worker-override` flag.
```
python Av1anStaxRipWrapper.py --set-worker-override
```
Follow the instructions shown in the cmd window and the script should store a configuration file in this location. `%LOCALAPPDATA%\Av1anStaxRipWrapper\override-workers.json`
### Check to see if it's working
The next time a job is being run, the script will show in its logs that the override file is found. For example:
```
=================================================
Av1anStaxRipWrapper
https://github.com/Kidsnd274/Av1anStaxRipWrapper
=================================================
[INFO] Found override-workers.json
[INFO] Overriding CPU Workers = 4 and CPU Thread Affinity = 2
[INFO] Automatic Thread Detection Disabled
THREADING INFORMATION:
Automatic Thread Detection: DISABLED
Starting av1an... Check new console window for progress
```
### Structure of override-workers.json
The json file stores two keys for worker count and thread affinity (at the moment both keys must exist). The values must be in integers.
Example:
```
{
"cpu_workers": 4,
"cpu_thread_affinity": 4
}
```