Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Keytoyze/Mug-Diffusion

High-quality and controllable charting AI for rhythm games, modifed from stable diffusion
https://github.com/Keytoyze/Mug-Diffusion

Last synced: 29 days ago
JSON representation

High-quality and controllable charting AI for rhythm games, modifed from stable diffusion

Host: GitHub
URL: https://github.com/Keytoyze/Mug-Diffusion
Owner: Keytoyze
License: mit
Created: 2023-03-22T13:15:50.000Z (over 1 year ago)
Default Branch: master
Last Pushed: 2023-11-23T07:21:44.000Z (about 1 year ago)
Last Synced: 2024-08-04T01:19:04.220Z (4 months ago)
Language: Python
Homepage:
Size: 7.94 MB
Stars: 191
Watchers: 6
Forks: 17
Open Issues: 8
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

ai-game-devtools - MuG Diffusion

README

        



# Mug Diffusion

_🎶 A charting AI for rhythm games. 🤖_





  

    

  

  

    

  

  

    

  

  

    

  

  

    

  





  English

  |

  中文



![](asset/bg.jpg)

MuG Diffusion is a charting AI for rhythm games based on [Stable Diffusion](https://github.com/CompVis/latent-diffusion/) (one of the most powerful AIGC models) with a large modification to incorporate audio waves. Given an audio file, MuG Diffusion is able to generate high-quality diverse charts, which is aligned with the music and highly controllable. Currently, it supports 4K vertical scroll rhythm game (VSRG) only, with the following control options:

- Difficulty: supporting both [osu! star rating system](https://osu.ppy.sh/wiki/en/Beatmap/Star_rating) and [Etterna MSD system](https://etternaonline.com/).

- Style: ranked beatmaps ([osu!](https://osu.ppy.sh/)) / stable charts ([Malody](https://m.mugzone.net/)), or other beatmap styles. 

- LNs: the ratio of the number of long notes to the total.

- Patterns: supporting all patterns in Etterna MSD system, including chordjack, stamina, stream, jumpstream, handstream and technical.

MuG Diffusion aims to support other rhythm games in the future (osu!standard, 5-8K VSRG, maimai, etc), and hopes to provide a beneficial AIGC tool for all the charters and players. 

![](asset/screenshot1.png)

![](asset/screenshot2.png)

## Installation and Running

### Bundled Executable

I packaged a bundled executable containing all the dependencies and model weights in the Windows platform, which is available at:

- [Google Drive](https://drive.google.com/file/d/1-TmLsveLAjRCPwd0iwXS7V1v61MlQ7DM/view?usp=share_link)

- [hiosu](https://dl2.hiosu.com/d/kuit/MugDiffusion.zip)

Unzip the file and double click "Mug Diffusion.exe", which will open a browser interface for controlling. It takes around 30 seconds on my computer (NVidia 3050Ti, 4GB memory) to generate four charts for a 3-minute-long audio.

### Running from Source

If you use other platforms, other GPU types or want to run from source, here are the instructions.

- Install [Python](https://www.python.org/downloads/)

- Install [PyTorch](https://pytorch.org/get-started/locally/)

- Install other requirements:

```commandline

pip install -r requirements.txt

```

- Install [FFmpeg](https://ffmpeg.org/download.html), make sure that `ffmpeg` command can execute correctly.

- Download the bundled executable, and copy the file `models/ckpt/model.ckpt` and `models/ckpt/model.yaml` to `{REPOSITORY_ROOT}/models/ckpt/*`.

- Run the WebUI:

```commandline

python webui.py

```

## Model Structure and Methodology

![](asset/structure.png)

## Acknowledgement

In order to ensure the fairness and transparency of training, the dataset list is published in [here](https://mugdiffusion.keytoix.vip/dataset.html). 

Thank all the Charters / Mappers in the community. It's you who endowed MuG Diffusion with intelligence. Besides, I would like to thank the [Malody](https://m.mugzone.net/) development teams (and many other supporters that cannot be listed due to space limit TAT) for the financial support.

Thank [raber](https://github.com/zengrber) for webui development, [RiceSS](https://osu.ppy.sh/users/8271436) for logo design, and many testers for their support. 

Special thanks: 

- [kangalio](https://github.com/kangalio/): for [MinaCalc-standalone](https://github.com/kangalio/minacalc-standalone), which is a component of MSD controlling system.

- [Evening](https://github.com/Eve-ning/): for [reamberPy](https://github.com/Eve-ning/reamberPy), which provides a very intuitive visualization of generated charts.

## Credits

Charts created through MuG Diffusion are fully open source, explicitly falling under the [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) Universal Public Domain Dedication. **The model weights and charts created are non-commercial.**

Besides, all charts created by AI are tagged with `AIMode: MuG Diffusion vx.x.x` in the `[Meta]` section. **Please keep its integrity or mark it explicitly unless you modify the most of the notes, otherwise you will be at risk of abusing AI.**