{"id":13811987,"url":"https://github.com/riffusion/riffusion-hobby","last_synced_at":"2026-01-12T01:58:01.884Z","repository":{"id":64836593,"uuid":"570696986","full_name":"riffusion/riffusion-hobby","owner":"riffusion","description":"Stable diffusion for real-time music generation","archived":false,"fork":false,"pushed_at":"2024-07-22T18:09:39.000Z","size":8453,"stargazers_count":3862,"open_issues_count":71,"forks_count":466,"subscribers_count":42,"default_branch":"main","last_synced_at":"2026-01-11T18:34:36.824Z","etag":null,"topics":["ai","audio","diffusers","diffusion","music","stable-diffusion"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/riffusion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-25T21:16:13.000Z","updated_at":"2026-01-11T15:33:18.000Z","dependencies_parsed_at":"2024-07-22T21:38:15.545Z","dependency_job_id":"3496f679-753f-470a-bd69-c8b83fef42a4","html_url":"https://github.com/riffusion/riffusion-hobby","commit_stats":null,"previous_names":["riffusion/riffusion-hobby","riffusion/riffusion"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/riffusion/riffusion-hobby","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riffusion%2Friffusion-hobby","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riffusion%2Friffusion-hobby/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riffusion%2Friffusion-hobby/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riffusion%2Friffusion-hobby/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/riffusion","download_url":"https://codeload.github.com/riffusion/riffusion-hobby/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riffusion%2Friffusion-hobby/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28331431,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T00:36:25.062Z","status":"ssl_error","status_checked_at":"2026-01-12T00:36:15.229Z","response_time":60,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","audio","diffusers","diffusion","music","stable-diffusion"],"created_at":"2024-08-04T04:00:44.165Z","updated_at":"2026-01-12T01:58:01.869Z","avatar_url":"https://github.com/riffusion.png","language":"Python","funding_links":[],"categories":["Music \u0026 Audio","GitHub projects","Python","Music Generation \u0026 AI"],"sub_categories":["Text-to-Speech"],"readme":"# :guitar: Riffusion (hobby)\n\n:no_entry: This project is no longer actively maintained.\n\n\u003ca href=\"https://github.com/riffusion/riffusion/actions/workflows/ci.yml?query=branch%3Amain\"\u003e\u003cimg alt=\"CI status\" src=\"https://github.com/riffusion/riffusion/actions/workflows/ci.yml/badge.svg\" /\u003e\u003c/a\u003e\n\u003cimg alt=\"Python 3.9 | 3.10\" src=\"https://img.shields.io/badge/Python-3.9%20%7C%203.10-blue\" /\u003e\n\u003ca href=\"https://github.com/riffusion/riffusion/tree/main/LICENSE\"\u003e\u003cimg alt=\"MIT License\" src=\"https://img.shields.io/badge/License-MIT-yellowgreen\" /\u003e\u003c/a\u003e\n\nRiffusion is a library for real-time music and audio generation with stable diffusion.\n\nRead about it at https://www.riffusion.com/about and try it at https://www.riffusion.com/.\n\nThis is the core repository for riffusion image and audio processing code.\n\n * Diffusion pipeline that performs prompt interpolation combined with image conditioning\n * Conversions between spectrogram images and audio clips\n * Command-line interface for common tasks\n * Interactive app using streamlit\n * Flask server to provide model inference via API\n * Various third party integrations\n\nRelated repositories:\n* Web app: https://github.com/riffusion/riffusion-app\n* Model checkpoint: https://huggingface.co/riffusion/riffusion-model-v1\n\n## Citation\n\nIf you build on this work, please cite it as follows:\n\n```\n@article{Forsgren_Martiros_2022,\n  author = {Forsgren, Seth* and Martiros, Hayk*},\n  title = {{Riffusion - Stable diffusion for real-time music generation}},\n  url = {https://riffusion.com/about},\n  year = {2022}\n}\n```\n\n## Install\n\nTested in CI with Python 3.9 and 3.10.\n\nIt's highly recommended to set up a virtual Python environment with `conda` or `virtualenv`:\n```\nconda create --name riffusion python=3.9\nconda activate riffusion\n```\n\nInstall Python dependencies:\n```\npython -m pip install -r requirements.txt\n```\n\nIn order to use audio formats other than WAV, [ffmpeg](https://ffmpeg.org/download.html) is required.\n```\nsudo apt-get install ffmpeg          # linux\nbrew install ffmpeg                  # mac\nconda install -c conda-forge ffmpeg  # conda\n```\n\nIf torchaudio has no backend, you may need to install `libsndfile`. See [this issue](https://github.com/riffusion/riffusion/issues/12).\n\nIf you have an issue, try upgrading [diffusers](https://github.com/huggingface/diffusers). Tested with 0.9 - 0.11.\n\nGuides:\n* [Simple Install Guide for Windows](https://www.reddit.com/r/riffusion/comments/zrubc9/installation_guide_for_riffusion_app_inference/)\n\n## Backends\n\n### CPU\n`cpu` is supported but is quite slow.\n\n### CUDA\n`cuda` is the recommended and most performant backend.\n\nTo use with CUDA, make sure you have torch and torchaudio installed with CUDA support. See the\n[install guide](https://pytorch.org/get-started/locally/) or\n[stable wheels](https://download.pytorch.org/whl/torch_stable.html).\n\nTo generate audio in real-time, you need a GPU that can run stable diffusion with approximately 50\nsteps in under five seconds, such as a 3090 or A10G.\n\nTest availability with:\n\n```python3\nimport torch\ntorch.cuda.is_available()\n```\n\n### MPS\nThe `mps` backend on Apple Silicon is supported for inference but some operations fall back to CPU,\nparticularly for audio processing. You may need to set\n`PYTORCH_ENABLE_MPS_FALLBACK=1`.\n\nIn addition, this backend is not deterministic.\n\nTest availability with:\n\n```python3\nimport torch\ntorch.backends.mps.is_available()\n```\n\n## Command-line interface\n\nRiffusion comes with a command line interface for performing common tasks.\n\nSee available commands:\n```\npython -m riffusion.cli -h\n```\n\nGet help for a specific command:\n```\npython -m riffusion.cli image-to-audio -h\n```\n\nExecute:\n```\npython -m riffusion.cli image-to-audio --image spectrogram_image.png --audio clip.wav\n```\n\n## Riffusion Playground\n\nRiffusion contains a [streamlit](https://streamlit.io/) app for interactive use and exploration.\n\nRun with:\n```\npython -m riffusion.streamlit.playground\n```\n\nAnd access at http://127.0.0.1:8501/\n\n\u003cimg alt=\"Riffusion Playground\" style=\"width: 600px\" src=\"https://i.imgur.com/OOMKBbT.png\" /\u003e\n\n## Run the model server\n\nRiffusion can be run as a flask server that provides inference via API. This server enables the [web app](https://github.com/riffusion/riffusion-app) to run locally.\n\nRun with:\n\n```\npython -m riffusion.server --host 127.0.0.1 --port 3013\n```\n\nYou can specify `--checkpoint` with your own directory or huggingface ID in diffusers format.\n\nUse the `--device` argument to specify the torch device to use.\n\nThe model endpoint is now available at `http://127.0.0.1:3013/run_inference` via POST request.\n\nExample input (see [InferenceInput](https://github.com/hmartiro/riffusion-inference/blob/main/riffusion/datatypes.py#L28) for the API):\n```\n{\n  \"alpha\": 0.75,\n  \"num_inference_steps\": 50,\n  \"seed_image_id\": \"og_beat\",\n\n  \"start\": {\n    \"prompt\": \"church bells on sunday\",\n    \"seed\": 42,\n    \"denoising\": 0.75,\n    \"guidance\": 7.0\n  },\n\n  \"end\": {\n    \"prompt\": \"jazz with piano\",\n    \"seed\": 123,\n    \"denoising\": 0.75,\n    \"guidance\": 7.0\n  }\n}\n```\n\nExample output (see [InferenceOutput](https://github.com/hmartiro/riffusion-inference/blob/main/riffusion/datatypes.py#L54) for the API):\n```\n{\n  \"image\": \"\u003c base64 encoded JPEG image \u003e\",\n  \"audio\": \"\u003c base64 encoded MP3 clip \u003e\"\n}\n```\n\n## Tests\nTests live in the `test/` directory and are implemented with `unittest`.\n\nTo run all tests:\n```\npython -m unittest test/*_test.py\n```\n\nTo run a single test:\n```\npython -m unittest test.audio_to_image_test\n```\n\nTo preserve temporary outputs for debugging, set `RIFFUSION_TEST_DEBUG`:\n```\nRIFFUSION_TEST_DEBUG=1 python -m unittest test.audio_to_image_test\n```\n\nTo run a single test case within a test:\n```\npython -m unittest test.audio_to_image_test -k AudioToImageTest.test_stereo\n```\n\nTo run tests using a specific torch device, set `RIFFUSION_TEST_DEVICE`. Tests should pass with\n`cpu`, `cuda`, and `mps` backends.\n\n## Development Guide\nInstall additional packages for dev with `python -m pip install -r requirements_dev.txt`.\n\n* Linter: `ruff`\n* Formatter: `black`\n* Type checker: `mypy`\n\nThese are configured in `pyproject.toml`.\n\nThe results of `mypy .`, `black .`, and `ruff .` *must* be clean to accept a PR.\n\nCI is run through GitHub Actions from `.github/workflows/ci.yml`.\n\nContributions are welcome through pull requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friffusion%2Friffusion-hobby","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Friffusion%2Friffusion-hobby","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friffusion%2Friffusion-hobby/lists"}