{"id":19459741,"url":"https://github.com/carperai/drlx","last_synced_at":"2025-04-07T15:08:40.879Z","repository":{"id":189232516,"uuid":"647961936","full_name":"CarperAI/DRLX","owner":"CarperAI","description":"Diffusion Reinforcement Learning Library","archived":false,"fork":false,"pushed_at":"2024-02-13T04:14:06.000Z","size":4681,"stargazers_count":184,"open_issues_count":9,"forks_count":8,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-31T11:08:16.121Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CarperAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-31T22:53:31.000Z","updated_at":"2025-03-30T13:16:55.000Z","dependencies_parsed_at":"2023-08-18T22:40:19.315Z","dependency_job_id":"9eeb0f8f-cbc2-4e37-8d56-60769242f00f","html_url":"https://github.com/CarperAI/DRLX","commit_stats":null,"previous_names":["carperai/drlx"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2FDRLX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2FDRLX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2FDRLX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2FDRLX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CarperAI","download_url":"https://codeload.github.com/CarperAI/DRLX/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247675597,"owners_count":20977376,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T17:33:55.017Z","updated_at":"2025-04-07T15:08:40.830Z","avatar_url":"https://github.com/CarperAI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Diffusion Reinforcement Learning X\n\nDRLX is a library for distributed training of diffusion models via RL. It is meant to wrap around 🤗 Hugging Face's [Diffusers](https://huggingface.co/docs/diffusers/) library and uses [Accelerate](https://huggingface.co/docs/accelerate/) for Multi-GPU and Multi-Node (as of yet untested)\n\n**News (09/27/2023): Check out our blog post with some recent experiments [here](https://carper.ai/enhancing-diffusion-models-with-reinforcement-learning/)!**\n\n📖 **[Documentation](https://DRLX.readthedocs.io)**\n\n# Setup\n\nFirst make sure you've installed [OpenCLIP](https://github.com/openai/CLIP.git). Afterwards, you can install the library from pypi:\n\n```sh\npip install drlx\n```\n\nor from source:\n\n```sh\npip install git+https://github.com/CarperAI/DRLX.git\n```\n\n# How to use\n\nCurrently we have only tested the library with Stable Diffusion 1.4, 1.5, and 2.1, but the plug and play nature of it means that realistically any denoiser from most pipelines should be usable. Models saved with DRLX are compatible with the pipeline they originated from and can be loaded like any other pretrained model. Currently the only algorithm supported for training is [DDPO](https://arxiv.org/abs/2305.13301).\n\n```python\nfrom drlx.reward_modelling.aesthetics import Aesthetics\nfrom drlx.pipeline.pickapic_prompts import PickAPicPrompts\nfrom drlx.trainer.ddpo_trainer import DDPOTrainer\nfrom drlx.configs import DRLXConfig\n\n# We import a reward model, a prompt pipeline, the trainer and config\n\npipe = PickAPicPrompts()\nconfig = DRLXConfig.load_yaml(\"configs/my_cfg.yml\")\ntrainer = DDPOTrainer(config)\n\ntrainer.train(pipe, Aesthetics())\n```\n\nAnd then to use a trained model for inference:\n\n```python\npipe = StableDiffusionPipeline.from_pretrained(\"out/ddpo_exp\")\nprompt = \"A mad panda scientist\"\nimage = pipe(prompt).images[0]\nimage.save(\"test.jpeg\")\n```\n\n# Accelerated Training\n\n```bash\naccelerate config\naccelerate launch -m [your module]\n```\n\n# Roadmap  \n\n- [x] Initial launch and DDPO\n- [x] PickScore Tuned Models\n- [ ] DPO\n- [ ] SDXL support\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarperai%2Fdrlx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcarperai%2Fdrlx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarperai%2Fdrlx/lists"}