{"id":27948365,"url":"https://github.com/pku-alignment/align-anything","last_synced_at":"2025-05-14T15:00:39.041Z","repository":{"id":248375264,"uuid":"828505099","full_name":"PKU-Alignment/align-anything","owner":"PKU-Alignment","description":"Align Anything: Training All-modality Model with Feedback","archived":false,"fork":false,"pushed_at":"2025-05-01T07:39:06.000Z","size":113741,"stargazers_count":3601,"open_issues_count":24,"forks_count":418,"subscribers_count":261,"default_branch":"main","last_synced_at":"2025-05-07T14:57:57.925Z","etag":null,"topics":["chameleon","dpo","large-language-models","multimodal","rlhf","vision-language-model"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2412.15838","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PKU-Alignment.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-14T11:05:19.000Z","updated_at":"2025-05-07T14:33:05.000Z","dependencies_parsed_at":"2024-09-12T11:30:27.310Z","dependency_job_id":"1c73b7d1-5db3-466d-a530-fc34a7a997b1","html_url":"https://github.com/PKU-Alignment/align-anything","commit_stats":null,"previous_names":["pku-alignment/align-anything"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2Falign-anything","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2Falign-anything/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2Falign-anything/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2Falign-anything/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PKU-Alignment","download_url":"https://codeload.github.com/PKU-Alignment/align-anything/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254168217,"owners_count":22026120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chameleon","dpo","large-language-models","multimodal","rlhf","vision-language-model"],"created_at":"2025-05-07T14:57:55.216Z","updated_at":"2025-05-14T15:00:35.527Z","avatar_url":"https://github.com/PKU-Alignment.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- markdownlint-disable first-line-h1 --\u003e\r\n\u003c!-- markdownlint-disable html --\u003e\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n  \u003cimg src=\"assets/logo.jpg\" width=\"390\"/\u003e\r\n  \u003cdiv\u003e\u0026nbsp;\u003c/div\u003e\r\n  \u003cdiv align=\"center\"\u003e\r\n    \u003cb\u003e\u003cfont size=\"5\"\u003eproject website\u003c/font\u003e\u003c/b\u003e\r\n    \u003csup\u003e\r\n      \u003ca href=\"https://space.bilibili.com/3493095748405551?spm_id_from=333.337.search-card.all.click\"\u003e\r\n        \u003ci\u003e\u003cfont size=\"4\"\u003eHOT\u003c/font\u003e\u003c/i\u003e\r\n      \u003c/a\u003e\r\n    \u003c/sup\u003e\r\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\r\n    \u003cb\u003e\u003cfont size=\"5\"\u003ePKU-Alignment Team\u003c/font\u003e\u003c/b\u003e\r\n    \u003csup\u003e\r\n      \u003ca href=\"https://space.bilibili.com/3493095748405551?spm_id_from=333.337.search-card.all.click\"\u003e\r\n        \u003ci\u003e\u003cfont size=\"4\"\u003ewelcome\u003c/font\u003e\u003c/i\u003e\r\n      \u003c/a\u003e\r\n    \u003c/sup\u003e\r\n  \u003c/div\u003e\r\n  \u003cdiv\u003e\u0026nbsp;\u003c/div\u003e\r\n\r\n[![PyPI](https://img.shields.io/pypi/v/align-anything?logo=pypi)](https://pypi.org/project/align-anything)\r\n[![License](https://img.shields.io/github/license/PKU-Alignment/align-anything?label=license)](#license)\r\n\r\n[📘Documentation](https://align-anything.readthedocs.io/) |\r\n[🛠️Quick Start](#quick-start) |\r\n[🚀Algorithms](#algorithms) |\r\n[👀Evaluation](https://github.com/PKU-Alignment/eval-anything) |\r\n[🤔Reporting Issues](#report-issues)\r\n\r\n\u003c/div\u003e\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n\r\n[Our All-Modality Alignment Datasets](https://huggingface.co/datasets/PKU-Alignment/align-anything)\r\n\r\n\u003c/div\u003e\r\n\r\nAlign-Anything aims to align any modality large models (any-to-any models) with human intentions and values. \r\n\r\n- **Highly Modular Framework** allowing users to easily modify and customize the code for different tasks (see [framework design](https://align-anything.readthedocs.io/)).\r\n- **Various Modality Model Fine-Tuning** for diverse multi-modal (image/video/audio) models (see [scripts](./scripts)).\r\n- **Different Alignment Methods.** Different alignment algorithms, including SFT, DPO, PPO, and others.\r\n- **Multi-Modal CLI.** Multi-modal CLI for image, audio, and video modalities (see [multi-modal CLI](#multi-modal-cli)).\r\n- **O1-like Training.** O1-like training based on [DollyTails](https://huggingface.co/datasets/PKU-Alignment/DollyTails-12K) (see [scripts/llama_sft_o1.sh](./scripts)).\r\n- **Rule-based RL.** Rule-based RL encouraged by [Deepseek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1).\r\n\r\n**Note:** We provide a [quick start guide](https://align-anything.readthedocs.io/) for users to quickly get the code structure and development details.\r\n\r\n## 📣 News\r\n\r\n- **[Future]** ⚡️⚡️⚡️ We will integrate more cutting-edge MLLMs into our framework and develop our VLA part. Stay tuned!\r\n\r\n- **[2025.04.14]** 📜📜📜 We release the tutorial on SFT training for `text-image-to-text` models. Check out the [cookbook_en](./cookbooks/en/text_image_to_text_sft.ipynb) (for English) and [cookbook_zh](./cookbooks/zh/text_image_to_text_sft.ipynb) (for Chinese).\r\n\r\n- **[2025.04.07]** 🥳🥳🥳 Align-Anything now serves as the homework platform for the PKU course [Large Language Models Basics and Alignment](https://pku-llm.ai/), supporting on both Nvidia GPU and Huawei Ascend NPU. The corresponding tutorial will be released soon!\r\n\r\n\u003e Align-Anything目前已成为北京大学本硕博课程《大模型基础与对齐》的课程作业平台，支持在Nvidia GPU和华为昇腾NPU上进行训练与评估。对应教程将持续发布！\r\n\r\n- **[2025.03.31]** ✅✅✅ We enhance the installation process for both Nvidia GPU and Huawei Ascend NPU. Please refer to the [Quick Start](#quick-start) for details.\r\n\r\n- **[2025.03.31]** 🛠️🛠️🛠️ We separate the evaluation component from align-anything and establish [eval-anything](https://github.com/PKU-Alignment/eval-anything) as a dedicated repository for large-scale evaluation of any-to-any models.\r\n\r\n- **[2025.03.31]** 🚀🚀🚀 We support wrapping the `actor` model with [vLLM engine](https://github.com/vllm-project/vllm) for sequence generation in `text-to-text ppo` training. It greatly accelerates the ppo training process. Our results show that with vLLM engine, it only takes 22 minutes to finish ppo, while the baseline case needs ~150 minutes.\r\n\r\n    \u003e 😊 Our implementation is encouraged by [OpenRLHF](https://github.com/OpenRLHF/OpenRLHF), which is a great project for RLHF training.\r\n\r\n- **[2025.03.27]** 📜📜📜 We release the tutorial on DPO training for `text-to-text` models. Check out the [cookbook_en](./cookbooks/en/text_to_text_dpo.ipynb) (for English) and [cookbook_zh](./cookbooks/zh/text_to_text_dpo.ipynb) (for Chinese).\r\n\r\n- **[2025.03.15]** 📜📜📜 We release the tutorial for extending modality from `text-to-text` to `text-image-to-text` models. Check out the [cookbook_en](./cookbooks/en/modality_scaling.ipynb) (for English) and [cookbook_zh](./cookbooks/zh/modality_scaling.ipynb) (for Chinese).\r\n\r\n  \u003e We will release other tutorials in the future. Stay tuned! 😊\r\n\r\n- **[2025.03.15]** We have supported seamless migration to Slurm clusters! Check out our example [here](#training-on-slurm) to get started.\r\n\r\n- **[2025.03.14]** 🛠️🛠️🛠️ We have supported Safe RLHF-V for `Text + Image -\u003e Text` modality models.\r\n\r\n- **[2025.03.12]** 🛠️🛠️🛠️ We have supported resume training for DPO and SFT, see [here](https://github.com/PKU-Alignment/align-anything/pull/153).\r\n\r\n- **[2025.03.11]** 🎉🎉🎉 We support the installation of **Huawei Ascend** dependencies through pre-set Docker image.\r\n\r\n- **[2025.03.02]** 🎉🎉🎉 We have implemented alignment training for Vision-Language-Action Models in embodied intelligence, see [VLA Trainer](https://github.com/PKU-Alignment/align-anything/tree/main/align_anything/trainers/text_video_to_action), with more features coming soon!\r\n\r\n- **[2025.02.28]** 🤝🤝🤝 We supported the training and inference of align-anything on Huawei Ascend NPU.\r\n\r\n  \u003e 近期 align-anything 团队正在和华为昇腾团队积极联合开发，基于 VLLMs-Ascend 上的全模态推理和对齐微调。\r\n\r\n\r\n\u003cdetails\u003e\u003csummary\u003eMore News\u003c/summary\u003e\r\n\r\n- **[2025.02.28]** 🤗🤗🤗 We open-sourced [🤗Align-DS-V](https://huggingface.co/PKU-Alignment/Align-DS-V), an experimental vision-language model based on [DeepSeek-R1-Distill-Llama-8B](https://github.com/deepseek-ai/DeepSeek-R1), which enhances reasoning by incorporating additional modalities into the language model. The model has already surpassed **18,000+** downloads!\r\n- **[2025.02.28]** We supported the alignment fine-tuning of DeepSeek’s Unified Multimodal Understanding and Generation Models, as well as the SFT and DPO of the [**Janus-Series**](https://github.com/deepseek-ai/Janus). You can find the examples in the `./scripts` and `./projects/janus` directories.\r\n- **[2025.02.19]** We supported the alignment methods **GRPO** used in DeepSeek R1. See [GRPO Trainer](https://github.com/PKU-Alignment/align-anything/blob/main/align_anything/trainers/text_to_text/grpo.py).\r\n- **[2025.01.21]** We supported the alignment fine-tuning of **MiniCPM-o** (audio \u0026 image), also included in [the official repository’s README recommendations](https://github.com/OpenBMB/MiniCPM-o#with-align-anything-).\r\n- **[2025.01.17]** 🔥🔥🔥 We supported the fine-tuning of **O1-like reasoning in the text2text modality** (see [DollyTails](https://huggingface.co/datasets/PKU-Alignment/DollyTails-12K)), with multimodal and additional modalities coming soon!\r\n- **[2024.10.11]** We supported the alignment fine-tuning of the latest **Emu3** model.\r\n- **[2024.08.29]** 💡💡💡 We supported learning from language feedback (different from binary feedback). For more details, see [lang-feedback](https://github.com/PKU-Alignment/align-anything/tree/main/projects/lang_feedback).\r\n- **[2024.10.10]** We support SFT for `Any -\u003e Any` modality models Emu3.\r\n- **[2024.09.24]** We support SFT, DPO, RM and PPO for `Text + Video -\u003e Text` modality models.\r\n- **[2024.09.13]** We support SFT, DPO, RM and PPO for `Text + Audio -\u003e Text` modality models.\r\n- **[2024.08.17]** We support DPO and PPO for `Text+Image -\u003e Text+Image` modality models.\r\n- **[2024.08.15]** We support a new function in the evaluation module: the `models_pk` script in [here](./scripts/models_pk.sh), which enables comparing the performance of two models across different benchmarks.\r\n- **[2024.08.06]** We restructure the framework to support any modality evaluation and the supported benchmark list is [here](https://github.com/PKU-Alignment/align-anything/tree/main/align_anything/evaluation/benchmarks).\r\n- **[2024.08.06]** We support `Text+Image -\u003e Text+Image` modality for the SFT trainer and Chameleon models.\r\n- **[2024.07.23]** We support `Text -\u003e Image`, `Text -\u003e Audio`, and `Text -\u003e Video` modalities for the SFT trainer and DPO trainer.\r\n- **[2024.07.22]** We support the **Chameleon** model for the SFT trainer and DPO trainer!\r\n- **[2024.07.17]** We open-source the Align-Anything-Instruction-100K dataset for text modality. This dataset is available in both [English](https://huggingface.co/datasets/PKU-Alignment/Align-Anything-Instruction-100K) and [Chinese](https://huggingface.co/datasets/PKU-Alignment/Align-Anything-Instruction-100K-zh) versions, each sourced from different data sets and meticulously refined for quality by GPT-4.\r\n- **[2024.07.14]** We open-source the align-anything framework.\r\n\r\n\u003c/details\u003e\r\n\r\n## Quick Start\r\n\r\n### Easy Installation\r\n\r\n```bash\r\n# clone the repository\r\ngit clone git@github.com:PKU-Alignment/align-anything.git\r\ncd align-anything\r\n\r\n# create virtual env\r\nconda create -n align-anything python==3.11\r\nconda activate align-anything\r\n```\r\n\r\n#### On Nvidia GPU\r\n\r\n- **`[Optional]`** We recommend installing [CUDA](https://anaconda.org/nvidia/cuda) in the conda environment and set the environment variable.\r\n\r\n```bash\r\n# We tested on the H800 computing cluster, and this version of CUDA works well.\r\n# You can adjust this version according to the actual situation of the computing cluster.\r\n\r\nconda install nvidia/label/cuda-12.2.0::cuda\r\nexport CUDA_HOME=$CONDA_PREFIX\r\n```\r\n\r\n\u003e If your CUDA installed in a different location, such as `/usr/local/cuda/bin/nvcc`, you can set the environment variables as follows:\r\n\r\n```bash\r\nexport CUDA_HOME=\"/usr/local/cuda\"\r\n```\r\n\r\nFinally, install `align-anything` by:\r\n\r\n```bash\r\npip3 install -e .\r\n\r\npip3 install vllm==0.7.2 # to run ppo on vllm engine\r\n```\r\n\r\n#### On Huawei Ascend NPU\r\n\r\nYou can build on Huawei Ascend NPU by simply:\r\n\r\n```bash\r\npip3 install -e .[ascend]\r\n```\r\n\r\nThe current test environment for Ascend is:\r\n\r\n- Python 3.10.6\r\n- CANN 8.0.rc3\r\n- Architecture: aarch64\r\n- Hardware: 8x Ascend-SNT9B ARM (192 cores, 1536GB memory)\r\n\r\n\u003cdetails\u003e\r\n  \u003csummary\u003e[Optional] Install ascend dependencies using our docker image\u003c/summary\u003e\r\n\r\n1. **Current Ascend Machine Environment Configuration**\r\n   The current environment configuration for the Ascend Machine is as follows:\r\n\r\n   ```\r\n   - Python version: 3.10.6\r\n   - CANN version: 8.0.rc3\r\n   - Architecture: aarch64\r\n   - Hardware: 8x Ascend-SNT9B ARM (192 cores, 1536GB memory)\r\n   - Ascend Driver Version: 23.0.7\r\n   - AscendHAL Version: 7.35.19\r\n   - AICPU Version: 1.0\r\n   - TDT Version: 1.0\r\n   - Log Version: 1.0\r\n   - Profiler Version: 2.0\r\n   - DVPP Kernels Version: 1.1\r\n   - TSFW Version: 1.0\r\n   - Inner Version: V100R001C15SPC012B220\r\n   - Compatible Versions: V100R001C30, V100R001C13, V100R001C15\r\n   - Compatible Firmware Versions: [7.0.0, 7.1.99]\r\n   - Package Version: 23.0.7\r\n   ```\r\n\r\n2. **Create the Docker Container**\r\n   To get started with the pre-configured environment, you can use the `setup_docker.sh` script located in the `./scripts` directory to pull the Docker image and create a container with all necessary environments set up:\r\n\r\n   ```\r\n   cd scripts\r\n   bash setup_docker.sh\r\n   ```\r\n\r\n   This will automatically pull the Docker image and create a Docker container where all the dependencies and configurations for running the framework are already set up.\r\n\r\n3. **Warning**\r\n   **Environment Compatibility**: The environment mentioned above is tested and verified to work. If you attempt to run the setup on other environments, you may encounter issues. In such cases, you will need to perform debugging and adjustments yourself to ensure compatibility with your specific environment.\r\n\r\n\u003c/details\u003e\r\n\r\n\r\nIf you encounter any issues, please refer to the [FAQ](https://github.com/PKU-Alignment/align-anything/discussions/167) for solutions.\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003e[Optional] Other Dependencies\u003c/summary\u003e\r\n\r\n- `pip install -e .[text-to-audio]`: Install the text-to-audio dependencies.\r\n- `pip install -e .[minicpmv]`: Install the minicpmv dependencies.\r\n- `pip install -e .[minicpmo]`: Install the minicpmo dependencies.\r\n\r\n\u003c/details\u003e\r\n\r\n### Training\r\n\r\nWe provide some scripts for quick start, you can find them in the `./scripts` directory. These scripts would automatically download the model and dataset, and run the training or evaluation.\r\n\r\nFor example, `scripts/llava/llava_dpo.sh` is the script for `Text + Image -\u003e Text` modality, you can run it by:\r\n\r\n```bash\r\ncd scripts\r\nbash llava/llava_dpo.sh\r\n```\r\n\r\n**Note:** The scripts will automatically download the model and dataset from huggingface. If you are prohibited from the internet, please try to use the `HF Mirror`:\r\n\r\n```bash\r\nexport HF_ENDPOINT=https://hf-mirror.com\r\n```\r\n\r\n#### Training on Slurm\r\n\r\n\u003e We fully support seamless migration to Slurm. If you plan to run training on a Slurm-managed cluster, we invite you to use our example Slurm training script:\r\n\u003e\r\n\u003e ```bash\r\n\u003e cd scripts\r\n\u003e bash slurm/slurm_llava_dpo.sh\r\n\u003e ```\r\n\u003e\r\n\u003e This script is pre-configured with suitable Slurm parameters. You only need to adjust the settings (such as the `job name`, `partition`, `account`, `path` and `resource allocations`) to match your cluster configuration.\r\n\r\n## Algorithms\r\n\r\nWe support basic alignment algorithms for different modalities, each of which may involve additional algorithms. For instance, in the text modality, we have also implemented SimPO, KTO, and others.\r\n\r\n| Modality                           | SFT | RM  | DPO | PPO |\r\n| ---------------------------------- | --- | --- | --- | --- |\r\n| `Text -\u003e Text (t2t)`               | ✔️  | ✔️  | ✔️  | ✔️  |\r\n| `Text+Image -\u003e Text (ti2t)`        | ✔️  | ✔️  | ✔️  | ✔️  |\r\n| `Text+Image -\u003e Text+Image (ti2ti)` | ✔️  | ✔️  | ✔️  | ✔️  |\r\n| `Text+Audio -\u003e Text (ta2t)`        | ✔️  | ✔️  | ✔️  | ✔️  |\r\n| `Text+Video -\u003e Text (tv2t)`        | ✔️  | ✔️  | ✔️  | ✔️  |\r\n| `Text -\u003e Image (t2i)`              | ✔️  | ⚒️  | ✔️  | ⚒️  |\r\n| `Text -\u003e Video (t2v)`              | ✔️  | ⚒️  | ✔️  | ⚒️  |\r\n| `Text -\u003e Audio (t2a)`              | ✔️  | ⚒️  | ✔️  | ⚒️  |\r\n| `Text+Video -\u003e Action (tv2act)`    | ✔️  | ⚒️  | ⚒️  | ⚒️  |\r\n\r\n## New Feature: Align VLA\r\n\r\n|              | \u003cdetails\u003e\u003csummary\u003eprompt\u003c/summary\u003enavigate to a basketball\u003c/details\u003e                                          | \u003cdetails\u003e\u003csummary\u003eprompt\u003c/summary\u003efind to a basketball\u003c/details\u003e                                              | \u003cdetails\u003e\u003csummary\u003eprompt\u003c/summary\u003elocate a vase.\u003c/details\u003e                                                    | \u003cdetails\u003e\u003csummary\u003eprompt\u003c/summary\u003efind a spray bottle and pick up that spray bottle\u003c/details\u003e                 |\r\n| ------------ | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |\r\n| Baseline     | \u003cimg src=\"assets/text_video_to_action/unsafevideo1.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e | \u003cimg src=\"assets/text_video_to_action/unsafevideo2.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e | \u003cimg src=\"assets/text_video_to_action/unsafevideo3.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e | \u003cimg src=\"assets/text_video_to_action/unsafevideo4.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e |\r\n| **AlignVLA** | \u003cimg src=\"assets/text_video_to_action//safevideo1.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e  | \u003cimg src=\"assets/text_video_to_action//safevideo2.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e  | \u003cimg src=\"assets/text_video_to_action//safevideo3.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e  | \u003cimg src=\"assets/text_video_to_action//safevideo4.gif\" alt=\"Image 8\" style=\"max-width: 100%; height: auto;\"\u003e  |\r\n\r\n\u003e Alignment fine-tuning can significantly enhance the security performance of the VLA model.\r\n\r\n### Downloading the training data\r\n\r\n```bash\r\npython -m align_anything.utils.spoc_utils.download_training_data --save_dir /path/to/data  --types fifteen\r\n```\r\n\r\nThen decompress the compressed data package.\r\n\r\n### Training\r\n\r\n\r\nmodify ``HOME_PREFIX`` in ``align-anything/scripts/vla/spoc_sft.sh`` to your local data path.\r\n\r\n\r\n```bash\r\nbash scripts/vla/spoc_sft.sh\r\n```\r\n\r\n\r\n## Citation\r\n\r\nPlease cite the repo if you find the data or code in this repo useful 😊\r\n\r\n```bibtex\r\n@inproceedings{ji2024align,\r\n  title={Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback},\r\n  author={Jiaming Ji and Jiayi Zhou and Hantao Lou and Boyuan Chen and Donghai Hong and Xuyao Wang and Wenqi Chen and Kaile Wang and Rui Pan and Jiahao Li and Mohan Wang and Josef Dai and Tianyi Qiu and Hua Xu and Dong Li and Weipeng Chen and Jun Song and Bo Zheng and Yaodong Yang},\r\n  year={2024},\r\n  url={https://arxiv.org/abs/2412.15838}\r\n}\r\n```\r\n\r\n## Report Issues\r\n\r\nIf you have any questions in the process of using align-anything, don't hesitate to ask your questions on [the GitHub issue page](https://github.com/PKU-Alignment/align-anything/issues/new/choose), we will reply to you in 2-3 working days.\r\n\r\n# License\r\n\r\nalign-anything is released under Apache License 2.0.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpku-alignment%2Falign-anything","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpku-alignment%2Falign-anything","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpku-alignment%2Falign-anything/lists"}