{"id":15033450,"url":"https://github.com/togethercomputer/moa","last_synced_at":"2025-05-14T02:08:30.257Z","repository":{"id":243898026,"uuid":"810431023","full_name":"togethercomputer/MoA","owner":"togethercomputer","description":"Together Mixture-Of-Agents (MoA) –  65.1% on AlpacaEval with OSS models","archived":false,"fork":false,"pushed_at":"2025-01-07T03:21:19.000Z","size":23240,"stargazers_count":2742,"open_issues_count":23,"forks_count":368,"subscribers_count":36,"default_branch":"main","last_synced_at":"2025-05-06T13:45:08.914Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/togethercomputer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-04T17:23:26.000Z","updated_at":"2025-05-06T13:19:16.000Z","dependencies_parsed_at":"2025-01-15T23:22:40.967Z","dependency_job_id":null,"html_url":"https://github.com/togethercomputer/MoA","commit_stats":{"total_commits":30,"total_committers":7,"mean_commits":4.285714285714286,"dds":0.6,"last_synced_commit":"f72c1d14ccb69b4590f83461030b4962630dbaac"},"previous_names":["togethercomputer/moa"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/togethercomputer%2FMoA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/togethercomputer%2FMoA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/togethercomputer%2FMoA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/togethercomputer%2FMoA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/togethercomputer","download_url":"https://codeload.github.com/togethercomputer/MoA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254053178,"owners_count":22006717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T20:21:15.271Z","updated_at":"2025-05-14T02:08:25.246Z","avatar_url":"https://github.com/togethercomputer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mixture-of-Agents (MoA)\n\n[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](LICENSE)\n[![arXiv](https://img.shields.io/badge/ArXiv-2406.04692-b31b1b.svg)](https://arxiv.org/abs/2406.04692)\n[![Discord](https://img.shields.io/badge/Discord-Together%20AI-blue?logo=discord\u0026logoColor=white)](https://discord.com/invite/9Rk6sSeWEG)\n[![Twitter](https://img.shields.io/twitter/url/https/twitter.com/togethercompute.svg?style=social\u0026label=Follow%20%40togethercompute)](https://twitter.com/togethercompute)\n\n\u003cimg alt=\"MoA architecture\" src=\"./assets/moa.jpg\"\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#overview\"\u003e\u003cstrong\u003eOverview\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"#quickstart-moa-in-50-loc\"\u003e\u003cstrong\u003eQuickstart\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"#multi-layer-moa-example\"\u003e\u003cstrong\u003eAdvanced example\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"#interactive-cli-demo\"\u003e\u003cstrong\u003eInteractive CLI Demo\u003c/strong\u003e\u003c/a\u003e\n  ·\n  \u003ca href=\"#evaluation\"\u003e\u003cstrong\u003eEvaluation\u003c/strong\u003e\u003c/a\u003e\n  ·\n  \u003ca href=\"#results\"\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/a\u003e\n  .\n  \u003ca href=\"#credits\"\u003e\u003cstrong\u003eCredits\u003c/strong\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Overview\n\nMixture of Agents (MoA) is a novel approach that leverages the collective strengths of multiple LLMs to enhance performance, achieving state-of-the-art results. By employing a layered architecture where each layer comprises several LLM agents, **MoA significantly outperforms GPT-4 Omni’s 57.5% on AlpacaEval 2.0 with a score of 65.1%**, using only open-source models!\n\n## Quickstart: MoA in 50 LOC\n\nTo get to get started with using MoA in your own apps, see `moa.py`. In this simple example, we'll use 2 layers and 4 LLMs. You'll need to:\n\n1. Install the Together Python library: `pip install together`\n2. Get your [Together API Key](https://api.together.xyz/settings/api-keys) \u0026 export it: `export TOGETHER_API_KEY=`\n3. Run the python file: `python moa.py`\n\n\u003cimg alt=\"MoA explained\" src=\"./assets/moa-explained.png\"\u003e\n\n## Multi-layer MoA Example\n\nIn the previous example, we went over how to implement MoA with 2 layers (4 LLMs answering and one LLM aggregating). However, one strength of MoA is being able to go through several layers to get an even better response. In this example, we'll go through how to run MoA with 3+ layers in `advanced-moa.py`.\n\n```python\npython advanced-moa.py\n```\n\n\u003cimg alt=\"MoA – 3 layer example\" src=\"./assets/moa-3layer.png\"\u003e\n\n## Interactive CLI Demo\n\nThis interactive CLI demo showcases a simple multi-turn chatbot where the final response is aggregated from various reference models.\n\nTo run the interactive demo, follow these 3 steps:\n\n1. Export Your API Key: `export TOGETHER_API_KEY={your_key}`\n2. Install Requirements: `pip install -r requirements.txt`\n3. Run the script: `python bot.py`\n\nThe CLI will prompt you to input instructions interactively:\n\n1. Start by entering your instruction at the \"\u003e\u003e\u003e\" prompt.\n2. The system will process your input using the predefined reference models.\n3. It will generate a response based on the aggregated outputs from these models.\n4. You can continue the conversation by inputting more instructions, with the system maintaining the context of the multi-turn interaction.\n\n### [Optional] Additional Configuration\n\nThe demo will ask you to specify certain options but if you want to do additional configuration, you can specify these parameters:\n\n- `--aggregator`: The primary model used for final response generation.\n- `--reference_models`: List of models used as references.\n- `--temperature`: Controls the randomness of the response generation.\n- `--max_tokens`: Maximum number of tokens in the response.\n- `--rounds`: Number of rounds to process the input for refinement. (num rounds == num of MoA layers - 1)\n- `--num_proc`: Number of processes to run in parallel for faster execution.\n- `--multi_turn`: Boolean to toggle multi-turn interaction capability.\n\n## Evaluation\n\nWe provide scripts to quickly reproduce some of the results presented in our paper\nFor convenience, we have included the code from [AlpacaEval](https://github.com/tatsu-lab/alpaca_eval),\n[MT-Bench](https://github.com/lm-sys/FastChat), and [FLASK](https://github.com/kaistAI/FLASK), with necessary modifications.\nWe extend our gratitude to these projects for creating the benchmarks.\n\n### Preparation\n\n```bash\n# install requirements\npip install -r requirements.txt\ncd alpaca_eval\npip install -e .\ncd FastChat\npip install -e \".[model_worker,llm_judge]\"\ncd ..\n\n# setup api keys\nexport TOGETHER_API_KEY=\u003cTOGETHER_API_KEY\u003e\nexport OPENAI_API_KEY=\u003cOPENAI_API_KEY\u003e\n```\n\n### Run AlpacaEval 2\n\nTo run AlpacaEval 2, execute the following scripts:\n\n```\nbash run_eval_alpaca_eval.sh\n```\n\n### Run MT-Bench\n\nFor a minimal example of MT-Bench evaluation, run:\n\n```\nbash run_eval_mt_bench.sh\n```\n\n### Run FLASK\n\nFor a minimal example of FLASK evaluation, run:\n\n```\nbash run_eval_flask.sh\n```\n\n### Results\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/alpaca_and_mtbench.png\" alt=\"alpaca_mtbench\" style=\"width: 100%; display: block; margin-left: auto; margin-right: auto;\" /\u003e\n  \u003cbr\u003e\n\u003c/div\u003e\n\nWe achieved top positions on both the AlpacaEval 2.0 leaderboard and MT-Bench. Notably, on AlpacaEval 2.0, using solely open-source models, we achieved a margin of 7.6% absolute improvement from 57.5% (GPT-4 Omni) to 65.1% (MoA).\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/flask.png\" alt=\"flask\" style=\"width: 50%; display: block; margin-left: auto; margin-right: auto;\" /\u003e\n  \u003cbr\u003e\n\u003c/div\u003e\n\nFLASK offers fine-grained evaluation of models across multiple dimensions. Our MoA method significantly outperforms the original Qwen1.5-110B-Chat on harmlessness, robustness, correctness, efficiency, factuality, commonsense, insightfulness, completeness. Additionally, MoA also outperforms GPT-4 Omni in terms of correctness, factuality, insightfulness, completeness, and metacognition.\n\nPlease feel free to contact us if you have difficulties in reproducing the results.\n\n## Credits\n\nNotably, this work was made possible by the collaborative spirit and contributions of active organizations in the AI field. We appreciate the efforts of Meta AI, Mistral AI, Microsoft, Alibaba Cloud, and DataBricks for developing the Llama 3, Mixtral, WizardLM 2, Qwen 1.5, and DBRX models. Additionally, we extend our gratitude to Tatsu Labs, LMSYS, and KAIST AI for developing the AlpacaEval, MT-Bench, and FLASK evaluation benchmarks.\n\n## License\n\nThis project is licensed under the Apache 2.0 License - see the LICENSE file for details.\n\n## Citation\n\nIf you find this work helpful, please consider citing:\n\n```bibtex\n@article{wang2024mixture,\n  title={Mixture-of-Agents Enhances Large Language Model Capabilities},\n  author={Wang, Junlin and Wang, Jue and Athiwaratkun, Ben and Zhang, Ce and Zou, James},\n  journal={arXiv preprint arXiv:2406.04692},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftogethercomputer%2Fmoa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftogethercomputer%2Fmoa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftogethercomputer%2Fmoa/lists"}