{"id":13465356,"url":"https://github.com/microsoft/UFO","last_synced_at":"2025-03-25T16:31:43.070Z","repository":{"id":221726882,"uuid":"740316987","full_name":"microsoft/UFO","owner":"microsoft","description":"A UI-Focused Agent for Windows OS Interaction.","archived":false,"fork":false,"pushed_at":"2025-03-13T07:47:55.000Z","size":34534,"stargazers_count":6614,"open_issues_count":25,"forks_count":849,"subscribers_count":492,"default_branch":"main","last_synced_at":"2025-03-18T20:09:33.918Z","etag":null,"topics":["agent","automation","copilot","gui","llm","windows"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2402.07939","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-08T05:07:52.000Z","updated_at":"2025-03-18T19:33:00.000Z","dependencies_parsed_at":"2024-02-09T18:24:33.582Z","dependency_job_id":"87c5c7ec-32ac-4d84-94ce-0a13ad006ae4","html_url":"https://github.com/microsoft/UFO","commit_stats":null,"previous_names":["microsoft/ufo"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FUFO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FUFO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FUFO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FUFO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/UFO/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245500247,"owners_count":20625532,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","automation","copilot","gui","llm","windows"],"created_at":"2024-07-31T15:00:28.190Z","updated_at":"2025-03-25T16:31:43.063Z","avatar_url":"https://github.com/microsoft.png","language":"Python","readme":"\u003ch1 align=\"center\"\u003e\n    \u003cb\u003eUFO\u003c/b\u003e \u003cimg src=\"./assets/ufo_blue.png\" alt=\"UFO Image\" width=\"40\"\u003e: A \u003cb\u003eU\u003c/b\u003eI-\u003cb\u003eFo\u003c/b\u003ecused Agent for Windows OS Interaction\n\u003c/h1\u003e\n\n\n\u003cdiv align=\"center\"\u003e\n\n[![arxiv](https://img.shields.io/badge/Paper-arXiv:202402.07939-b31b1b.svg)](https://arxiv.org/abs/2402.07939)\u0026ensp;\n![Python Version](https://img.shields.io/badge/Python-3776AB?\u0026logo=python\u0026logoColor=white-blue\u0026label=3.10%20%7C%203.11)\u0026ensp;\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\u0026ensp;\n[![Documentation](https://img.shields.io/badge/Documentation-%230ABAB5?style=flat\u0026logo=readthedocs\u0026logoColor=black)](https://microsoft.github.io/UFO/)\u0026ensp;\n[![YouTube](https://img.shields.io/badge/YouTube-white?logo=youtube\u0026logoColor=%23FF0000)](https://www.youtube.com/watch?v=QT_OhygMVXU)\u0026ensp;\n\u003c!-- [![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/UFO_Agent)](https://twitter.com/intent/follow?screen_name=UFO_Agent) --\u003e\n\u003c!-- ![Welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)\u0026ensp; --\u003e\n\n\u003c/div\u003e\n\n**UFO** is a **UI-Focused** multi-agent framework to fulfill user requests on **Windows OS** by seamlessly navigating and operating within individual or spanning multiple applications.\n\n\u003ch1 align=\"center\"\u003e\n    \u003cimg src=\"./assets/overview_n.png\"/\u003e \n\u003c/h1\u003e\n\n\n## 🕌 Framework\n\u003cb\u003eUFO\u003c/b\u003e \u003cimg src=\"./assets/ufo_blue.png\" alt=\"UFO Image\" width=\"24\"\u003e operates as a multi-agent framework, encompassing:\n- \u003cb\u003eHostAgent 🤖\u003c/b\u003e, tasked with choosing an application for fulfilling user requests. This agent may also switch to a different application when a request spans multiple applications, and the task is partially completed in the preceding application. \n- \u003cb\u003eAppAgent 👾\u003c/b\u003e, responsible for iteratively executing actions on the selected applications until the task is successfully concluded within a specific application. \n- \u003cb\u003eApplication Automator 🎮\u003c/b\u003e, is tasked with translating actions from HostAgent and AppAgent into interactions with the application and through UI controls, native APIs or AI tools. Check out more details [here](https://microsoft.github.io/UFO/automator/overview/).\n\nBoth agents leverage the multi-modal capabilities of GPT-4V(o) to comprehend the application UI and fulfill the user's request. For more details, please consult our [technical report](https://arxiv.org/abs/2402.07939) and [documentation](https://microsoft.github.io/UFO/).\n\u003ch1 align=\"center\"\u003e\n    \u003cimg src=\"./assets/framework_v2.png\"/\u003e \n\u003c/h1\u003e\n\n\n## 📢 News\n- 📅 2025-01-21: Version **v1.2.1** Released! We’re excited to announce the release of **v1.2.1**! 🎉 This update includes:\n    1. **Bug Fixes**: Resolved issues in `requirements.txt` for smoother setup.\n    2. **Multi-Action Mode**: Introducing a powerful new feature to execute **multiple actions** in a single inference step! Enable this mode by setting `ACTION_SEQUENCE=True` in `config_dev.yaml` and enjoy a more efficient workflow.\n- 📅 2024-12-13: We have a **New Release for v1.2.0!**! Checkout our new features and improvements:\n    1. **Large Action Model (LAM) Data Collection:** We have released the code and sample data for Large Action Model (LAM) data collection with UFO! Please checkout our [new paper](https://arxiv.org/abs/2412.10047), [code](dataflow/README.md) and [documentation](https://microsoft.github.io/UFO/dataflow/overview/) for more details.    \n    2. **Bash Command Support:** HostAgent also support bash command now!\n    3. **Bug Fixes:** We have fixed some bugs, error handling, and improved the overall performance.\n- 📅 2024-09-08: We have a **New Release for v1.1.0!**, to allows UFO to click on any region of the application and reduces its latency by up tp 1/3!\n- 📅 2024-07-06: We have a **New Release for v1.0.0!**.  You can check out our [documentation](https://microsoft.github.io/UFO/). We welcome your contributions and feedback!\n- 📅 2024-06-28: We are thrilled to announce that our official introduction video is now available on [YouTube](https://www.youtube.com/watch?v=QT_OhygMVXU)!\n\u003c!-- - 📅 2024-06-25: **New Release for v0.2.1!**  We are excited to announce the release of version 0.2.1! This update includes several new features and improvements:\n    1. **HostAgent Refactor:** We've refactored the HostAgent to enhance its efficiency in managing AppAgents within UFO.\n    2. **Evaluation Agent:** Introducing an evaluation agent that assesses task completion and provides real-time feedback.\n    3. **Google Gemini \u0026\u0026 Claude Support:** UFO now supports Google Gemini and Cluade as the inference engine. Refer to our detailed guide in [Gemini documentation](https://microsoft.github.io/UFO/supported_models/gemini/) or [Claude documentation](https://microsoft.github.io/UFO/supported_models/claude/).\n    4. **Customized User Agents:** Users can now create customized agents by simply answering a few questions.\n- 📅 2024-05-21: We have reached 5K stars!✨\n- 📅 2024-05-08: **New Release for v0.1.1!** We've made some significant updates! Previously known as AppAgent and ActAgent, we've rebranded them to HostAgent and AppAgent to better align with their functionalities. Explore the latest enhancements:\n    1. **Learning from Human Demonstration:** UFO now supports learning from human demonstration! Utilize the [Windows Step Recorder](https://support.microsoft.com/en-us/windows/record-steps-to-reproduce-a-problem-46582a9b-620f-2e36-00c9-04e25d784e47) to record your steps and demonstrate them for UFO. Refer to our detailed guide in [README.md](https://microsoft.github.io/UFO/creating_app_agent/demonstration_provision/) for more information.\n    2. **Win32 Support:** We've incorporated support for [Win32](https://learn.microsoft.com/en-us/windows/win32/controls/window-controls) as a control backend, enhancing our UI automation capabilities.\n    3. **Extended Application Interaction:** UFO now goes beyond UI controls, allowing interaction with your application through keyboard inputs and native APIs! Presently, we support Word ([examples](/ufo/prompts/apps/word/api.yaml)), with more to come soon. Customize and build your own interactions.\n    4. **Control Filtering:** Streamline LLM's action process by using control filters to remove irrelevant control items. Enable them in [config_dev.yaml](/ufo/config/config_dev.yaml) under the `control filtering` section at the bottom.\n- 📅 2024-03-25: **New Release for v0.0.1!** Check out our exciting new features.\n    1. We now support creating your help documents for each Windows application to become an app expert. Check the [documentation](https://microsoft.github.io/UFO/creating_app_agent/help_document_provision/) for more details!\n    2. UFO now supports RAG from offline documents and online Bing search.\n    3. You can save the task completion trajectory into its memory for UFO's reference, improving its future success rate!\n    4. You can customize different GPT models for HostAgent and AppAgent. Text-only models (e.g., GPT-4) are now supported! --\u003e\n- 📅 ...\n- 📅 2024-02-14: Our [technical report](https://arxiv.org/abs/2402.07939) is online!\n- 📅 2024-02-10: UFO is released on GitHub🎈. Happy Chinese New year🐉!\n\n\n## 🌐 Media Coverage \n\nUFO sightings have garnered attention from various media outlets, including:\n- [Microsoft's UFO abducts traditional user interfaces for a smarter Windows experience](https://the-decoder.com/microsofts-ufo-abducts-traditional-user-interfaces-for-a-smarter-windows-experience/)\n- [🚀 UFO \u0026 GPT-4-V: Sit back and relax, mientras GPT lo hace todo🌌](https://www.linkedin.com/posts/gutierrezfrancois_ai-ufo-microsoft-activity-7176819900399652865-pLoo?utm_source=share\u0026utm_medium=member_desktop)\n- [The AI PC - The Future of Computers? - Microsoft UFO](https://www.youtube.com/watch?v=1k4LcffCq3E)\n- [下一代Windows系统曝光：基于GPT-4V，Agent跨应用调度，代号UFO](https://baijiahao.baidu.com/s?id=1790938358152188625\u0026wfr=spider\u0026for=pc)\n- [下一代智能版 Windows 要来了？微软推出首个 Windows Agent，命名为 UFO！](https://blog.csdn.net/csdnnews/article/details/136161570)\n- [Microsoft発のオープンソース版「UFO」登場！　Windowsを自動操縦するAIエージェントを試す](https://internet.watch.impress.co.jp/docs/column/shimizu/1570581.html)\n- ...\n\nThese sources provide insights into the evolving landscape of technology and the implications of UFO phenomena on various platforms.\n\n\n## 💥 Highlights\n\n- [x] **First Windows Agent** - UFO is the pioneering agent framework capable of translating user requests in natural language into actionable operations on Windows OS.\n- [x] **Agent as an Expert** - UFO is enhanced by Retrieval Augmented Generation (RAG) from heterogeneous sources, including offline help documents, online search engines, and human demonstrations, making the agent an application \"expert\".\n- [x] **Rich Skill Set** - UFO is equipped with a diverse set of skills to support comprehensive automation, such as mouse, keyboard, native API, and \"Copilot\".\n- [x] **Interactive Mode** - UFO facilitates multiple sub-requests from users within the same session, enabling the seamless completion of complex tasks.\n- [x] **Agent Customization** - UFO allows users to customize their own agents by providing additional information. The agent will proactively query users for details when necessary to better tailor its behavior.\n- [x] **Scalable AppAgent Creation** - UFO offers extensibility, allowing users and app developers to create their own AppAgents in an easy and scalable way.\n\n\n## ✨ Getting Started\n\n\n### 🛠️ Step 1: Installation\nUFO requires **Python \u003e= 3.10** running on **Windows OS \u003e= 10**. It can be installed by running the following command:\n```bash\n# [optional to create conda environment]\n# conda create -n ufo python=3.10\n# conda activate ufo\n\n# clone the repository\ngit clone https://github.com/microsoft/UFO.git\ncd UFO\n# install the requirements\npip install -r requirements.txt\n# If you want to use the Qwen as your LLMs, uncomment the related libs.\n```\n\n### ⚙️ Step 2: Configure the LLMs\nBefore running UFO, you need to provide your LLM configurations **individually for HostAgent and AppAgent**. You can create your own config file `ufo/config/config.yaml`, by copying the `ufo/config/config.yaml.template` and editing config for **HOST_AGENT** and **APP_AGENT** as follows: \n\n\n#### OpenAI\n```bash\nVISUAL_MODE: True, # Whether to use the visual mode\nAPI_TYPE: \"openai\" , # The API type, \"openai\" for the OpenAI API.  \nAPI_BASE: \"https://api.openai.com/v1/chat/completions\", # The the OpenAI API endpoint.\nAPI_KEY: \"sk-\",  # The OpenAI API key, begin with sk-\nAPI_VERSION: \"2024-02-15-preview\", # \"2024-02-15-preview\" by default\nAPI_MODEL: \"gpt-4-vision-preview\",  # The only OpenAI model\n```\n\n#### Azure OpenAI (AOAI)\n```bash\nVISUAL_MODE: True, # Whether to use the visual mode\nAPI_TYPE: \"aoai\" , # The API type, \"aoai\" for the Azure OpenAI.  \nAPI_BASE: \"YOUR_ENDPOINT\", #  The AOAI API address. Format: https://{your-resource-name}.openai.azure.com\nAPI_KEY: \"YOUR_KEY\",  # The aoai API key\nAPI_VERSION: \"2024-02-15-preview\", # \"2024-02-15-preview\" by default\nAPI_MODEL: \"gpt-4-vision-preview\",  # The only OpenAI model\nAPI_DEPLOYMENT_ID: \"YOUR_AOAI_DEPLOYMENT\", # The deployment id for the AOAI API\n```\nYou can also non-visial model (e.g., GPT-4) for each agent, by setting `VISUAL_MODE: False` and proper `API_MODEL` (openai) and `API_DEPLOYMENT_ID` (aoai). You can also optionally set an backup LLM engine in the field of `BACKUP_AGENT` if the above engines failed during the inference.\n\n\n####  Non-Visual Model Configuration\nYou can utilize non-visual models (e.g., GPT-4) for each agent by configuring the following settings in the `config.yaml` file:\n\n- ```VISUAL_MODE: False # To enable non-visual mode.```\n- Specify the appropriate `API_MODEL` (OpenAI) and `API_DEPLOYMENT_ID` (AOAI) for each agent.\n\nOptionally, you can set a backup language model (LLM) engine in the `BACKUP_AGENT` field to handle cases where the primary engines fail during inference. Ensure you configure these settings accurately to leverage non-visual models effectively.\n\n#### NOTE 💡 \nUFO also supports other LLMs and advanced configurations, such as customize your own model, please check the [documents](https://microsoft.github.io/UFO/supported_models/overview/) for more details. Because of the limitations of model input, a lite version of the prompt is provided to allow users to experience it, which is configured in `config_dev.yaml`.\n\n### 📔 Step 3: Additional Setting for RAG (optional).\nIf you want to enhance UFO's ability with external knowledge, you can optionally configure it with an external database for retrieval augmented generation (RAG) in the `ufo/config/config.yaml` file. \n\nWe provide the following options for RAG to enhance UFO's capabilities:\n- [Offline Help Document](https://microsoft.github.io/UFO/advanced_usage/reinforce_appagent/learning_from_help_document/) Enable UFO to retrieve information from offline help documents.\n- [Online Bing Search Engine](https://microsoft.github.io/UFO/advanced_usage/reinforce_appagent/learning_from_bing_search/): Enhance UFO's capabilities by utilizing the most up-to-date online search results.\n- [Self-Experience](https://microsoft.github.io/UFO/advanced_usage/reinforce_appagent/experience_learning/): Save task completion trajectories into UFO's memory for future reference.\n- [User-Demonstration](https://microsoft.github.io/UFO/advanced_usage/reinforce_appagent/learning_from_demonstration/): Boost UFO's capabilities through user demonstration.\n\nConsult their respective documentation for more information on how to configure these settings.\n\n\u003c!-- #### RAG from Offline Help Document\nBefore enabling this function, you need to create an offline indexer for your help document. Please refer to the [README](./learner/README.md) to learn how to create an offline vectored database for retrieval. You can enable this function by setting the following configuration:\n```bash\n## RAG Configuration for the offline docs\nRAG_OFFLINE_DOCS: True  # Whether to use the offline RAG.\nRAG_OFFLINE_DOCS_RETRIEVED_TOPK: 1  # The topk for the offline retrieved documents\n```\nAdjust `RAG_OFFLINE_DOCS_RETRIEVED_TOPK` to optimize performance.\n\n\n####  RAG from Online Bing Search Engine\nEnhance UFO's ability by utilizing the most up-to-date online search results! To use this function, you need to obtain a Bing search API key. Activate this feature by setting the following configuration:\n```bash\n## RAG Configuration for the Bing search\nBING_API_KEY: \"YOUR_BING_SEARCH_API_KEY\"  # The Bing search API key\nRAG_ONLINE_SEARCH: True  # Whether to use the online search for the RAG.\nRAG_ONLINE_SEARCH_TOPK: 5  # The topk for the online search\nRAG_ONLINE_RETRIEVED_TOPK: 1 # The topk for the online retrieved documents\n```\nAdjust `RAG_ONLINE_SEARCH_TOPK` and `RAG_ONLINE_RETRIEVED_TOPK` to get better performance.\n\n\n#### RAG from Self-Demonstration\nSave task completion trajectories into UFO's memory for future reference. This can improve its future success rates based on its previous experiences!\n\nAfter completing a task, you'll see the following message:\n```\nWould you like to save the current conversation flow for future reference by the agent?\n[Y] for yes, any other key for no.\n```\nPress `Y` to save it into its memory and enable memory retrieval via the following configuration:\n```bash\n## RAG Configuration for experience\nRAG_EXPERIENCE: True  # Whether to use the RAG from its self-experience.\nRAG_EXPERIENCE_RETRIEVED_TOPK: 5  # The topk for the offline retrieved documents\n```\n\n#### RAG from User-Demonstration\nBoost UFO's capabilities through user demonstration! Utilize Microsoft Steps Recorder to record step-by-step processes for achieving specific tasks. With a simple command processed by the record_processor (refer to the [README](./record_processor/README.md)), UFO can store these trajectories in its memory for future reference, enhancing its learning from user interactions.\n\nYou can enable this function by setting the following configuration:\n```bash\n## RAG Configuration for demonstration\nRAG_DEMONSTRATION: True  # Whether to use the RAG from its user demonstration.\nRAG_DEMONSTRATION_RETRIEVED_TOPK: 5  # The topk for the demonstration examples.\n``` --\u003e\n\n\n### 🎉 Step 4: Start UFO\n\n#### ⌨️ You can execute the following on your Windows command Line (CLI):\n\n```bash\n# assume you are in the cloned UFO folder\npython -m ufo --task \u003cyour_task_name\u003e\n```\n\nThis will start the UFO process and you can interact with it through the command line interface. \nIf everything goes well, you will see the following message:\n\n```bash\nWelcome to use UFO🛸, A UI-focused Agent for Windows OS Interaction. \n _   _  _____   ___\n| | | ||  ___| / _ \\\n| | | || |_   | | | |\n| |_| ||  _|  | |_| |\n \\___/ |_|     \\___/\nPlease enter your request to be completed🛸:\n```\n#### ⚠️Reminder:  ####\n- Before UFO executing your request, please make sure the targeted applications are active on the system.\n- The GPT-V accepts screenshots of your desktop and application GUI as input. Please ensure that no sensitive or confidential information is visible or captured during the execution process. For further information, refer to [DISCLAIMER.md](./DISCLAIMER.md).\n\n\n###  Step 5 🎥: Execution Logs \n\nYou can find the screenshots taken and request \u0026 response logs in the following folder:\n```\n./ufo/logs/\u003cyour_task_name\u003e/\n```\nYou may use them to debug, replay, or analyze the agent output.\n\n\n## ❓Get help \n* Please first check our our documentation [here](https://microsoft.github.io/UFO/).\n* ❔GitHub Issues (prefered)\n* For other communications, please contact [ufo-agent@microsoft.com](mailto:ufo-agent@microsoft.com).\n---\n\n\n\u003c!-- ## 🎬 Demo Examples\n\nWe present two demo videos that complete user request on Windows OS using UFO. For more case study, please consult our [technical report](https://arxiv.org/abs/2402.07939).\n\n#### 1️⃣🗑️ Example 1: Deleting all notes on a PowerPoint presentation.\nIn this example, we will demonstrate how to efficiently use UFO to delete all notes on a PowerPoint presentation with just a few simple steps. Explore this functionality to enhance your productivity and work smarter, not harder!\n\n\nhttps://github.com/microsoft/UFO/assets/11352048/cf60c643-04f7-4180-9a55-5fb240627834\n\n\n\n#### 2️⃣📧 Example 2: Composing an email using text from multiple sources.\nIn this example, we will demonstrate how to utilize UFO to extract text from Word documents, describe an image, compose an email, and send it seamlessly. Enjoy the versatility and efficiency of cross-application experiences with UFO!\n\n\nhttps://github.com/microsoft/UFO/assets/11352048/aa41ad47-fae7-4334-8e0b-ba71c4fc32e0 --\u003e\n\n\n\n\n\n## 📊 Evaluation\n\nPlease consult the [WindowsBench](https://arxiv.org/pdf/2402.07939.pdf) provided in Section A of the Appendix within our technical report. Here are some tips (and requirements) to aid in completing your request:\n\n- Prior to UFO execution of your request, ensure that the targeted application is active (though it may be minimized).\n- Please note that the output of GPT-V may not consistently align with the same request. If unsuccessful with your initial attempt, consider trying again.\n\n\n\n## 📚 Citation\nOur technical report paper can be found [here](https://arxiv.org/abs/2402.07939). Note that previous AppAgent and ActAgent in the paper are renamed to HostAgent and AppAgent in the code base to better reflect their functions.\nIf you use UFO in your research, please cite our paper:\n```\n@article{ufo,\n  title={{UFO: A UI-Focused Agent for Windows OS Interaction}},\n  author={Zhang, Chaoyun and Li, Liqun and He, Shilin and Zhang, Xu and Qiao, Bo and  Qin, Si and Ma, Minghua and Kang, Yu and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei and  Zhang, Qi},\n  journal={arXiv preprint arXiv:2402.07939},\n  year={2024}\n}\n```\n\n## 📝 Todo List\n- [x] RAG enhanced UFO.\n- [x] Support more control using Win32 API.\n- [x] [Documentation](https://microsoft.github.io/UFO/).\n- [ ] Support local host GUI interaction model.\n- [ ] Chatbox GUI for UFO.\n\n\n\n## 🎨 Related Projects\n1. If you're interested in data analytics agent frameworks, check out [TaskWeaver](https://github.com/microsoft/TaskWeaver?tab=readme-ov-file), a code-first LLM agent framework designed for seamlessly planning and executing data analytics tasks.\n\n2. For more information on GUI agents, refer to our survey paper: [Large Language Model-Brained GUI Agents: A Survey](https://arxiv.org/abs/2411.18279). You can also explore the survey through:\n- [Paper](https://arxiv.org/abs/2411.18279)\n- [GitHub Repository](https://github.com/vyokky/LLM-Brained-GUI-Agents-Survey)\n- [Searchable Website](https://vyokky.github.io/LLM-Brained-GUI-Agents-Survey/)\n\n## ⚠️ Disclaimer\nBy choosing to run the provided code, you acknowledge and agree to the following terms and conditions regarding the functionality and data handling practices in [DISCLAIMER.md](./DISCLAIMER.md)\n\n\n## \u003cimg src=\"./assets/ufo_blue.png\" alt=\"logo\" width=\"30\"\u003e Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft \ntrademarks or logos is subject to and must follow \n[Microsoft's Trademark \u0026 Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n","funding_links":[],"categories":["Python","**Section 2** : Azure OpenAI and Reference Architecture","[UFO](https://github.com/microsoft/UFO)","Papers","A01_文本生成_文本对话","Platforms/API","🚀 AI Tools for Vim, Neovim, and Terminal","Agent Frameworks","Task Automation \u0026 Workflow Orchestration","AI开源项目"],"sub_categories":["**Microsoft Azure OpenAI relevant LLM Framework**","Links","Models","大语言对话模型及数据","Advanced Components","Frameworks \u0026 Models","Inventory Automation Agents","AI Agent"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2FUFO","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2FUFO","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2FUFO/lists"}