{"id":21647796,"url":"https://github.com/prajwalshettydev/unrealgenaisupport","last_synced_at":"2025-10-04T18:50:31.600Z","repository":{"id":263710615,"uuid":"889942997","full_name":"prajwalshettydev/UnrealGenAISupport","owner":"prajwalshettydev","description":"An Unreal Engine plugin for LLM/GenAI models \u0026 MCP UE5 server. Supports Claude Desktop App, Windsurf  \u0026 Cursor, also includes OpenAI's GPT4o, DeepseekR1, Claude Sonnet 3.7 APIs and Grok 4, with plans to add Gemini, audio \u0026 realtime APIs soon. UnrealMCP is also here!! Automatic blueprint and scene generation from AI!! ","archived":false,"fork":false,"pushed_at":"2025-09-01T00:03:04.000Z","size":15423,"stargazers_count":260,"open_issues_count":8,"forks_count":45,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-09-01T02:33:02.776Z","etag":null,"topics":["claude","deepseek","game-development","gemini","generative-ai","gpt-4","grok","lightweight","llama","llm","mcp","mcp-server","meta","openai","sonnet3-7","ue5","unreal-engine-5","unreal-engine-plugin","whisper","xai"],"latest_commit_sha":null,"homepage":"https://github.com/prajwalshettydev/UnrealGenAISupport/wiki","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prajwalshettydev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"prajwalshettydev"}},"created_at":"2024-11-17T16:25:35.000Z","updated_at":"2025-09-01T00:03:08.000Z","dependencies_parsed_at":"2024-12-18T17:18:58.124Z","dependency_job_id":"354b9275-9fbb-4ba4-9437-bb7f4e742b14","html_url":"https://github.com/prajwalshettydev/UnrealGenAISupport","commit_stats":null,"previous_names":["prajwalshettydev/unrealgenaisupport"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/prajwalshettydev/UnrealGenAISupport","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prajwalshettydev%2FUnrealGenAISupport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prajwalshettydev%2FUnrealGenAISupport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prajwalshettydev%2FUnrealGenAISupport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prajwalshettydev%2FUnrealGenAISupport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prajwalshettydev","download_url":"https://codeload.github.com/prajwalshettydev/UnrealGenAISupport/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prajwalshettydev%2FUnrealGenAISupport/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278358477,"owners_count":25973949,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["claude","deepseek","game-development","gemini","generative-ai","gpt-4","grok","lightweight","llama","llm","mcp","mcp-server","meta","openai","sonnet3-7","ue5","unreal-engine-5","unreal-engine-plugin","whisper","xai"],"created_at":"2024-11-25T06:51:39.531Z","updated_at":"2025-10-04T18:50:31.572Z","avatar_url":"https://github.com/prajwalshettydev.png","language":"C++","funding_links":["https://github.com/sponsors/prajwalshettydev","https://github.com/sponsors/prajwalshettydev/button"],"categories":[],"sub_categories":[],"readme":"# Unreal Engine Generative AI Support Plugin\n\n[![Discord](https://img.shields.io/badge/Discord-7289DA?style=for-the-badge\u0026logo=discord\u0026logoColor=white)](https://discord.com/invite/KBWmkCKv5U)\n[![License: MIT](https://img.shields.io/badge/License-MIT-007EC7?style=for-the-badge)](LICENSE)\n## Usage Examples:\n#### MCP Example:\nClaude spawning scene objects and controlling their transformations and materials, generating blueprints, functions, variables, adding components, running python scripts etc.\n\n\u003cimg src=\"Docs/UnrealMcpDemo.gif\" width=\"480\"/\u003e\n\n#### API Example:\nA project called become human, where NPCs are OpenAI agentic instances. Built using this plugin.\n![Become Human](Docs/BHDemoGif.gif)\n\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"Docs/Repo Card - Updated.png\" width=\"512\"/\u003e\u003c/p\u003e\n\nEvery month, hundreds of new AI models are released by various organizations, making it hard to keep up with the latest advancements.\n\nThe Unreal Engine Generative AI Support Plugin allows you to focus on game development without worrying about the LLM/GenAI integration layer.\n\nCurrently integrating Model Control Protocol (MCP) with Unreal Engine 5.5.\n\nThis project aims to build a long-term support (LTS) plugin for various cutting-edge LLM/GenAI models and foster a\ncommunity around it. It currently includes OpenAI's GPT-4o, Deepseek R1, Claude Sonnet 4, Claude Opus 4, and GPT-4o-mini for Unreal Engine 5.1 or higher, with plans to add\n, real-time APIs, Gemini, MCP, and Grok 3 APIs soon. The plugin will focus exclusively on APIs useful for\ngame development, evals and interactive experiences. All suggestions and contributions are welcome. The plugin can also be used for setting up new evals and ways to compare models in game battlefields.\n\n## Current Progress:\n\n\u003e [!WARNING]  \n\u003e This plugin is still under rapid development.\n\u003e 1) Do not use it in production environments. ⚠️\n\u003e 2) Do not use it without version control. ⚠️\n\n\n**I will continue to keep this repo updated with the latest features and models as they become available. Contributions are welcome.**\n**If you want a production ready plugin, with more features and guaranteed stability please checkout the [Gen AI Pro](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) plugin, as it costs a lot of API credits for me to test different models per feature and per engine version to make sure everything works well and compatible. Otherwise, I feel this plugin is good enough for many use cases (including the examples shown in the beginning) and you can use it for free, forever.**\n\n**Currently working on fixing the issues with MCP (especially the node generation) and adding more features to it.**\n\n\n### LLM/GenAI API Support:\n\n- OpenAI API Support:\n    - OpenAI Chat API ✅ \n      [(models-ref)](https://platform.openai.com/docs/models)\n        - `gpt-5` ✅,\n        - `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano` Model ✅\n        - `gpt-4o`, `gpt-4o-mini`  Model ✅\n        - `o4-mini` Model ✅\n        - `o3`, `o3-pro`, `o3-mini`, `o1` Model ✅ \n    - OpenAI DALL-E API  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Responses API 🛠️\n    - OpenAI Vision API  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - OpenAI Realtime API  \n        - `gpt-4o-realtime-preview` `gpt-4o-mini-realtime-preview` Model  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - OpenAI Structured Outputs ✅\n    - OpenAI Whisper \u0026 TTS API  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Multimodal API Support  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Text Streaming  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n- Anthropic Claude API Support:\n    - Claude Chat API ✅\n        - `claude-4-latest`, `claude-3-7-sonnet-latest`, `claude-3-5-sonnet`, `claude-3-5-haiku-latest`, `claude-3-opus-latest` Model ✅\n    - Multimodal/Vision API Support  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n- XAI (Grok 3) API Support:\n    - XAI Chat Completions API ✅\n        - `grok-3-latest`, `grok-3-mini-beta` Model ✅\n        - `grok-3`, `grok-3-mini`, `grok-3-fast`, `grok-3-mini-fast`, `grok-2-vision-1212`, `grok-2-1212`.\n        - `grok-4` Reasoning API  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - XAI Image API 🚧\n    - Text Streaming API  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Multimodal API Support  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n- Google Gemini API Support:\n    - Gemini Chat API \n        - `gemini-2.0-flash-lite`, `gemini-2.0-flash` `gemini-1.5-flash`  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n        - Gemini 2.5 Pro Model  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Gemini Imagen API:  \n      - `imagen-3.0-generate-002`  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n      - `nano-banana` [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Google TTS \u0026 Transcription API: [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Multimodal API Support  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n- Meta AI API Support:\n    - Llama 4 herd: ❌\n      - Llama 4 Behemoth, Llama 4 Maverick, Llama 4 Scout ❌\n      - llama3.3-70b, llama3.1-8b Model❌\n    - Local Llama API 🚧🤝\n- Deepseek API Support:\n    - Deepseek Chat API ✅\n        - `deepseek-chat` (DeepSeek-V3.1) Model ✅\n    - Deepseek Reasoning API, R1 ✅\n        - `deepseek-reasoning-r1` Model ✅\n        - `deepseek-reasoning-r1` CoT Streaming ❌\n    - Independently Hosted Deepseek Models ❌\n- Baidu API Support:\n    - Baidu Chat API ❌\n        - `baidu-chat` Model ❌\n- 3D generative model APIs:\n    -  TripoSR by StabilityAI 🚧\n- Plugin Documentation 🛠️🤝\n- Plugin Example Project 🛠️ [here](https://github.com/prajwalshettydev/unreal-llm-api-test-project)\n- Version Control Support\n    - Perforce Support 🛠️\n    - Git Submodule Support ✅ \n- LTS Branching 🚧\n    - Stable Branch with Bug Fixes 🚧\n    - Dedicated Contributor for LTS 🚧\n- Lightweight Plugin (In Builds) \n    - No External Dependencies ✅\n    - Build Flags to enable/disable APIs 🚧\n    - Submodules per API Organization 🚧\n    - Exclude MCP from build 🚧\n- Testing \n    - Automated Testing  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Different Platforms  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n    - Different Engine Versions  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin) ☑️\n\n### Unreal MCP (Model Control Protocol):\n\n- Clients Support ✅\n    - Claude Desktop App Support ✅\n    - Cursor IDE Support ✅\n    - OpenAI Operator API Support 🚧\n- Blueprints Auto Generation 🛠️\n    - Creating new blueprint of types ✅\n    - Adding new functions, function/blueprint variables ✅\n    - Adding nodes and connections 🛠️ (buggy, issues open)\n    - Advanced Blueprints Generation 🛠️\n- Level/Scene Control for LLMs 🛠️\n    - Spawning Objects and Shapes ✅\n    - Moving, rotating and scaling objects ✅\n    - Changing materials and color ✅\n    - Advanced scene features 🛠️\n- Generative AI:\n    - Prompt to 3D model fetch and spawn 🛠️ \n- Control:\n    - Ability to run Python scripts ✅\n    - Ability to run Console Commands ✅\n- UI:\n    - Widgets generation 🛠️\n    - UI Blueprint generation 🛠️\n- Project Files:\n    - Create/Edit project files/folders ️✅\n    - Delete existing project files ❌\n- Others:\n    - Project Cleanup 🛠️ \n\nWhere,\n- ✅ - Completed\n- ☑️ -  [(available in pro)](https://muddyterrain.com/t/genai-fab?utm_source=github.com\u0026utm_medium=repo-free\u0026utm_campaign=genai-plugin)\n- 🛠️ - In Progress\n- 🚧 - Planned\n- 🤝 - Need Contributors\n- ❌ - Won't Support For Now\n\n## Table of Contents\n\n- [Setting API Keys](#setting-api-keys)\n    - [For Editor](#for-editor)\n    - [For Packaged Builds](#for-packaged-builds)\n- [Setting up MCP](#setting-up-mcp)\n- [Adding the plugin to your project](#adding-the-plugin-to-your-project)\n    - [With Git](#with-git)\n    - [With Perforce](#with-perforce)\n    - [With Unreal Marketplace](#with-unreal-marketplace)\n- [Fetching the Latest Plugin Changes](#fetching-the-latest-plugin-changes)\n    - [With Git](#with-git-1)\n    - [With Perforce](#with-perforce-1)\n- [Usage](#usage)\n    - [OpenAI](#openai)\n        - [1. Chat](#1-chat)\n        - [2. Structured Outputs](#2-structured-outputs)\n    - [DeepSeek API](#deepseek-api)\n        - [1. Chat and Reasoning](#1-chat-and-reasoning)\n    - [Anthropic API](#anthropic-api)\n        - [1. Chat](#1-chat-1)\n    - [XAI's Grok 3 API](#xais-grok-3-api)\n        - [1. Chat](#1-chat-2)\n    - [Model Control Protocol (MCP)](#model-control-protocol-mcp)\n- [Known Issues](#known-issues)\n- [Config Window](#config-window)\n- [Contribution Guidelines](#contribution-guidelines)\n    - [Setting up for Development](#setting-up-for-development)\n    - [Project Structure](#project-structure)\n- [References](#references)\n\n## Setting API Keys:\n\u003e [!NOTE]  \n\u003e There is no need to set the API key for testing the MCP features in Claude app. Anthropic key only needed for Claude API.\n\n### For Editor:\n\nSet the environment variable `PS_\u003cORGNAME\u003e` to your API key.\n#### For Windows:\n```cmd\nsetx PS_\u003cORGNAME\u003e \"your api key\"\n```\n\n#### For Linux/MacOS:\n\n1. Run the following command in your terminal, replacing yourkey with your API key.\n    ```bash\n    echo \"export PS_\u003cORGNAME\u003e='yourkey'\" \u003e\u003e ~/.zshrc\n    ```\n\n2. Update the shell with the new variable:\n    ```bash\n    source ~/.zshrc\n    ```\n\nPS: Don't forget to restart the Editor and ALSO the connected IDE after setting the environment variable.\n\nWhere `\u003cORGNAME\u003e` can be:\n`PS_OPENAIAPIKEY`, `PS_DEEPSEEKAPIKEY`, `PS_ANTHROPICAPIKEY`, `PS_METAAPIKEY`, `PS_GOOGLEAPIKEY` etc.\n\n### For Packaged Builds:\n\nStoring API keys in packaged builds is a security risk. This is what the OpenAI API documentation says about it:\n\u003e\"Exposing your OpenAI API key in client-side environments like browsers or mobile apps allows malicious users to take that key and make requests on your behalf – which may lead to unexpected charges or compromise of certain account data. Requests should always be routed through your own backend server where you can keep your API key secure.\"\n\nRead more about it [here](https://help.openai.com/en/articles/5112595-best-practices-for-api-key-safety).\n\nFor test builds you can call the `GenSecureKey::SetGenAIApiKeyRuntime` either in c++ or blueprints function with your API key in the packaged build.\n\n## Setting up MCP:\n\n\u003e [!NOTE]  \n\u003e If your project only uses the LLM APIs and not the MCP, you can skip this section.\n\n\u003e [!CAUTION]  \n\u003e Discalimer: If you are using the MCP feature of the plugin, it will directly let the Claude Desktop App control your Unreal Engine project.\n\u003e Make sure you are aware of the security risks and only use it in a controlled environment.\n\u003e\n\u003e Please backup your project before using the MCP feature and use version control to track changes.\n\n\n##### 1. Install any one of the below clients: \n* Claude Desktop App from [here](https://claude.anthropic.com/).\n* Cursor IDE from [here](https://www.cursor.com/).\n\n##### 2. Setup the mcp config json:\n###### For Claude Desktop App:\n`claude_desktop_config.json` file in Claude Desktop App's installation directory. (might ask claude where its located for your platform!)\nThe file will look something like this:\n```json\n{\n    \"mcpServers\": {\n      \"unreal-handshake\": {\n        \"command\": \"python\",\n        \"args\": [\"\u003cyour_project_directoy_path\u003e/Plugins/GenerativeAISupport/Content/Python/mcp_server.py\"],\n        \"env\": {\n          \"UNREAL_HOST\": \"localhost\",\n          \"UNREAL_PORT\": \"9877\" \n        }\n      }\n    }\n}\n```\n###### For Cursor IDE:\n`.cursor/mcp.json` file in your project directory. The file will look something like this:\n```json\n{\n    \"mcpServers\": {\n      \"unreal-handshake\": {\n        \"command\": \"python\",\n        \"args\": [\"\u003cyour_project_directoy_path\u003e/Plugins/GenerativeAISupport/Content/Python/mcp_server.py\"],\n        \"env\": {\n          \"UNREAL_HOST\": \"localhost\",\n          \"UNREAL_PORT\": \"9877\" \n        }\n      }\n    }\n}\n```\n##### 3. Install MCP[CLI] from with either pip or cv.\n```bash\npip install mcp[cli]\n```\n##### 4. Enable python plugin in Unreal Engine. (Edit -\u003e Plugins -\u003e Python Editor Script Plugin)\n\n##### 5. [OPTIONAL] Enable AutoStart MCP server on editor open\n\n\u003cimg src=\"Docs/Settings.png\" width=\"782\"/\u003e\n\n\n## Adding the plugin to your project:\n\n### With Git:\n\n1. Add the Plugin Repository as a Submodule in your project's repository.\n\n   ```cmd\n   git submodule add https://github.com/prajwalshettydev/UnrealGenAISupport Plugins/GenerativeAISupport\n   ```\n\n2. Regenerate Project Files:\n   Right-click your .uproject file and select Generate Visual Studio project files.\n3. Enable the Plugin in Unreal Editor:\n   Open your project in Unreal Editor. Go to Edit \u003e Plugins. Search for the Plugin in the list and enable it.\n4. For Unreal C++ Projects, include the Plugin's module in your project's Build.cs file:\n\n   ```cpp\n   PrivateDependencyModuleNames.AddRange(new string[] { \"GenerativeAISupport\" });\n   ```\n\n### With Perforce:\n\nStill in development..\n\n### With Unreal Marketplace:\nComing soon, for free, in the Unreal Engine Marketplace.\n\n## Fetching the Latest Plugin Changes:\n\n### With Git:\n\nyou can pull the latest changes with:\n\n```cmd\ncd Plugins/GenerativeAISupport\ngit pull origin main\n```\n\nOr update all submodules in the project:\n\n```cmd\ngit submodule update --recursive --remote\n```\n\n### With Perforce:\n\nStill in development..\n\n## Usage:\nThere is a example Unreal project that already implements the plugin. You can find it [here](https://github.com/prajwalshettydev/unreal-llm-api-test-project).\n\n### OpenAI:\n\nCurrently the plugin supports Chat and Structured Outputs from OpenAI API. Both for C++ and Blueprints.\nTested models are `gpt-4o`, `gpt-4o-mini`, `gpt-4.5`, `o1-mini`, `o1`, `o3-mini-high`.\n\n#### 1. Chat:\n\n   ##### C++ Example:\n```cpp\n    void SomeDebugSubsystem::CallGPT(const FString\u0026 Prompt, \n        const TFunction\u003cvoid(const FString\u0026, const FString\u0026, bool)\u003e\u0026 Callback)\n    {\n        FGenChatSettings ChatSettings;\n        ChatSettings.Model = TEXT(\"gpt-4o-mini\");\n        ChatSettings.MaxTokens = 500;\n        ChatSettings.Messages.Add(FGenChatMessage{ TEXT(\"system\"), Prompt });\n    \n        FOnChatCompletionResponse OnComplete = FOnChatCompletionResponse::CreateLambda(\n            [Callback](const FString\u0026 Response, const FString\u0026 ErrorMessage, bool bSuccess)\n        {\n            Callback(Response, ErrorMessage, bSuccess);\n        });\n    \n        UGenOAIChat::SendChatRequest(ChatSettings, OnComplete);\n    }\n```\n\n   ##### Blueprint Example:\n\n\u003cimg src=\"Docs/BpExampleOAIChat.png\" width=\"782\"/\u003e\n\n#### 2. Structured Outputs:\n   ##### C++ Example 1:\n   Sending a custom schema json directly to function call\n   ```cpp\n   FString MySchemaJson = R\"({\n   \"type\": \"object\",\n   \"properties\": {\n       \"count\": {\n           \"type\": \"integer\",\n           \"description\": \"The total number of users.\"\n       },\n       \"users\": {\n           \"type\": \"array\",\n           \"items\": {\n               \"type\": \"object\",\n               \"properties\": {\n                   \"name\": { \"type\": \"string\", \"description\": \"The user's name.\" },\n                   \"heading_to\": { \"type\": \"string\", \"description\": \"The user's destination.\" }\n               },\n               \"required\": [\"name\", \"role\", \"age\", \"heading_to\"]\n           }\n       }\n   },\n   \"required\": [\"count\", \"users\"]\n   })\";\n   \n   UGenAISchemaService::RequestStructuredOutput(\n       TEXT(\"Generate a list of users and their details\"),\n       MySchemaJson,\n       [](const FString\u0026 Response, const FString\u0026 Error, bool Success) {\n          if (Success)\n          {\n              UE_LOG(LogTemp, Log, TEXT(\"Structured Output: %s\"), *Response);\n          }\n          else\n          {\n              UE_LOG(LogTemp, Error, TEXT(\"Error: %s\"), *Error);\n          }\n       }\n   );\n   ```\n   ##### C++ Example 2:\n   Sending a custom schema json from a file\n   ```cpp\n   #include \"Misc/FileHelper.h\"\n   #include \"Misc/Paths.h\"\n   FString SchemaFilePath = FPaths::Combine(\n       FPaths::ProjectDir(),\n       TEXT(\"Source/:ProjectName/Public/AIPrompts/SomeSchema.json\")\n   );\n   \n   FString MySchemaJson;\n   if (FFileHelper::LoadFileToString(MySchemaJson, *SchemaFilePath))\n   {\n       UGenAISchemaService::RequestStructuredOutput(\n           TEXT(\"Generate a list of users and their details\"),\n           MySchemaJson,\n           [](const FString\u0026 Response, const FString\u0026 Error, bool Success) {\n              if (Success)\n              {\n                  UE_LOG(LogTemp, Log, TEXT(\"Structured Output: %s\"), *Response);\n              }\n              else\n              {\n                  UE_LOG(LogTemp, Error, TEXT(\"Error: %s\"), *Error);\n              }\n           }\n       );\n   }\n   ```\n\n##### Blueprint Example:\n\u003cimg src=\"Docs/BpExampleOAIStructuredOp.png\" width=\"782\"/\u003e\n\n### DeepSeek API:\n\nCurrently the plugin supports Chat and Reasoning from DeepSeek API. Both for C++ and Blueprints.\nPoints to note:\n* System messages are currently mandatory for the reasoning model. API otherwise seems to return null\n* Also, from the documentation: \"Please note that if the reasoning_content field is included in the sequence of input messages, the API will return a 400 error.\n  Read more about it [here](https://api-docs.deepseek.com/guides/reasoning_model)\"\n\n\u003e [!WARNING]  \n\u003e While using the R1 reasoning model, make sure the Unreal's HTTP timeouts are not the default values at 30 seconds.\n\u003e As these API calls can take longer than 30 seconds to respond. Simply setting the `HttpRequest-\u003eSetTimeout(\u003cN Seconds\u003e);` is not enough\n\u003e So the following lines need to be added to your project's `DefaultEngine.ini` file:\n\u003e ```ini\n\u003e [HTTP]\n\u003e HttpConnectionTimeout=180\n\u003e HttpReceiveTimeout=180\n\u003e ```\n\n#### 1. Chat and Reasoning:\n##### C++ Example:\n\n   ```cpp\n    FGenDSeekChatSettings ReasoningSettings;\n    ReasoningSettings.Model = EDeepSeekModels::Reasoner; // or EDeepSeekModels::Chat for Chat API\n    ReasoningSettings.MaxTokens = 100;\n    ReasoningSettings.Messages.Add(FGenChatMessage{TEXT(\"system\"), TEXT(\"You are a helpful assistant.\")});\n    ReasoningSettings.Messages.Add(FGenChatMessage{TEXT(\"user\"), TEXT(\"9.11 and 9.8, which is greater?\")});\n    ReasoningSettings.bStreamResponse = false;\n    UGenDSeekChat::SendChatRequest(\n        ReasoningSettings,\n        FOnDSeekChatCompletionResponse::CreateLambda(\n            [this](const FString\u0026 Response, const FString\u0026 ErrorMessage, bool bSuccess)\n            {\n                if (!UTHelper::IsContextStillValid(this))\n                {\n                    return;\n                }\n\n                // Log response details regardless of success\n                UE_LOG(LogTemp, Warning, TEXT(\"DeepSeek Reasoning Response Received - Success: %d\"), bSuccess);\n                UE_LOG(LogTemp, Warning, TEXT(\"Response: %s\"), *Response);\n                if (!ErrorMessage.IsEmpty())\n                {\n                    UE_LOG(LogTemp, Error, TEXT(\"Error Message: %s\"), *ErrorMessage);\n                }\n            })\n    );\n   ```\n\n##### Blueprint Example:\n\u003cimg src=\"Docs/BpExampleDeepseekChat.png\" width=\"782\"/\u003e\n\n### Anthropic API:\nCurrently the plugin supports Chat from Anthropic API. Both for C++ and Blueprints.\nTested models are `claude-sonnet-4-20250514`, `claude-opus-4-20250514`, `claude-3-7-sonnet-latest`, `claude-3-5-sonnet`, `claude-3-5-haiku-latest`, `claude-3-opus-latest`.\n\n#### 1. Chat:\n##### C++ Example:\n```cpp\n    // ---- Claude Chat Test ----\n    FGenClaudeChatSettings ChatSettings;\n    ChatSettings.Model = EClaudeModels::Claude_3_7_Sonnet; // Use Claude 3.7 Sonnet model\n    ChatSettings.MaxTokens = 4096;\n    ChatSettings.Temperature = 0.7f;\n    ChatSettings.Messages.Add(FGenChatMessage{TEXT(\"system\"), TEXT(\"You are a helpful assistant.\")});\n    ChatSettings.Messages.Add(FGenChatMessage{TEXT(\"user\"), TEXT(\"What is the capital of France?\")});\n    \n    UGenClaudeChat::SendChatRequest(\n        ChatSettings,\n        FOnClaudeChatCompletionResponse::CreateLambda(\n            [this](const FString\u0026 Response, const FString\u0026 ErrorMessage, bool bSuccess)\n            {\n                if (!UTHelper::IsContextStillValid(this))\n                {\n                    return;\n                }\n    \n                if (bSuccess)\n                {\n                    UE_LOG(LogTemp, Warning, TEXT(\"Claude Chat Response: %s\"), *Response);\n                }\n                else\n                {\n                    UE_LOG(LogTemp, Error, TEXT(\"Claude Chat Error: %s\"), *ErrorMessage);\n                }\n            })\n    );\n```\n\n##### Blueprint Example:\n\u003cimg src=\"Docs/BpExampleClaudeChat.png\" width=\"782\"/\u003e\n\n### XAI's Grok 3 API:\nCurrently the plugin supports Chat from XAI's Grok 3 API. Both for C++ and Blueprints.\n\n#### 1. Chat:\n```cpp\n\tFGenXAIChatSettings ChatSettings;\n\tChatSettings.Model = TEXT(\"grok-3-latest\");\n\t\tChatSettings.Messages.Add(FGenXAIMessage{\n\t\tTEXT(\"system\"),\n\t\tTEXT(\"You are a helpful AI assistant for a game. Please provide concise responses.\")\n\t});\n\tChatSettings.Messages.Add(FGenXAIMessage{TEXT(\"user\"), TEXT(\"Create a brief description for a forest level in a fantasy game\")});\n\tChatSettings.MaxTokens = 1000;\n\n\tUGenXAIChat::SendChatRequest(\n\t\tChatSettings,\n\t\tFOnXAIChatCompletionResponse::CreateLambda(\n\t\t\t[this](const FString\u0026 Response, const FString\u0026 ErrorMessage, bool bSuccess)\n\t\t\t{\n\t\t\t\tif (!UTHelper::IsContextStillValid(this))\n\t\t\t\t{\n\t\t\t\t\treturn;\n\t\t\t\t}\n\t\t\t\t\n\t\t\t\tUE_LOG(LogTemp, Warning, TEXT(\"XAI Chat response: %s\"), *Response);\n\t\t\t\t\n\t\t\t\tif (!bSuccess)\n\t\t\t\t{\n\t\t\t\t\tUE_LOG(LogTemp, Error, TEXT(\"XAI Chat error: %s\"), *ErrorMessage);\n\t\t\t\t}\n\t\t\t})\n\t);\n```\n\n## Model Control Protocol (MCP):\nThis is currently work in progress. The plugin supports various clients like Claude Desktop App, Cursor etc.\n### Usage:\n\n#### If Autostart MCP server is enabled: (In plugin's settings)\n##### 1. Open the Unreal Engine Editor.\n##### 2. Open the Claude Desktop App or Cursor IDE or Windsor. \n\nThat's it! You can now use the MCP features of the plugin.\n\n#### If Autostart MCP server is disabled:\n\n##### 1. Run the MCP server from the plugin's python directory.\n```bash\npython \u003cyour_project_directoy\u003e/Plugins/GenerativeAISupport/Content/Python/mcp_server.py\n```\n##### 2. Run the MCP client by opening or restarting the Claude desktop app or Cursor IDE.\n\n##### 3. Open a new Unreal Engine project and run the below python script from the plugin's python directory.\n\n\u003e Tools -\u003e Run Python Script -\u003e Select the `Plugins/GenerativeAISupport/Content/Python/unreal_socket_server.py` file.\n\n#### 4. Now you should be able to prompt the Claude Desktop App to use Unreal Engine.\n\n## Known Issues:\n- Nodes fail to connect properly with MCP\n- No undo redo support for MCP\n- No streaming support for Deepseek reasoning model\n- No complex material generation support for the create material tool\n- Issues with running some llm generated valid python scripts\n- When LLM compiles a blueprint no proper error handling in its response\n- Issues spawning certain nodes, especially with getters and setters\n- Doesn't open the right context window during scene and project files edit. \n- Doesn't dock the window properly in the editor for blueprints.\n\n## Config Window:\n(Still wip)\n\u003cimg src=\"Docs/EditorWindow.png\" width=\"782\"/\u003e\n\n## Contribution Guidelines:\n\n### Setting up for Development:\n\n1. Install `unreal` python package and setup the IDE's python interpreter for proper intellisense.\n```bash\npip install unreal\n```\n\nMore details will be added soon.\n\n### Project Structure:\n\nMore details will be added soon.\n\n## References:\n\n* Env Var set logic\n  from: [OpenAI-Api-Unreal by KellanM](https://github.com/KellanM/OpenAI-Api-Unreal/blob/main/Source/OpenAIAPI/Private/OpenAIUtils.cpp)\n* MCP Server inspiration\n  from: [Blender-MCP by ahujasid](https://github.com/ahujasid/blender-mcp)\n\n\n## Quick Links:\n\n- [OpenAI API Documentation](https://platform.openai.com/docs/api-reference)\n- [Anthropic API Documentation](https://docs.anthropic.com/en/docs/about-claude/models)\n- [XAI API Documentation](https://docs.x.ai/api)\n- [Google Gemini API Documentation](https://ai.google.dev/gemini-api/docs/models/gemini)\n- [Meta AI API Documentation](https://docs.llama-api.com/quickstart#available-models)\n- [Deepseek API Documentation](https://api-docs.deepseek.com/)\n- [Model Control Protocol (MCP) Documentation](https://modelcontextprotocol.io/)\n- [TripoSt Documentation](https://huggingface.co/stabilityai/TripoSR)\n\n## Support This Project\n\n\u003ciframe src=\"https://github.com/sponsors/prajwalshettydev/button\" title=\"Sponsor prajwalshettydev\" height=\"32\" width=\"114\" style=\"border: 0; border-radius: 6px;\"\u003e\u003c/iframe\u003e\n\nIf you find UnrealGenAISupport helpful, consider sponsoring me to keep the project going! Click the \"Sponsor\" button above to contribute.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprajwalshettydev%2Funrealgenaisupport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprajwalshettydev%2Funrealgenaisupport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprajwalshettydev%2Funrealgenaisupport/lists"}