{"id":13589361,"url":"https://github.com/sha3sha3/UE-EasyKafka","last_synced_at":"2025-04-08T09:32:37.609Z","repository":{"id":133041634,"uuid":"576786990","full_name":"sha3sha3/UE-EasyKafka","owner":"sha3sha3","description":"EasyKafka is a Kafka/Redpanda client sub-system for unreal engine. It supports producing and consuming records through blueprint and C++.","archived":false,"fork":false,"pushed_at":"2024-02-20T11:02:58.000Z","size":24947,"stargazers_count":36,"open_issues_count":5,"forks_count":8,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-06T09:39:36.313Z","etag":null,"topics":["cloud","kafka","microservice","microservices","networking","redpanda","unreal","unreal-engine","unreal-engine-4","unreal-engine-5","unrealengine"],"latest_commit_sha":null,"homepage":"https://www.unrealengine.com/marketplace/en-US/product/client-for-kafka","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sha3sha3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-11T01:05:08.000Z","updated_at":"2024-10-12T05:10:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"cf779d0d-ef71-4a44-9db1-e01ddbbb4e29","html_url":"https://github.com/sha3sha3/UE-EasyKafka","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sha3sha3%2FUE-EasyKafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sha3sha3%2FUE-EasyKafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sha3sha3%2FUE-EasyKafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sha3sha3%2FUE-EasyKafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sha3sha3","download_url":"https://codeload.github.com/sha3sha3/UE-EasyKafka/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247814160,"owners_count":21000511,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","kafka","microservice","microservices","networking","redpanda","unreal","unreal-engine","unreal-engine-4","unreal-engine-5","unrealengine"],"created_at":"2024-08-01T16:00:28.864Z","updated_at":"2025-04-08T09:32:36.227Z","avatar_url":"https://github.com/sha3sha3.png","language":"C++","funding_links":[],"categories":["Networking","Game Development"],"sub_categories":["Unreal Engine: Resources"],"readme":"# This project is no longer maintained for some platforms on GitHub, but you may obtain the latest maintained version on [Unreal Marketplace](https://www.unrealengine.com/marketplace/en-US/product/client-for-kafka). \n\n\u003ch1 align=\"center\"\u003e\n  \u003cbr\u003e\n  \u003ca href=\"https://discord.com/invite/MtGpAdM\"\u003e\n    \u003cimg src=\"Resources/Icon128.png\" alt=\"EasyKafakLogo\" width=\"200\"\u003e\n  \u003c/a\u003e\n  \u003cbr\u003e\n  Kafka Client for Unreal Engine 4/5\n  \u003cbr\u003e\n\u003c/h1\u003e\n\n# EasyKafka subsystem\n\nEasyKafka is a Kafka/Redpanda client sub-system for unreal engine. It supports producing and consuming records through blueprint and C++.\n\n# Supported Platforms\n\n - Windows x86_64\n - Hololens 2 (Windows ARM64)\n - Linux x86_64\n - Linux ARM64\n\n# C++ Modules Link\n\nLink the plugin modules to your project through `\u003cYourModule\u003e.build.cs`:\n\n```cs\nCppStandard = CppStandardVersion.Cpp17;//avoid using boost\nif(Target.Platform == UnrealTargetPlatform.HoloLens || Target.Platform == UnrealTargetPlatform.Win64)\n\tbUseRTTI = true;\n\n PrivateDependencyModuleNames.AddRange( new string[]\n{\n    \"EasyKafka\",\n    \"KafkaLib\",\n    \"KafkaConsumer\",\n    \"KafkaProducer\",\n    \"KafkaAdmin\"\n});\n```\n\n# Kafka Consumer Basic Usage\n\n**PAY ATTENTION TO THE BLOCKING METHODS.**\n## C++\n\nCreate Consumer with default configuration:\n\n```cpp\n#include \"EasyKafkaSubsystem.h\"\n\nTSharedPtr\u003cFEasyKafkaModule\u003e EasyKafka = GEngine-\u003eGetEngineSubsystem\u003cUEasyKafkaSubsystem\u003e()-\u003eGetEasyKafka();\nEasyKafka-\u003eGetConsumer()-\u003eCreateConsumer(`\u003cBOOTSTRAP_SERVERS_COMMA_SEPARATED\u003e`, `\u003cUSERNAME\u003e`, `\u003cTOKEN/PASSWORD\u003e`, (int)EKafkaLogLevel::ERR);\n```\n\nCreate Consumer with configuration:\n\n```cpp\n#include \"EasyKafkaSubsystem.h\"\n\nTSharedPtr\u003cFEasyKafkaModule\u003e EasyKafka = GEngine-\u003eGetEngineSubsystem\u003cUEasyKafkaSubsystem\u003e()-\u003eGetEasyKafka();\n\nTMap\u003cEKafkaConsumerConfig, FString\u003e KafkaConfiguration =\n{\n\t{EKafkaConsumerConfig::CLIENT_ID,\"34235\"},\n\t{EKafkaConsumerConfig::SOCKET_TIMEOUT_MS,\"10000\"}\n};\nEasyKafka-\u003eGetConsumer()-\u003eCreateConsumer(`\u003cBOOTSTRAP_SERVERS_COMMA_SEPARATED\u003e`, `\u003cUSERNAME\u003e`, `\u003cTOKEN/PASSWORD\u003e`, KafkaConfiguration, (int)EKafkaLogLevel::ERR);\n```\n\nConsume messages:\n\n```cpp\nEasyKafka-\u003eGetConsumer()-\u003eOnNewMessage().AddLambda([](const TArray\u003cFConsumerRecord\u003e\u0026 Messages)\n{\n\tfor (FConsumerRecord Message : Messages)\n\t{\n\t\tUE_LOG(LogTemp, Display, TEXT(\"New Message %s \\n\"), *Message.Value);//process messages\n\t}\n});\n\nEasyKafka-\u003eGetConsumer()-\u003eSubscribe(\n{\n\t\"topic\",\n\t\"topic1\",\n\t\"topic2\"\n});\n\nEasyKafka-\u003eGetConsumer()-\u003eStartConsuming();\n```\n**ATTENTION: MAKE SURE TO COMMIT FROM THE CONSUMER RUNNABLE THREAD BEFORE PROCESSING RECORDS IF YOU DISABLED AUTOCOMMIT.**\n\n## Blueprint\n\n\u003cimg src=\"Images/ConsumerBP.png\" alt=\"ConsumerBP\"\u003e\n\n# Kafka Producer Basic Usage\n\n**PAY ATTENTION TO THE BLOCKING METHODS.**\n## C++\n\nCreate Producer with default configuration:\n\n```cpp\n#include \"EasyKafkaSubsystem.h\"\n\nTSharedPtr\u003cFEasyKafkaModule\u003e EasyKafka = GEngine-\u003eGetEngineSubsystem\u003cUEasyKafkaSubsystem\u003e()-\u003eGetEasyKafka();\nEasyKafka-\u003eGetProducer()-\u003eCreateProducer(`\u003cBOOTSTRAP_SERVERS_COMMA_SEPARATED\u003e`, `\u003cUSERNAME\u003e`, `\u003cTOKEN/PASSWORD\u003e`, (int)EKafkaLogLevel::ERR);\n```\n\nCreate Producer with configuration:\n\n```cpp\n#include \"EasyKafkaSubsystem.h\"\n\nTSharedPtr\u003cFEasyKafkaModule\u003e EasyKafka = GEngine-\u003eGetEngineSubsystem\u003cUEasyKafkaSubsystem\u003e()-\u003eGetEasyKafka();\n\nTMap\u003cEKafkaProducerConfig, FString\u003e KafkaConfiguration =\n{\n\t{EKafkaProducerConfig::MESSAGE_TIMEOUT_MS,\"5000\"},\n\t{EKafkaProducerConfig::REQUEST_TIMEOUT_MS,\"5000\"}\n};\nEasyKafka-\u003eGetProducer()-\u003eCreateProducer(`\u003cBOOTSTRAP_SERVERS_COMMA_SEPARATED\u003e`, `\u003cUSERNAME\u003e`, `\u003cTOKEN/PASSWORD\u003e`, KafkaConfiguration, (int)EKafkaLogLevel::ERR);\n```\n\non record produced/failed to produce callback\n\n```cpp\nEasyKafka-\u003eGetProducer()-\u003eOnProduce().AddLambda([](const FProducerCallback\u0026 Callback)\n{\n\n\tif (Callback.bError)\n\t{\n\t\tUE_LOG(LogTemp, Error, TEXT(\"Error producing recordId: %d \\nError Message: %s\\n\"), Callback.RecordMetadata.RecordId, *Callback.ErrorMessage);\n\t}\n\telse\n\t{\n\t\tUE_LOG(LogTemp, Display, TEXT(\"RecordId: %d produced.\\n\"), Callback.RecordMetadata.RecordId);\n\t}\n});\n```\nproduce record async\n\n```cpp\nEasyKafka-\u003eGetProducer()-\u003eProduceRecord('\u003cTOPIC\u003e', '\u003c\"RECORD_VALUE\u003e');\n\n/*\nMore control over your record\nSuch as headers,Id...\n*/\n\nFProducerRecord record;\nrecord.Key = \"key\";\nrecord.Topic = \"topic\";\nrecord.Value = \"value\";\nrecord.Id = 2312;//Unique id to identify this record OnProduce callback;\nrecord.Headers = FRecordHeader(\n{ \n\t{\"KeyOne\",\"ValueOne\"},\n\t{\"KeyTwo\",\"ValueTwo\"}\n});\n\nEasyKafka-\u003eGetProducer()-\u003eProduceRecord(record);\n```\n## Blueprint\n\n\u003cimg src=\"Images/ProducerBP.png\" alt=\"ProducerBP\"\u003e\n\n# Kafka Admin Basic Usage\n\n**ALL THE METHODS ARE BLOCKING, ASYNC TO BE ADDED.**\n## C++\nCreate Admin with default configuration:\n\n```cpp\n#include \"EasyKafkaSubsystem.h\"\n\nTSharedPtr\u003cFEasyKafkaModule\u003e EasyKafka = GEngine-\u003eGetEngineSubsystem\u003cUEasyKafkaSubsystem\u003e()-\u003eGetEasyKafka();\nEasyKafka-\u003eGetAdmin()-\u003eCreateAdmin(`\u003cBOOTSTRAP_SERVERS_COMMA_SEPARATED\u003e`, `\u003cUSERNAME\u003e`, `\u003cTOKEN/PASSWORD\u003e`, (int)EKafkaLogLevel::ERR);\n```\nCreate Admin with configuration:\n\n```cpp\n#include \"EasyKafkaSubsystem.h\"\n\nTSharedPtr\u003cFEasyKafkaModule\u003e EasyKafka = GEngine-\u003eGetEngineSubsystem\u003cUEasyKafkaSubsystem\u003e()-\u003eGetEasyKafka();\n\nTMap\u003cEKafkaAdminConfig, FString\u003e KafkaConfiguration =\n{\n\t{EKafkaAdminConfig::SOCKET_TIMEOUT_MS,\"10000\"}\n};\nEasyKafka-\u003eGetAdmin()-\u003eCreateAdmin(`\u003cBOOTSTRAP_SERVERS_COMMA_SEPARATED\u003e`, `\u003cUSERNAME\u003e`, `\u003cTOKEN/PASSWORD\u003e`, KafkaConfiguration, (int)EKafkaLogLevel::ERR);\n```\nSimple Admin request example:\n\n```cpp\nconst TArray\u003cFString\u003e TopicsToDelete = { \"Topic1Name\", \"Topic2Name\" };\nFAdminRequestResult Result = EasyKafka-\u003eGetAdmin()-\u003eDeleteTopics(TopicsToDelete);\n\nif (Result.bError)\n{\n\tUE_LOG(LogTemp, Error, TEXT(\"Error deleting topics:  %s\\n\"), *Result.ErrorMessage);\n}\n```\n## Blueprint\n\n\u003cimg src=\"Images/AdminBP.png\" alt=\"ProducerBP\"\u003e\n\n## Find it helpful?\n\nGive us a ⭐️!\n\u003c!---\n# License\n\nLicensed under BSD 2-Clause [License](LICENSE.txt).\n\n```\nCopyright (c) 2022-2023 MrShaaban, Mohamad Shaaban.\n```\n--\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsha3sha3%2FUE-EasyKafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsha3sha3%2FUE-EasyKafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsha3sha3%2FUE-EasyKafka/lists"}