{"id":16681591,"url":"https://github.com/slonopotamus/uest","last_synced_at":"2025-03-21T18:32:38.711Z","repository":{"id":251269249,"uuid":"836906378","full_name":"slonopotamus/UEST","owner":"slonopotamus","description":"Unreal Engine Suckless Testing","archived":false,"fork":false,"pushed_at":"2025-01-28T12:53:37.000Z","size":141,"stargazers_count":19,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-18T03:51:23.811Z","etag":null,"topics":["testing","unreal-engine"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slonopotamus.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-01T20:02:47.000Z","updated_at":"2025-03-06T13:51:12.000Z","dependencies_parsed_at":"2024-08-01T22:27:35.522Z","dependency_job_id":"47160829-14f5-43cf-92a9-f7ffb4049c2f","html_url":"https://github.com/slonopotamus/UEST","commit_stats":null,"previous_names":["slonopotamus/uest"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slonopotamus%2FUEST","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slonopotamus%2FUEST/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slonopotamus%2FUEST/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slonopotamus%2FUEST/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slonopotamus","download_url":"https://codeload.github.com/slonopotamus/UEST/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244848999,"owners_count":20520625,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["testing","unreal-engine"],"created_at":"2024-10-12T14:04:44.731Z","updated_at":"2025-03-21T18:32:38.648Z","avatar_url":"https://github.com/slonopotamus.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"= UEST (Unreal Engine Suckless Testing)\n:icons: font\n\nifdef::env-github[]\n:caution-caption: :fire:\n:important-caption: :exclamation:\nendif::[]\n\nThis project aims to provide a testing framework for Unreal Engine that does not suck.\n\n== Goals\n\n* Simple things should be simple, complex things should be possible\n* Extensibility\n* Async execution support\n* IDE and CI first-class support\n* User-visible API ergonomics is more important than internal implementation\n\n== Dependencies\n\nThis project relies on C++20 features.\nTested against Unreal Engine 5.4.\n\n== Usage\n\n=== Defining tests\n\n.Simple test\n[source,cpp]\n----\n#include \"UEST.h\"\n\nTEST(MyFancyTest)\n{\n    // test body goes here\n    ASSERT_THAT(...);\n}\n----\n\n.Test class with multiple methods\n[source,cpp]\n----\n#include \"UEST.h\"\n\nTEST_CLASS(MyFancyTestClass)\n{\n    TEST_METHOD(Method1)\n    {\n        // test body goes here\n        ASSERT_THAT(...);\n    }\n\n    TEST_METHOD(Method2)\n    {\n        // test body goes here\n        ASSERT_THAT(...);\n    }\n\n    // put helper fields or methods here\n}\n----\n\nIf you want to execute a common piece of logic before and after each test method in a test class, you can do that using `BEFORE_EACH`/`AFTER_EACH` macros:\n\n[source,cpp]\n----\n#include \"UEST.h\"\n\nTEST_CLASS(MyFancyTestClass)\n{\n    BEFORE_EACH()\n    {\n        // Place code that will be executed before each test method of this class\n    }\n\n    AFTER_EACH()\n    {\n        // Place code that will be executed after each test method of this class\n    }\n\n    ...\n}\n----\n\n=== Assertions\n\nAll UEST assertions are done through `ASSERT_THAT(Expression, Matcher)`.\nFailed assertion performs `return`, aborting further test execution.\n\n.Available matchers\n`ASSERT_THAT(Value, Is::True)`:: Tests that `Value` is `true`.\n`ASSERT_THAT(Value, Is::False)`:: Tests that `Value` is `false`.\n`ASSERT_THAT(Value, Is::Null)`:: Tests that `Value` is `nullptr`.\n`ASSERT_THAT(Value, Is::EqualTo(Expected))`:: Tests that `Value` is equal to `Expected`.\n`ASSERT_THAT(Value, Is::LessThan(OtherValue))`:: Tests that `Value` is less than `OtherValue`.\n`ASSERT_THAT(Value, Is::LessThanOrEqualTo(OtherValue))` or `ASSERT_THAT(Value, Is::AtMost(OtherValue)`:: Tests that `Value` is less than or equal to `OtherValue`.\n`ASSERT_THAT(Value, Is::GreaterThan(OtherValue))`:: Tests that `Value` is greater than `OtherValue`.\n`ASSERT_THAT(Value, Is::GreaterThanOrEqualTo(OtherValue))` or `ASSERT_THAT(Value, Is::AtLeast(OtherValue)`:: Tests that `Value` is greater than or equal to `OtherValue`.\n`ASSERT_THAT(Value, Is::Zero)`:: Shortcut for `ASSERT_THAT(Value, Is::EqualTo(0))`.\n`ASSERT_THAT(Value, Is::Positive)`:: Shortcut for `ASSERT_THAT(Value, Is::GreaterThan(0))`.\n`ASSERT_THAT(Value, Is::Negative)`:: Shortcut for `ASSERT_THAT(Value, Is::LessThan(0))`.\n`ASSERT_THAT(Value, Is::InRange(From, To))`:: Tests that `Value` is greater than or equal to `From` and is less than or equal to `To`.\n`ASSERT_THAT(Value, Is::Empty)`:: Tests that `Value` is empty using its `IsEmpty()` method.\nUse this for `FString` or collections (`TArray`, `TMap`, etc).\n`ASSERT_THAT(Value, Is::Valid)`:: Tests that `Value` is valid using its `IsValid()` method.\nUse this for `TSharedPtr`, `TWeakObjectPtr` or `TWeakPtr`.\n`ASSERT_THAT(Value, Is::NaN)`:: Tests that `Value` is floating NaN.\nSupports both float and double.\n\nIMPORTANT: Because of the https://github.com/llvm/llvm-project/issues/73093[bug in Clang template type deduction] in versions older than 19.0, matchers with parameters (`LessThan`, `GreaterThan`, `EqualTo` and so on) require explicit template type specification: `ASSERT_THAT(0, Is::LessThan\u003cint\u003e(1))`.\n\nYou can also negate assertions using `ASSERT_THAT(Value, Is::Not::\u003cmatcher\u003e)`.\n\nNegated assertion example:\n[source,cpp]\n----\nASSERT_THAT(Value, Is::Not::Null);\n----\n\n#TODO: Document how to write custom matchers#\n\n== Running tests\n\nUEST is seamlessly integrated into Unreal Engine testing infrastructure, so you can run them using standard Session Frontend or IDE integration plugins.\n\n=== Testing game worlds\n\nUEST provides a convenient way to test game worlds, both standalone and multiplayer.\n\n.Basic usage\n[source,cpp]\n----\nTEST(MyGame, SimpleMultiplayerTest)\n{\n\tauto Tester = FScopedGame().Create();\n\n\t// You can create a dedicated server\n\tUGameInstance* Server = Tester.CreateGame(EScopedGameType::Server, TEXT(\"/Engine/Maps/Entry\"));\n\n\t// You can connect a client to it\n\tUGameInstance* Client = Tester.CreateClientFor(Server);\n\tASSERT_THAT(Client, Is::Not::Null);\n\n\t// Actually, you can connect as many clients as you want!\n\tfor (int32 Index = 0; Index \u003c 10; ++Index)\n\t{\n\t\tTester.CreateClientFor(Server);\n\t}\n\n\t// You can access game worlds\n\tUWorld* ServerWorld = Server-\u003eGetWorld();\n\tASSERT_THAT(ServerWorld, Is::Not::Null);\n\tUWorld* ClientWorld = Client-\u003eGetWorld();\n\tASSERT_THAT(ClientWorld, Is::Not::Null);\n\n\t// You can access actors in worlds\n\tAPlayerController* ClientPC = ClientWorld-\u003eGetFirstPlayerController();\n\tASSERT_THAT(ClientPC, Is::Not::Null);\n\n\t// You can lookup matching replicated actors in paired worlds\n\tAPlayerController* ServerPC = Tester.FindReplicatedObjectIn(ClientPC, Server-\u003eGetWorld());\n\tASSERT_THAT(ServerPC, Is::Not::Null);\n\n\t// You can advance game time\n\tTester.Tick(1);\n\n\t// You can shut down individual game instances\n\tTester.DestroyGame(Client);\n\n\t// You can also create standalone game worlds\n\tUGameInstance* Standalone = Tester.CreateGame(EScopedGameType::Client, TEXT(\"/Engine/Maps/Entry\"));\n\n\t// Tester automatically cleans everything up when goes out of scope\n}\n----\n\n== Further development plans\n\n* More matchers\n* Add `ASSERT_MULTIPLE` that allows performing multiple assertions without interrupting execution between them, also known as \"soft assertions\".\n* Add API to disable tests (with `EAutomationTestFlags::Disabled` under the hood)\n* Add API for asynchronous/latent tests\n\n== Analysis of existing Unreal Engine solutions\n\nAs of 5.4, Unreal Engine has 4 (FOUR, that's not a typo) APIs for writing tests and all are very far from being good for various reasons.\n\nLet's analyze them one-by-one.\n\n=== Automation Test\n\n[source,cpp]\n----\nIMPLEMENT_SIMPLE_AUTOMATION_TEST(FMyTest, \"MyGame.MyTest\", EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::ProductFilter)\nbool FMyTest::RunTest(const FString\u0026 Parameters)\n{\n    UTEST_TRUE_EXPR(true);\n\n    return true;\n}\n----\n\n.The good\n* VisualStudio and JetBrains Rider know how to run this.\n* `UTEST*` macros interrupt test execution (though these macros are useless for all other test frameworks because of non-void `return false;`)\n\n.The bad\n* Assertions do not capture expression that is being tested.\nYou have to write descriptive messages by hand.\n* Overcomplicated way to add multiple tests with common logic.\n\n.The ugly\n* You need to write your test name *three times* as if it isn't clear enough what test name actually is.\n* Requires lots of typing.\nMacro could easily declare `RunTest` signature automatically.\nAlso, almost nobody wants to use custom flags.\n* You must return a `bool` from the test.\nIf test reports an error, it should be marked as failing.\nIf there are no errors, it should be marked as successful.\nThis bool adds a completely useless (and even harmful) way to *fail without a message*.\n* Nontrivial assertions (like `UTEST_EQUAL_EXPR`) are unable to print exact values of actual/expected.\n* Inadequate support for async tests.\nAs soon as something becomes async, test body transforms into `ADD_LATENT_AUTOMATION_COMMAND` monster without an easy way of passing data between commands.\n\n=== Automation Spec\n\n[source,cpp]\n----\nDEFINE_SPEC(MySpec, \"MyGame.MySpec\", EAutomationTestFlags::ProductFilter | EAutomationTestFlags::ApplicationContextMask)\nvoid MyCustomSpec::Define()\n{\n    TestTrue(TEXT(\"True should be true\"), true);\n}\n----\n\n.The good\n* Understood by VS and Rider\n* `void` return type\n* Better async execution support, but not the best.\nProgramming community developed much better techniques than callback hell.\n* May attract people that are familiar with spec-based approach from other areas.\n\n.The ugly\n* Declaring test name three times again\n* Flags again\n* No builtin way to interrupt test execution when assertion fails, so people have to invent their own assrtion macros.\n\n=== Low Level\n\n[source,cpp]\n----\nTEST_CASE(\"MyGame.MyTest\", \"[ApplicationContextMask][ProductFilter]\")\n{\n    REQUIRE(true);\n}\n----\n\n.The good\n* Test name is written only once...\nWell, no.\n+\n--\nThe caveat is that `TEST_CASE` macro uses a very broken way to generate unique class names.\nThey collide across compilation units and namespaces, and you end up asking yourself \"why my test doesn't register at all\".\nInstead, Epics tell users to use `TEST_CASE_NAMED`, where you need to write test name _twice_.\nThat way, you end up with the same test class name collision chances as other approaches.\n--\n\n.The bad\n* Not understood by Rider (https://youtrack.jetbrains.com/issue/RIDER-110897[RIDER-110897])\n\n.The ugly\n* String tags, really?\nI am more than sure people will make typos and spend multiple hours trying to figure out why their test doesn't run.\n* Assertions are a joke.\n+\n--\nJust look at it:\n\n[source,cpp]\n----\n#define REQUIRE(Expr) if (!(Expr)) { FAutomationTestFramework::Get().GetCurrentTest()-\u003eAddError(TEXT(\"Required condition failed, interrupting test\")); return; }\n----\n\nYep, you guessed it right, all you will get for failed assertion is \"Required condition failed, interrupting test\"\n--\n\n=== CQTest\n\n[source,cpp]\n----\nTEST(MyTest, \"MyGame\")\n{\n    ASSERT_THAT(IsTrue(true));\n}\n----\n\n.The good\n* Test name is written only once\n* No more flags\n* `AreEqual` assertion is extensible and can print arbitrary types in error messages\n* `void` test body\n* Nice way to add multiple test methods to a single test class\n\n.The bad\n* Not understood by Rider (https://youtrack.jetbrains.com/issue/RSCPP-36039/Support-Unreal-Engine-CQTest-framework)[RSCPP-36039]).\nNot sure about VS, would not be surprised if situation is the same.\n* Async execution is as bad as in Automation Test style\n* `clang-format` is unable to properly indent `TEST_CLASS` with nested `TEST_METHOD`\n\n.The ugly\n* Assertions do not capture tested expression.\n`Expected condition to be true.`, seriously?\n* Inadequate way to add custom assertions.\nYou need to use custom macros instead of `TEST` and `TEST_CLASS` because they hardcode `FNoDiscardAsserter`.\nAnd this framework claims they are about composition instead of inheritance!\nThere was absolutely zero reason to tie test class to a _single_ asserter.\nAsserter could easily be absolutely external class to the test itself, see NUnit for example.\n\n// TODO: Write about AFunctionalTest, DaedalicTestAutomationPlugin, Gauntlet\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslonopotamus%2Fuest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslonopotamus%2Fuest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslonopotamus%2Fuest/lists"}