{"id":19315657,"url":"https://github.com/ellraiser/love-test","last_synced_at":"2025-04-22T16:32:34.005Z","repository":{"id":197881947,"uuid":"699572953","full_name":"ellraiser/love-test","owner":"ellraiser","description":"test suite for the LÖVE API","archived":false,"fork":false,"pushed_at":"2025-03-01T20:34:29.000Z","size":597,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-02T01:42:24.642Z","etag":null,"topics":["love","love2d","testing","testing-framework","testing-tools"],"latest_commit_sha":null,"homepage":"https://love2d.org/","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ellraiser.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-02T22:40:39.000Z","updated_at":"2025-03-01T20:34:33.000Z","dependencies_parsed_at":"2023-11-23T00:21:27.221Z","dependency_job_id":"beea68da-6179-4b64-9264-6f3ca76fb13e","html_url":"https://github.com/ellraiser/love-test","commit_stats":null,"previous_names":["ellraiser/love-test"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ellraiser%2Flove-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ellraiser%2Flove-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ellraiser%2Flove-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ellraiser%2Flove-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ellraiser","download_url":"https://codeload.github.com/ellraiser/love-test/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250277358,"owners_count":21404033,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["love","love2d","testing","testing-framework","testing-tools"],"created_at":"2024-11-10T01:07:49.131Z","updated_at":"2025-04-22T16:32:33.597Z","avatar_url":"https://github.com/ellraiser.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lövetest\nTest suite for the [Löve](https://github.com/love2d/love) APIs, based off of [this issue](https://github.com/love2d/love/issues/1745).\n\nCurrently written for [Löve 12](https://github.com/love2d/love/tree/12.0-development), which is still in development. As such the test suite may fail if you try to run it with an older version of Löve due to it trying to call methods that don't exist.\n\nWhile the test suite is part of the main Löve repo, the test suite has it's own repo here so that it can be used with other builds like [love-potion](https://github.com/lovebrew/lovepotion). If you would like to contribute to the test suite please raise a PR on this repo.\n\n---\n\n## Features\n- [x] Simple pass/fail tests written in Lua with minimal setup \n- [x] Ability to run all tests with a simple command\n- [x] Ability to see how many tests are passing/failing\n- [x] Ability to run a subset of tests\n- [x] Ability to easily run an individual test\n- [x] Ability to see all visual results at a glance\n- [x] Compare graphics test output with an expected output\n- [x] Automatic testing that happens after every commit\n- [x] No platform-specific dependencies / scripts\n\n---\n\n## Coverage\nThis is the status of all module tests.  \nSee the **Todo** section for outstanding tasks if you want to contribute!\n| Module            | Done | Skip | Modules          | Done | Skip |\n| ----------------- | ---- | ---- | ---------------- | ---- | ---- |\n| 🟢 audio          |   31 |   0  | 🟢 mouse          |   18 |   0  |\n| 🟢 data           |   12 |   0  | 🟢 physics        |   26 |   0  |\n| 🟢 event          |    4 |   2  | 🟢 sensor         |    1 |   0  |\n| 🟢 filesystem     |   33 |   2  | 🟢 sound          |    4 |   0  |\n| 🟢 font           |    7 |   0  | 🟢 system         |    7 |   2  |\n| 🟢 graphics       |  105 |   1  | 🟢 thread         |    5 |   0  |\n| 🟢 image          |    5 |   0  | 🟢 timer          |    6 |   0  |\n| 🟢 joystick       |    6 |   0  | 🟢 touch          |    3 |   0  |\n| 🟢 keyboard       |   10 |   0  | 🟢 video          |    2 |   0  |\n| 🟢 love           |    6 |   0  | 🟢 window         |   34 |   2  |\n| 🟢 math           |   20 |   0  | \n\n\u003e The following modules are covered but at a basic level as we can't emulate hardware input nicely for all platforms + virtual runners:  \n\u003e `joystick`, `keyboard`, `mouse`, `sensor` and `touch`\n\n---\n\n## Running Tests\nThe testsuite aims to keep things as simple as possible, and just runs all the tests inside Löve to match how they'd be used by developers in-engine.\nTo run the tests, download the repo and then run the main.lua as you would a Löve game, i.e:\n\nWINDOWS: `\u0026 'c:\\Program Files\\LOVE\\love.exe' PATH_TO_TESTING_FOLDER/main.lua --console`  \nMACOS: `/Applications/love.app/Contents/MacOS/love PATH_TO_TESTING_FOLDER/main.lua`  \nLINUX: `./love.AppImage PATH_TO_TESTING_FOLDER/main.lua`\n\nBy default all tests will be run for all modules.  \nIf you want to specify a module/s you can use:  \n`--modules filesystem,audio`  \nIf you want to specify only 1 specific method only you can use:  \n`--method filesystem write`\n\nAll results will be printed in the console per method as PASS, FAIL, or SKIP with total assertions met on a module level and overall level.  \n\nWhen finished, the following files will be generated in the `/output` directory with a summary of the test results:\n- an `XML` file in the style of [JUnit XML](https://www.ibm.com/docs/en/developer-for-zos/14.1?topic=formats-junit-xml-format)\n- a `HTML` file that shows the report + any visual test results\n- a `Markdown` file you can use with [this github action](https://github.com/ellraiser/love-test-report)\n\u003e An example of all types of output can be found in the `/examples`  \n\u003e The visual results of any graphic tests can be found in `/output/actual`\n\n---\n\n## Architecture\nEach method and object has it's own test method written in `/tests` under the matching module name.\n\nWhen you run the tests, a single TestSuite object is created which handles the progress + totals for all the tests.  \nEach module has a TestModule object created, and each test method has a TestMethod object created which keeps track of assertions for that method. You can currently do the following assertions:\n- **assertNotNil**(value)\n- **assertEquals**(expected, actual, label)\n- **assertTrue**(value, label)\n- **assertFalse**(value, label)\n- **assertNotEquals**(expected, actual, label)\n- **assertRange**(actual, min, max, label)\n- **assertMatch**({option1, option2, option3 ...}, actual, label) \n- **assertGreaterEqual**(expected, actual, label)\n- **assertLessEqual**(expected, actual, label)\n- **assertObject**(table)\n- **assertCoords**(expected, actual, label)\n\nExample test method:\n```lua\n-- love.filesystem.read test method\n-- all methods should be put under love.test.MODULE.METHOD, matching the API\nlove.test.filesystem.read = function(test)\n  -- setup any data needed then run any asserts using the passed test object\n  local content, size = love.filesystem.read('resources/test.txt')\n  test:assertNotNil(content)\n  test:assertEquals('helloworld', content, 'check content match')\n  test:assertEquals(10, size, 'check size match')\n  content, size = love.filesystem.read('resources/test.txt', 5)\n  test:assertNotNil(content)\n  test:assertEquals('hello', content, 'check content match')\n  test:assertEquals(5, size, 'check size match')\n  -- no need to return anything or cleanup, GCC is called after each method\nend\n```\n\nEach test is run inside it's own coroutine - you can use `test:waitFrames(frames)` or `test:waitSeconds(seconds)` to pause the test for a small period if you need to check things that won't happen for a few frames/seconds.\n\nAfter each test method is ran, the assertions are totalled up, printed, and we move onto the next method! Once all methods in the suite are run a total pass/fail/skip is given for that module and we move onto the next module (if any)\n\nFor sanity-checking, if it's currently not covered or it's not possible to test the method we can set the test to be skipped with `test:skipTest(reason)` - this way we still see the method listed in the test output without it affected the pass/fail totals\n\n---\n\n## Todo\nIf you would like to contribute to the test suite please raise a PR in this repo.\n\nThere is a list of outstanding methods that require test coverage in `todo.md`, expanding on any existing tests is also very welcome!\n\n---\n\n## Graphics Tolerance\nBy default all graphic tests are run with pixel precision and 0 rgba tolerance.  \n\nHowever there are a couple of methods that on some platforms require some slight tolerance to allow for tiny differences in rendering.\n| Test                        |    OS     |      Exception      | Reason |\n| --------------------------  | --------- | ------------------- | ------ |\n| love.graphics.drawInstanced |  Windows  |   1rgba tolerance   | On Windows there's a couple pixels a tiny bit off, most likely due to complexity of the mesh drawn |\n| love.graphics.setBlendMode  |  Win/Lin  |   1rgba tolerance   | Blendmodes have some small varience on some machines |\n\n---\n\n## Runner Exceptions\nThe automated tests through Github work for the most part however there are a few exceptions that have to be accounted for due to limitations of the VMs and the graphics emulation used.  \n\nThese exceptions are either skipped, or handled by using a 1px or 1/255rgba tolerance - when run locally on real hardware, these tests pass fine at the default 0 tolerance.  \nYou can specify the test suite is being run on a runner by adding the `--isRunner` flag in your workflow file, i.e.:  \n`\u0026 'c:\\Program Files\\LOVE\\love.exe' PATH_TO_TESTING_FOLDER/main.lua --console --all --isRunner`\n| Test                       |    OS     |      Exception      | Reason |\n| -------------------------- | --------- | ------------------- | ------ |\n| love.graphics.setWireframe |   MacOS   |    1px tolerance    | Wireframes are offset by 1,1 when drawn |\n| love.graphica.arc          |   MacOS   |       Skipped       | Arc curves are drawn slightly off at really low scale  |\n| love.graphics.setLineStyle |   Linux   |   1rgba tolerance   | 'Rough' lines blend differently with the background rgba |\n| love.audio.RecordingDevice |    All    |       Skipped       | Recording devices can't be emulated on runners |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fellraiser%2Flove-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fellraiser%2Flove-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fellraiser%2Flove-test/lists"}