{"id":22240181,"url":"https://github.com/limeraiin/unity-machine-learning-test-comparison","last_synced_at":"2026-05-05T21:39:00.460Z","repository":{"id":174525878,"uuid":"366830279","full_name":"limeraiin/unity-machine-learning-test-comparison","owner":"limeraiin","description":"In this machine learning experiment, I have analyzed the effect of using floating point numbers instead of integers in AI actions. ","archived":false,"fork":false,"pushed_at":"2021-05-12T21:14:01.000Z","size":289,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-30T09:29:10.582Z","etag":null,"topics":["machine-learning","machine-learning-projects","ml","pytorch","tensorboard","unity3d"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/limeraiin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-12T19:24:33.000Z","updated_at":"2024-03-12T15:48:27.000Z","dependencies_parsed_at":"2023-07-28T21:15:53.497Z","dependency_job_id":null,"html_url":"https://github.com/limeraiin/unity-machine-learning-test-comparison","commit_stats":null,"previous_names":["limeraiin/unity-machine-learning-test-comparison"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limeraiin%2Funity-machine-learning-test-comparison","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limeraiin%2Funity-machine-learning-test-comparison/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limeraiin%2Funity-machine-learning-test-comparison/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limeraiin%2Funity-machine-learning-test-comparison/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/limeraiin","download_url":"https://codeload.github.com/limeraiin/unity-machine-learning-test-comparison/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245440671,"owners_count":20615673,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","machine-learning-projects","ml","pytorch","tensorboard","unity3d"],"created_at":"2024-12-03T03:29:10.897Z","updated_at":"2026-05-05T21:38:55.433Z","avatar_url":"https://github.com/limeraiin.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Continuous Vs. Discrete Actions in AI Behaviour \nIn this machine learning experiment, I have analyzed the effect of using floating point numbers instead of integers in AI actions. \n\n## About the Experiment\n\nI have created a very basic machine learning scenario. The AI has to prevent colliding with obstacles and the ceiling. It gets a little reward for each successful dodging and a big punishment for every colliding.  \n\nUsed tools: Pytorch, ML Agents (1.1.0-preview 3), Tensorboard, Unity3D.\n\n### Test field:\n\u003cp align=\"center\"\u003e\n   \u003cimg src=\"https://user-images.githubusercontent.com/64265868/118036872-c9049b80-b375-11eb-8db3-eeefac0763c0.png\" alt=\"drawing\" width=\"1000\"/\u003e\n\u003c/p\u003e\n\n8 different playgrounds are used to speed up the learning process. Children of each agent has a vertically attached \"Ray Perception Sensor\" component. There have been no observations other than that.\n\nThere are rewarding walls that agents have to collide with behind every obstacle.  \n(Obsticles are randomly spawned)\n\n## Results\n\nBoth tests ran for one million steps. Blue and orange lines represent the mean reward for float and integer actions of the AI respectively.  \n(Each reward has a value of 0.1f, punishments have -1f)\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/64265868/118035386-d15bd700-b373-11eb-8858-84aa18f43515.jpg\" alt=\"drawing\" width=\"600\"/\u003e\n\u003c/p\u003e\n\nIt was pretty hard to control the jumps even while the agent was in human control. As can be seen above, using float data type for AI actions shortened the learning period dramatically. After around 800k steps, test runs started to complete with little to no mistakes but they didn't contribute to the mean reward graph.\n\nThe mean loss of the value update correlates to how well the model is able to predict the value of each state. This should increase while the agent is learning, and then decrease once the reward stabilizes.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/64265868/118036278-0d436c00-b375-11eb-912f-2cb735ba0e4b.jpg\" alt=\"drawing\" width=\"600\"/\u003e\n\u003c/p\u003e\n\n## Working Example\n\nRunning a test using the brain of the AI with float type actions after 1 million learning step:\n\n\n![test3](https://user-images.githubusercontent.com/64265868/118038183-79bf6a80-b377-11eb-8160-1a21d6620db8.gif)\n\n\n\n![test4](https://user-images.githubusercontent.com/64265868/118038201-7e841e80-b377-11eb-891f-1700da1c8079.gif)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flimeraiin%2Funity-machine-learning-test-comparison","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flimeraiin%2Funity-machine-learning-test-comparison","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flimeraiin%2Funity-machine-learning-test-comparison/lists"}