{"id":19401123,"url":"https://github.com/google-research/falken","last_synced_at":"2025-05-16T15:07:25.591Z","repository":{"id":46078303,"uuid":"367164407","full_name":"google-research/falken","owner":"google-research","description":"Falken provides developers with a service that allows them to train AI that can play their games","archived":false,"fork":false,"pushed_at":"2025-02-17T10:23:38.000Z","size":12813,"stargazers_count":255,"open_issues_count":28,"forks_count":36,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-12T14:16:57.185Z","etag":null,"topics":["cpp","games","imitation-learning","ml","unity3d"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-13T20:17:06.000Z","updated_at":"2025-04-01T22:12:22.000Z","dependencies_parsed_at":"2022-09-23T07:01:08.663Z","dependency_job_id":"3ae799ca-617e-4ae9-900a-36db615d6901","html_url":"https://github.com/google-research/falken","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Ffalken","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Ffalken/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Ffalken/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Ffalken/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-research","download_url":"https://codeload.github.com/google-research/falken/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254553958,"owners_count":22090417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","games","imitation-learning","ml","unity3d"],"created_at":"2024-11-10T11:17:17.718Z","updated_at":"2025-05-16T15:07:25.571Z","avatar_url":"https://github.com/google-research.png","language":"Python","readme":"# Falken\n\n![Main branch](https://github.com/google-research/falken/actions/workflows/build_on_push.yaml/badge.svg)\n![Latest release](https://github.com/google-research/falken/actions/workflows/build_release.yaml/badge.svg)\n\n**NOTE**: This is not an officially supported Google product.\n\nFalken provides developers with a service that allows them to train AI that can\nplay their games. Unlike traditional RL frameworks that learn through rewards\nor batches of offline training, Falken is based on training AI via realtime,\nhuman interactions.\n\nBecause human time is precious, the system is designed to seamlessly transition\nbetween humans playing the game (training) and the AI playing the game\n(inference). This allows us to support a variety of training styles, from\nproviding a large batch of demonstrations up front to providing “just in time”\ndemonstrations when the AI encounters a state it does not know how to handle.\n\nBehind the scenes, Falken automatically handles logging of experience,\nseparates out human demonstrations, trains new models, serves the best models\nfor on-device inference, and maintains a version history.\n\n## Components\n\nThis project consists of:\n\n * [A local service](service/README.md) that collects experience, trains and\n   serves models.\n * A [C++](sdk/cpp/README.md) SDK that can be integrated into C++ games to use\n   the service and an example environment,\n   [Hello Falken](environments/cpp/hello_falken/README.md) that demonstrates\n   use of the C++ SDK.\n * A [Unity](sdk/unity/README.md) SDK that can be integrated into Unity games to\n   use the service and\n   [example environments](environments/unity/demos/README.md) with\n   [Hello Falken](environments/unity/demos/Assets/HelloFalken) being the most\n   simple to get started.\n * A [simple web dashboard](dashboard/README.md) that connects to the service to\n   provides a way to visualize traces from sessions for debugging purposes.\n\n## Getting started\n\nThe fastest way to ensure that everything is working properly is to\nlaunch a local service then build and run Hello Falken environment.\n\nIn C++:\n * [Launch the service](service/README.md#launch-the-service)\n * [Build and run the environment](environments/cpp/hello_falken/README.md)\n\nIn Unity:\n * [Launch the service](service/README.md#launch-the-service)\n * [Configure and run the environment](environments/unity/demos/Assets/HelloFalken/README.md)\n\n## Concepts\n\n### Stand up and Initialize Falken\n\nThe first step in using Falken is to\n[launch the service](service/README.md#launch-the-service) and connect to the\nservice from the client SDK (e.g C++ or Unity) using your JSON configuration.\nAll subsequent API calls from the client will automatically use this\ninformation.\n\n### Define Actions\nActions allow you to define how Falken’s AI interacts with your game and how it\nlearns from humans who play it. In Falken, Actions can be either continuous or\ndiscrete.\n\n * **Continuous Actions** are used to represent analog controls like a joystick\n   that controls aiming or an analog trigger that controls the gas pedal in a\n   car. All continuous actions are required to include a developer defined min\n   and max value.\n * **Discrete Actions** can be used to represent simple buttons (like whether\n  the fire button is pressed) or discrete selections (like which weapon to\n  equip). We generally recommend developers create Actions which represent\n  logical interactions (jump, shoot, move, etc) rather than physical inputs (A\n  button, left stick). As such, you may have more Actions in Falken than there\n  are physical inputs for your game.\n\n### Define Observations\n\nObservations allow you to define how Falken’s AI perceives your game both while\nit is playing and while it is “watching” a human play. Falken’s Observations are\nbased on the Entity/Attribute model, where an Entity is simply a container for\nattributes, each of which have a corresponding name, type, and value.\n\n * **Position:** All Entities include an optional, pre-defined 3D position\n   attribute (vector). If used, this position should generally be in world\n   space.\n * **Rotation:** All Entities include an optional, pre-defined 3D rotation\n   attribute (quaternion). If used, this rotation should generally be in world\n   space.\n * **Number:** Number attributes represent quantities in the world that can be\n   counted, either as whole numbers (ints) or real numbers (floats). All Number\n   attributes are required to have a developer-defined minimum and maximum\n   value.\n * **Category:** Categorical attributes represent values with discrete states,\n   like a simple bool or the entries in an enum.\n * **Feelers:** Feelers represent a set of evenly spaced samples reaching out\n   from an entity. They are commonly used to sense the 3D environment around an\n   entity (usually the player).\n\nDefining the best Observations for your game can be tricky and can impact\nFalken's performance, so we recommend starting with the bare minimum a human\nwould require to play a game and slowly refining the set of Observations to\nimprove efficacy. In practice we've found that simple observations can yield\nmuch better performance.\n\n### Create a Brain\n\nIn Falken, a Brain represents the idea of the thing that observes, learns, and\nacts. It’s also helpful to think about each Brain as being designed to\naccomplish a specific kind of task, like completing a lap in a racing game or\ndefeating enemies while moving through a level in an FPS.\n\nWhen creating a Brain, you provide a specific set of Actions and a specific set\nof Observations that define how the brain perceives the world and what actions\nit can take.\n\n### Start a Session. Start an Episode.\n\nIn Falken, a Session represents a period of time during which Falken was\nlearning how to play your game, playing your game, or both. Sessions are\ncomposed of one or more Episodes, which represent a sequence of interactions\nwith a clear beginning, middle, and end (like starting a race, driving around\nthe track, and crossing the finish line).\n\nFalken supports a number of different Session types, including:\n\n * **Interactive Training:** In this mode, Falken learns by watching a human\n   play the game. Players can teach Falken by playing entire Episodes from\n   beginning to end or they can let Falken play and only take over to suggest\n   a correction in behavior whenever Falken makes a mistake.\n * **Evaluation:** In this mode, Falken plays the game autonomously in order to\n   determine the best model that can be produced for your game given the data\n   provided in earlier Interactive Training mode.\n * **Inference:** In this mode, Falken simply plays your game as much as you\n   like and intentionally disables all learning features. This guarantees that\n   you get consistent results, and is ideal for testing your game at scale.\n\nAll Sessions also require you to define a maximum number of Steps per Episode.\nYou’ll want to set this number to be a comfortable amount more than one a human\nwould require. For example, if you call `Episode::Step()` every frame, your game\nruns at 30fps, and a human would take 300 seconds to complete an episode, then\nyou should set a max steps to about 9000 or 12000 to ensure the AI has enough\ntime to complete the task. If the number of steps is too high then Falken will\ntake longer to detect when an agent is stuck when using an evaluation session.\n\n### Step the Episode\n\nEvery time you want Falken to learn from a human or interact with your game,\nyou need to need to call Step(). Step() always operates on a set of\nObservations from your game. If a human is playing, then you also provide the\nActions that the human performed, and Falken will learn from them. If you want\nFalken to play your game, don’t pass in any Actions to Step() but instead use\nthe Actions returned by Step() and apply them to your game as if a human had\nperformed them.\n\nFor most games, you’ll call Falken’s Step() every time you update your game\nsimulation. However, some games might want to call Step() less often, like when\nthe player takes a turn or performs an important action.\n\n### End the Episode. Reset the Game.\n\nEpisodes in Falken end when the player succeeds, fails, or the maximum number\nof steps has been exceeded. You provide this context when you end the Episode.\n\n * **Success:** The player (human or AI) has successfully accomplished the goal\n   (e.g. completed a lap or finished the level).\n * **Failure:** The player (human or AI) failed to accomplish the goal\n   (e.g. car crashed or player was killed by an enemy).\n * **Gave Up:** The episode reached its max steps without succeeding or failing.\n * **Aborted:** The episode was unexpectedly terminated in the middle.\n\nMost Sessions involve many Episodes (either for training Falken or testing your\ngame), so when an Episode completes, you’ll need to reset your game state before\nyou bring the next Episode. Falken leaves the definition of this reset up to you\nand does not require that every Episode in a Session start with identical state.\nIn general, you want each episode to start with similar but not identical state\n(similar to what many games do for human players) as this helps Falken most\neffectively learn from and test your game.\n\n### Ending one Session. Starting a new Session.\n\nOnce you are done with a Session, you simply Stop it. If an Episode is still\nactive when a Session is stopped, the Episode will be Aborted.\n\nWhen you Stop a learning-type session (InteractiveTraining or Evaluation), the\nBrain makes a record of what it has learned and *\"gets smarter\"*. Additional\nSessions created with this brain will start from this more intelligent state and\nuse that as the basis for additional learning or playing.\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research%2Ffalken","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-research%2Ffalken","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research%2Ffalken/lists"}