{"id":20388184,"url":"https://github.com/ivy-llc/gym","last_synced_at":"2025-04-12T10:36:10.781Z","repository":{"id":37083182,"uuid":"331022472","full_name":"ivy-llc/gym","owner":"ivy-llc","description":"Fully differentiable RL environments, written in Ivy.","archived":false,"fork":false,"pushed_at":"2023-08-28T07:06:41.000Z","size":263,"stargazers_count":64,"open_issues_count":1,"forks_count":8,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-04-11T00:48:20.856Z","etag":null,"topics":["deep-learning","gym-environment","ivy","jax","machine-learning","mxnet","numpy","python","pytorch","reinforcement-learning","tensorflow"],"latest_commit_sha":null,"homepage":"https://unify.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ivy-llc.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-01-19T15:21:19.000Z","updated_at":"2025-01-09T18:25:55.000Z","dependencies_parsed_at":"2023-02-13T23:31:28.326Z","dependency_job_id":"e184bce1-72ac-4d4c-b972-7153b97233e7","html_url":"https://github.com/ivy-llc/gym","commit_stats":{"total_commits":176,"total_committers":17,"mean_commits":"10.352941176470589","dds":0.5227272727272727,"last_synced_commit":"ae6295e427845c126d84150e09cde18f40f12d7c"},"previous_names":["ivy-dl/gym","transpile-ai/gym","ivy-llc/gym","unifyai/gym"],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivy-llc%2Fgym","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivy-llc%2Fgym/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivy-llc%2Fgym/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivy-llc%2Fgym/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ivy-llc","download_url":"https://codeload.github.com/ivy-llc/gym/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248553561,"owners_count":21123473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","gym-environment","ivy","jax","machine-learning","mxnet","numpy","python","pytorch","reinforcement-learning","tensorflow"],"created_at":"2024-11-15T03:07:46.850Z","updated_at":"2025-04-12T10:36:10.729Z","avatar_url":"https://github.com/ivy-llc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":".. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/logo.png?raw=true#gh-light-mode-only\n   :width: 100%\n   :class: only-light\n\n.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/logo_dark.png?raw=true#gh-dark-mode-only\n   :width: 100%\n   :class: only-dark\n\n.. raw:: html\n\n    \u003cbr/\u003e\n    \u003ca href=\"https://pypi.org/project/ivy-gym\"\u003e\n        \u003cimg class=\"dark-light\" style=\"float: left; padding-right: 4px; padding-bottom: 4px;\" src=\"https://badge.fury.io/py/ivy-gym.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/unifyai/gym/actions?query=workflow%3Adocs\"\u003e\n        \u003cimg class=\"dark-light\" style=\"float: left; padding-right: 4px; padding-bottom: 4px;\" src=\"https://github.com/unifyai/gym/actions/workflows/docs.yml/badge.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/unifyai/gym/actions?query=workflow%3Anightly-tests\"\u003e\n        \u003cimg class=\"dark-light\" style=\"float: left; padding-right: 4px; padding-bottom: 4px;\" src=\"https://github.com/unifyai/gym/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://discord.gg/G4aR9Q7DTN\"\u003e\n        \u003cimg class=\"dark-light\" style=\"float: left; padding-right: 4px; padding-bottom: 4px;\" src=\"https://img.shields.io/discord/799879767196958751?color=blue\u0026label=%20\u0026logo=discord\u0026logoColor=white\"\u003e\n    \u003c/a\u003e\n    \u003cbr clear=\"all\" /\u003e\n\n**Fully differentiable reinforcement learning environments, written in Ivy.**\n\n.. raw:: html\n\n    \u003cdiv style=\"display: block;\" align=\"center\"\u003e\n        \u003cimg class=\"dark-light\" width=\"6%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/empty.png\"\u003e\n        \u003ca href=\"https://jax.readthedocs.io\"\u003e\n            \u003cimg class=\"dark-light\" width=\"13%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/jax_logo.png\"\u003e\n        \u003c/a\u003e\n        \u003cimg class=\"dark-light\" width=\"12%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/empty.png\"\u003e\n        \u003ca href=\"https://www.tensorflow.org\"\u003e\n            \u003cimg class=\"dark-light\" width=\"13%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/tensorflow_logo.png\"\u003e\n        \u003c/a\u003e\n        \u003cimg class=\"dark-light\" width=\"12%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/empty.png\"\u003e\n        \u003ca href=\"https://pytorch.org\"\u003e\n            \u003cimg class=\"dark-light\" width=\"13%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/pytorch_logo.png\"\u003e\n        \u003c/a\u003e\n        \u003cimg class=\"dark-light\" width=\"12%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/empty.png\"\u003e\n        \u003ca href=\"https://numpy.org\"\u003e\n            \u003cimg class=\"dark-light\" width=\"13%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/numpy_logo.png\"\u003e\n        \u003c/a\u003e\n        \u003cimg class=\"dark-light\" width=\"6%\" style=\"float: left;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/supported/empty.png\"\u003e\n    \u003c/div\u003e\n\nContents\n--------\n\n* `Overview`_\n* `Run Through`_\n* `Optimization Demos`_\n* `Get Involved`_\n\nOverview\n--------\n\n.. _docs: https://unify.ai/docs/gym/\n\n**What is Ivy Gym?**\n\nIvy Gym opens the door for intersectional research between supervised learning (SL), reinforcement learning (RL),\nand trajectory optimization (TO),\nby implementing RL environments in a fully differentiable manner.\n\nSpecifically, Ivy gym provides differentiable implementations of the classic control tasks from OpenAI Gym,\nas well as a new Swimmer task, which illustrates the simplicity of creating new tasks using Ivy.\nThe differentiable nature of the environments means that the cumulative reward can be directly optimized for in a supervised manner,\nwithout need for reinforcement learning, which is the de facto approach for optimizing cumulative rewards. Check out the docs_ for more info!\n\nThe library is built on top of the Ivy machine learning framework.\nThis means all environments simultaneously support:\nJax, Tensorflow, PyTorch, MXNet, and Numpy.\n\n**Ivy Libraries**\n\nThere are a host of derived libraries written in Ivy, in the areas of mechanics, 3D vision, robotics, gym environments,\nneural memory, pre-trained models + implementations, and builder tools with trainers, data loaders and more. Click on the icons below to learn more!\n\n.. raw:: html\n\n    \u003cdiv style=\"display: block;\"\u003e\n        \u003ca href=\"https://github.com/unifyai/mech\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_mech_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_mech.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/vision\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_vision_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_vision.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/robot\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_robot_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_robot.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/gym\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_gym_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_gym.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n\n        \u003cbr clear=\"all\" /\u003e\n\n        \u003ca href=\"https://pypi.org/project/ivy-mech\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-mech.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://pypi.org/project/ivy-vision\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-vision.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://pypi.org/project/ivy-robot\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-robot.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://pypi.org/project/ivy-gym\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\"width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-gym.svg\"\u003e\n        \u003c/a\u003e\n\n        \u003cbr clear=\"all\" /\u003e\n\n        \u003ca href=\"https://github.com/unifyai/mech/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\"src=\"https://github.com/unifyai/mech/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/vision/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/vision/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/robot/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/robot/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/gym/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/gym/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n\n        \u003cbr clear=\"all\" /\u003e\n\n        \u003ca href=\"https://github.com/unifyai/memory\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_memory_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_memory.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/builder\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_builder_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_builder.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/models\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_models_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_models.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/ecosystem\"\u003e\n            \u003cpicture\u003e\n                \u003csource class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_ecosystem_dark.png\"\u003e\n                \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://raw.githubusercontent.com/unifyai/unifyai.github.io/main/img/externally_linked/logos/ivy_ecosystem.png\"\u003e\n            \u003c/picture\u003e\n        \u003c/a\u003e\n\n        \u003cbr clear=\"all\" /\u003e\n\n        \u003ca href=\"https://pypi.org/project/ivy-memory\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-memory.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://pypi.org/project/ivy-builder\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-builder.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://pypi.org/project/ivy-models\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://badge.fury.io/py/ivy-models.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/ecosystem/actions?query=workflow%3Adocs\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/ecosystem/actions/workflows/docs.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n\n        \u003cbr clear=\"all\" /\u003e\n\n        \u003ca href=\"https://github.com/unifyai/memory/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/memory/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/builder/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/builder/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://github.com/unifyai/models/actions?query=workflow%3Anightly-tests\"\u003e\n            \u003cimg class=\"dark-light\" width=\"15%\" style=\"float: left; margin: 0% 5%;\" src=\"https://github.com/unifyai/models/actions/workflows/nightly-tests.yml/badge.svg\"\u003e\n        \u003c/a\u003e\n\n        \u003cbr clear=\"all\" /\u003e\n\n    \u003c/div\u003e\n    \u003cbr clear=\"all\" /\u003e\n\n**Quick Start**\n\nIvy gym can be installed like so: ``pip install ivy-gym``\n\n.. _demos: https://github.com/unifyai/gym/tree/main/ivy_gym_demos\n.. _optimization: https://github.com/unifyai/gym/tree/main/ivy_gym_demos/optimization\n\nTo quickly see the different environments provided, we suggest you check out the demos_!\nWe suggest you start by running the script ``run_through.py``,\nand read the \"Run Through\" section below which explains this script.\n\nFor demos which optimize performance on the different tasks, we suggest you run either\n``optimize_trajectory.py`` or ``optimize_policy.py`` in the optimization_ demos folder.\n\nRun Through\n-----------\n\nThe different environments can be visualized via a simple script,\nwhich executes random motion for 250 steps in one of the environments.\nThe script is available in the demos_ folder, as file ``run_through.py``.\nFirst, we select a random backend framework to use for the examples, from the options\n``jax``, ``tensorflow``, ``torch``, ``mxnet`` or ``numpy``,\nand use this to set the ivy backend framework.\n\n.. code-block:: python\n\n    import ivy\n    fw = ivy.choose_random_backend() \n    ivy.set_backend(fw)\n\nWe then select an environment to use and execute 250 random actions,\nwhile rendering the environment after each step.\n\nBy default, the demos all use the ``CartPole`` environment, but this can be changed using the ``--env`` argument,\nchoosing from the options ``CartPole``, ``Pendulum``, ``MountainCar``, ``Reacher`` or ``Swimmer``.\n\n.. code-block:: python\n\n    env = getattr(ivy_gym, env_str)()\n\n    env.reset()\n    ac_dim = env.action_space.shape[0]\n    for _ in range(250):\n        ac = ivy.random_uniform(low=-1, high=1, shape=(ac_dim,))\n        env.step(ac)\n        env.render()\n\nHere, we briefly discuss each of the five environments,\nbefore showing example episodes from a learnt policy network.\nWe use a learnt policy in these visualizations rather than random actions as used in the script,\nbecause we find this to be more descriptive for visually explaining each task.\nWe also plot the instantaneous reward corresponding to each frame.\n\n**CartPole**\n\nFor this task, a pole is attached by an un-actuated joint to a cart, which moves along a frictionless track.\nThe system is controlled by applying a force to the cart.\nA reward is given based on the angle of the pendulum from being upright.\nExample trajectories are given below.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"40%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/cartpole.gif?raw=true'\u003e\n    \u003c/p\u003e\n\n**MountainCar**\n\nFor this task, a car is on a one-dimensional track, positioned between two \"mountains\".\nThe goal is to drive up the mountain on the right.\nHowever, the car's engine is not strong enough to scale the mountain in a single pass.\nTherefore, the only way to succeed is to drive back and forth to build up momentum.\nHere, the reward is greater if you spend less energy to reach the goal.\nExample trajectories are given below.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"40%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/mountain_car.gif?raw=true'\u003e\n    \u003c/p\u003e\n\n**Pendulum**\n\nFor this task, an inverted pendulum starts in a random position, and the goal is to swing it up so it stays upright.\nAgain, a reward is given based on the angle of the pendulum from being upright.\nExample trajectories are given below.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"40%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/pendulum.gif?raw=true'\u003e\n    \u003c/p\u003e\n\n**Reacher**\n\nFor this task, a 2-link robot arm must reach a target position.\nReward is given based on the distance of the end effector to the target.\nExample trajectories are given below.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"40%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/reacher.gif?raw=true'\u003e\n    \u003c/p\u003e\n\n**Swimmer**\n\nWe implemented this task ourselves, in order to highlight the simplicity of creating new custom environments.\nFor this task, a fish must swim to reach a target 2D positions whilst avoiding sharp obstacles.\nReward is given for being close to the target, and negative reward is given for colliding with the sharp objects.\nExample trajectories are given below.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"40%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/swimmer.gif?raw=true'\u003e\n    \u003c/p\u003e\n\nOptimization Demos\n------------------\n\nWe provide two demo scripts which optimize performance on these tasks in a supervised manner,\neither via trajectory optimization or policy optimization.\n\nIn the case of trajectory optimization, we optimize for a specific starting state of the environment,\nwhereas for policy optimization we train a policy network which is conditioned on the environment state,\nand the starting state is then randomized between training steps.\n\nRather than presenting the code here, we show visualizations of the demos.\nThe scripts for these demos can be found in the optimization_ demos folder.\n\n**Trajectory Optimization**\n\nIn this demo, we show trajectories on each of the five ivy gym environments during the course of trajectory optimization.\nThe optimization iteration is shown in the bottom right, along with the step in the environment.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"75%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/demo_a.gif?raw=true'\u003e\n    \u003c/p\u003e\n\n**Policy Optimization**\n\nIn this demo, we show trajectories on each of the five ivy gym environments during the course of policy optimization.\nThe optimization iteration is shown in the bottom right, along with the step in the environment.\n\n.. raw:: html\n\n    \u003cp align=\"center\"\u003e\n        \u003cimg width=\"75%\" style=\"display: block;\" src='https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/ivy_gym/demo_b.gif?raw=true'\u003e\n    \u003c/p\u003e\n\nGet Involved\n-----------\n\nWe hope the differentiable environments in this library are useful to a wide range of machine learning developers.\nHowever, there are many more tasks which could be implemented.\n\nIf there are any particular tasks you feel are missing,\nor you would like to implement your own custom task,\nthen we are very happy to accept pull requests!\n\nWe look forward to working with the community on expanding and improving the Ivy gym library.\n\nCitation\n--------\n\n::\n\n    @article{lenton2021ivy,\n      title={Ivy: Templated deep learning for inter-framework portability},\n      author={Lenton, Daniel and Pardo, Fabio and Falck, Fabian and James, Stephen and Clark, Ronald},\n      journal={arXiv preprint arXiv:2102.02886},\n      year={2021}\n    }\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivy-llc%2Fgym","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fivy-llc%2Fgym","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivy-llc%2Fgym/lists"}