{"id":15293036,"url":"https://github.com/denys88/rl_games","last_synced_at":"2025-05-14T00:07:09.733Z","repository":{"id":37017411,"uuid":"165469880","full_name":"Denys88/rl_games","owner":"Denys88","description":"RL implementations","archived":false,"fork":false,"pushed_at":"2025-05-03T22:04:47.000Z","size":379364,"stargazers_count":1083,"open_issues_count":52,"forks_count":173,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-05-03T23:18:21.460Z","etag":null,"topics":["deep-learning","pytorch","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Denys88.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-01-13T05:35:44.000Z","updated_at":"2025-05-03T22:04:48.000Z","dependencies_parsed_at":"2024-06-28T00:12:44.783Z","dependency_job_id":"34367b22-254e-4b65-9463-7fddaf85be88","html_url":"https://github.com/Denys88/rl_games","commit_stats":{"total_commits":722,"total_committers":22,"mean_commits":32.81818181818182,"dds":0.5193905817174516,"last_synced_commit":"b483bd62982f668e3fb4d457b418e56fae38ebf2"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Denys88%2Frl_games","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Denys88%2Frl_games/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Denys88%2Frl_games/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Denys88%2Frl_games/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Denys88","download_url":"https://codeload.github.com/Denys88/rl_games/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254043686,"owners_count":22004994,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","pytorch","reinforcement-learning"],"created_at":"2024-09-30T16:37:50.259Z","updated_at":"2025-05-14T00:07:09.716Z","avatar_url":"https://github.com/Denys88.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RL Games: High performance RL library \n\n## Discord Channel Link \n* https://discord.gg/hnYRq7DsQh\n\n## Papers and related links\n\n* Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning: https://arxiv.org/abs/2108.10470\n* DeXtreme: Transfer of Agile In-Hand Manipulation from Simulation to Reality: https://dextreme.org/ https://arxiv.org/abs/2210.13702\n* Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger: https://s2r2-ig.github.io/ https://arxiv.org/abs/2108.09779\n* Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge? \u003chttps://arxiv.org/abs/2011.09533\u003e\n* Superfast Adversarial Motion Priors (AMP) implementation: https://twitter.com/xbpeng4/status/1506317490766303235 https://github.com/NVIDIA-Omniverse/IsaacGymEnvs\n* OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation: https://cremebrule.github.io/oscar-web/ https://arxiv.org/abs/2110.00704\n* EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine: https://arxiv.org/abs/2206.10558 and https://github.com/sail-sg/envpool\n* TimeChamber: A Massively Parallel Large Scale Self-Play Framework: https://github.com/inspirai/TimeChamber\n\n\n## Some results on the different environments \n\n* [NVIDIA Isaac Gym](docs/ISAAC_GYM.md)\n\n![Ant_running](https://user-images.githubusercontent.com/463063/125260924-a5969800-e2b5-11eb-931c-116cc90d4bbe.gif)\n![Humanoid_running](https://user-images.githubusercontent.com/463063/125266095-4edf8d00-e2ba-11eb-9c1a-4dc1524adf71.gif)\n\n![Allegro_Hand_400](https://user-images.githubusercontent.com/463063/125261559-38373700-e2b6-11eb-80eb-b250a0693f0b.gif)\n![Shadow_Hand_OpenAI](https://user-images.githubusercontent.com/463063/125262637-328e2100-e2b7-11eb-99af-ea546a53f66a.gif)\n\n* [Dextreme](https://dextreme.org/)\n\n![Allegro_Hand_real_world](https://user-images.githubusercontent.com/463063/216529475-3adeddea-94c3-4ac0-99db-00e7df4ba54b.gif)\n\n* [DexPBT](https://sites.google.com/view/dexpbt)\n\n![AllegroKuka](https://github.com/Denys88/rl_games/assets/463063/3c073a0a-69e7-4696-b86f-64c4c1a7e288)\n\n* [Starcraft 2 Multi Agents](docs/SMAC.md) \n* [BRAX](docs/BRAX.md) \n* [Mujoco Envpool](docs/MUJOCO_ENVPOOL.md) \n* [DeepMind Envpool](docs/DEEPMIND_ENVPOOL.md) \n* [Atari Envpool](docs/ATARI_ENVPOOL.md) \n* [Random Envs](docs/OTHER.md) \n\n\nImplemented in Pytorch:\n\n* PPO with the support of asymmetric actor-critic variant\n* Support of end-to-end GPU accelerated training pipeline with Isaac Gym and Brax\n* Masked actions support\n* Multi-agent training, decentralized and centralized critic variants\n* Self-play \n\n Implemented in Tensorflow 1.x (was removed in this version):\n\n* Rainbow DQN\n* A2C\n* PPO\n\n## Quickstart: Colab in the Cloud\n\nExplore RL Games quick and easily in colab notebooks:\n\n* [Mujoco training](https://colab.research.google.com/github/Denys88/rl_games/blob/master/notebooks/mujoco_envpool_training.ipynb) Mujoco envpool training example.\n* [Brax training](https://colab.research.google.com/github/Denys88/rl_games/blob/master/notebooks/brax_training.ipynb) Brax training example, with keeping all the observations and actions on GPU.\n* [Onnx discrete space export example with Cartpole](https://colab.research.google.com/github/Denys88/rl_games/blob/master/notebooks/train_and_export_onnx_example_discrete.ipynb) envpool training example.\n* [Onnx continuous space export example with Pendulum](https://colab.research.google.com/github/Denys88/rl_games/blob/master/notebooks/train_and_export_onnx_example_continuous.ipynb) envpool training example.\n* [Onnx continuous space with LSTM export example with Pendulum](https://colab.research.google.com/github/Denys88/rl_games/blob/master/notebooks/train_and_export_onnx_example_lstm_continuous.ipynb) envpool training example.\n\n## Installation\n\nFor maximum training performance a preliminary installation of Pytorch 2.2 or newer with CUDA 12.1 or newer is highly recommended:\n\n```conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia``` or:\n```pip install pip3 install torch torchvision```\n\nThen:\n\n```pip install rl-games```\n\nTo run CPU-based environments either Ray or envpool are required ```pip install envpool``` or ```pip install ray```\nTo run Mujoco, Atari games or Box2d based environments training they need to be additionally installed with ```pip install gym[mujoco]```, ```pip install gym[atari]``` or ```pip install gym[box2d]``` respectively.\n\nTo run Atari also ```pip install opencv-python``` is required. In addition installation of envpool for maximum simulation and training perfromance of Mujoco and Atari environments is highly recommended: ```pip install envpool```\n\n## Citing\n\nIf you use rl-games in your research please use the following citation:\n\n```bibtex\n@misc{rl-games2021,\ntitle = {rl-games: A High-performance Framework for Reinforcement Learning},\nauthor = {Makoviichuk, Denys and Makoviychuk, Viktor},\nmonth = {May},\nyear = {2021},\npublisher = {GitHub},\njournal = {GitHub repository},\nhowpublished = {\\url{https://github.com/Denys88/rl_games}},\n}\n```\n\n\n## Development setup\n\n```bash\npoetry install\n# install cuda related dependencies\npoetry run pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html\n```\n\n## Training\n**NVIDIA Isaac Gym**\n\nDownload and follow the installation instructions of Isaac Gym: https://developer.nvidia.com/isaac-gym \nAnd IsaacGymEnvs: https://github.com/NVIDIA-Omniverse/IsaacGymEnvs\n\n*Ant*\n\n```python train.py task=Ant headless=True```\n```python train.py task=Ant test=True checkpoint=nn/Ant.pth num_envs=100```\n\n*Humanoid*\n\n```python train.py task=Humanoid headless=True```\n```python train.py task=Humanoid test=True checkpoint=nn/Humanoid.pth num_envs=100```\n\n*Shadow Hand block orientation task*\n\n```python train.py task=ShadowHand headless=True```\n```python train.py task=ShadowHand test=True checkpoint=nn/ShadowHand.pth num_envs=100```\n\n**Other**\n\n*Atari Pong*\n\n```bash\npoetry install -E atari\npoetry run python runner.py --train --file rl_games/configs/atari/ppo_pong.yaml\npoetry run python runner.py --play --file rl_games/configs/atari/ppo_pong.yaml --checkpoint nn/PongNoFrameskip.pth\n```\n\n*Brax Ant*\n\n```bash\npoetry install -E brax\npoetry run pip install --upgrade \"jax[cuda]==0.3.13\" -f https://storage.googleapis.com/jax-releases/jax_releases.html\npoetry run python runner.py --train --file rl_games/configs/brax/ppo_ant.yaml\npoetry run python runner.py --play --file rl_games/configs/brax/ppo_ant.yaml --checkpoint runs/Ant_brax/nn/Ant_brax.pth\n```\n\n## Experiment tracking\n\nrl_games support experiment tracking with [Weights and Biases](https://wandb.ai).\n\n```bash\npoetry install -E atari\npoetry run python runner.py --train --file rl_games/configs/atari/ppo_breakout_torch.yaml --track\nWANDB_API_KEY=xxxx poetry run python runner.py --train --file rl_games/configs/atari/ppo_breakout_torch.yaml --track\npoetry run python runner.py --train --file rl_games/configs/atari/ppo_breakout_torch.yaml --wandb-project-name rl-games-special-test --track\npoetry run python runner.py --train --file rl_games/configs/atari/ppo_breakout_torch.yaml --wandb-project-name rl-games-special-test -wandb-entity openrlbenchmark --track\n```\n\n\n## Multi GPU\n\nWe use `torchrun` to orchestrate any multi-gpu runs.\n\n```bash\ntorchrun --standalone --nnodes=1 --nproc_per_node=2 runner.py --train --file rl_games/configs/ppo_cartpole.yaml\n```\n\n## Config Parameters\n\n| Field | Example Value | Default | Description |\n| ---------------------- | ------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| seed | 8 | None | Seed for pytorch, numpy etc. |\n| algo | | | Algorithm block. |\n| name | a2c_continuous | None | Algorithm name. Possible values are: sac, a2c_discrete, a2c_continuous |\n| model | | | Model block. |\n| name | continuous_a2c_logstd | None | Possible values: continuous_a2c ( expects sigma to be (0, +inf), continuous_a2c_logstd ( expects sigma to be (-inf, +inf), a2c_discrete, a2c_multi_discrete |\n| network | | | Network description. |\n| name | actor_critic | | Possible values: actor_critic or soft_actor_critic. |\n| separate | False | | Whether use or not separate network with same same architecture for critic. In almost all cases if you normalize value it is better to have it False |\n| space | | | Network space |\n| continuous | | | continuous or discrete |\n| mu_activation | None | | Activation for mu. In almost all cases None works the best, but we may try tanh. |\n| sigma_activation | None | | Activation for sigma. Will be threated as log(sigma) or sigma depending on model. |\n| mu_init | | | Initializer for mu. |\n| name | default | | |\n| sigma_init | | | Initializer for sigma. if you are using logstd model good value is 0. |\n| name | const_initializer | | |\n| val | 0 | | |\n| fixed_sigma | True | | If true then sigma vector doesn't depend on input. |\n| cnn | | | Convolution block. |\n| type | conv2d | | Type: right now two types supported: conv2d or conv1d |\n| activation | elu | | activation between conv layers. |\n| initializer | | | Initialier. I took some names from the tensorflow. |\n| name | glorot_normal_initializer | | Initializer name |\n| gain | 1.4142 | | Additional parameter. |\n| convs | | | Convolution layers. Same parameters as we have in torch. |\n| filters | 32 | | Number of filters. |\n| kernel_size | 8 | | Kernel size. |\n| strides | 4 | | Strides |\n| padding | 0 | | Padding |\n| filters | 64 | | Next convolution layer info. |\n| kernel_size | 4 | | |\n| strides | 2 | | |\n| padding | 0 | | |\n| filters | 64 | | |\n| kernel_size | 3 | | |\n| strides | 1 | | |\n| padding | 0 | |\n| mlp | | | MLP Block. Convolution is supported too. See other config examples. |\n| units | | | Array of sizes of the MLP layers, for example: [512, 256, 128] |\n| d2rl | False | | Use d2rl architecture from https://arxiv.org/abs/2010.09163. |\n| activation | elu | | Activations between dense layers. |\n| initializer | | | Initializer. |\n| name | default | | Initializer name. |\n| rnn | | | RNN block. |\n| name | lstm | | RNN Layer name. lstm and gru are supported. |\n| units | 256 | | Number of units. |\n| layers | 1 | | Number of layers |\n| before_mlp | False | False | Apply rnn before mlp block or not. |\n| config | | | RL Config block. |\n| reward_shaper | | | Reward Shaper. Can apply simple transformations. |\n| min_val | -1 | | You can apply min_val, max_val, scale and shift. |\n| scale_value | 0.1 | 1 | |\n| normalize_advantage | True | True | Normalize Advantage. |\n| gamma | 0.995 | | Reward Discount |\n| tau | 0.95 | | Lambda for GAE. Called tau by mistake long time ago because lambda is keyword in python :( |\n| learning_rate | 3e-4 | | Learning rate. |\n| name | walker | | Name which will be used in tensorboard. |\n| save_best_after | 10 | | How many epochs to wait before start saving checkpoint with best score. |\n| score_to_win | 300 | | If score is \u003e=value then this value training will stop. |\n| grad_norm | 1.5 | | Grad norm. Applied if truncate_grads is True. Good value is in (1.0, 10.0) |\n| entropy_coef | 0 | | Entropy coefficient. Good value for continuous space is 0. For discrete is 0.02 |\n| truncate_grads | True | | Apply truncate grads or not. It stabilizes training. |\n| env_name | BipedalWalker-v3 | | Envinronment name. |\n| e_clip | 0.2 | | clip parameter for ppo loss. |\n| clip_value | False | | Apply clip to the value loss. If you are using normalize_value you don't need it. |\n| num_actors | 16 | | Number of running actors/environments. |\n| horizon_length | 4096 | | Horizon length per each actor. Total number of steps will be num_actors*horizon_length * num_agents (if env is not MA num_agents==1). |\n| minibatch_size | 8192 | | Minibatch size. Total number number of steps must be divisible by minibatch size. |\n| minibatch_size_per_env | 8 | | Minibatch size per env. If specified will overwrite total number number the default minibatch size with minibatch_size_per_env * nume_envs value. |\n| mini_epochs | 4 | | Number of miniepochs. Good value is in [1,10] |\n| critic_coef | 2 | | Critic coef. by default critic_loss = critic_coef * 1/2 * MSE. |\n| lr_schedule | adaptive | None | Scheduler type. Could be None, linear or adaptive. Adaptive is the best for continuous control tasks. Learning rate is changed changed every miniepoch |\n| kl_threshold | 0.008 | | KL threshould for adaptive schedule. if KL \u003c kl_threshold/2 lr = lr * 1.5 and opposite. |\n| normalize_input | True | | Apply running mean std for input. |\n| bounds_loss_coef | 0.0 | | Coefficient to the auxiary loss for continuous space. |\n| max_epochs | 10000 | | Maximum number of epochs to run. |\n| max_frames | 5000000 | | Maximum number of frames (env steps) to run. |\n| normalize_value | True | | Use value running mean std normalization. |\n| use_diagnostics | True | | Adds more information into the tensorboard. |\n| value_bootstrap | True | | Bootstraping value when episode is finished. Very useful for different locomotion envs. |\n| bound_loss_type | regularisation | None | Adds aux loss for continuous case. 'regularisation' is the sum of sqaured actions. 'bound' is the sum of actions higher than 1.1. |\n| bounds_loss_coef | 0.0005 | 0 | Regularisation coefficient |\n| use_smooth_clamp | False | | Use smooth clamp instead of regular for cliping |\n| zero_rnn_on_done | False | True | If False RNN internal state is not reset (set to 0) when an environment is rest. Could improve training in some cases, for example when domain randomization is on |\n| player | | | Player configuration block. |\n| render | True | False | Render environment |\n| deterministic | True | True | Use deterministic policy ( argmax or mu) or stochastic. |\n| use_vecenv | True | False | Use vecenv to create environment for player |\n| games_num | 200 | | Number of games to run in the player mode. |\n| env_config | | | Env configuration block. It goes directly to the environment. This example was take for my atari wrapper. |\n| skip | 4 | | Number of frames to skip |\n| name | BreakoutNoFrameskip-v4 | | The exact name of an (atari) gym env. An example, depends on the training env this parameters can be different. |\n| evaluation | True | False | Enables the evaluation feature for inferencing while training. |\n| update_checkpoint_freq | 100 | 100 | Frequency in number of steps to look for new checkpoints. |\n| dir_to_monitor | | | Directory to search for checkpoints in during evaluation. |\n\n## Custom network example: \n[simple test network](rl_games/envs/test_network.py) \nThis network takes dictionary observation.\nTo register it you can add code in your __init__.py\n\n```\nfrom rl_games.envs.test_network import TestNetBuilder \nfrom rl_games.algos_torch import model_builder\nmodel_builder.register_network('testnet', TestNetBuilder)\n```\n[simple test environment](rl_games/envs/test/rnn_env.py)\n[example environment](rl_games/envs/test/example_env.py) \n\nAdditional environment supported properties and functions \n\n| Field | Default Value | Description |\n| -------------------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| use_central_value | False | If true than returned obs is expected to be dict with 'obs' and 'state' |\n| value_size | 1 | Shape of the returned rewards. Network wil support multihead value automatically. |\n| concat_infos | False | Should default vecenv convert list of dicts to the dicts of lists. Very usefull if you want to use value_boostrapping. in this case you need to always return 'time_outs' : True or False, from the env. |\n| get_number_of_agents(self) | 1 | Returns number of agents in the environment |\n| has_action_mask(self) | False | Returns True if environment has invalid actions mask. |\n| get_action_mask(self) | None | Returns action masks if has_action_mask is true. Good example is [SMAC Env](rl_games/envs/test/smac_env.py) |\n\n\n## Release Notes\n\n1.6.1\n* Fixed Central Value RNN bug which occurs if you train ma multi agent environment.\n* Added Deepmind Control PPO benchmark.\n* Added a few more experimental ways to train value prediction (OneHot, TwoHot encoding and crossentropy loss instead of L2).\n* New methods didn't. It is impossible to turn it on from the yaml files. Once we find an env which trains better it will be added to the config.\n* Added shaped reward graph to the tensorboard.\n* Fixed bug with SAC not saving weights with save_frequency.\n* Added multi-node training support for GPU-accelerated training environments like Isaac Gym. No changes in training scripts are required. Thanks to @ankurhanda and @ArthurAllshire for assistance in implementation.\n* Added evaluation feature for inferencing during training. Checkpoints from training process can be automatically picked up and updated in the inferencing process when enabled.\n* Added get/set API for runtime update of rl training parameters. Thanks to @ArthurAllshire for the initial version of fast PBT code.\n* Fixed SAC not loading weights properly.\n* Removed Ray dependency for use cases it's not required.\n* Added warning for using deprecated 'seq_len' instead of 'seq_length' in configs with RNN networks.\n\n\n1.6.0\n\n* Added ONNX export colab example for discrete and continious action spaces. For continuous case LSTM policy example is provided as well.\n* Improved RNNs training in continuous space, added option `zero_rnn_on_done`.\n* Added NVIDIA CuLE support: https://github.com/NVlabs/cule\n* Added player config everride. Vecenv is used for inference.\n* Fixed multi-gpu training with central value.\n* Fixed max_frames termination condition, and it's interaction with the linear learning rate: https://github.com/Denys88/rl_games/issues/212\n* Fixed \"deterministic\" misspelling issue.\n* Fixed Mujoco and Brax SAC configs.\n* Fixed multiagent envs statistics reporting. Fixed Starcraft2 SMAC environments.\n\n1.5.2\n\n* Added observation normalization to the SAC.\n* Returned back adaptive KL legacy mode.\n\n1.5.1\n\n* Fixed build package issue.\n\n1.5.0\n\n* Added wandb support.\n* Added poetry support.\n* Fixed various bugs.\n* Fixed cnn input was not divided by 255 in case of the dictionary obs.\n* Added more envpool mujoco and atari training examples. Some of the results: 15 min Mujoco humanoid training, 2 min atari pong.\n* Added Brax and Mujoco colab training examples.\n* Added 'seed' command line parameter. Will override seed in config in case it's \u003e 0.\n* Deprecated `horovod` in favor of `torch.distributed` ([#171](https://github.com/Denys88/rl_games/pull/171)).\n\n1.4.0\n\n* Added discord channel https://discord.gg/hnYRq7DsQh :)\n* Added envpool support with a few atari examples. Works 3-4x time faster than ray.\n* Added mujoco results. Much better than openai spinning up ppo results.\n* Added tcnn(https://github.com/NVlabs/tiny-cuda-nn) support. Reduces 5-10% of training time in the IsaacGym envs. \n* Various fixes and improvements.\n\n1.3.2\n\n* Added 'sigma' command line parameter. Will override sigma for continuous space in case if fixed_sigma is True.\n\n1.3.1\n\n* Fixed SAC not working\n\n1.3.0\n\n* Simplified rnn implementation. Works a little bit slower but much more stable. \n* Now central value can be non-rnn if policy is rnn.\n* Removed load_checkpoint from the yaml file. now --checkpoint works for both train and play.\n\n1.2.0\n\n* Added Swish (SILU) and GELU activations, it can improve Isaac Gym results for some of the envs.\n* Removed tensorflow and made initial cleanup of the old/unused code.\n* Simplified runner.\n* Now networks are created in the algos with load_network method.\n\n1.1.4\n\n* Fixed crash in a play (test) mode in player, when simulation and rl_devices are not the same.\n* Fixed variuos multi gpu errors.\n\n1.1.3\n\n* Fixed crash when running single Isaac Gym environment in a play (test) mode.\n* Added config parameter ```clip_actions``` for switching off internal action clipping and rescaling\n\n1.1.0\n\n* Added to pypi: ```pip install rl-games```\n* Added reporting env (sim) step fps, without policy inference. Improved naming.\n* Renames in yaml config for better readability: steps_num to horizon_length amd lr_threshold to kl_threshold\n\n\n\n## Troubleshouting\n\n* Some of the supported envs are not installed with setup.py, you need to manually install them\n* Starting from rl-games 1.1.0 old yaml configs won't be compatible with the new version: \n * ```steps_num``` should be changed to ```horizon_length``` amd ```lr_threshold``` to ```kl_threshold```\n\n## Known issues\n\n* Running a single environment with Isaac Gym can cause crash, if it happens switch to at least 2 environments simulated in parallel\n \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenys88%2Frl_games","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdenys88%2Frl_games","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenys88%2Frl_games/lists"}