Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tensorflow/agents

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.
https://github.com/tensorflow/agents

bandits contextual-bandits dqn multi-armed-bandits reinforcement-learning rl-algorithms tensorflow tf-agents

Last synced: 6 days ago
JSON representation

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.

Host: GitHub
URL: https://github.com/tensorflow/agents
Owner: tensorflow
License: apache-2.0
Created: 2018-11-17T00:29:12.000Z (almost 6 years ago)
Default Branch: master
Last Pushed: 2024-09-30T15:29:26.000Z (about 1 month ago)
Last Synced: 2024-10-08T09:44:49.783Z (27 days ago)
Topics: bandits, contextual-bandits, dqn, multi-armed-bandits, reinforcement-learning, rl-algorithms, tensorflow, tf-agents
Language: Python
Homepage:
Size: 12.8 MB
Stars: 2,783
Watchers: 79
Forks: 720
Open Issues: 199
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

awesome-deep-rl - Google TF-Agents - TF-Agents is a library for Reinforcement Learning in TensorFlow. (Libraries)
awesome-robotic-tooling - tf-agents - A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning. (Sensor Processing / Machine Learning)
awesome-list - TF-Agents - A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning. (Reinforcement Learning / Others)
awesome-datascience - TF-Agents
awesome-google-colab - TF Agents - TF-Agents is a library for Reinforcement Learning in TensorFlow (Technologies)
awesome-python-machine-learning-resources - GitHub - 22% open · ⏱️ 24.08.2022): (强化学习)
StarryDivineSky - tensorflow/agents - Agents是TensorFlow中的强化学习库 (时间序列 / 网络服务_其他)
awesome-production-machine-learning - TF-Agents - A reliable, scalable and easy to use TensorFlow library for contextual bandits and reinforcement learning. (Industry Strength RL)

README

        # TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.

[![PyPI tf-agents](https://badge.fury.io/py/tf-agents.svg)](https://badge.fury.io/py/tf-agents)

![PyPI - Python Version](https://img.shields.io/pypi/pyversions/tf-agents)

[TF-Agents](https://github.com/tensorflow/agents) makes implementing, deploying,

and testing new Bandits and RL algorithms easier. It provides well tested and

modular components that can be modified and extended. It enables fast code

iteration, with good test integration and benchmarking.

To get started, we recommend checking out one of our Colab tutorials. If you

need an intro to RL (or a quick recap),

[start here](docs/tutorials/0_intro_rl.ipynb). Otherwise, check out our

[DQN tutorial](docs/tutorials/1_dqn_tutorial.ipynb) to get an agent up and

running in the Cartpole environment. API documentation for the current stable

release is on

[tensorflow.org](https://www.tensorflow.org/agents/api_docs/python/tf_agents).

TF-Agents is under active development and interfaces may change at any time.

Feedback and comments are welcome.

## Table of contents

Agents


Tutorials


Multi-Armed Bandits


Examples


Installation


Contributing


Releases


Principles


Contributors


Citation


Disclaimer




## Agents

In TF-Agents, the core elements of RL algorithms are implemented as `Agents`. An

agent encompasses two main responsibilities: defining a Policy to interact with

the Environment, and how to learn/train that Policy from collected experience.

Currently the following algorithms are available under TF-Agents:

*   [DQN: __Human level control through deep reinforcement learning__ Mnih et

    al., 2015](https://deepmind.com/research/dqn/)

*   [DDQN: __Deep Reinforcement Learning with Double Q-learning__ Hasselt et

    al., 2015](https://arxiv.org/abs/1509.06461)

*   [DDPG: __Continuous control with deep reinforcement learning__ Lillicrap et

    al., 2015](https://arxiv.org/abs/1509.02971)

*   [TD3: __Addressing Function Approximation Error in Actor-Critic Methods__

    Fujimoto et al., 2018](https://arxiv.org/abs/1802.09477)

*   [REINFORCE: __Simple Statistical Gradient-Following Algorithms for

    Connectionist Reinforcement Learning__ Williams,

    1992](https://www-anw.cs.umass.edu/~barto/courses/cs687/williams92simple.pdf)

*   [PPO: __Proximal Policy Optimization Algorithms__ Schulman et al., 2017](https://arxiv.org/abs/1707.06347)

*   [SAC: __Soft Actor Critic__ Haarnoja et al., 2018](https://arxiv.org/abs/1812.05905)



## Tutorials

See [`docs/tutorials/`](docs/tutorials) for tutorials on the major components

provided.



## Multi-Armed Bandits

The TF-Agents library contains a comprehensive Multi-Armed Bandits suite,

including Bandits environments and agents. RL agents can also be used on Bandit

environments. There is a tutorial in

[`bandits_tutorial.ipynb`](https://github.com/tensorflow/agents/tree/master/docs/tutorials/bandits_tutorial.ipynb).

and ready-to-run examples in

[`tf_agents/bandits/agents/examples/v2`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/examples/v2).



## Examples

End-to-end examples training agents can be found under each agent directory.

e.g.:

*   DQN:

    [`tf_agents/agents/dqn/examples/v2/train_eval.py`](https://github.com/tensorflow/agents/tree/master/tf_agents/agents/dqn/examples/v2/train_eval.py)



## Installation

TF-Agents publishes nightly and stable builds. For a list of releases read the

Releases section. The commands below cover installing

TF-Agents stable and nightly from [pypi.org](https://pypi.org) as well as from a

GitHub clone.

> :warning: If using Reverb (replay buffer), which is very common,

TF-Agents will only work with Linux.

> Note: Python 3.11 requires pygame 2.1.3+.

### Stable

Run the commands below to install the most recent stable release. API

documentation for the release is on

[tensorflow.org](https://www.tensorflow.org/agents/api_docs/python/tf_agents).

```shell

$ pip install --user tf-agents[reverb]

# Use keras-2

$ export TF_USE_LEGACY_KERAS=1

# Use this tag get the matching examples and colabs.

$ git clone https://github.com/tensorflow/agents.git

$ cd agents

$ git checkout v0.18.0

```

If you want to install TF-Agents with versions of Tensorflow or

[Reverb](https://github.com/deepmind/reverb) that are flagged as not compatible

by the pip dependency check, use the following pattern below at your own risk.

```shell

$ pip install --user tensorflow

$ pip install --user tf-keras

$ pip install --user dm-reverb

$ pip install --user tf-agents

```

If you want to use TF-Agents with TensorFlow 1.15 or 2.0, install version 0.3.0:

```shell

# Newer versions of tensorflow-probability require newer versions of TensorFlow.

$ pip install tensorflow-probability==0.8.0

$ pip install tf-agents==0.3.0

```

### Nightly

Nightly builds include newer features, but may be less stable than the versioned

releases. The nightly build is pushed as `tf-agents-nightly`. We suggest

installing nightly versions of TensorFlow (`tf-nightly`) and TensorFlow

Probability (`tfp-nightly`) as those are the versions TF-Agents nightly are

tested against.

To install the nightly build version, run the following:

```shell

# Use keras-2

$ export TF_USE_LEGACY_KERAS=1

# `--force-reinstall helps guarantee the right versions.

$ pip install --user --force-reinstall tf-nightly

$ pip install --user --force-reinstall tf-keras-nightly

$ pip install --user --force-reinstall tfp-nightly

$ pip install --user --force-reinstall dm-reverb-nightly

# Installing with the `--upgrade` flag ensures you'll get the latest version.

$ pip install --user --upgrade tf-agents-nightly

```

### From GitHub

After cloning the repository, the dependencies can be installed by running `pip

install -e .[tests]`. TensorFlow needs to be installed independently: `pip

install --user tf-nightly`.



## Contributing

We're eager to collaborate with you! See [`CONTRIBUTING.md`](CONTRIBUTING.md)

for a guide on how to contribute. This project adheres to TensorFlow's

[code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to

uphold this code.



## Releases

TF Agents has stable and nightly releases. The nightly releases are often fine

but can have issues due to upstream libraries being in flux. The table below

lists the version(s) of TensorFlow that align with each TF Agents' release.

Release versions of interest:

  * 0.19.0 supports tensorflow-2.15.0.

  * 0.18.0 dropped Python 3.8 support.

  * 0.16.0 is the first version to support Python 3.11.

  * 0.15.0 is the last release compatible with Python 3.7.

  * If using numpy < 1.19, then use TF-Agents 0.15.0 or earlier.

  * 0.9.0 is the last release compatible with Python 3.6.

  * 0.3.0 is the last release compatible with Python 2.x.

Release | Branch / Tag                                               | TensorFlow 
------- | ---------------------------------------------------------- 
Nightly | [master](https://github.com/tensorflow/agents)             | tf-nightly 
0.19.0  | [v0.19.0](https://github.com/tensorflow/agents/tree/v0.19.0) | 2.15.0 
0.18.0  | [v0.18.0](https://github.com/tensorflow/agents/tree/v0.18.0) | 2.14.0 
0.17.0  | [v0.17.0](https://github.com/tensorflow/agents/tree/v0.17.0) | 2.13.0 
0.16.0  | [v0.16.0](https://github.com/tensorflow/agents/tree/v0.16.0) | 2.12.0 
0.15.0  | [v0.15.0](https://github.com/tensorflow/agents/tree/v0.15.0) | 2.11.0 
0.14.0  | [v0.14.0](https://github.com/tensorflow/agents/tree/v0.14.0) | 2.10.0 
0.13.0  | [v0.13.0](https://github.com/tensorflow/agents/tree/v0.13.0) | 2.9.0 
0.12.0  | [v0.12.0](https://github.com/tensorflow/agents/tree/v0.12.0) | 2.8.0 
0.11.0  | [v0.11.0](https://github.com/tensorflow/agents/tree/v0.11.0) | 2.7.0 
0.10.0  | [v0.10.0](https://github.com/tensorflow/agents/tree/v0.10.0) | 2.6.0 
0.9.0   | [v0.9.0](https://github.com/tensorflow/agents/tree/v0.9.0) | 2.6.0 
0.8.0   | [v0.8.0](https://github.com/tensorflow/agents/tree/v0.8.0) | 2.5.0 
0.7.1   | [v0.7.1](https://github.com/tensorflow/agents/tree/v0.7.1) | 2.4.0 
0.6.0   | [v0.6.0](https://github.com/tensorflow/agents/tree/v0.6.0) | 2.3.0 
0.5.0   | [v0.5.0](https://github.com/tensorflow/agents/tree/v0.5.0) | 2.2.0 
0.4.0   | [v0.4.0](https://github.com/tensorflow/agents/tree/v0.4.0) | 2.1.0 
0.3.0   | [v0.3.0](https://github.com/tensorflow/agents/tree/v0.3.0) | 1.15.0 and 2.0.0.

Version | dm-reverb Version | ------------------ | ----------- | dm-reverb-nightly | 0.14.0 | 0.13.0 | 0.12.0 | 0.11.0 | 0.10.0 | 0.9.0 | 0.8.0 | 0.7.0 | 0.6.0 | | | | | | | |



## Principles

This project adheres to [Google's AI principles](PRINCIPLES.md). By

participating, using or contributing to this project you are expected to adhere

to these principles.



## Contributors

We would like to recognize the following individuals for their code

contributions, discussions, and other work to make the TF-Agents library.

* James Davidson

* Ethan Holly

* Toby Boyd

* Summer Yue

* Robert Ormandi

* Kuang-Huei Lee

* Alexa Greenberg

* Amir Yazdanbakhsh

* Yao Lu

* Gaurav Jain

* Christof Angermueller

* Mark Daoust

* Adam Wood



## Citation

If you use this code, please cite it as:

```

@misc{TFAgents,

  title = {{TF-Agents}: A library for Reinforcement Learning in TensorFlow},

  author = {Sergio Guadarrama and Anoop Korattikara and Oscar Ramirez and

     Pablo Castro and Ethan Holly and Sam Fishman and Ke Wang and

     Ekaterina Gonina and Neal Wu and Efi Kokiopoulou and Luciano Sbaiz and

     Jamie Smith and Gábor Bartók and Jesse Berent and Chris Harris and

     Vincent Vanhoucke and Eugene Brevdo},

  howpublished = {\url{https://github.com/tensorflow/agents}},

  url = "https://github.com/tensorflow/agents",

  year = 2018,

  note = "[Online; accessed 25-June-2019]"

}

```



## Disclaimer

This is not an official Google product.