https://github.com/lukehollis/three-mlagents
Three.js + Torch implementation of Unity's ML-Agents framework for agent training and visualization in browser
https://github.com/lukehollis/three-mlagents
deep-learning deep-reinforcement-learning machine-learning neural-networks reinforcement-learning three-js threejs torch
Last synced: 9 months ago
JSON representation
Three.js + Torch implementation of Unity's ML-Agents framework for agent training and visualization in browser
- Host: GitHub
- URL: https://github.com/lukehollis/three-mlagents
- Owner: lukehollis
- Created: 2025-06-29T14:20:30.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-10-04T22:22:55.000Z (9 months ago)
- Last Synced: 2025-10-04T23:28:40.891Z (9 months ago)
- Topics: deep-learning, deep-reinforcement-learning, machine-learning, neural-networks, reinforcement-learning, three-js, threejs, torch
- Language: Python
- Homepage: https://lukehollis.github.io/three-mlagents/
- Size: 39.3 MB
- Stars: 23
- Watchers: 0
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Three ML-Agents
[](https://lukehollis.github.io/three-mlagents/)
[](#license)
A **JavaScript / Python** re-implementation of the core [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents) examples. Test out the [Live Demos](https://lukehollis.github.io/three-mlagents/).
The goal of this repository is to make the ideas from Unity ML-Agents easily approachable from the browser (React + Three.js) and from lightweight Python back-ends (FastAPI) to be used for future study.
## Examples
View all the demos on the live demos page: [Live Demos](https://lukehollis.github.io/three-mlagents/).
Select demos are here.
### MineCraft
[Live demo](https://lukehollis.github.io/three-mlagents/minecraft)
### Glider
[Live demo](https://lukehollis.github.io/three-mlagents/glider)
### Astrodynamics
[Live demo](https://lukehollis.github.io/three-mlagents/astrodynamics)
### Fish
[Live demo](https://lukehollis.github.io/three-mlagents/fish)
### Self-driving Car (Interpretability)
[Live demo](https://lukehollis.github.io/three-mlagents/self-driving-car)
### Labyrinth
[Live demo](https://lukehollis.github.io/three-mlagents/labyrinth)
| Demo | Live Demo |
|--------------------------------|--------------------------------------------------------------------|
| Basic 1-D Move to Goal | [Live Demo](https://lukehollis.github.io/three-mlagents/basic) |
| 3DBall Balance | [Live Demo](https://lukehollis.github.io/three-mlagents/ball3d) |
| GridWorld Navigation | [Live Demo](https://lukehollis.github.io/three-mlagents/gridworld) |
| Push-Block | [Live Demo](https://lukehollis.github.io/three-mlagents/push) |
| Wall Jump | [Live Demo](https://lukehollis.github.io/three-mlagents/walljump) |
| Ant (Crawler) | [Live Demo](https://lukehollis.github.io/three-mlagents/crawler) |
| Swimmer / Worm | [Live Demo](https://lukehollis.github.io/three-mlagents/worm) |
| Brick Break | [Live Demo](https://lukehollis.github.io/three-mlagents/brickbreak)|
| Food Collector | [Live Demo](https://lukehollis.github.io/three-mlagents/foodcollector) |
| Bicycle | [Live Demo](https://lukehollis.github.io/three-mlagents/bicycle) |
| Intersection | [Live Demo](https://lukehollis.github.io/three-mlagents/intersection) |
## Project layout
```
three-mlagents/
│
├─ client/ ← Vite + React + Three.js front-end (examples live here)
│ └─ src/examples
│ ├─ Index.jsx ← landing page listing all examples
│ └─ Basic.jsx ← 1-D "move-to-goal" environment matching Unity Basic
│
├─ api/ ← FastAPI micro-service (Python ≥3.9)
│ ├─ main.py ← gym-style REST API for each environment
│ └─ requirements.txt
│
└─ ml-agents/ ← Upstream Unity project kept for reference only (no build step)
```
You only need **client** and **api** to run the demos. The `ml-agents` directory remains untouched so you can cross-reference the original C#/Unity code (see `Examples/Basic/Scripts/BasicController.cs` for this particular demo).
---
## Quick start
### 1. Front-end (browser)
```bash
cd client
npm install # installs React, Three.js, @react-three/fiber, @react-three/drei …
npm run dev # opens http://localhost:5173
```
You will land on `/` – a list of examples. Click **/basic** to open the first scene. Use **← / →** to move the agent cube; rewards replicate the logic of the original `BasicController` script:
* Each step: **−0.01**
* Small goal (position 7): **+0.1** and episode terminates
* Large goal (position 17): **+1** and episode terminates
### 2. Back-end (optional)
For browser-only play the API is **not required**. If you want to let external RL algorithms interact with the environment, spin up the FastAPI service:
```bash
cd api
pip install -r requirements.txt
uvicorn main:app --reload
```
Endpoints:
* `POST /basic/reset` → `{ "position": 10 }`
* `POST /basic/step` → `{ "position": int, "reward": float, "done": bool }`
The contract mirrors OpenAI Gym (`position` plays the role of the observation).
---
## MineCraft RL + LLM Example - Additional Setup
The **MineCraft** example (#11) combines reinforcement learning with large language models (LLMs) for intelligent agent behavior. This requires additional setup beyond the basic installation.
### Option 1: Local Ollama (Recommended for Development)
1. **Install Ollama**: Download and install from [ollama.ai](https://ollama.ai/)
2. **Pull the required model**:
```bash
ollama pull gemma3n:latest
```
3. **Configure the example**: In `api/examples/minecraft.py`, set:
```python
USE_LOCAL_OLLAMA = True
```
4. **Start Ollama**: Make sure Ollama is running locally (usually starts automatically)
### Option 2: OpenRouter (Cloud-based)
1. **Get an OpenRouter API key**: Sign up at [openrouter.ai](https://openrouter.ai/) and get your API key
2. **Set environment variable**:
```bash
export OPENROUTER_API_KEY="your_api_key_here"
```
3. **Configure the example**: In `api/examples/minecraft.py`, set:
```python
USE_LOCAL_OLLAMA = False
```
4. **Note**: This option uses `anthropic/claude-sonnet-4` and requires credits/tokens
### Additional Dependencies
The MineCraft example also requires additional Python packages for embeddings and LLM functionality. Make sure these are included in your `api/requirements.txt`:
```
sentence-transformers # for text embeddings
openai # for LLM API calls (if using OpenRouter)
```
### Running the MineCraft Example
1. Start the API with LLM support:
```bash
cd api
pip install -r requirements.txt
uvicorn main:app --reload
```
2. Open the frontend and navigate to the MineCraft example
3. The agents will use LLMs for strategic decision-making, communication, and trading while using RL for basic movement and mining actions.
---
[](https://star-history.com/#lukehollis/three-mlagents&Date)
---
## License
MIT © 2025