https://github.com/ar-ray-code/vlaframe-inference-gateway-plugin
https://github.com/ar-ray-code/vlaframe-inference-gateway-plugin
Last synced: 24 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/ar-ray-code/vlaframe-inference-gateway-plugin
- Owner: Ar-Ray-code
- License: apache-2.0
- Created: 2026-04-10T09:13:06.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2026-04-21T06:28:55.000Z (3 months ago)
- Last Synced: 2026-04-27T01:08:54.179Z (2 months ago)
- Language: C++
- Size: 14.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# VLAFrame Inference Gateway Plugin
A VLAFrame pluginlib plugin that offloads inference to a remote gRPC inference-gateway server. Implements `VLAFramePlugin` by forwarding images and joint state via gRPC Predict RPC and returning the action chunk from the server response.
## Overview
```
VLAFrameNode (ROS 2)
└─ InferenceGatewayPlugin
│ gRPC Predict (images + joint_state)
▼
inference-gateway (:50051)
│ round-robin
▼
backend server1 / server2 ...
```
- Images are converted from BGR to RGB, then sent as raw bytes
- `joint_state` is sent as the `position` field
- The response `data[rows * cols]` is reshaped into a `[rows][cols]` action chunk
## Prerequisites
- The inference-gateway server must be running (verify with `docker ps`)
- If not running, start it as follows:
```bash
cd /
# Start backend server
SERVER_ID=local1 OUTPUT_ROWS=6 OUTPUT_COLS=6 \
docker compose -f docker-compose.server.yml up --build -d
# Start gateway
BACKEND_SERVERS="local1=:50052" \
docker compose -f docker-compose.gateway.yml up --build -d
```
## Usage
Example:
```bash
ros2 run vla_frame_node vla_frame_node_exec --ros-args \
-p plugin_type:=inference_gateway_plugin/InferenceGatewayPlugin \
-p model_path:=host.docker.internal:50051 \
-p image_height:=480 \
-p image_width:=640 \
-p joint_state_topic:=/follower/joint_states \
-p arm_trajectory_topic:=/follower/arm_controller/joint_trajectory \
-p gripper_trajectory_topic:=/follower/gripper_controller/joint_trajectory \
-p 'image_topics:=["/camera0/image_raw", "/camera1/image_raw"]' \
-p num_arm_joints:=5 \
-p num_gripper_joints:=1
```
## Parameters (`config/inference_gateway_params.yaml`)
| Parameter | Description | Default |
|---|---|---|
| `plugin_type` | pluginlib class name | `inference_gateway_plugin/InferenceGatewayPlugin` |
| `model_path` | Gateway address (host:port) | `host.docker.internal:50051` |
| `image_height` | Input image height (for metadata) | `480` |
| `image_width` | Input image width (for metadata) | `640` |
| `inference_rate` | Inference rate (Hz) | `10.0` |
| `num_arm_joints` | Number of arm joints | `5` |
| `num_gripper_joints` | Number of gripper joints | `1` |
| `use_state_normalization` | State normalization (handled server-side) | `false` |
| `use_action_normalization` | Action normalization (handled server-side) | `false` |
If `model_path` is empty, it falls back to `localhost:50051`.
## Network Configuration
To connect from a Docker container to the gateway running on the host, add the following to `compose.yaml`:
```yaml
extra_hosts:
- "host.docker.internal:host-gateway"
```
This allows the container to reach the gateway at `host.docker.internal:50051`.