https://github.com/gpustack/gpustack-higress-plugin
https://github.com/gpustack/gpustack-higress-plugin
Last synced: 3 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/gpustack/gpustack-higress-plugin
- Owner: gpustack
- Created: 2025-11-09T15:20:03.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2026-03-20T07:28:55.000Z (4 months ago)
- Last Synced: 2026-03-21T00:20:08.152Z (4 months ago)
- Language: Go
- Size: 30.3 KB
- Stars: 1
- Watchers: 0
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Higress plugins for GPUStack
## Project Overview
This repository contains custom Higress Proxy-Wasm plugins (extensions) designed for GPUStack, focusing on AI API traffic processing, observability, and enhanced gateway features. Each extension is implemented as a standalone module and can be deployed independently in a Higress-compatible environment.
## Available Extensions
- **gpustack-token-usage**
- Collects and injects token usage statistics into AI API streaming responses (SSE), including time to first token, average token latency, and tokens per second. Supports real client IP injection and path-based filtering. See [`extensions/gpustack-token-usage/README.md`](./extensions/gpustack-token-usage/README.md) for details.
- **gpustack-set-header-pre-route**
- Automatically injects the route name and model name into HTTP request headers before routing, based on configurable path suffixes or prefixes. Useful for model service and API gateway scenarios. See [`extensions/gpustack-set-header-pre-route/README.md`](./extensions/gpustack-set-header-pre-route/README.md) for details.
## Usage
1. Build the desired extension(s) to generate wasm files.
2. Upload the wasm file(s) to the Higress console and configure parameters as needed.
3. Apply the extension(s) to target routes for immediate effect.
## Notes
- All extensions are designed for Proxy-Wasm compatible gateways.
- See each extension's README for specific configuration and usage instructions.