https://github.com/inference-gateway/ui
The UI for the inference-gateway, providing a user-friendly interface to interact with and visualize inference results and manage models
https://github.com/inference-gateway/ui
inference-gateway ui
Last synced: 3 months ago
JSON representation
The UI for the inference-gateway, providing a user-friendly interface to interact with and visualize inference results and manage models
- Host: GitHub
- URL: https://github.com/inference-gateway/ui
- Owner: inference-gateway
- License: mit
- Created: 2025-03-27T22:12:54.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-07-31T15:01:43.000Z (5 months ago)
- Last Synced: 2025-07-31T15:02:25.034Z (5 months ago)
- Topics: inference-gateway, ui
- Language: TypeScript
- Homepage: https://docs.inference-gateway.com/
- Size: 1.01 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
Inference Gateway UI
The Inference Gateway UI is a Next.js application that provides a user-friendly interface to interact with AI models through the [Inference Gateway](https://github.com/inference-gateway/inference-gateway) service. It enables easy access to various language models through a consistent interface, streamlining the process of sending requests and receiving responses.
## Table of Contents
- [Table of Contents](#table-of-contents)
- [Key Features](#key-features)
- [Development](#development)
- [Configuration](#configuration)
- [General Settings](#general-settings)
- [Storage Settings](#storage-settings)
- [Authentication Settings](#authentication-settings)
- [Keycloak Auth Provider](#keycloak-auth-provider)
- [Docker](#docker)
- [Kubernetes Deployment](#kubernetes-deployment)
- [Quick Installation](#quick-installation)
- [Deployment Options](#deployment-options)
- [1. Combined Deployment with Inference Gateway Backend](#1-combined-deployment-with-inference-gateway-backend)
- [2. UI-Only Deployment (Connecting to External Gateway)](#2-ui-only-deployment-connecting-to-external-gateway)
- [3. Deployment with Ingress for External Access](#3-deployment-with-ingress-for-external-access)
- [Key Configuration Parameters](#key-configuration-parameters)
- [Complete Example](#complete-example)
- [Deployment](#deployment)
- [Related Projects](#related-projects)
- [License](#license)
- [Contributing](#contributing)
## Key Features
- 🎨 **Modern Interface**: Clean, responsive design built with Next.js 15 and Tailwind CSS
- 🔌 **Seamless API Integration**: Connects directly to the Inference Gateway backend API
- 🛠️ **Model Selection**: Support for multiple language models through an intuitive model selector
- 💬 **Chat Interface**: Intuitive chat UI for interacting with AI models
- 🧰 **Tool Support**: Enables AI models to use tools, including web search capability
- 🔍 **Web Search**: Integrated web search functionality for models to retrieve current information
- 🔒 **Authentication**: Optional authentication using NextAuth.js
- 📱 **Responsive Design**: Works on desktop and mobile devices
- 🐳 **Docker Support**: Easy containerization and deployment
- ☸️ **Kubernetes Ready**: Includes configurations for Kubernetes deployment
- 🧩 **Component Library**: Built with shadcn/ui components and Radix UI primitives
- 🔄 **State Management**: Efficient state management with React hooks
## Development
```bash
# Install dependencies
npm install
# Run development server
npm run dev
```
The development server will be available at http://localhost:3000.
## Configuration
The UI can be configured using the following environment variables:
### General Settings
| Environment Variable | Default Value | Description |
| --------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------- |
| NODE_ENV | `development` | Node environment |
| PORT | `3000` | Port to run the application on |
| HOSTNAME | `0.0.0.0` | Hostname to bind to |
| INFERENCE_GATEWAY_URL | `http://localhost:8080` | URL of the Inference Gateway API |
| LOG_LEVEL | `debug` | Logging level for both client and server-side code (defaults to 'debug' in development, 'info' in production) |
### Storage Settings
| Environment Variable | Default Value | Description |
| -------------------- | ------------- | -------------------------------------------------- |
| STORAGE_TYPE | `local` | Storage type for chat history |
| DB_CONNECTION_URL | - | Connection URL for storage (required for postgres) |
Examples:
- **Local storage** (default): No connection URL needed
- **PostgreSQL**: `postgresql://username:password@host:port/database`
### Authentication Settings
| Environment Variable | Default Value | Description |
| ------------------------------ | ----------------------- | ------------------------------------------ |
| AUTH_ENABLE | `false` | Enable authentication |
| AUTH_SECURE_COOKIES | `false` | Use secure cookies (set to true for HTTPS) |
| NEXTAUTH_URL | `http://localhost:3000` | URL of this application (for NextAuth) |
| NEXTAUTH_SECRET | - | Secret used to encrypt session cookies |
| NEXTAUTH_TRUST_HOST | `true` | Trust the host header from the proxy |
| NEXTAUTH_REFRESH_TOKEN_ENABLED | `true` | Enable refresh token rotation |
### Keycloak Auth Provider
| Environment Variable | Default Value | Description |
| -------------------------------- | ---------------------------------------- | ------------------ |
| AUTH_OIDC_KEYCLOAK_CLIENT_ID | `app-client` | OIDC client ID |
| AUTH_OIDC_KEYCLOAK_CLIENT_SECRET | - | OIDC client secret |
| AUTH_OIDC_KEYCLOAK_ISSUER | `http://localhost:8080/realms/app-realm` | OIDC issuer URL |
| AUTH_OIDC_KEYCLOAK_SCOPES | `openid profile email` | OIDC scopes |
### Github Auth Provider
| Environment Variable | Default Value | Description |
| ------------------------------ | ---------------------------------------- | ------------------ |
| AUTH_OIDC_GITHUB_CLIENT_ID | `app-client` | OIDC client ID |
| AUTH_OIDC_GITHUB_CLIENT_SECRET | - | OIDC client secret |
| AUTH_OIDC_GITHUB_ISSUER | `http://localhost:8080/realms/app-realm` | OIDC issuer URL |
| AUTH_OIDC_GITHUB_SCOPES | `read:user user:email` | OIDC scopes |
### Google Auth Provider
| Environment Variable | Default Value | Description |
| ------------------------------ | ------------------------------------------------------------------------------------------------- | ------------------ |
| AUTH_OIDC_GOOGLE_CLIENT_ID | `app-client` | OIDC client ID |
| AUTH_OIDC_GOOGLE_CLIENT_SECRET | - | OIDC client secret |
| AUTH_OIDC_GOOGLE_ISSUER | `http://localhost:8080/realms/app-realm` | OIDC issuer URL |
| AUTH_OIDC_GOOGLE_SCOPES | `https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile` | OIDC scopes |
## Docker
Pre-built container images are available on the GitHub Container Registry. You can use these images directly:
```bash
# Pull the pre-built image
docker pull ghcr.io/inference-gateway/ui:latest
# Run the container with the pre-built image
docker run -p 3000:3000 \
-e INFERENCE_GATEWAY_URL=http://localhost:8080 \
ghcr.io/inference-gateway/ui:latest
```
Alternatively, you can build the image locally:
```bash
# Build the Docker image locally
docker build -t inference-gateway-ui --target dev .
# Run the container with the locally built image
docker run -p 3000:3000 \
-v $(pwd):/app \
-e INFERENCE_GATEWAY_URL=http://localhost:8080 \
inference-gateway-ui
```
## Kubernetes Deployment
The Inference Gateway UI provides a Helm chart for easy deployment to Kubernetes environments. The chart is available as an OCI artifact, which can be deployed directly without adding a Helm repository.
### Quick Installation
```bash
helm upgrade --install inference-gateway-ui \
oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
--version 0.7.1 \
--create-namespace \
--namespace inference-gateway \
--set gateway.enabled=true \
--set gateway.envFrom.secretRef=inference-gateway
```
### Deployment Options
The Helm chart supports multiple deployment scenarios:
#### 1. Combined Deployment with Inference Gateway Backend
Deploy the UI with the Inference Gateway backend as a dependency (recommended):
```bash
helm upgrade --install inference-gateway-ui \
oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
--create-namespace \
--namespace inference-gateway \
--set gateway.enabled=true \
--set gateway.envFrom.secretRef=inference-gateway
```
#### 2. UI-Only Deployment (Connecting to External Gateway)
Deploy the UI separately and connect it to an existing Inference Gateway instance:
```bash
helm upgrade --install inference-gateway-ui \
oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
--create-namespace \
--namespace inference-gateway \
--set gateway.enabled=false \
--set-string "env[0].name=INFERENCE_GATEWAY_URL" \
--set-string "env[0].value=http://your-gateway-service:8080"
```
#### 3. Deployment with Ingress for External Access
Enable ingress for accessing the UI from outside the cluster:
```bash
helm upgrade --install inference-gateway-ui \
oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
--create-namespace \
--namespace inference-gateway \
--set gateway.enabled=true \
--set gateway.envFrom.secretRef=inference-gateway \
--set ingress.enabled=true \
--set ingress.className=nginx \
--set "ingress.hosts[0].host=ui.inference-gateway.local" \
--set "ingress.hosts[0].paths[0].path=/" \
--set "ingress.hosts[0].paths[0].pathType=Prefix" \
--set "ingress.tls[0].secretName=inference-gateway-ui-tls" \
--set "ingress.tls[0].hosts[0]=ui.inference-gateway.local"
```
### Key Configuration Parameters
| Parameter | Description | Default |
| ------------------- | ---------------------------------------------------------------------- | ------------------------------ |
| `replicaCount` | Number of UI replicas to deploy | `1` |
| `image.repository` | UI container image repository | `ghcr.io/inference-gateway/ui` |
| `image.tag` | UI container image tag (defaults to chart appVersion if not specified) | `""` |
| `gateway.enabled` | Deploy the Inference Gateway backend with the UI | `true` |
| `ingress.enabled` | Enable ingress for external access | `false` |
| `ingress.className` | Ingress controller class to use | `nginx` |
| `resources` | CPU/Memory resource limits and requests | See values.yaml |
| `config.*` | Environment variables for the UI configuration | See values.yaml |
### Complete Example
For a complete example including a local Kubernetes setup with k3d, ingress configuration, and more, refer to the [Kubernetes example directory](./examples/kubernetes/).
```bash
# Example with comprehensive configuration
helm upgrade --install inference-gateway-ui \
oci://ghcr.io/inference-gateway/charts/inference-gateway-ui \
--version 0.7.1 \
--create-namespace \
--namespace inference-gateway \
--set replicaCount=1 \
--set gateway.enabled=true \
--set gateway.envFrom.secretRef=inference-gateway \
--set resources.limits.cpu=500m \
--set resources.limits.memory=512Mi \
--set resources.requests.cpu=100m \
--set resources.requests.memory=128Mi \
--set-string "env[0].name=NODE_ENV" \
--set-string "env[0].value=production" \
--set-string "env[1].name=NEXT_TELEMETRY_DISABLED" \
--set-string "env[1].value=1"
```
## Deployment
The application is automatically packaged as a Docker image and published to GitHub Container Registry (ghcr.io) when a new release is created.
To pull the latest release:
```bash
docker pull ghcr.io/inference-gateway/ui:latest
```
## Related Projects
- [Inference Gateway](https://github.com/inference-gateway/inference-gateway) - The main gateway service
- [Documentation](https://docs.inference-gateway.com) - Project documentation
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Contributing
Contributions are welcome! Please see our [Contributing Guide](CONTRIBUTING.md) for details on how to get started, coding standards, development workflow, and more.