https://github.com/tron4x/neuronhub
AI Multi-Model Management Platform
https://github.com/tron4x/neuronhub
bullmq-processing nextjs nodejs ollama postgresql redis
Last synced: 5 months ago
JSON representation
AI Multi-Model Management Platform
- Host: GitHub
- URL: https://github.com/tron4x/neuronhub
- Owner: tron4x
- License: agpl-3.0
- Created: 2026-01-23T12:23:28.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-01-25T21:25:37.000Z (5 months ago)
- Last Synced: 2026-01-25T22:54:05.189Z (5 months ago)
- Topics: bullmq-processing, nextjs, nodejs, ollama, postgresql, redis
- Language: TypeScript
- Homepage:
- Size: 5.03 MB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
NeuronHub 🧠
Multi-Model Intelligence — Your Central LLM Management Platform
---
## ✨ Overview
NeuronHub is a powerful, self-hosted platform for managing multiple Large Language Models (LLMs) from a single interface. Whether you're running local models with Ollama or connecting to cloud providers like OpenAI and Anthropic, NeuronHub provides a unified dashboard to manage, compare, and interact with all your AI models.
---
## 🎯 Why NeuronHub?
> A complete LLM operations platform — manage, compare, train, and integrate your AI models.
### 🚀 Core Capabilities
### 🎨 **Model Studio**
Visual editor to shape your AI's personality. Adjust communication style, response length, creativity via sliders. Define behavior rules (what the model must/must not do). Add knowledge and facts. **No prompt engineering required.**
### 🧪 **Behavior Testing**
Create test suites to verify your model behaves correctly. "Does it refuse medical diagnoses?" "Does it know its identity?" Run automated tests before deploying. Track pass/fail rates over versions.
### ⚖️ **Model Comparison**
Run the same prompt against multiple models simultaneously. See responses side-by-side. Find the best model for your use case without guesswork.
### 📝 **Prompt Templates**
18+ ready-to-use templates across 6 categories: Development, Language, Creative, Analysis, Business. Create your own with `{{variables}}`. Share across your team.
### 🎓 **Model Training**
Fine-tune Ollama models via UI. Upload training data, select a preset, click train. Background worker handles the job. Your tuned model appears in the list — no CLI needed.
### 🔌 **MCP Integration**
17 tools for IDE integration. Use your models from VS Code (Cline), Claude Desktop, or Zed. Manage prompts, run comparisons, trigger training — without leaving your editor.
### 💡 Perfect For
| Use Case | How NeuronHub Helps |
|----------|---------------------|
| **Prompt Engineers** | 18 templates, custom prompt creation, test across models instantly |
| **Developers** | MCP integration, REST API, model comparison for choosing the right LLM |
| **Teams** | Shared prompt library, centralized model management, usage tracking |
| **AI Trainers** | Fine-tune Ollama models via UI, 8 training presets, progress tracking |
| **Enterprises** | Self-hosted, no data leaves your network, API key management |
### 🚀 Get Started in 60 Seconds
```bash
git clone https://github.com/tron4x/neuronhub.git && cd neuronhub
docker-compose up -d
# Open http://localhost:3001 🎉
```
---
## 📸 Screenshots
🖼️ Click to view screenshots
Dashboard - Your central AI command center
LLM Model Status ( Ollama )
Chat - Full markdown rendering and code highlighting
Model Studio - Visual personality editor with sliders
Model Studio - Define behavior rules for your AI
Model Comparison - Test multiple models side-by-side
Model Training - Fine-tune models with live progress
---
## 🚀 Features
| Feature | Description |
|---------|-------------|
| **🤖 Multi-Model Management** | Manage local (Ollama, Open WebUI) and cloud LLMs (OpenAI, Anthropic) in one place |
| **🔍 Model Discovery** | Automatic detection of local Ollama models, Docker containers (Ollama, LocalAI, vLLM, TGI), and Open WebUI |
| **💬 Chat Interface** | Beautiful chat interface with any configured model |
| **⚖️ Model Comparison** | Compare responses from multiple models side-by-side |
| **📝 Prompt Templates** | Create, manage, and reuse prompt templates with 18+ predefined templates |
| **🎯 Training Presets** | 8 predefined training configurations for common use cases |
| **📊 Usage Tracking** | Monitor token usage, request counts, and costs |
| **🔑 API Key Management** | Securely store and manage provider API keys |
| **🌙 Dark/Light Mode** | Beautiful UI with theme support |
| **🔌 MCP Integration** | Full Model Context Protocol support for AI assistants |
| **🐳 Docker Ready** | One-command deployment with Docker Compose |
## 🛠️ Tech Stack
| Layer | Technologies |
|-------|--------------|
| **Frontend** | Next.js 16.1, React 19.2, Tailwind CSS 3.4, Framer Motion 12 |
| **Backend** | Next.js API Routes, Prisma 7.2 ORM |
| **Database** | PostgreSQL |
| **Cache/Queue** | Redis, BullMQ 5.6 |
| **Container** | Docker, Docker Compose |
| **MCP** | Model Context Protocol SDK |
| **Language** | TypeScript 5.9 |
## 📦 Installation
### Prerequisites
- **Git** - to clone the repository
- **Node.js 22+** - JavaScript runtime ([nodejs.org](https://nodejs.org/))
- **Docker & Docker Compose** - for containerized deployment ([docker.com](https://www.docker.com/))
- **Ollama** (optional) - for local LLM models ([ollama.ai](https://ollama.ai/))
🦙 Ollama Setup (for local models)
If you want to use local LLM models with Ollama:
**1. Install Ollama**
| Platform | Installation |
|----------|--------------|
| **macOS** | `brew install ollama` or download from [ollama.ai](https://ollama.ai/) |
| **Linux** | `curl -fsSL https://ollama.ai/install.sh \| sh` |
| **Windows** | Download installer from [ollama.ai/download](https://ollama.ai/download) |
**2. Start Ollama**
Ollama must be running before NeuronHub can discover and use local models:
🍎 macOS
```bash
# Option 1: Start as background service (recommended)
brew services start ollama
# Option 2: Run manually in terminal
ollama serve
# Check if running
brew services list | grep ollama
# or
curl http://localhost:11434/api/tags
```
🐧 Linux
```bash
# Option 1: Start as systemd service (recommended)
sudo systemctl start ollama
sudo systemctl enable ollama # Auto-start on boot
# Option 2: Run manually in terminal
ollama serve
# Check if running
systemctl status ollama
# or
curl http://localhost:11434/api/tags
```
🪟 Windows
```powershell
# Option 1: Ollama runs automatically after installation
# Check the system tray for the Ollama icon
# Option 2: Start manually via command line
ollama serve
# Check if running (PowerShell)
Invoke-RestMethod http://localhost:11434/api/tags
```
> **Note:** On Windows, Ollama typically starts automatically as a background process after installation.
> Look for the Ollama icon in the system tray.
**3. Pull models**
Download at least one model before using NeuronHub:
```bash
# Example: Pull popular models
ollama pull llama3.2
ollama pull mistral
ollama pull qwen3:14b
# List downloaded models
ollama list
```
> **Note:** NeuronHub will automatically discover all pulled Ollama models via the `/api/models/discover` endpoint.
### 🚀 One-Click Start (Mac/Linux)
The easiest way to start NeuronHub is using the `start.sh` script:
```bash
# Clone repository
git clone https://github.com/tron4x/neuronhub.git
cd neuronhub
# Create .env file
cp .env.example .env
# Make executable (only needed once)
chmod +x start.sh
# Start NeuronHub
./start.sh
```
### Quick Start
```bash
# Clone repository
git clone https://github.com/tron4x/neuronhub.git
cd neuronhub
# Install dependencies
npm install
# Create .env file
cp .env.example .env
# Start PostgreSQL and Redis
docker-compose up -d postgres redis
# Initialize database
npx prisma db push
npx prisma generate
# Start development server
npm run dev
```
🌐 Open http://localhost:3001 in your browser
**Requirements:** Docker Desktop (Mac) or Docker Engine (Linux) must be installed and running.
The script will:
- ✅ Check if Docker is running
- ✅ Build and start all containers (App, Worker, PostgreSQL, Redis)
- ✅ Display service status and connection info
- ✅ Show useful commands
🌐 Open http://localhost:3001 when ready!
### Docker Deployment (Manual)
```bash
# Start all services (App, Worker, PostgreSQL, Redis)
docker-compose up -d
# View logs
docker-compose logs -f app
# Stop all services
docker-compose down
```
## ⚙️ Configuration
Create a `.env` file in the root directory:
```env
# Database
DATABASE_URL="postgresql://neuronhub:neuronhub_secret@localhost:5433/neuronhub"
# Redis
REDIS_URL="redis://localhost:6380"
# API Keys (optional - add as needed)
OPENAI_API_KEY="sk-..."
ANTHROPIC_API_KEY="sk-ant-..."
GOOGLE_API_KEY="..."
# Auth
AUTH_SECRET="your-super-secret-key-here"
NEXTAUTH_URL="http://localhost:3001"
```
## 📱 Application Routes
| Route | Description |
|-------|-------------|
| `/` | Dashboard with statistics and quick actions |
| `/chat` | Chat interface with model selection |
| `/compare` | Side-by-side model comparison |
| `/models` | Model management and configuration |
| `/prompts` | Prompt template library |
| `/training` | Fine-tuning job management |
| `/api-keys` | API key management |
| `/settings` | System settings |
## 🐳 Docker Services
| Service | Port | Description |
|---------|------|-------------|
| `app` | 3001 | Next.js Application |
| `postgres` | 5433 | PostgreSQL Database |
| `redis` | 6380 | Redis Cache & Queue |
| `worker` | - | Background Job Worker (BullMQ) |
⚡ Background Worker (BullMQ)
NeuronHub includes a powerful background worker powered by **BullMQ** for processing asynchronous tasks. The worker runs as a separate Docker container and handles long-running operations without blocking the main application.
### Worker Queues
| Queue | Description |
|-------|-------------|
| `training-jobs` | Processes AI model fine-tuning jobs with progress tracking |
| `inference-jobs` | Handles batch/async LLM inference requests |
| `model-discovery` | Discovers available models from Ollama, Open WebUI, Docker containers (LocalAI, vLLM, TGI) |
### Features
- **🔄 Automatic Retry** - Failed jobs are automatically retried with exponential backoff
- **📊 Progress Tracking** - Real-time progress updates for training jobs (0-100%)
- **📝 Job Logging** - All job activities are logged to the database
- **⚡ Concurrency** - Multiple jobs can run in parallel
- **🛡️ Graceful Shutdown** - Jobs are properly cleaned up on shutdown
- **💾 Persistence** - Jobs survive restarts (Redis-backed)
### Worker Commands
```bash
# Start worker in development
npm run worker:dev
# Start worker in production
npm run worker
# View worker logs (Docker)
docker-compose logs -f worker
```
### Architecture
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Next.js │────▶│ Redis │◀────│ Worker │
│ API │ │ (BullMQ) │ │ (worker.ts)│
└─────────────┘ └─────────────┘ └─────────────┘
│
▼
┌─────────────┐
│ PostgreSQL │
│ (Results) │
└─────────────┘
```
🤖 MCP Server Integration
NeuronHub includes a full **Model Context Protocol (MCP)** server that allows any MCP-compatible AI assistant to interact with your LLM platform.
**Supported Clients:** Claude Desktop, Cline (VS Code), Zed, any MCP-compatible client
### Setup for Cline (VS Code)
Add to `cline_mcp_settings.json`:
```json
{
"mcpServers": {
"neuronhub": {
"command": "node",
"args": ["/build/index.js"],
"env": {
"NEURONHUB_URL": "http://localhost:3001"
}
}
}
}
```
> **Note:** Replace `` with the actual path where you installed the neuronhub-mcp package.
### Setup for Claude Desktop
Add to `claude_desktop_config.json`:
```json
{
"mcpServers": {
"neuronhub": {
"command": "node",
"args": ["/build/index.js"],
"env": {
"NEURONHUB_URL": "http://localhost:3001"
}
}
}
}
```
### MCP Inspector (Debugging & Testing)
The **MCP Inspector** is a powerful tool for debugging and testing MCP servers. Use it to verify your NeuronHub MCP connection works correctly.
#### Installation
```bash
npm install -g @modelcontextprotocol/inspector
```
#### Running the Inspector
```bash
# Start MCP Inspector with NeuronHub
npx @modelcontextprotocol/inspector node /build/index.js
```
#### Inspector Parameters
| Parameter | Description | Example |
|-----------|-------------|---------|
| **Command** | The runtime to execute | `node` |
| **Arguments** | Path to the MCP server entry point | `/build/index.js` |
| **Environment Variables** | Configure the MCP server | See below |
> **Note:** Replace `` with the actual path where you cloned/installed the neuronhub-mcp package (e.g., `~/Documents/neuronhub-mcp` or `/opt/neuronhub-mcp`).
#### Environment Variables for Inspector
When using the MCP Inspector, you can set these environment variables:
| Variable | Description | Default |
|----------|-------------|---------|
| `NEURONHUB_URL` | NeuronHub API base URL | `http://localhost:3001` |
| `DEBUG` | Enable debug logging | `false` |
#### Example: Full Inspector Command
```bash
# With environment variable (replace path with your actual path)
NEURONHUB_URL=http://localhost:3001 npx @modelcontextprotocol/inspector node ~/neuronhub-mcp/build/index.js
# Or start Inspector UI first, then configure in the web interface
npx @modelcontextprotocol/inspector
```
The Inspector opens a web UI at `http://localhost:5173` where you can:
- ✅ **List all available tools** - See all 17 MCP tools
- ✅ **Test tool calls** - Execute tools with custom parameters
- ✅ **View responses** - Inspect JSON responses
- ✅ **Debug errors** - See detailed error messages
- ✅ **Monitor logs** - Real-time server logs
#### Screenshot: Inspector Interface
```
┌─────────────────────────────────────────────────────────────┐
│ MCP Inspector [⚙️ Settings] │
├─────────────────────────────────────────────────────────────┤
│ Server: neuronhub-mcp Status: Connected │
├─────────────────────────────────────────────────────────────┤
│ Tools (17) │
│ ├── list_models │
│ ├── get_model │
│ ├── activate_model │
│ ├── chat │
│ ├── inference │
│ └── ... │
├─────────────────────────────────────────────────────────────┤
│ [Execute Tool] [Clear Logs] [Refresh] │
└─────────────────────────────────────────────────────────────┘
```
### Available MCP Tools (17 tools)
| Category | Tool | Description |
|----------|------|-------------|
| **Models** | `list_models` | List all registered LLM models |
| | `get_model` | Get detailed model information |
| | `activate_model` | Activate a model for inference |
| | `deactivate_model` | Deactivate a model |
| | `discover_models` | Discover local & Docker models (Ollama, Open WebUI, LocalAI, vLLM, TGI) |
| **Chat** | `chat` | Send a chat message to a model |
| | `inference` | Run inference with full parameters |
| | `compare_models` | Compare multiple model responses |
| **Prompts** | `list_prompts` | List all prompt templates |
| | `get_prompt` | Get a specific template |
| | `create_prompt` | Create a new template |
| **Training** | `list_training_jobs` | List fine-tuning jobs |
| | `get_training_job` | Get job details |
| | `create_training_job` | Start a new job |
| | `cancel_training_job` | Cancel a running job |
| **System** | `get_stats` | Get platform statistics |
| | `health_check` | Check API health |
## 🔌 REST API
All endpoints are accessible at `http://localhost:3001/api/`
📋 Models API
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/models` | List all models |
| `POST` | `/api/models` | Create a new model |
| `GET` | `/api/models/:id` | Get model by ID |
| `PATCH` | `/api/models/:id` | Update model |
| `DELETE` | `/api/models/:id` | Delete model |
| `POST` | `/api/models/discover` | Discover local models |
| `GET` | `/api/models/health` | Check health of all models |
| `POST` | `/api/models/health` | Update model status based on health |
💬 Chat & Inference API
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/chat` | Chat with a model |
| `POST` | `/api/inference` | Run inference |
| `POST` | `/api/compare` | Compare multiple models |
| `POST` | `/api/embedding` | Generate text embeddings |
| `GET` | `/api/embedding` | List available embedding models |
🖥️ Hardware API
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/hardware` | Detect GPU/training hardware capabilities |
📝 Prompts API
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/prompts` | List all prompts |
| `POST` | `/api/prompts` | Create a prompt |
| `GET` | `/api/prompts/:id` | Get prompt by ID |
| `PATCH` | `/api/prompts/:id` | Update prompt |
| `DELETE` | `/api/prompts/:id` | Delete prompt |
🎓 Training API
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/training` | List training jobs |
| `POST` | `/api/training` | Create training job |
| `GET` | `/api/training/:id` | Get job by ID |
| `PATCH` | `/api/training/:id` | Update/cancel job |
| `DELETE` | `/api/training/:id` | Delete job |
📝 Predefined Templates
NeuronHub includes **18 predefined prompt templates** and **8 training presets** to help you get started quickly.
### Prompt Templates (18 templates in 6 categories)
| Category | Templates |
|----------|-----------|
| **📋 Professional** | Document Specialist (System/User/Assistant) |
| **💻 Development** | Code Reviewer, Code Explainer, Bug Hunter, Test Generator, API Designer |
| **🌐 Language** | Professional Translator, Grammar & Style Checker, Text Summarizer |
| **✍️ Creative** | Story Writer, Blog Post Writer, Professional Email Writer |
| **📊 Analysis** | Data Analyst, Research Assistant |
| **💼 Business** | Customer Support Agent, FAQ Generator |
### Training Presets (8 presets)
| Preset | Description | Config |
|--------|-------------|--------|
| **Quick Test** | Fast training for testing | 1 epoch, LoRA |
| **Document Specialist** | Optimized for document analysis | 5 epochs, Fine-tune |
| **Code Assistant** | Train for code understanding | 3 epochs, LoRA |
| **Chat Assistant** | Conversational fine-tuning | 4 epochs, Fine-tune |
| **Instruction Following** | Better instruction adherence | 5 epochs, Fine-tune |
| **Memory Efficient** | For limited GPU memory | 3 epochs, QLoRA |
| **High Quality** | Longer training for best results | 10 epochs, Fine-tune |
| **Embedding Model** | Train sentence embeddings | 5 epochs, Embedding |
Access templates via:
- **New Prompt** → Select from dropdown
- **New Training Job** → Select preset
## 🎓 Model Training / Fine-Tuning
NeuronHub supports fine-tuning of language models. You can create training jobs via the UI or API.
### ⚠️ Important: Understanding Training Types
NeuronHub offers **three different training methods**. It's critical to understand what each one actually does:
| Type | What it REALLY does | Model Format | GPU Required | New Model Created? |
|------|---------------------|--------------|--------------|-------------------|
| **FINE_TUNE** | Ollama Modelfile customization | GGUF (Ollama) | ❌ No | ❌ No, creates a variant |
| **LoRA** | Real neural network training | HuggingFace | ✅ Yes | ✅ Yes, new weights |
| **QLoRA** | 4-bit quantized LoRA training | HuggingFace | ✅ NVIDIA only | ✅ Yes, new weights |
---
#### 🔧 FINE_TUNE (Ollama Modelfile Customization)
**What it does:** Creates an Ollama Modelfile that wraps your base model with:
- A custom **system prompt** (extracted from your training data)
- **Parameter tweaks** (temperature, top_p, etc.)
- **Few-shot examples** (from your training data)
**What it does NOT do:**
- ❌ Does NOT modify the neural network weights
- ❌ Does NOT create a truly new model
- ❌ Does NOT learn new knowledge
**How it works:**
```dockerfile
# Generated Modelfile
FROM qwen3:14b # Original model stays unchanged
SYSTEM """You are a document specialist...""" # Custom behavior
PARAMETER temperature 0.7 # Parameter tweaks
PARAMETER top_p 0.9
MESSAGE user "Example question" # Few-shot examples
MESSAGE assistant "Example answer"
```
**Result:** A model variant like `qwen3-14b-tuned-1737572XXX` that:
- Uses the **same weights** as the base model
- Has a **different personality** via system prompt
- Appears as a separate model in Ollama (`ollama list`)
**Best for:** Quick customization, no GPU needed, works with any Ollama model.
---
#### 🧠 LoRA / QLoRA (Real Neural Network Training)
**What it does:** Actually modifies the model's neural network weights using:
- **PEFT** (Parameter-Efficient Fine-Tuning) library
- **Low-Rank Adaptation** - trains small "adapter" matrices
- **HuggingFace Transformers** for model loading
**What it does:**
- ✅ Learns NEW patterns from your training data
- ✅ Creates NEW model weights (adapter + merged)
- ✅ Can learn things the base model didn't know
**How it works:**
```
1. Load HuggingFace model (e.g., mistralai/Ministral-8B)
2. Freeze base weights
3. Add small trainable LoRA adapters (~1-5% of params)
4. Train on your JSONL data for N epochs
5. Save adapter → Merge with base → Convert to GGUF
6. Register in Ollama
```
**Requirements:**
- **LoRA:** Any GPU (CUDA, Apple MPS, or even CPU - but slow)
- **QLoRA:** NVIDIA GPU with CUDA (uses 4-bit quantization)
- **HuggingFace model** (NOT Ollama models)
**Best for:** When you need the model to truly learn new knowledge or behaviors.
---
🖥️ LoRA/QLoRA Training by Platform (Docker)
All training runs in Docker containers. Choose your platform:
### 🐧 Linux (Recommended)
Linux with NVIDIA GPU provides the best training experience.
**Requirements:**
- Docker & Docker Compose
- NVIDIA Driver 525+
- NVIDIA Container Toolkit
**Setup NVIDIA Docker:**
```bash
# Install NVIDIA Container Toolkit
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
# Start NeuronHub
docker-compose up -d
# Verify GPU in container
docker-compose exec worker nvidia-smi
```
**Supported:**
- FINE_TUNE ✅ | LoRA ✅ | QLoRA ✅ | Unsloth ✅
---
### 🪟 Windows
Windows uses Docker Desktop with WSL2 backend.
**Requirements:**
- Windows 10/11 Pro or Enterprise
- Docker Desktop 4.x with WSL2 backend
- NVIDIA GPU + Driver 525+ (optional)
**Setup:**
```powershell
# 1. Install Docker Desktop from docker.com
# 2. Enable WSL2 backend in Docker Desktop settings
# 3. For GPU: Enable "Use WSL 2 based engine" + install NVIDIA drivers
# Start NeuronHub
docker-compose up -d
# Verify (PowerShell)
docker-compose exec worker python3 -c "import torch; print('CUDA:', torch.cuda.is_available())"
```
**Supported:**
- FINE_TUNE ✅ | LoRA ✅ (GPU/CPU) | QLoRA ✅ (NVIDIA only)
---
### 🍎 macOS
Docker on macOS runs in a Linux VM without direct GPU access.
**Requirements:**
- Docker Desktop for Mac
**Setup:**
```bash
# Start NeuronHub
docker-compose up -d
# Training runs on CPU in Docker (no MPS access)
docker-compose exec worker python3 -c "import torch; print('MPS:', torch.backends.mps.is_available())"
# Output: MPS: False (expected in Docker)
```
**Supported:**
- FINE_TUNE ✅ | LoRA ⚠️ (CPU - slow) | QLoRA ❌
> **Note:** Docker on Mac cannot access Apple Silicon GPU. FINE_TUNE (Ollama Modelfile) works fast on any system. For faster LoRA training on Mac, consider cloud GPUs.
---
### ☁️ Cloud GPUs
Cloud GPUs provide the best training performance.
| Provider | Instance | GPU | VRAM | Example Use Case |
|----------|----------|-----|------|------------------|
| AWS | p3.2xlarge | V100 | 16GB | e.g. LoRA 7B models |
| AWS | p4d.24xlarge | A100 | 40GB | e.g. larger models |
| GCP | a2-highgpu-1g | A100 | 40GB | e.g. fine-tuning |
| Azure | NC24ads_A100_v4 | A100 | 80GB | e.g. 70B+ models |
| RunPod | RTX 4090 | 4090 | 24GB | e.g. budget option |
| Lambda | A100 | A100 | 40/80GB | e.g. general training |
> **Note:** These are example configurations. Actual requirements depend on model size and training type.
**Quick Setup:**
```bash
# On cloud VM (Ubuntu)
curl -fsSL https://get.docker.com | sh
# Install NVIDIA Container Toolkit (see Linux section)
git clone https://github.com/tron4x/neuronhub.git && cd neuronhub
docker-compose up -d
```
---
### Platform Summary
| Platform | FINE_TUNE | LoRA | QLoRA | Unsloth |
|----------|-----------|------|-------|---------|
| Linux + NVIDIA | ✅ | ✅ | ✅ | ✅ |
| Windows + NVIDIA | ✅ | ✅ | ✅ | ✅ |
| macOS (Docker) | ✅ | ⚠️ CPU | ❌ | ❌ |
| Cloud GPU | ✅ | ✅ | ✅ | ✅ |
---
#### 📊 Comparison Example
| Scenario | Use FINE_TUNE | Use LoRA/QLoRA |
|----------|---------------|----------------|
| "Make model respond in German" | ✅ System prompt | ❌ Overkill |
| "Give model a personality" | ✅ System prompt | ❌ Overkill |
| "Teach model domain knowledge" | ❌ Can't learn | ✅ Yes |
| "Train on 10,000 examples" | ❌ Only few-shot | ✅ Full training |
| "No GPU available" | ✅ Works | ❌ Needs GPU |
| "Quick test in 1 minute" | ✅ Fast | ❌ Takes hours |
---
### Training via Web UI
1. Navigate to **Training** in the main menu (or `/training`)
2. Click **"New Job"**
3. Configure your training:
- **Select Model**: Choose the base model to fine-tune
- **Training Type**:
- `FINE_TUNE` - Full fine-tuning
- `LORA` - Low-Rank Adaptation (efficient)
- `QLORA` - Quantized LoRA (memory efficient)
- **Dataset Path**: Path or URL to your training data (JSONL, CSV, Parquet)
- **Hyperparameters**: Epochs, Batch Size, Learning Rate
4. Click **"Start Training"**
### Ollama Modelfile Customization
NeuronHub supports **real Ollama training** through Modelfile customization. When you create a training job for an Ollama model, the worker:
1. **Extracts Training Data**: Parses your JSONL file for system prompts and examples
2. **Creates Modelfile**: Generates an Ollama Modelfile with:
```dockerfile
FROM deepseek-r1:8b # Your base model
SYSTEM """Your custom system prompt extracted from training data"""
PARAMETER temperature 0.7 # Based on learning rate
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 4096
# Few-shot examples from training data
MESSAGE user Example question...
MESSAGE assistant Example answer...
```
3. **Creates New Model**: Runs `ollama create -tuned`
4. **Validates Model**: Tests the new model with a sample prompt
5. **Registers in NeuronHub**: Adds the tuned model to your model library
**Result**: A new model variant like `deepseek-r1-8b-tuned-1737572XXX` that appears in your Models page!
### Creator Attribution (Permanent)
When you create a tuned model, your attribution is **permanently embedded** in the model:
```
"You are a helpful AI assistant. This model was fine-tuned by tron4x using NeuronHub.
Base model: qwen3:14b."
```
**What this means:**
- ✅ The system prompt is permanently stored in the model
- ✅ Every time someone uses the model, the attribution is active
- ✅ You can verify with: `ollama show your-tuned-model`
- ✅ Asking "Who created you?" will mention tron4x
**Storage location:**
- Model files: `~/.ollama/models/`
- Configuration embedded in model manifest
**Note:** This is Modelfile customization (system prompt + parameters). For actual neural network weight fine-tuning (LoRA/QLoRA), you'd need GPU + training frameworks like unsloth. The Modelfile approach is the quickest way to personalize a model's behavior and identity.
### Running the Training Worker
The worker must be running to process training jobs:
```bash
# Make sure Redis is running
docker-compose up -d redis
# Start the worker (development)
npm run worker:dev
# Or with Docker
docker-compose up -d worker
# View worker logs
docker-compose logs -f worker
```
### Training Process Flow
```
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ QUEUED │────▶│ PREPARING │────▶│ TRAINING │
│ (10%) │ │ (20%) │ │ (40%) │
└──────────────┘ └──────────────┘ └──────────────┘
│
▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ COMPLETED │◀────│ VALIDATING │◀────│ ollama create│
│ (100%) │ │ (90%) │ │ (80%) │
└──────────────┘ └──────────────┘ └──────────────┘
```
🔧 Training via API, MCP & Examples
### Training via API
```bash
# Create a fine-tuning job
curl -X POST http://localhost:3001/api/training \
-H "Content-Type: application/json" \
-d '{
"modelId": "YOUR_MODEL_ID",
"name": "My Fine-Tune Job",
"trainingType": "LORA",
"epochs": 3,
"batchSize": 4,
"learningRate": 0.0001
}' | jq
```
### Training Data Format
Prepare your training data in JSONL format:
```jsonl
{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi! How can I help you?"}]}
{"messages": [{"role": "user", "content": "What is AI?"}, {"role": "assistant", "content": "AI stands for Artificial Intelligence..."}]}
```
Or in conversational format:
```jsonl
{"prompt": "What is machine learning?", "completion": "Machine learning is a subset of AI..."}
{"prompt": "Explain neural networks", "completion": "Neural networks are computing systems..."}
```
### Training via MCP
```
# List all training jobs
Use the list_training_jobs tool
# Create a new training job
Use the create_training_job tool with:
- modelId: The ID of the base model
- trainingType: FINE_TUNE, LORA, or QLORA
- epochs: Number of training epochs (default: 3)
- batchSize: Batch size (default: 4)
- learningRate: Learning rate (default: 0.0001)
# Check job status
Use the get_training_job tool with jobId
# Cancel a running job
Use the cancel_training_job tool with jobId
```
### Example: Complete Training Workflow
```bash
# 1. First, get the model ID you want to fine-tune
curl -s http://localhost:3001/api/models | jq '.[] | {id, name}'
# 2. Create training job
curl -X POST http://localhost:3001/api/training \
-H "Content-Type: application/json" \
-d '{
"modelId": "cm123abc",
"name": "Customer Support Fine-Tune",
"description": "Fine-tune for customer support responses",
"trainingType": "LORA",
"epochs": 5,
"batchSize": 8,
"learningRate": 0.00005
}' | jq
# 3. Check training progress
curl -s http://localhost:3001/api/training | jq '.[] | {id, status, progress, model: .model.displayName}'
# 4. Get detailed job info
curl -s http://localhost:3001/api/training/JOB_ID | jq
# 5. Cancel job if needed
curl -X PATCH http://localhost:3001/api/training/JOB_ID \
-H "Content-Type: application/json" \
-d '{"status": "CANCELLED"}'
```
### Training Job Status
| Status | Description |
|--------|-------------|
| `QUEUED` | Job is waiting to start |
| `PREPARING` | Preparing training environment |
| `TRAINING` | Training in progress |
| `VALIDATING` | Validating trained model |
| `COMPLETED` | Training finished successfully |
| `FAILED` | Training failed (check error message) |
| `CANCELLED` | Job was cancelled by user |
🔑 API Keys & Settings
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/api-keys` | List API keys |
| `POST` | `/api/api-keys` | Create API key |
| `DELETE` | `/api/api-keys/:id` | Delete key |
| `GET` | `/api/health` | Health check |
| `GET` | `/api/stats` | Usage statistics |
| `GET/PATCH` | `/api/settings` | Get/Update settings |
🔧 curl Examples
### Models
```bash
# List all models
curl -s http://localhost:3001/api/models | jq
# List active models only
curl -s "http://localhost:3001/api/models?status=ACTIVE" | jq
# Get model names and parameters
curl -s http://localhost:3001/api/models | jq '.[] | {name, displayName, contextLength, maxTokens, temperature}'
# Get specific model by ID
curl -s http://localhost:3001/api/models/MODEL_ID | jq
# Activate a model
curl -X PATCH http://localhost:3001/api/models/MODEL_ID \
-H "Content-Type: application/json" \
-d '{"status": "ACTIVE"}'
# Delete a model
curl -X DELETE http://localhost:3001/api/models/MODEL_ID
```
### Model Discovery
```bash
# Discover local models (Ollama, Docker containers)
curl -X POST http://localhost:3001/api/models/discover | jq
# List discovered models with context length
curl -s http://localhost:3001/api/models/discover | jq '.[] | {name, source, contextLength, metadata}'
```
### Chat & Inference
```bash
# Simple chat
curl -X POST http://localhost:3001/api/chat \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3:14b",
"message": "Hello, how are you?"
}' | jq
# Chat with system prompt
curl -X POST http://localhost:3001/api/chat \
-H "Content-Type: application/json" \
-d '{
"model": "mistral:7b",
"message": "Explain quantum computing",
"systemPrompt": "You are a physics professor. Explain simply.",
"temperature": 0.7
}' | jq
# Full inference with parameters
curl -X POST http://localhost:3001/api/inference \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3:14b",
"messages": [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "Write a haiku"}
],
"temperature": 0.8,
"maxTokens": 100
}' | jq
```
### Compare Models
```bash
# Compare responses from multiple models
curl -X POST http://localhost:3001/api/compare \
-H "Content-Type: application/json" \
-d '{
"models": ["qwen3:14b", "mistral:7b"],
"prompt": "What is the meaning of life?"
}' | jq
```
### Statistics & Health
```bash
# Get platform statistics
curl -s http://localhost:3001/api/stats | jq
# Health check
curl -s http://localhost:3001/api/health | jq
```
### Prompts
```bash
# List all prompts
curl -s http://localhost:3001/api/prompts | jq
# Create a new prompt template
curl -X POST http://localhost:3001/api/prompts \
-H "Content-Type: application/json" \
-d '{
"name": "Code Review",
"description": "Review code for best practices",
"category": "development",
"systemPrompt": "You are an expert code reviewer",
"userPromptTemplate": "Review this {{language}} code:\n\n{{code}}"
}' | jq
```
## 🤝 Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
## 📄 License
This project is licensed under the **GNU Affero General Public License v3.0 (AGPL-3.0)** - see the [LICENSE](LICENSE) file for details.
**What this means:**
- ✅ You can use, modify, and distribute the code
- ✅ You must provide source code for any changes
- ✅ If you run a modified version as a web service, you must release the source code
- ❌ You cannot make this code proprietary
## 🚀 What's New in v1.3.2
### ⚡ Chat Streaming
Real-time streaming responses - watch AI responses appear word-by-word!
- **Streaming Toggle** - Enable/disable in Chat Settings (⚙️ icon)
- **Stop Button** - Cancel generation mid-response (red button appears while generating)
- **Auto-Scroll** - Chat automatically follows new text
- Works with Ollama and OpenAI models
### 📦 Prompt Import/Export
Share and backup your prompt templates:
- **Export** - Download all prompts as JSON (⬇️ button)
- **Import** - Upload JSON file to import prompts (⬆️ button)
- **Import Modes**:
- Skip existing - Keep your prompts, only add new ones
- Overwrite - Replace existing prompts with imported versions
- Smart UI - Export button disabled when no prompts exist
### 🎨 UI Improvements
- Cleaner chat interface - AI responses now blend with page background
- Improved markdown rendering during streaming
## 🚀 Coming Next
| Feature | Description | Status |
|---------|-------------|--------|
| **🧬 Embeddings** | Generate and manage text embeddings for semantic search | ✅ Implemented |
| **🔄 Model Versioning** | Track different versions of fine-tuned models | 📋 Planned |
| **📊 Advanced Analytics** | Model performance metrics and dashboards | 📋 Planned |
| **🌐 Distributed Training** | Multi-GPU training across nodes | 📋 Planned |
> 💡 **Want to suggest a feature?** [Open an issue](https://github.com/tron4x/neuronhub/issues) on GitHub!
## 🙏 Technologies Used
**LLM Providers:**
- [Ollama](https://ollama.ai/) - Local LLM runtime (MIT License)
- [OpenAI](https://openai.com/) - GPT models API
- [Anthropic](https://anthropic.com/) - Claude models API
**Frontend:**
- [Next.js](https://nextjs.org/) - React framework by Vercel
- [React](https://react.dev/) - UI library
- [Tailwind CSS](https://tailwindcss.com/) - Utility-first CSS
- [Framer Motion](https://www.framer.com/motion/) - Animations
**Backend:**
- [Prisma](https://prisma.io/) - Database ORM
- [PostgreSQL](https://www.postgresql.org/) - Database
- [Redis](https://redis.io/) - Cache & Queue
- [BullMQ](https://bullmq.io/) - Job queue
**ML/Training:**
- [HuggingFace Transformers](https://huggingface.co/transformers/) - Model loading
- [PEFT](https://huggingface.co/docs/peft/) - LoRA training
- [PyTorch](https://pytorch.org/) - ML framework
**Standards:**
- [Model Context Protocol](https://modelcontextprotocol.io/) - MCP standard
---
Made with ❤️ by tron4x
NeuronHub is developed and maintained by tron4x. While we strive for quality,
bugs may occur. We actively monitor and address reported issues.
Your feedback helps make NeuronHub better! 🚀
Thank you for your support and feedback! 🙏