https://github.com/liliang-cn/lmstudio-go
A comprehensive Go client library for LM Studio, providing easy access to local language models for chat completions, embeddings, tool calling, and model management.
https://github.com/liliang-cn/lmstudio-go
lmstudio local-llm
Last synced: 24 days ago
JSON representation
A comprehensive Go client library for LM Studio, providing easy access to local language models for chat completions, embeddings, tool calling, and model management.
- Host: GitHub
- URL: https://github.com/liliang-cn/lmstudio-go
- Owner: liliang-cn
- Created: 2025-08-27T08:07:05.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2025-08-27T08:08:02.000Z (about 1 month ago)
- Last Synced: 2025-08-28T16:00:40.778Z (about 1 month ago)
- Topics: lmstudio, local-llm
- Language: Go
- Homepage:
- Size: 59.6 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LM Studio Go SDK
[](https://pkg.go.dev/github.com/liliang-cn/lmstudio-go)
[](https://goreportcard.com/report/github.com/liliang-cn/lmstudio-go)
[](https://opensource.org/licenses/MIT)
A comprehensive Go client library for [LM Studio](https://lmstudio.ai/), providing easy access to local language models for chat completions, embeddings, tool calling, and model management.
## Features
- **Chat Completions**: Synchronous, streaming, and async chat completions
- **Embeddings**: Generate text embeddings with similarity calculations
- **Tool Calling**: Function calling with automatic tool registry and handling
- **Structured Responses**: JSON schema generation and validation
- **Model Management**: Load, unload, and query model status with real-time progress
- **WebSocket Support**: Real-time model loading with progress callbacks
- **JavaScript-like API**: Familiar namespace-based API (`client.LLM`, `client.Embedding`)
- **Model Repository**: Search, download, and manage models
- **Session Management**: Persistent chat sessions with history
- **Type Safety**: Full type definitions and compile-time safety
- **Comprehensive Testing**: Extensive test coverage with mock servers
## Installation
```bash
go get github.com/liliang-cn/lmstudio-go
```
## Quick Start
### Basic Chat Completion
```go
package main
import (
"context"
"fmt"
"log"
"github.com/liliang-cn/lmstudio-go"
)
func main() {
// Create client with default settings (localhost:1234)
client := lmstudio.NewClient()
// Simple chat
response, err := client.Chat.SimpleChat(
context.Background(),
"your-model-name",
"What is the capital of France?",
)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Response: %s\n", response)
}
```
### Custom Configuration
```go
config := &lmstudio.Config{
BaseURL: "http://localhost:8080",
Timeout: 60 * time.Second,
UseWebSocket: true, // Enable WebSocket for full model management
SessionsDir: "./my-sessions", // For session persistence
}
client := lmstudio.NewClientWithConfig(config)
```
### JavaScript-like API
The SDK provides a familiar namespace-based API similar to the official JavaScript SDK:
```go
// Get any loaded LLM model (like: client.llm.model())
llmModel, err := client.LLM.Model(ctx)
if err != nil {
log.Fatal(err)
}
// Use the model
response, err := llmModel.SimpleChat(ctx, "Hello!")
// Get specific model or auto-load (like: client.llm.model("model-key"))
specificModel, err := client.LLM.Model(ctx, "qwen3-4b")
// List all loaded LLM models
llmHandles, err := client.LLM.ListLoaded(ctx)
// Get any loaded embedding model (like: client.embedding.model())
embeddingModel, err := client.Embedding.Model(ctx)
embeddings, err := embeddingModel.CreateEmbedding(ctx, "Hello world")
// List all loaded embedding models
embeddingHandles, err := client.Embedding.ListLoaded(ctx)
```
## Advanced Usage
### Streaming Chat
```go
ctx := context.Background()
stream, err := client.Chat.Stream(ctx, &lmstudio.ChatRequest{
Model: "your-model",
Messages: []lmstudio.Message{
{Role: "user", Content: "Tell me a story"},
},
})
if err != nil {
log.Fatal(err)
}
for {
select {
case chunk := <-stream.Response:
if len(chunk.Choices) > 0 && chunk.Choices[0].Delta.Content != nil {
fmt.Print(*chunk.Choices[0].Delta.Content)
}
case err := <-stream.Errors:
if err != nil {
log.Fatal(err)
}
return
case <-ctx.Done():
return
}
}
```
### Chat with Options
```go
response, err := client.Chat.ChatWithHistory(ctx, "your-model", messages,
lmstudio.WithTemperature(0.7),
lmstudio.WithMaxTokens(100),
lmstudio.WithTopP(0.9),
)
```
### Embeddings
```go
// Single embedding
embedding, err := client.Embeddings.CreateSingle(ctx, "embedding-model", "Hello world")
if err != nil {
log.Fatal(err)
}
// Batch embeddings
texts := []string{"Hello", "World", "AI"}
embeddings, err := client.Embeddings.CreateBatch(ctx, "embedding-model", texts)
// Calculate similarity
similarity, err := lmstudio.CosineSimilarity(embeddings[0], embeddings[1])
```
### Tool Calling
```go
// Create tool registry
registry := lmstudio.NewToolRegistry()
// Define a tool
weatherTool := lmstudio.NewFunctionTool(
"get_weather",
"Get current weather for a location",
map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"location": map[string]interface{}{
"type": "string",
"description": "City name",
},
},
"required": []string{"location"},
},
)
// Register tool with handler
registry.RegisterTool(*weatherTool, func(toolCall *lmstudio.ToolCall) (string, error) {
var args struct {
Location string `json:"location"`
}
if err := lmstudio.ParseToolCallArguments(toolCall, &args); err != nil {
return "", err
}
return fmt.Sprintf("Weather in %s: 22°C, Sunny", args.Location), nil
})
// Chat with tool handling
messages := []lmstudio.Message{
{Role: "user", Content: "What's the weather in Paris?"},
}
response, err := client.Chat.ChatWithToolsAndHandle(ctx, "your-model", messages, registry, 5)
```
### Structured Responses
```go
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
City string `json:"city"`
}
var person Person
structuredResp, err := client.Chat.CompleteWithSchema(
ctx,
"your-model",
[]lmstudio.Message{
{Role: "user", Content: "Generate a person: John, 30 years old, from New York"},
},
&person,
)
if structuredResp.Valid {
fmt.Printf("Person: %+v\n", person)
}
```
### Model Management
```go
// List available models
models, err := client.Models.List(ctx)
for _, model := range models {
fmt.Printf("Model: %s, Status: %s\n", model.ID, model.Status)
}
// WebSocket model loading with real-time progress
loadOptions := &lmstudio.ModelLoadOptions{
Identifier: "my-model-instance",
TTL: 1800, // Auto-unload after 30 minutes
Config: map[string]interface{}{
"contextLength": 2048,
"gpu": map[string]interface{}{
"ratio": 0.8, // 80% GPU usage
},
},
OnProgress: func(message string) {
fmt.Printf("📊 %s\n", message)
},
}
loadedModel, err := client.Models.LoadWithProgress(ctx, "qwen/qwen3-4b", loadOptions)
if err != nil {
log.Fatal(err)
}
// Use the loaded model
response, err := client.Chat.SimpleChat(ctx, loadedModel.ID(), "Hello!")
// Unload when done
err = loadedModel.Unload(ctx)
// Check if loaded
isLoaded, err := client.Models.IsLoaded(ctx, "model-id")
```
### Session Management
```go
// Create a session
session, err := client.Sessions.manager.CreateSession("My Chat", "your-model")
// Chat within session (maintains history)
response, err := client.Sessions.ChatWithSession(
ctx,
session.ID,
"Hello, remember our conversation",
)
// Export session
err = client.Sessions.ExportSession(session.ID, "./chat-export.json")
// List all sessions
sessions, err := client.Sessions.manager.ListSessions()
```
### Model Repository
```go
// Search for models
searchResp, err := client.Repository.Search(ctx, &lmstudio.SearchRequest{
Query: "llama",
ModelType: "llm",
Limit: 10,
})
// Get popular models
popular, err := client.Repository.ListPopular(ctx, 5)
// Download a model with progress tracking
err = client.Repository.Download(ctx, &lmstudio.DownloadRequest{
ModelID: "microsoft/DialoGPT-medium",
DestinePath: "./models/",
}, func(progress *lmstudio.DownloadProgress) {
fmt.Printf("Progress: %.1f%% (%d/%d bytes)\n",
progress.Progress*100,
progress.BytesDownloaded,
progress.TotalBytes,
)
})
```
## Error Handling
The SDK provides structured error handling:
```go
response, err := client.Chat.Complete(ctx, request)
if err != nil {
if lmErr, ok := err.(*lmstudio.Error); ok {
// LM Studio API error
fmt.Printf("API Error [%d]: %s\n", lmErr.Code, lmErr.Message)
} else {
// Other errors (network, parsing, etc.)
fmt.Printf("Error: %v\n", err)
}
}
```
## Testing
The SDK includes comprehensive tests with mock HTTP servers:
```bash
go test ./...
go test -v ./... # Verbose output
go test -race ./... # Race condition detection
```
## Examples
See the [examples](./examples) directory for complete working examples:
- [Basic Chat](./examples/basic/main.go) - Simple chat completions
- [JavaScript-like API](./examples/js-like-api/main.go) - Namespace-based model management
- [WebSocket Model Loading](./examples/websocket-test/main.go) - Real-time model loading
- [WebSocket Quick Test](./examples/websocket-quick-test/main.go) - WebSocket functionality demo
- [Model Listing](./examples/model-listing/main.go) - List and manage models
- [Comprehensive Test](./examples/comprehensive/main.go) - Advanced features showcase
## Documentation
Full API documentation is available on [pkg.go.dev](https://pkg.go.dev/github.com/liliang-cn/lmstudio-go).
Generate local documentation:
```bash
go doc -all github.com/liliang-cn/lmstudio-go
```
## Contributing
1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass: `go test ./...`
5. Run linting: `golangci-lint run`
6. Submit a pull request
## Requirements
- Go 1.23 or later
- LM Studio running locally (default: `localhost:1234`)
## License
MIT License - see [LICENSE](LICENSE) file for details.
## Related Projects
- [lmstudio-python](https://github.com/lmstudio-ai/lmstudio-python) - Official Python SDK
- [lmstudio-js](https://github.com/lmstudio-ai/lmstudio-js) - Official JavaScript SDK
## Changelog
### v1.1.0 (Latest)
- **WebSocket Support**: Real-time model loading with progress callbacks
- **JavaScript-like API**: Familiar namespace-based API (`client.LLM`, `client.Embedding`)
- **Enhanced Model Management**: Load, unload with TTL and progress tracking
- **Model Handles**: Direct model operations with simplified API
- **Dual-mode Architecture**: HTTP fallback + WebSocket enhanced functionality
### v1.0.0
- Initial release
- Chat completions with streaming support
- Embeddings API
- Tool calling functionality
- Structured response support
- Model management
- Session persistence
- Model repository integration
- Comprehensive test coverage