{"id":23838080,"url":"https://github.com/spongeengine/lmsharp","last_synced_at":"2025-09-07T14:33:01.084Z","repository":{"id":270263259,"uuid":"909704300","full_name":"SpongeEngine/LMSharp","owner":"SpongeEngine","description":"A unified .NET client library for running LLMs (Large Language Models) locally. LocalAI.NET provides a single, consistent API for interacting with popular local LLM providers like KoboldCpp, Ollama, LM Studio, and Text Generation WebUI.","archived":false,"fork":false,"pushed_at":"2024-12-30T16:24:53.000Z","size":47,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-01-02T16:26:51.340Z","etag":null,"topics":["ai","ai-client","csharp","dotnet","koboldcpp","language-models","llm","llm-client","lm-studio","local-llm","offline-ai","ollama","openai-compatible-api","self-hosted-ai","text-generation-webui"],"latest_commit_sha":null,"homepage":"https://www.nuget.org/packages/LocalAI.NET","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SpongeEngine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-29T14:38:40.000Z","updated_at":"2025-01-01T19:24:13.000Z","dependencies_parsed_at":"2024-12-30T01:11:16.292Z","dependency_job_id":null,"html_url":"https://github.com/SpongeEngine/LMSharp","commit_stats":null,"previous_names":["spongeengine/localai.net","spongeengine/lmsharp"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpongeEngine%2FLMSharp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpongeEngine%2FLMSharp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpongeEngine%2FLMSharp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpongeEngine%2FLMSharp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SpongeEngine","download_url":"https://codeload.github.com/SpongeEngine/LMSharp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232221122,"owners_count":18490730,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-client","csharp","dotnet","koboldcpp","language-models","llm","llm-client","lm-studio","local-llm","offline-ai","ollama","openai-compatible-api","self-hosted-ai","text-generation-webui"],"created_at":"2025-01-02T16:27:04.862Z","updated_at":"2025-01-02T16:27:06.139Z","avatar_url":"https://github.com/SpongeEngine.png","language":"C#","readme":"# LocalAI.NET (In Progress)\n[![NuGet](https://img.shields.io/nuget/v/LocalAI.NET.svg)](https://www.nuget.org/packages/LocalAI.NET)\n[![NuGet Downloads](https://img.shields.io/nuget/dt/LocalAI.NET.svg)](https://www.nuget.org/packages/LocalAI.NET)\n[![License](https://img.shields.io/github/license/SpongeEngine/LocalAI.NET)](LICENSE)\n[![.NET](https://img.shields.io/badge/.NET-6.0%20%7C%207.0%20%7C%208.0%2B-512BD4)](https://dotnet.microsoft.com/download)\n[![Tests](https://github.com/SpongeEngine/LocalAI.NET/actions/workflows/test.yml/badge.svg)](https://github.com/SpongeEngine/LocalAI.NET/actions/workflows/test.yml)\n\nA unified .NET client library for running LLMs (Large Language Models) locally. LocalAI.NET provides a single, consistent API for interacting with popular local LLM providers like KoboldCpp, Ollama, LM Studio, and Text Generation WebUI. Perfect for developers who want to:\n\n- Run AI models locally without cloud dependencies\n- Switch between different local LLM providers without changing code\n\nLocalAI.NET acts as a universal wrapper around local LLM providers, offering:\n- Single API: Use the same code regardless of the underlying LLM provider\n- Provider Flexibility: Easily switch between KoboldCpp, Ollama, LM Studio, or Text Generation WebUI\n- Production Ready: Built-in retry policies, circuit breakers, and error handling\n- Modern .NET: Async/await, streaming support, and comprehensive logging\n- Privacy First: All AI operations run locally - your data never leaves your machine\n\n📦 [View Package on NuGet](https://www.nuget.org/packages/LocalAI.NET)\n\n## Feature Comparison\n| Feature | LocalAI.NET | OpenAI.NET | LLamaSharp | OllamaSharp |\n|---------|-------------|------------|------------|-------------|\n| Local LLM Support | ✅ | ❌ | ✅ | ✅ |\n| Multiple Providers | ✅ | ❌ | ❌ | ❌ |\n| KoboldCpp Support | ✅ | ❌ | ❌ | ❌ |\n| Ollama Support | ✅ | ❌ | ❌ | ✅ |\n| LM Studio Support | ✅ | ❌ | ❌ | ❌ |\n| Text Gen WebUI Support | ✅ | ❌ | ❌ | ❌ |\n| Streaming | ✅ | ✅ | ✅ | ✅ |\n| OpenAI Compatible | ✅ | ✅ | ❌ | ✅ |\n| Progress Tracking | ✅ | ❌ | ❌ | ❌ |\n| Retry Policies | ✅ | ❌ | ❌ | ❌ |\n| Circuit Breaker | ✅ | ❌ | ❌ | ❌ |\n| .NET Standard 2.0 | ❌ | ✅ | ✅ | ✅ |\n| .NET 6.0+ | ✅ | ✅ | ✅ | ✅ |\n\n## Supported Providers\n- [KoboldCpp](https://github.com/LostRuins/koboldcpp): Both native and OpenAI-compatible modes\n- [Ollama](https://github.com/ollama/ollama): Run Llama 2, Code Llama, and other models locally (using OllamaSharp).\n- [LM Studio](https://lmstudio.ai): Local deployment of various open-source models\n- [Text Generation WebUI](https://github.com/oobabooga/text-generation-webui): Popular web interface for running local models\n\n## Installation\nInstall LocalAI.NET via NuGet:\n```bash\ndotnet add package LocalAI.NET\n```\n\n## Quick Start\n```csharp\nusing LocalAI.NET.Client;\nusing LocalAI.NET.Models.Configuration;\n\n// Create client with KoboldCpp provider\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:5000\",\n    ProviderOptions = new KoboldCppNativeOptions\n    {\n        ContextSize = 2048,\n        UseGpu = true,\n        RepetitionPenalty = 1.1f\n    }\n};\n\nusing var client = new LocalAIClient(options);\n\n// Generate text completion\nstring response = await client.CompleteAsync(\"Write a short story about a robot:\");\n\n// Stream completion tokens\nawait foreach (var token in client.StreamCompletionAsync(\"Once upon a time...\"))\n{\n    Console.Write(token);\n}\n\n// List available models\nvar models = await client.GetAvailableModelsAsync();\nforeach (var model in models)\n{\n    Console.WriteLine($\"Model: {model.Name} (Provider: {model.Provider})\");\n    Console.WriteLine($\"Context Length: {model.Capabilities.MaxContextLength}\");\n}\n```\n\n## Provider Configuration\n\n### KoboldCpp (Native)\n```csharp\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:5000\",\n    ProviderOptions = new KoboldCppNativeOptions\n    {\n        ContextSize = 2048,\n        UseGpu = true,\n        RepetitionPenalty = 1.1f,\n        RepetitionPenaltyRange = 320,\n        TrimStop = true,\n        Mirostat = new MirostatSettings\n        {\n            Mode = 2,\n            Tau = 5.0f,\n            Eta = 0.1f\n        }\n    }\n};\n```\n\n### KoboldCpp (OpenAI-compatible)\n```csharp\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:5000\",\n    ProviderOptions = new KoboldCppOpenAiOptions\n    {\n        ContextSize = 2048,\n        UseGpu = true,\n        ModelName = \"koboldcpp\",\n        UseChatCompletions = true\n    }\n};\n```\n\n### Ollama\n```csharp\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:11434\",\n    ProviderOptions = new OllamaOptions\n    {\n        ConcurrentRequests = 1\n    }\n};\n```\n\n### LM Studio\n```csharp\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:1234\",\n    ProviderOptions = new LMStudioOptions\n    {\n        UseOpenAIEndpoint = true\n    }\n};\n```\n\n### Text Generation WebUI\n```csharp\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:7860\",\n    ProviderOptions = new TextGenWebOptions\n    {\n        UseOpenAIEndpoint = true\n    }\n};\n```\n\n## Completion Options\n```csharp\nvar options = new CompletionOptions\n{\n    ModelName = \"wizardLM\",         // Optional model name\n    MaxTokens = 200,                // Max tokens to generate\n    Temperature = 0.7f,             // Randomness (0.0-1.0)\n    TopP = 0.9f,                    // Nucleus sampling threshold\n    StopSequences = new[] { \"\\n\" }  // Sequences that stop generation\n};\n\nstring response = await client.CompleteAsync(\"Your prompt here\", options);\n```\n\n## Progress Tracking\n```csharp\nclient.OnProgress += (progress) =\u003e\n{\n    switch (progress.State)\n    {\n        case LocalAIProgressState.Starting:\n            Console.WriteLine(\"Starting completion...\");\n            break;\n        case LocalAIProgressState.Processing:\n            Console.WriteLine($\"Processing: {progress.Message}\");\n            break;\n        case LocalAIProgressState.Streaming:\n            Console.WriteLine(\"Receiving tokens...\");\n            break;\n        case LocalAIProgressState.Complete:\n            Console.WriteLine(\"Completion finished!\");\n            break;\n        case LocalAIProgressState.Failed:\n            Console.WriteLine($\"Error: {progress.Message}\");\n            break;\n    }\n};\n```\n\n## Error Handling\n```csharp\ntry\n{\n    var response = await client.CompleteAsync(\"Test prompt\");\n}\ncatch (LocalAIException ex)\n{\n    Console.WriteLine($\"LocalAI API error: {ex.Message}\");\n    if (ex.StatusCode.HasValue)\n    {\n        Console.WriteLine($\"Status code: {ex.StatusCode}\");\n    }\n    if (ex.Provider != null)\n    {\n        Console.WriteLine($\"Provider: {ex.Provider}\");\n    }\n}\ncatch (Exception ex)\n{\n    Console.WriteLine($\"General error: {ex.Message}\");\n}\n```\n\n## Advanced Configuration\n```csharp\nvar options = new LocalAIOptions\n{\n    BaseUrl = \"http://localhost:5000\",\n    ApiKey = \"optional_api_key\",\n    Timeout = TimeSpan.FromMinutes(2),\n    MaxRetryAttempts = 3,\n    RetryDelay = TimeSpan.FromSeconds(2),\n    Logger = loggerInstance,\n    JsonSettings = new JsonSerializerSettings(),\n    ProviderOptions = new KoboldCppNativeOptions()\n};\n```\n\n## License\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Contributing\nContributions are welcome! Please feel free to submit a Pull Request.\n\nPlease see [CONTRIBUTING.md](CONTRIBUTING.md) for details on:\n- How to publish to NuGet\n- Development guidelines\n- Code style\n- Testing requirements\n- Pull request process\n\n## Support\nFor issues and feature requests, please use the [GitHub issues page](https://github.com/SpongeEngine/LocalAI.NET/issues).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspongeengine%2Flmsharp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspongeengine%2Flmsharp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspongeengine%2Flmsharp/lists"}