https://github.com/kzu/streamai
A sample of how to implement streaming responses from OpenAI using SignalR
https://github.com/kzu/streamai
Last synced: over 1 year ago
JSON representation
A sample of how to implement streaming responses from OpenAI using SignalR
- Host: GitHub
- URL: https://github.com/kzu/streamai
- Owner: kzu
- License: mit
- Created: 2024-02-10T22:06:55.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-17T19:14:59.000Z (over 2 years ago)
- Last Synced: 2025-04-04T13:49:22.583Z (over 1 year ago)
- Language: C#
- Size: 14.6 KB
- Stars: 27
- Watchers: 3
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: license.txt
Awesome Lists containing this project
README
# StreamAI
An example using a minimal ASP.NET Core server to stream responses
from OpenAI to a console app, over SignalR.
Implemented using the suggested approach in the
[Microsoft Copilot implementation blog post](https://devblogs.microsoft.com/dotnet/building-ai-powered-bing-chat-with-signalr-and-other-open-source-tools/#deep-dive-how-do-we-use-signalr),
using no third-party libraries (even for OpenAI) beyond plain SignalR.
https://github.com/kzu/StreamAI/assets/169707/114b8f6f-d7d2-4d98-82af-ed1c06d541ee
## Running
Before running the server, set your OpenAI API key by running:
```shell
dotnet user-secrets set OpenAI:Key
```
Then open the solution in Visual Studio and set both projects
as startup projects.
# Who Needs Streaming?
If you think you need to stream responses because the LLM is too slow, make yourself a favor and try [groq](https://wow.groq.com/).
I can hardly think of a scenario where I'd need streaming anymore once that tech can run GPT-4 level models.