{"id":19175496,"url":"https://github.com/jzo001/webapistreaming","last_synced_at":"2025-05-07T19:10:04.698Z","repository":{"id":89201074,"uuid":"605783210","full_name":"JZO001/WebApiStreaming","owner":"JZO001","description":"How to get data as a stream from a WebAPI (.NET)","archived":false,"fork":false,"pushed_at":"2023-02-24T10:17:56.000Z","size":9,"stargazers_count":33,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-07T19:09:57.362Z","etag":null,"topics":["csharp","dotnet-core","iasyncenumerable","streaming","streaming-api","streaming-data","webapi-core"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JZO001.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-23T22:21:49.000Z","updated_at":"2025-01-24T16:47:11.000Z","dependencies_parsed_at":"2023-06-14T10:00:08.080Z","dependency_job_id":null,"html_url":"https://github.com/JZO001/WebApiStreaming","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JZO001%2FWebApiStreaming","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JZO001%2FWebApiStreaming/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JZO001%2FWebApiStreaming/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JZO001%2FWebApiStreaming/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JZO001","download_url":"https://codeload.github.com/JZO001/WebApiStreaming/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252940934,"owners_count":21828769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csharp","dotnet-core","iasyncenumerable","streaming","streaming-api","streaming-data","webapi-core"],"created_at":"2024-11-09T10:23:31.048Z","updated_at":"2025-05-07T19:10:02.894Z","avatar_url":"https://github.com/JZO001.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# WebAPI Streaming\nHow to get data as a stream from a WebAPI (.NET)\n\nA remote web API can be consumed as a stream, which can take longer,\ndepends an other server process, and the necessary time to generate\nthe requested content.\n\nIt is a real scenario, if we get event, data pieces from a services, etc.\n\n\nIn this project I will demonstrate, how we can use a streaming API,\nhow we can iterate through on the incoming dataset with the IAsyncEnumerable interface.\n\n\n## The server side, using a controller\n\nThis controller generates a set of weather data, than send the dataset\nback to the client as a stream. Every data item serialized into Json format,\nand finally it sends '[DONE]' string to indicate, the streaming is over.\n\n\n```c#\n[ApiController]\n[Route(\"[controller]\")]\npublic class WeatherForecastController : ControllerBase\n{\n    private static readonly string[] Summaries = new[]\n    {\n        \"Freezing\", \"Bracing\", \"Chilly\", \"Cool\", \"Mild\", \n        \"Warm\", \"Balmy\", \"Hot\", \"Sweltering\", \"Scorching\"\n    };\n\n    private readonly ILogger\u003cWeatherForecastController\u003e _logger;\n\n    public WeatherForecastController(ILogger\u003cWeatherForecastController\u003e logger)\n    {\n        _logger = logger;\n    }\n\n    [HttpGet]\n    public async Task GetAsync()\n    {\n        Response.StatusCode = 200;\n        Response.ContentType = \"text/html\";\n\n        List\u003cWeatherForecast\u003e list = Enumerable.Range(1, 5)\n            .Select(index =\u003e new WeatherForecast\n        {\n            Date = DateTime.Now.AddDays(index),\n            TemperatureC = Random.Shared.Next(-20, 55),\n            Summary = Summaries[Random.Shared.Next(Summaries.Length)]\n        })\n        .ToList();\n\n        StreamWriter sw;\n        await using ((sw = new StreamWriter(Response.Body))\n            .ConfigureAwait(false))\n        {\n            foreach (WeatherForecast item in list)\n            {\n                // Thread.Sleep simulates a long running process, \n                // which generates some kind of output\n                Thread.Sleep(1000);\n\n                await sw.WriteLineAsync(item.ToString()).ConfigureAwait(false);\n                await sw.FlushAsync().ConfigureAwait(false);\n            }\n            await sw.WriteLineAsync(\"[DONE]\").ConfigureAwait(false);\n        };\n\n    }\n\n}\n```\n\n\n## The client side #1, implementing the caller API\n\nOn the client side, we need a caller code, which can read and handle the incoming data.\nCheck out the 'yield' instruction when passing every incoming data item towards the API caller.\nIf the code detects the '[DONE]' maker, the reading ends.\n\n\n```c#\npublic class Api\n{\n\n    public async IAsyncEnumerable\u003cstring\u003e StreamedAsync([System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken = default)\n    {\n        using (HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, \"https://localhost:7176/weatherforecast\"))\n        {\n            using (HttpClient httpClient = new HttpClient())\n            {\n                //httpClient.Timeout = Timeout.InfiniteTimeSpan;\n                using (HttpResponseMessage response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false))\n                {\n                    if (response.IsSuccessStatusCode)\n                    {\n                        using (Stream contentStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))\n                        {\n                            using (StreamReader reader = new StreamReader(contentStream))\n                            {\n                                string? line = null;\n\n                                while ((line = await reader.ReadLineAsync().ConfigureAwait(false)) != null \u0026\u0026\n                                    !cancellationToken.IsCancellationRequested)\n                                {\n                                    if (\"[DONE]\".Equals(line.Trim())) break;\n\n                                    if (!string.IsNullOrWhiteSpace(line))\n                                    {\n                                        yield return line;\n                                    }\n                                }\n                            }\n                        }\n                    }\n                    else\n                    {\n                        string? jsonResult = await response.Content.ReadAsStringAsync().ConfigureAwait(false);\n                        yield return jsonResult;\n                    }\n                }\n            }\n        }\n    }\n\n}\n```\n\n\n## The client side #2, using the client side API\n\nFinally, a code can use the Api class to acquire the data, using IAsyncEnumerable\u003cstring\u003e on a proper way.\n\nThis code iterates on the incoming data asynchronously and display the given data as a text, \none by one, after a data item arrived. This approch does not wait to arrive all of the data,\nit immediatelly displays what we got from the server.\n\n\n```c#\nstatic async Task Main(string[] args)\n{\n    Console.WriteLine(\"Press a key to start\");\n    Console.ReadKey();\n\n    Api api = new Api();\n    await foreach (string line in api.StreamedAsync(CancellationToken.None))\n    {\n        Console.WriteLine(line);\n    }\n\n    Console.WriteLine(\"[DONE] press a key to exit\");\n    Console.ReadKey();\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjzo001%2Fwebapistreaming","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjzo001%2Fwebapistreaming","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjzo001%2Fwebapistreaming/lists"}