{"id":45056651,"url":"https://github.com/xyctruth/stream","last_synced_at":"2026-02-19T09:44:23.989Z","repository":{"id":37959973,"uuid":"467455424","full_name":"xyctruth/stream","owner":"xyctruth","description":"A Stream processing library based on Go 1.18+ Generics (parallel, pipeline, lazy) :cherries:","archived":false,"fork":false,"pushed_at":"2023-07-03T15:39:17.000Z","size":149,"stargazers_count":180,"open_issues_count":1,"forks_count":15,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-06-18T16:58:10.577Z","etag":null,"topics":["concurrency","functional","generics","go","golang","lazy","parallel","parallel-programming","pipeline","stream"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/xyctruth/stream","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xyctruth.png","metadata":{"files":{"readme":"README-ZH.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-08T09:59:46.000Z","updated_at":"2024-05-29T10:44:25.000Z","dependencies_parsed_at":"2024-06-18T16:57:50.655Z","dependency_job_id":null,"html_url":"https://github.com/xyctruth/stream","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/xyctruth/stream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyctruth%2Fstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyctruth%2Fstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyctruth%2Fstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyctruth%2Fstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xyctruth","download_url":"https://codeload.github.com/xyctruth/stream/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyctruth%2Fstream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29609525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T06:47:36.664Z","status":"ssl_error","status_checked_at":"2026-02-19T06:45:47.551Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","functional","generics","go","golang","lazy","parallel","parallel-programming","pipeline","stream"],"created_at":"2026-02-19T09:44:18.758Z","updated_at":"2026-02-19T09:44:23.984Z","avatar_url":"https://github.com/xyctruth.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Stream\n\n[![Build](https://github.com/xyctruth/stream/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/xyctruth/stream/actions/workflows/test.yml)\n[![codecov](https://codecov.io/gh/xyctruth/stream/branch/main/graph/badge.svg?token=ZHMPMQP0CP)](https://codecov.io/gh/xyctruth/stream)\n\n\u003e [English](./README.md) / [中文](./README-ZH.md)\n\n`Stream` 是一个基于 Go 1.18+ 泛型的流式处理库, 它支持并行处理流中的数据.\n\n## 特性\n\n- `并行流`: 并行处理流中的数据,保持流中元素原始顺序\n- `流水线`: 组合多个操作以减少元素循环,更早地短路\n- `惰性调用`: 中间操作是惰性的\n\n## 安装\n\n需要安装 Go 1.18+ 版本\n\n```go\nimport \"github.com/xyctruth/stream\"\n```\n\n## 入门\n\n```go\n// constraints.Ordered\ns := stream.NewSliceByOrdered([]string{\"d\", \"a\", \"b\", \"c\", \"a\"}).\n    Filter(func(s string) bool { return s != \"b\" }).\n    Map(func(s string) string { return \"class_\" + s }).\n    Sort().\n    Distinct().\n    ToSlice()\n```\n\n## 类型约束\n\n`any` 接受任何类型的元素, 所以不能使用 `==` `!=` `\u003e` `\u003c` 比较元素, 导致你不能使用 Sort(), Find()...等函数 ,但是你可以使用 SortFunc(fn), FindFunc(fn)... 代替\n\n```go\nstream.NewSlice([]int{1, 2, 3, 7, 1})\n```\n\n`comparable` 接收的类型可以使用 `==` `!=` 比较元素, 但仍然不能使用 `\u003e` `\u003c` 比较元素, 因此你不能使用 Sort(), Min()...等函数 ,但是你可以使用 SortFunc(fn), MinFunc()... 代替\n\n```go\nstream.NewSliceByComparable([]int{1, 2, 3, 7, 1})\n```\n\n`constraints.Ordered` 接收的类型可以使用 `==` `!=` `\u003e` `\u003c`, 所以可以使用所有的函数\n\n```go\nstream.NewSliceByOrdered([]int{1, 2, 3, 7, 1})\n```\n\n## 类型转换\n\n有些时候我们需要使用 `Map` ,`Reduce` 转换切片元素的类型,但是很遗憾目前 Golang 并不支持结构体的方法有额外的类型参数,所有类型参数必须在结构体中声明。在 Golang 支持之前我们暂时使用临时方案解决这个问题。\n\n```go\n// SliceMappingStream  Need to convert the type of slice elements.\n// - E elements type\n// - MapE map elements type\n// - ReduceE reduce elements type\ntype SliceMappingStream[E any, MapE any, ReduceE any] struct {\n    SliceStream[E]\n}\n\ns := stream.NewSliceByMapping[int, string, string]([]int{1, 2, 3, 4, 5}).\n    Filter(func(v int) bool { return v \u003e3 }).\n    Map(func(v int) string { return \"mapping_\" + strconv.Itoa(v) }).\n    Reduce(func(r string, v string) string { return r + v })\n```\n\n## 并行\n\n`Parallel` 函数接收一个 `goroutines int` 参数. 如果 goroutines\u003e1 则开启并行, 否则关闭并行, 默认流是关闭并行的。\n\n并行会将流中的元素平均划分多个的分区, 并创建相同数量的 goroutine 执行, 并且会保证处理完成后流中元素保持原始顺序.\n\n```go\ns := stream.NewSliceByOrdered([]string{\"d\", \"a\", \"b\", \"c\", \"a\"}).\n    Parallel(10).\n    Filter(func(s string) bool {\n    // 一些耗时操作\n    return s != \"b\"\n    }).\n    Map(func(s string) string {\n    // 一些耗时操作\n    return \"class_\" + s\n    }).\n    ForEach(\n    func(index int, s string) {\n    // 一些耗时操作\n    },\n    ).ToSlice()\n```\n\n### 并行类型\n\n- `First`: 一旦获得第一个返回值，并行处理就结束. `For: AllMatch, AnyMatch, FindFunc`\n- `ALL`: 所有元素都需要并行处理，得到所有返回值，然后并行结束. `For: Map, Filter`\n- `Action`: 所有元素需要并行处理，不需要返回值. `For: ForEach, Action`\n\n### 并行 Goroutines 数量\n\n开启并行 goroutine 数量在面对 CPU 操作与 IO 操作有着不同的选择。 一般面对 CPU 操作时 goroutine 数量不需要设置大于 CPU 核心数，而 IO 操作时 goroutine 数量可以设置远远大于 CPU 核心数.\n\n#### CPU 操作\n\n[BenchmarkParallelByCPU](./benchmark_test.go)\n\n```go\nNewSlice(s).Parallel(goroutines).ForEach(func(i int, v int) {\n    sort.Ints(newArray(1000)) // 模拟 CPU 耗时操作\n})\n```\n使用6个cpu核心进行基准测试\n```go\ngo test -run=^$ -benchtime=5s -cpu=6  -bench=^BenchmarkParallelByCPU\n\ngoos: darwin\ngoarch: amd64\npkg: github.com/xyctruth/stream\ncpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz\nBenchmarkParallelByCPU/no_Parallel(0)-6         \t     710\t   8265106 ns/op\nBenchmarkParallelByCPU/goroutines(2)-6          \t    1387\t   4333929 ns/op\nBenchmarkParallelByCPU/goroutines(4)-6          \t    2540\t   2361783 ns/op\nBenchmarkParallelByCPU/goroutines(6)-6          \t    3024\t   2100158 ns/op\nBenchmarkParallelByCPU/goroutines(8)-6          \t    2347\t   2531435 ns/op\nBenchmarkParallelByCPU/goroutines(10)-6         \t    2622\t   2306752 ns/op\n```\n\n#### IO 操作\n\n[BenchmarkParallelByIO](./benchmark_test.go)\n\n```go\nNewSlice(s).Parallel(goroutines).ForEach(func(i int, v int) {\n    time.Sleep(time.Millisecond) // 模拟 IO 耗时操作\n})\n```\n使用6个cpu核心进行基准测试\n```go\ngo test -run=^$ -benchtime=5s -cpu=6  -bench=^BenchmarkParallelByIO\n\ngoos: darwin\ngoarch: amd64\npkg: github.com/xyctruth/stream\ncpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz\nBenchmarkParallelByIO/no_parallel(0)-6          \t      52\t 102023558 ns/op\nBenchmarkParallelByIO/goroutines(2)-6           \t     100\t  55807303 ns/op\nBenchmarkParallelByIO/goroutines(4)-6           \t     214\t  27868725 ns/op\nBenchmarkParallelByIO/goroutines(6)-6           \t     315\t  18925789 ns/op\nBenchmarkParallelByIO/goroutines(8)-6           \t     411\t  14439700 ns/op\nBenchmarkParallelByIO/goroutines(10)-6          \t     537\t  11164758 ns/op\nBenchmarkParallelByIO/goroutines(50)-6          \t    2629\t   2310602 ns/op\nBenchmarkParallelByIO/goroutines(100)-6         \t    5094\t   1221887 ns/op\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyctruth%2Fstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxyctruth%2Fstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyctruth%2Fstream/lists"}