{"id":19419205,"url":"https://github.com/mariomac/gostream","last_synced_at":"2025-04-24T14:31:42.545Z","repository":{"id":42450817,"uuid":"434550283","full_name":"mariomac/gostream","owner":"mariomac","description":"A Go port of the Java Streams API. Type-safe and functional Go Streams processing for Go 1.18+. ","archived":false,"fork":false,"pushed_at":"2022-12-02T17:51:20.000Z","size":216,"stargazers_count":161,"open_issues_count":2,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-03T06:12:40.751Z","etag":null,"topics":["generics","go","golang","stream-processing","type-parameters"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mariomac.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-03T10:09:05.000Z","updated_at":"2025-03-09T12:38:18.000Z","dependencies_parsed_at":"2023-01-23T08:45:57.505Z","dependency_job_id":null,"html_url":"https://github.com/mariomac/gostream","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariomac%2Fgostream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariomac%2Fgostream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariomac%2Fgostream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariomac%2Fgostream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mariomac","download_url":"https://codeload.github.com/mariomac/gostream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250643421,"owners_count":21464172,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generics","go","golang","stream-processing","type-parameters"],"created_at":"2024-11-10T13:16:45.324Z","updated_at":"2025-04-24T14:31:42.199Z","avatar_url":"https://github.com/mariomac.png","language":"Go","funding_links":[],"categories":["Stream Processing","流处理"],"sub_categories":["HTTP Clients","HTTP客户端","Other Software"],"readme":"# Go Streams API\n\nType safe Stream processing library inspired in the [Java Streams API](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html).\n\n## Table of contents\n\n* [Requirements](#requirements)\n* [Usage examples](#usage-examples)\n* [Limitations](#limitations)\n* [Performance](#performance)\n* [Completion status](#completion-status)\n* [Extra credits](#extra-credits)\n\n## Requirements\n\n* Go 1.18 or higher\n\nThis library makes intensive usage of [Type Parameters (generics)](https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md) so it is not compatible with any Go version lower than 1.18.\n\n## Usage examples\n\nFor more details about the API, and until [pkg.go.dev](https://pkg.go.dev/github.com/mariomac/gostream), \nis able to parse documentation for functions and types using generics, you can have a quick look\nto the generated [go doc text descriptions](./docs), or just let that the embedded document browser of your\nIDE does the job.\n\n### Example 1: basic creation, transformation and iteration\n\n1. Creates a literal stream containing all the integers from 1 to 11.\n2. From the Stream, selects all the integers that are prime\n3. For each filtered int, prints a message.\n\n```go\nimport (\n  \"fmt\"\n  \"github.com/mariomac/gostream/stream\"\n)\n\nfunc main() {\n  stream.Of(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11).\n    Filter(isPrime).\n    ForEach(func(n int) {\n      fmt.Printf(\"%d is a prime number\\n\", n)\n    })\n}\n\nfunc isPrime(n int) bool {\n  for i := 2; i \u003c n/2; i++ {\n    if n%i == 0 {\n      return false\n    }\n  }\n  return true\n}\n```\n\nOutput: \n```\n1 is a prime number\n2 is a prime number\n3 is a prime number\n5 is a prime number\n7 is a prime number\n11 is a prime number\n```\n\n### Example 2: generation, map, limit and slice conversion\n\n1. Creates an **infinite** stream of random integers (no problem, streams are evaluated lazily!)\n2. Divides the random integer to get a number between 1 and 6\n3. Limits the infinite stream to 5 elements.\n4. Collects the stream items as a slice.\n\n```go\nrand.Seed(time.Now().UnixMilli())\nfmt.Println(\"let me throw 5 times a dice for you\")\n\nresults := stream.Generate(rand.Int).\n    Map(func(n int) int {\n        return n%6 + 1\n    }).\n    Limit(5).\n    ToSlice()\n\nfmt.Printf(\"results: %v\\n\", results)\n```\n\nOutput:\n```\nlet me throw 5 times a dice for you\nresults: [3 5 2 1 3]\n```\n\n### Example 3: Generation from an iterator, Map to a different type\n\n1. Generates an infinite stream composed by `1`, `double(1)`, `double(double(1))`, etc...\n   and cut it to 6 elements.\n2. Maps the numbers' stream to a strings' stream. Because, at the moment,\n   [go does not allow type parameters in methods](https://github.com/golang/go/issues/49085),\n   we need to invoke the `stream.Map` function instead of the `numbers.Map` method\n   because the contained type of the output stream (`string`) is different than the type of\n   the input stream (`int`).\n3. Converts the words stream to a slice and prints it.\n\n\n```go\nfunc main() {\n    numbers := stream.Iterate(1, double).Limit(6)\n    words := stream.Map(numbers, asWord).ToSlice()\n    fmt.Println(words)\n}\n\nfunc double(n int) int {\n    return 2 * n\n}\n\nfunc asWord(n int) string {\n    if n \u003c 10 {\n        return []string{\"zero\", \"one\", \"two\", \"three\", \"four\", \"five\",\n            \"six\", \"seven\", \"eight\", \"nine\"}[n]\n    } else {\n        return \"many\"\n    }\n}\n```\n\nOutput:\n```\n[one two four eight many many]\n```\n\n### Example 4: deduplication of elements\n\nFollowing example requires to compare the elements of the Stream, so the Stream needs to be\ncomposed by `comparable` elements (this is, accepted by the the `==` and `!=` operators):\n\n1. Instantiate a `Stream` of `comparable` items.\n2. Pass it to the `Distinct` method, that will return a copy of the original Stream without\n   duplicates\n3. Operating as any other stream.\n\n```go\nwords := stream.Distinct(\n  stream.Of(\"hello\", \"hello\", \"!\", \"ho\", \"ho\", \"ho\", \"!\"),\n).ToSlice()\n\nfmt.Printf(\"Deduplicated words: %v\\n\", words)\n```\n\nOutput:\n\n```\nDeduplicated words: [hello ! ho]\n```\n\n### Example 5: sorting from higher to lower\n\n1. Generate a stream of uint32 numbers.\n2. Picking up 5 elements.\n3. Sorting them by the inverse natural order (from higher to lower)\n   - It's **important** to limit the number of elements, avoiding invoking\n     `Sorted` over an infinite stream (otherwise it would panic).\n\n```go\nfmt.Println(\"picking up 5 random numbers from higher to lower\")\nstream.Generate(rand.Uint32).\n    Limit(5).\n    Sorted(order.Inverse(order.Natural[uint32])).\n    ForEach(func(n uint32) {\n        fmt.Println(n)\n    })\n```\n\nOutput:\n\n```\npicking up 5 random numbers from higher to lower\n4039455774\n2854263694\n2596996162\n1879968118\n1823804162\n```\n\n### Example 6: Reduce and helper functions\n\n1. Generate an infinite incremental Stream (1, 2, 3, 4...) using the `stream.Iterate`\n   instantiator and the `item.Increment` helper function.\n2. Limit the generated to 8 elements\n3. Reduce all the elements multiplying them using the item.Multiply helper function\n\n```go\nfac8, _ := stream.Iterate(1, item.Increment[int]).\n    Limit(8).\n    Reduce(item.Multiply[int])\nfmt.Println(\"The factorial of 8 is\", fac8)\n```\n\nOutput: \n\n```\nThe factorial of 8 is 40320\n```\n\n## Limitations\n\nDue to the initial limitations of Go generics, the API has the following limitations.\nWe will work on overcome them as long as new features are added to the Go type parameters\nspecification.\n\n* You can use `Map` and `FlatMap` as method as long as the output element has the same type of the input.\n  If you need to map to a different type, you need to use `stream.Map` or `stream.FlatMap` as functions.\n* There is no `Distinct` method. There is only a `stream.Distinct` function.\n* There is no `ToMap` method. There is only a `stream.ToMap` function.\n\n## Performance\n\nYou might want to check: [Performance comparison of Go functional stream libraries](https://macias.info/entry/202212020000_go_streams.md).\n\nStreams aren't the fastest option. They are aimed for complex workflows where you can\nsacrifice few microseconds for the sake of code organization and readability. Also disclaimer:\nfunctional streams don't have to always be the most readable option.\n\nThe following results show the difference in performance for an arbitrary set of operations\nin an imperative form versus the functional form using streams (see\n[stream/benchs_test.go file](stream/benchs_test.go)):\n\n```\n$ gotip test -bench=. -benchmem  ./...\ngoos: darwin\ngoarch: amd64\npkg: github.com/mariomac/gostream/stream\ncpu: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz\nBenchmarkImperative-4            2098518               550.6 ns/op          1016 B/op          7 allocs/op\nBenchmarkFunctional-4             293095              3653 ns/op            2440 B/op         23 allocs/op\n```\n\nIf you want a more performant, parallelizable alternative to create data processing pipelines (following\na programming model focused on Extract-Transform-Load, ETL), you\ncould give a try to my alternative project: [PIPES: Processing In Pipeline-Embedded Stages](https://github.com/mariomac/pipes).\n\n## Completion status\n\n* Stream instantiation functions\n  - [X] Comparable\n  - [X] Concat\n  - [X] Empty\n  - [X] Generate\n  - [X] Iterate\n  - [X] Of\n  - [X] OfMap\n  - [x] OfSlice\n  - [X] OfChannel\n* Stream transformers\n  - [X] Distinct\n  - [X] Filter\n  - [X] FlatMap\n  - [X] Limit\n  - [X] Map\n  - [X] Peek\n  - [X] Skip\n  - [X] Sorted\n* Collectors/Terminals\n  - [X] ToMap\n  - [X] ToSlice\n  - [X] AllMatch\n  - [X] AnyMatch\n  - [X] Count\n  - [X] FindFirst\n  - [X] ForEach\n  - [X] Max\n  - [X] Min\n  - [X] NoneMatch\n  - [X] Reduce\n* Auxiliary Functions\n  - [X] Add (for numbers)\n  - [X] Increment (for numbers)\n  - [X] IsZero\n  - [X] Multiply (for numbers)\n  - [X] Neg (for numbers)\n  - [X] Not (for bools)\n* Future\n  - [ ] Collectors for future standard generic data structures\n    - E.g. [ ] Join (for strings)\n  - [ ] Allow users implement their own Comparable or Ordered types\n  - [ ] More operations inspired in the Kafka Streams API\n  - [ ] Parallel streams \n    - [ ] FindAny\n\n\n## Extra credits\n\nThe Stream processing and aggregation functions are heavily inspired in the\n[Java Stream Specification](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html).\n\nStream code documentation also used \n[Stream Javadoc](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html) as an\nessential reference and might contain citations from it.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmariomac%2Fgostream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmariomac%2Fgostream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmariomac%2Fgostream/lists"}