{"id":42640850,"url":"https://github.com/spatialcurrent/go-pipe","last_synced_at":"2026-01-29T06:14:46.039Z","repository":{"id":45997350,"uuid":"187301229","full_name":"spatialcurrent/go-pipe","owner":"spatialcurrent","description":"go-pipe is a simple library for piping objects from iterators to writers.","archived":false,"fork":false,"pushed_at":"2021-11-21T22:42:49.000Z","size":48,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2023-12-11T09:19:46.257Z","etag":null,"topics":["big-data","bigdata","concurrency","data"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/spatialcurrent/go-pipe/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spatialcurrent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-18T01:37:50.000Z","updated_at":"2021-11-21T22:41:38.000Z","dependencies_parsed_at":"2022-09-10T06:12:22.922Z","dependency_job_id":null,"html_url":"https://github.com/spatialcurrent/go-pipe","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/spatialcurrent/go-pipe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatialcurrent%2Fgo-pipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatialcurrent%2Fgo-pipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatialcurrent%2Fgo-pipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatialcurrent%2Fgo-pipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spatialcurrent","download_url":"https://codeload.github.com/spatialcurrent/go-pipe/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatialcurrent%2Fgo-pipe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28865713,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-29T05:56:06.453Z","status":"ssl_error","status_checked_at":"2026-01-29T05:55:57.668Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","bigdata","concurrency","data"],"created_at":"2026-01-29T06:14:44.578Z","updated_at":"2026-01-29T06:14:46.026Z","avatar_url":"https://github.com/spatialcurrent.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![CircleCI](https://circleci.com/gh/spatialcurrent/go-pipe/tree/main.svg?style=svg)](https://circleci.com/gh/spatialcurrent/go-pipe/tree/main)\n[![Go Report Card](https://goreportcard.com/badge/spatialcurrent/go-pipe?style=flat-square)](https://goreportcard.com/report/github.com/spatialcurrent/go-pipe)\n[![PkgGoDev](https://pkg.go.dev/badge/mod/github.com/spatialcurrent/go-pipe)](https://pkg.go.dev/github.com/spatialcurrent/go-pipe)\n[![License](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://github.com/spatialcurrent/go-pipe/blob/master/LICENSE)\n\n# go-pipe\n\n# Description\n\n**go-pipe** is a simple library for piping data from iterators to writers.  Pipe uses the following interfaces, which allows reading and writing of objects to any number of inputs and outputs.\n\n**Iterator**\n\nThe Next method will return io.EOF once it has read all the objects from the input.\n\n```go\ntype Iterator interface {\n\tNext() (interface{}, error)\n}\n```\n\n**Writer**\n\nWriter is a writer that accepts input as individual objects.\n\n```go\ntype Writer interface {\n\tWriteObject(object interface{}) error\n\tFlush() error\n}\n```\n\n**BatchWriter**\n\nBatchWriter is a writer that accepts input in batches as an array or slice of objects.\n\n```go\ntype Writer interface {\n\tWriteObjects(objects interface{}) error\n\tFlush() error\n}\n```\n\n**go-pipe** includes concrete structs for writing to channels, slice, functions, and maps.  It is also used in [railgun](https://github.com/spatialcurrent/railgun) project along with [go-simple-serializer](https://github.com/spatialcurrent/go-simple-serializer) to process objects from files.\n\n# Usage\n\n**Go**\n\nInstall the package with:\n\n```shell\ngo get -d github.com/spatialcurrent/go-pipe/...\n```\n\nYou can import **go-pipe** as a library with:\n\n```go\nimport (\n  \"github.com/spatialcurrent/go-pipe/pkg/pipe\"\n)\n```\n\nSee [pipe](https://pkg.go.dev/github.com/spatialcurrent/go-pipe/pkg/pipe) in the docs for information on how to use Go API.\n\n# Examples\n\nSee the many [examples](https://pkg.go.dev/github.com/spatialcurrent/go-pipe/pkg/pipe/#pkg-examples) in the docs or the tests.\n\n# Testing\n\nRun test using `make test` or (`bash scripts/test.sh`), which runs unit tests, `go vet`, `go vet with shadow`, [errcheck](https://github.com/kisielk/errcheck), [staticcheck](https://staticcheck.io/), and [misspell](https://github.com/client9/misspell).\n\n# Contributing\n\n[Spatial Current, Inc.](https://spatialcurrent.io) is currently accepting pull requests for this repository.  We'd love to have your contributions!  Please see [Contributing.md](https://github.com/spatialcurrent/go-pipe/blob/master/CONTRIBUTING.md) for how to get started.\n\n# License\n\nThis work is distributed under the **MIT License**.  See **LICENSE** file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspatialcurrent%2Fgo-pipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspatialcurrent%2Fgo-pipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspatialcurrent%2Fgo-pipe/lists"}