{"id":17785052,"url":"https://github.com/eun/go-pgx-cursor-iterator","last_synced_at":"2025-03-16T04:30:56.915Z","repository":{"id":38318926,"uuid":"405078328","full_name":"Eun/go-pgx-cursor-iterator","owner":"Eun","description":"A golang package for fetching big chunks of rows from a postgres database using a cursor.","archived":false,"fork":false,"pushed_at":"2025-02-17T09:00:44.000Z","size":8623,"stargazers_count":3,"open_issues_count":2,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-27T05:04:22.122Z","etag":null,"topics":["database","go","golang","pgx","postgres","postgresql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Eun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Eun"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":"Eun","issuehunt":"eun","otechie":null,"custom":null}},"created_at":"2021-09-10T12:47:10.000Z","updated_at":"2025-02-17T09:00:47.000Z","dependencies_parsed_at":"2024-02-27T13:44:51.311Z","dependency_job_id":"4cc70e5c-0195-477f-bee8-63ebcfb58ff6","html_url":"https://github.com/Eun/go-pgx-cursor-iterator","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eun%2Fgo-pgx-cursor-iterator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eun%2Fgo-pgx-cursor-iterator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eun%2Fgo-pgx-cursor-iterator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eun%2Fgo-pgx-cursor-iterator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Eun","download_url":"https://codeload.github.com/Eun/go-pgx-cursor-iterator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243802944,"owners_count":20350316,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","go","golang","pgx","postgres","postgresql"],"created_at":"2024-10-27T08:23:05.350Z","updated_at":"2025-03-16T04:30:53.978Z","avatar_url":"https://github.com/Eun.png","language":"Go","funding_links":["https://github.com/sponsors/Eun","https://liberapay.com/Eun","https://issuehunt.io/r/eun"],"categories":[],"sub_categories":[],"readme":"# go-pgx-cursor-iterator\n[![Actions Status](https://github.com/Eun/go-pgx-cursor-iterator/workflows/push/badge.svg)](https://github.com/Eun/go-pgx-cursor-iterator/actions)\n[![Coverage Status](https://coveralls.io/repos/github/Eun/go-pgx-cursor-iterator/badge.svg?branch=master)](https://coveralls.io/github/Eun/go-pgx-cursor-iterator?branch=master)\n[![PkgGoDev](https://img.shields.io/badge/pkg.go.dev-reference-blue)](https://pkg.go.dev/github.com/Eun/go-pgx-cursor-iterator)\n[![go-report](https://goreportcard.com/badge/github.com/Eun/go-pgx-cursor-iterator)](https://goreportcard.com/report/github.com/Eun/go-pgx-cursor-iterator)\n---\nA golang package for fetching big chunks of rows from a postgres database using a cursor.\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\tcursoriterator \"github.com/Eun/go-pgx-cursor-iterator/v2\"\n\t\"github.com/jackc/pgx/v5/pgxpool\"\n)\n\ntype User struct {\n\tName string `db:\"name\"`\n\tRole string `db:\"role\"`\n}\n\nfunc main() {\n\tctx := context.Background()\n\tpool, _ := pgxpool.New(ctx, \"example-connection-url\")\n\n\tvalues := make([]User, 1000)\n\titer, err := cursoriterator.NewCursorIterator(pool, values, \"SELECT * FROM users WHERE role = $1\", \"Guest\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer iter.Close(ctx)\n\tfor iter.Next(ctx) {\n\t\tfmt.Printf(\"Name: %s\\n\", values[iter.ValueIndex()].Name)\n\t}\n\tif err := iter.Error(); err != nil {\n\t\tpanic(err)\n\t}\n}\n```\n\n## Behind the scenes\nWith the first `Next()` call the iterator will start a transaction and define the cursor.  \nAfter that it will fetch the first chunk of rows.\n\nWith the following `Next()` calls the iterator will first consume the already fetched rows.  \nIf it iterated over all (pre)fetched rows it will fetch the next chunk and repeats the process.\n\nWith a chunk/batch size of 100 items and 250 rows in the database,\nthe iterator will perform 3 fetches:\n1. Fetching 100 items (150 rows left)\n2. Fetching 100 items (50 rows left)\n3. Fetching 100 items, but will only return 50 (0 rows left)\n\nThe passed in `values` slice will be used as storage. So don't rely on the contents besides fetching the data from it.  \nTherefore, you should not reference an item in the `values` slice since it will most likely be replaced sooner or later.\n(depending on its size)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feun%2Fgo-pgx-cursor-iterator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feun%2Fgo-pgx-cursor-iterator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feun%2Fgo-pgx-cursor-iterator/lists"}