{"id":14108673,"url":"https://github.com/vikstrous/dataloadgen","last_synced_at":"2025-09-12T14:32:03.561Z","repository":{"id":42198271,"uuid":"466535014","full_name":"vikstrous/dataloadgen","owner":"vikstrous","description":"An implementation of Facebook's DataLoader in Go","archived":false,"fork":false,"pushed_at":"2025-06-07T19:12:58.000Z","size":328,"stargazers_count":93,"open_issues_count":0,"forks_count":13,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-07T20:19:08.131Z","etag":null,"topics":["dataloader","gqlgen","graphql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vikstrous.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-05T18:23:57.000Z","updated_at":"2025-06-07T19:11:39.000Z","dependencies_parsed_at":"2023-12-06T16:28:17.653Z","dependency_job_id":"a5673e18-2dae-4266-b175-8eaece9ff537","html_url":"https://github.com/vikstrous/dataloadgen","commit_stats":null,"previous_names":["vikstrous/dataloaden"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/vikstrous/dataloadgen","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vikstrous%2Fdataloadgen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vikstrous%2Fdataloadgen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vikstrous%2Fdataloadgen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vikstrous%2Fdataloadgen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vikstrous","download_url":"https://codeload.github.com/vikstrous/dataloadgen/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vikstrous%2Fdataloadgen/sbom","scorecard":{"id":921599,"data":{"date":"2025-08-11","repo":{"name":"github.com/vikstrous/dataloadgen","commit":"e93635f4a7bed7f70feba8efe81dcea3238168b9"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.6,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":2,"reason":"Found 4/17 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":10,"reason":"10 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: license.md:0","Info: FSF or OSI recognized license: MIT License: license.md:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 19 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-25T03:53:01.926Z","repository_id":42198271,"created_at":"2025-08-25T03:53:01.926Z","updated_at":"2025-08-25T03:53:01.926Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274824653,"owners_count":25356650,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataloader","gqlgen","graphql"],"created_at":"2024-08-14T10:01:34.294Z","updated_at":"2025-09-12T14:32:03.549Z","avatar_url":"https://github.com/vikstrous.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# dataloadgen\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/vikstrous/dataloadgen.svg)](https://pkg.go.dev/github.com/vikstrous/dataloadgen)\n\n`dataloadgen` is an implementation of a pattern popularized by [Facebook's DataLoader](https://github.com/graphql/dataloader).\n\nIt works as follows:\n* A Loader object is created per graphql request.\n* Each of many concurrently executing graphql resolver functions call `Load()` on the Loader object with different keys. Let's say `K1`, `K2`, `K3`\n* Each call to `Load()` with a new key is delayed slightly (a few milliseconds) so that the Loader can load them together.\n* The customizable `fetch` function of the loader takes a list of keys and loads data for all of them in a single batched request to the data storage layer. It might send `[K1,K2,K3]` and get back `[V1,V2,V3]`. The order of the keys must match the order of the values.\n  * Alternatively, the `mappedFetch` function of the loader takes a list of keys and returns a map instead of a list. It might send `[K1, K2, K3]` and get back `{K1: V1, K2: V2, K3: V3}`.\n* The Loader takes care of sending the right result to the right caller and the result is cached for the duration of the graphql request.\n\n\u003e [!NOTE]\n\u003e The `fetch` method expects the returned list to correspond to the provided keys in the same order. Alternatively, the `mappedFetch` allows returning a map, ensuring correct ordering and automatic creation of the `ErrNotFound` error for missing values.\n\nUsage:\n\n```sh\ngo get github.com/vikstrous/dataloadgen\n```\n\nSee the usage [example](https://pkg.go.dev/github.com/vikstrous/dataloadgen#example-Loader) in the documentation:\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strconv\"\n\n\t\"github.com/vikstrous/dataloadgen\"\n)\n\n// fetchFn/mappedFetchFn is shown as a function here, but it might work better as a method\n// ctx is the context from the first call to Load for the current batch\nfunc fetchFn(ctx context.Context, keys []string) (ret []int, errs []error) {\n    for _, key := range keys {\n        num, err := strconv.ParseInt(key, 10, 32)\n        ret = append(ret, int(num))\n        errs = append(errs, err)\n    }\n    return\n}\nfunc mappedFetchFn(ctx context.Context, keys []string) (ret map[string]int, err error) {\n\tret = make(map[string]int, len(keys))\n\terrs := make(map[string]error, len(keys))\n    for _, key := range keys {\n        num, err := strconv.ParseInt(key, 10, 32)\n        ret[key] = int(num)\n        errs[key] = err\n    }\n\t// You can also return a single error, returned for every key's load invocation, instead of this MappedFetchError.\n    err = dataloadgen.MappedFetchError[string](errs)\n    return\n}\n\nfunc main() {\n    ctx := context.Background()\n\t\n    // Per-request setup code. Either:\n    loader := dataloadgen.NewLoader(fetchFn)\n    // or\n    loader := dataloadgen.NewMappedLoader(mappedFetchFn)\n\t\n    // In every graphql resolver:\n    result, err := loader.Load(ctx, \"1\")\n    if err != nil {\n        panic(err)\n    }\n    fmt.Println(result)\n}\n```\n\n## Comparison to others\n\n* [dataloaden](https://github.com/vektah/dataloaden) uses code generation and has similar performance\n* [dataloader](https://github.com/graph-gophers/dataloader) does not use code generation but has much worse performance and is more difficult to use\n* [yckao/go-dataloader](https://github.com/yckao/go-dataloader) does not use code generation but has much worse performance and is very similar to dataloader.\n\nThe benchmarks in this repo show that this package is faster than all of the above and I also find it easier to use.\n\n\u003cdetails\u003e\n\u003csummary\u003eBenchmark data as CSV\u003c/summary\u003e\n\n```\nBenchmark,Package,iterations,ns/op,B/op,allocs/op\ninit-8,graph-gophers/dataloader,\"9,242,047.00\",130.50,208.00,3.00\ninit-8,vektah/dataloaden,\"1,000,000,000.00\",0.27,0.00,0.00\ninit-8,yckao/go-dataloader,\"3,153,999.00\",402.10,400.00,10.00\ninit-8,vikstrous/dataloadgen,\"10,347,595.00\",114.90,128.00,3.00\ncached-8,graph-gophers/dataloader,\"4,669.00\",\"222,072.00\",\"25,307.00\",522.00\ncached-8,vektah/dataloaden,\"1,243.00\",\"1,037,044.00\",\"5,234.00\",110.00\ncached-8,yckao/go-dataloader,\"2,312.00\",\"580,860.00\",\"2,273.00\",130.00\ncached-8,vikstrous/dataloadgen,\"1,552.00\",\"824,939.00\",776.00,15.00\nunique_keys-8,graph-gophers/dataloader,\"12,334.00\",\"97,118.00\",\"56,314.00\",945.00\nunique_keys-8,vektah/dataloaden,\"36,489.00\",\"32,507.00\",\"37,514.00\",227.00\nunique_keys-8,yckao/go-dataloader,\"8,055.00\",\"133,224.00\",\"50,180.00\",747.00\nunique_keys-8,vikstrous/dataloadgen,\"42,943.00\",\"27,257.00\",\"22,255.00\",230.00\n10_concurrently-8,graph-gophers/dataloader,326.00,\"11,119,367.00\",\"5,574,460.00\",\"164,247.00\"\n10_concurrently-8,vektah/dataloaden,100.00,\"194,627,574.00\",\"898,977.00\",\"19,502.00\"\n10_concurrently-8,yckao/go-dataloader,278.00,\"10,972,399.00\",\"314,963.00\",\"29,558.00\"\n10_concurrently-8,vikstrous/dataloadgen,643.00,\"8,249,158.00\",\"43,474.00\",806.00\nall_in_one_request-8,graph-gophers/dataloader,28.00,\"39,954,324.00\",\"27,475,136.00\",\"158,321.00\"\nall_in_one_request-8,vektah/dataloaden,328.00,\"3,713,407.00\",\"3,533,086.00\",\"41,368.00\"\nall_in_one_request-8,yckao/go-dataloader,132.00,\"9,060,571.00\",\"4,886,722.00\",\"102,564.00\"\nall_in_one_request-8,vikstrous/dataloadgen,375.00,\"3,206,175.00\",\"2,518,498.00\",\"41,582.00\"\n```\n\n\u003c/details\u003e\n\n![](init.png)\n![](cached.png)\n![](unique_keys.png)\n![](10_concurrently.png)\n![](all_in_one_request.png)\n\nTo run the benchmarks, run `go test -bench=. . -run BenchmarkAll -benchmem` from the benchmark directory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvikstrous%2Fdataloadgen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvikstrous%2Fdataloadgen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvikstrous%2Fdataloadgen/lists"}