{"id":16531399,"url":"https://github.com/marcbachmann/find-in-batches","last_synced_at":"2025-09-13T23:32:20.154Z","repository":{"id":33059405,"uuid":"36695885","full_name":"marcbachmann/find-in-batches","owner":"marcbachmann","description":"Batch find and processing method. DEPRECATED: Please use native async iterators","archived":false,"fork":false,"pushed_at":"2023-12-15T05:44:23.000Z","size":16,"stargazers_count":3,"open_issues_count":3,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-14T14:40:20.924Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcbachmann.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-06-01T23:48:32.000Z","updated_at":"2023-09-26T07:38:16.000Z","dependencies_parsed_at":"2023-12-15T06:50:13.396Z","dependency_job_id":null,"html_url":"https://github.com/marcbachmann/find-in-batches","commit_stats":{"total_commits":23,"total_committers":3,"mean_commits":7.666666666666667,"dds":0.08695652173913049,"last_synced_commit":"c933286e423877a9121d6e5061bef74ab395ec19"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcbachmann%2Ffind-in-batches","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcbachmann%2Ffind-in-batches/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcbachmann%2Ffind-in-batches/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcbachmann%2Ffind-in-batches/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcbachmann","download_url":"https://codeload.github.com/marcbachmann/find-in-batches/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232931899,"owners_count":18598665,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T18:08:43.628Z","updated_at":"2025-01-07T20:09:03.471Z","avatar_url":"https://github.com/marcbachmann.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# find-in-batches\n\nA flow control helper method that invokes a forEach function for each element that was returned in the passed find method. This is a wrapper around [async.js](https://github.com/caolan/async) and uses simple pagination to do its magic.\n\n## API\n\n```\nfindInBatches.each([options], findMethod, forEach, callback)\nfindInBatches.batch([options], findMethod, forBatch, callback)\nfindInBatches.all([options], findMethod, callback)\n```\n\n### findInBatches.each\n```js\nvar request = require('request')\nvar findInBatches = require('find-in-batches')\n\nvar options = {\n    offset: 0,            // a default offset\n    batchSize: 50,        // this gets passed to the findMethod as limit\n    maximum: Infinity,    // maximum count of documents that get loaded\n    concurrency: Infinity // how many forEach methods should run simultaneously\n}\n\nfindInBatches.each(options, findMethod, forEach, function(err){\n    if (err) return console.error('Failed to load the entries', err)\n    console.log('Loaded all the entries')\n});\n\nfunction findMethod (options, callback) {\n    // options = {offset: 0, limit: 50, page: 1} // on first search\n    // options = {offset: 50, limit: 50, page: 2} // on second iteration\n    var url = 'http://maps.gelbeseiten.de'\n    url += '/yp/subscriberlist_ajaxAction.yp?urlSegmentSubjectString=starbucks'\n    url += '\u0026recfrom=' + options.offset + '\u0026reccount=' + options.limit\n    request.get({url: url, json: true}, function(err, response, body){\n        // the callback must be called with either an error\n        // or an array as second argument\n        if (err) return callback(err)\n        if (!(body \u0026\u0026 body.data \u0026\u0026 body.data.subscribers)) return callback()\n        callback(null, body.data.subscribers.subscribers)\n    })\n}\n\n// forEach gets invoked for each element that gets returned in the findMethod.\n// e.g. if the findMethod would return `callback(null, [{id: 1, name: 'one'}, {id: 2, name: 'two'}])`,\n// the entry would equal `{id: 1, name: 'one'}`\nfunction forEach (entry, callback){\n    entry.address = entry.address || {}\n    entry.address.street = entry.address.street || {}\n    console.log(\n        \"Name: %s, Address: %s, %s\",\n        entry.name,\n        entry.address.street.name + ' ' + entry.address.street.houseno,\n        entry.address.location\n    )\n\n    callback()\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcbachmann%2Ffind-in-batches","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcbachmann%2Ffind-in-batches","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcbachmann%2Ffind-in-batches/lists"}