{"id":16625479,"url":"https://github.com/aleclarson/recrawl","last_synced_at":"2025-09-16T03:32:21.945Z","repository":{"id":57349330,"uuid":"138640946","full_name":"aleclarson/recrawl","owner":"aleclarson","description":"Filesystem crawler","archived":false,"fork":false,"pushed_at":"2022-10-20T16:01:19.000Z","size":304,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-13T04:05:54.766Z","etag":null,"topics":["crawler","fs","nodejs"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aleclarson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-25T19:32:04.000Z","updated_at":"2022-12-04T13:11:55.000Z","dependencies_parsed_at":"2022-09-17T06:41:27.043Z","dependency_job_id":null,"html_url":"https://github.com/aleclarson/recrawl","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aleclarson%2Frecrawl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aleclarson%2Frecrawl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aleclarson%2Frecrawl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aleclarson%2Frecrawl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aleclarson","download_url":"https://codeload.github.com/aleclarson/recrawl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":220108424,"owners_count":16595775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","fs","nodejs"],"created_at":"2024-10-12T04:05:50.783Z","updated_at":"2025-09-16T03:32:16.646Z","avatar_url":"https://github.com/aleclarson.png","language":"TypeScript","funding_links":["https://paypal.me/alecdotbiz"],"categories":[],"sub_categories":[],"readme":"# recrawl\n\n[![npm](https://img.shields.io/npm/v/recrawl.svg)](https://www.npmjs.com/package/recrawl)\n[![ci](https://github.com/aleclarson/recrawl/actions/workflows/release.yml/badge.svg)](https://github.com/aleclarson/recrawl/actions/workflows/release.yml)\n[![codecov](https://codecov.io/gh/aleclarson/recrawl/branch/master/graph/badge.svg)](https://codecov.io/gh/aleclarson/recrawl)\n[![Bundle size](https://badgen.net/bundlephobia/min/recrawl)](https://bundlephobia.com/result?p=recrawl)\n[![Code style: Prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg)](https://github.com/prettier/prettier)\n[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://paypal.me/alecdotbiz)\n\nCollect the descendants of a directory.\n\n```ts\nimport { recrawl } from 'recrawl'\n\n// Create a crawl function.\n// These are the default options.\nconst crawl = recrawl({\n  only: [],\n  skip: [],\n  deep: true,\n  follow: false,\n})\n\n// The result is an array when `follow` is false, else an object.\nconst files = crawl(root)\n\n// Provide your own array/object.\ncrawl(root, files)\n\n// Provide an iterator.\ncrawl(root, (file, link) =\u003e {\n  // The `file` argument is relative to the root.\n  // The `link` argument is null for non-symlinks. It will be absolute if the target is outside the root.\n})\n```\n\nYou can use the `crawl()` export if you don't want to reuse the configured crawler.\n\n```ts\nimport { crawl } from 'recrawl'\n\ncrawl(root, {\n  only: [],\n  skip: [],\n  deep: true,\n  follow: false,\n})\n```\n\n### Options\n\n- `only?: (string|RegExp)[]`\n- `skip?: (string|RegExp)[]`\n- `absolute?: boolean`\n- `deep?: boolean`\n- `depth?: number`\n- `enter?: function`\n- `filter?: function`\n- `follow?: boolean|number|function`\n- `adapter?: FileAdapter`\n\nThe `only` and `skip` options should be self-explanatory. Paths matching any of\nthe `only` patterns are good. When `only` is an empty array, all paths are good.\nPaths matching any of the `skip` patterns are bad. When `skip` is an empty\narray, no paths are bad. The `skip` patterns override the `only` patterns.\n\nThe `absolute` option converts matching file paths into their absolute form.\n\nTo avoid crawling sub-directories, set `deep` to false or `depth` to 0. You\nshould never define both `deep` and `depth`, because the `depth` option implies\n`deep` when it's greater than zero. If neither `deep` nor `depth` are defined,\nthe default depth is infinite.\n\nThe `enter` option is called whenever a directory is encountered. It's passed\nthe directory path and the current depth. You may return a falsy value to avoid\ncrawling a directory.\n\nThe `filter` option is called whenever a filename is encountered. It's passed\nthe filename and its basename. You may return a falsy value to skip a filename.\nThe `only` and `skip` options are applied before this option is called.\n\nTo follow all symlinks, set `follow` to true. For greater control, use a\nfunction. It's called whenever a symlink is encountered. You may return a falsy\nvalue to avoid following a symlink. It's passed the symlink path and the current\nlink depth. If you only need to limit the link depth, you can set `follow` to a\nnumber, where zero is equivalent to false.\n\nThe `adapter` option lets you provide your own filesystem.\n\n### Gotchas\n\n- Directory symlinks are treated the same as real directories\n- Directories are not affected by the `only` option\n\n### Pattern syntax\n\nRecrawl has its own take on globbing.\n\n1. When a path has no separators (`/`), only the basename is matched.\n\n```js\n'*.js' // matches 'a.js' and 'a/b.js'\n```\n\n2. Recursivity is implicit.\n\n```js\n'a/b' // identical to '**/a/b'\n```\n\n3. Use a leading separator to match against the root.\n\n```js\n'/*.js' // matches 'a.js' not 'a/b.js'\n```\n\n4. Use a trailing separator to match all descendants.\n\n```js\n'foo/' // matches 'foo/bar' and 'foo/bar/baz' etc\n```\n\n5. Regular expression syntax is supported. (except dot-all)\n\n```js\n'*.jsx?' // matches 'a.js' and 'b.jsx'\n'*.(js|ts)' // matches 'a.js' and 'b.ts'\n```\n\n6. Recursive globbing is supported.\n\n```js\n'foo/**/bar' // matches 'foo/bar' and 'foo/a/b/c/bar' etc\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faleclarson%2Frecrawl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faleclarson%2Frecrawl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faleclarson%2Frecrawl/lists"}