https://github.com/micromatch/glob-fs

file globbing for node.js. speedy and powerful alternative to node-glob. This library is experimental and does not work on windows!
https://github.com/micromatch/glob-fs
files fs glob glob-pattern match micromatch minimatch multimatch node node-glob patterns
Last synced: 3 months ago
JSON representation
file globbing for node.js. speedy and powerful alternative to node-glob. This library is experimental and does not work on windows!
Host: GitHub
URL: https://github.com/micromatch/glob-fs
Owner: micromatch
License: mit
Created: 2015-01-01T00:05:19.000Z (almost 11 years ago)
Default Branch: master
Last Pushed: 2022-01-08T05:25:22.000Z (over 3 years ago)
Last Synced: 2025-06-28T16:18:53.735Z (3 months ago)
Topics: files, fs, glob, glob-pattern, match, micromatch, minimatch, multimatch, node, node-glob, patterns
Language: JavaScript
Homepage: http://jonschlinkert.github.io/glob-fs
Size: 104 KB
Stars: 56
Watchers: 11
Forks: 17
Open Issues: 29
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project

README

          # glob-fs [![NPM version](https://badge.fury.io/js/glob-fs.svg)](http://badge.fury.io/js/glob-fs)

> file globbing for node.js. speedy and powerful alternative to node-glob.

## Usage

```js

var glob = require('glob-fs')({ gitignore: true });

var files = glob.readdirSync('**/*.js');

```

[Run actual examples](./examples/):

Jump to docs sections:

* [glob.readdir](#async)

* [glob.readPromise](#promise)

* [glob.readStream](#stream)

* [glob.readdirSync](#sync)

## Table of contents

* [Install](#install)

* [Usage](#usage)

* [API](#api)

* [Middleware](#middleware)

  - [Middleware examples](#middleware-examples)

  - [Middleware conventions](#middleware-conventions)

  - [Advice for middleware authors](#advice-for-middleware-authors)

* [Globbing examples](#globbing-examples)

  - [async](#async)

  - [promise](#promise)

  - [stream](#stream)

  - [sync](#sync)

* [Events](#events)

  - [Event examples](#event-examples)

* [FAQ](#faq)

* [TODO](#todo)

* [Community middleware](#community-middleware)

* [Related projects](#related-projects)

* [Running tests](#running-tests)

* [Contributing](#contributing)

* [Author](#author)

* [License](#license)

_(Table of contents generated by [verb](https://github.com/assemble/verb))_

## Install

Install with [npm](https://www.npmjs.com/)

```sh

$ npm i glob-fs --save

```

## Usage

**Params**

All "read" methods take a glob pattern and an `options` object.

* `pattern` **{String}**: Glob pattern to use for matching. (multiple pattern support is planned)

* `options` **{Object}**: Options for `glob-fs` or middleware.

**Examples:**

```js

// sync

var files = glob.readdirSync('*.js', {});

// async

glob.readdir('*.js', function(err, files) {

  console.log(files);

});

// stream

glob.readdirStream('*.js', {})

  .on('data', function(file) {

    console.log(file);

  });

// promise

glob.readdirPromise('*.js')

  .then(function(files) {

    console.log(file);

  });

```

## API

### [.readdir](lib/readers.js#L25)

Asynchronously glob files or directories that match the given `pattern`.

**Params**

* `pattern` **{String}**: Glob pattern

* `options` **{Object}**

* `cb` **{Function}**: Callback

**Example**

```js

var glob = require('glob-fs')({ gitignore: true });

glob.readdir('*.js', function (err, files) {

  //=> do stuff with `files`

});

```

### [.readdirSync](lib/readers.js#L59)

Synchronously glob files or directories that match the given `pattern`.

**Params**

* `pattern` **{String}**: Glob pattern

* `options` **{Object}**

* `returns` **{Array}**: Returns an array of files.

**Example**

```js

var glob = require('glob-fs')({ gitignore: true });

var files = glob.readdirSync('*.js');

//=> do stuff with `files`

```

### [.readdirStream](lib/readers.js#L90)

Stream files or directories that match the given glob `pattern`.

**Params**

* `pattern` **{String}**: Glob pattern

* `options` **{Object}**

* `returns` **{Stream}**

**Example**

```js

var glob = require('glob-fs')({ gitignore: true });

glob.readdirStream('*.js')

  .on('data', function (file) {

    console.log(file.path);

  })

  .on('error', console.error)

  .on('end', function () {

    console.log('end');

  });

```

### [Glob](index.js#L42)

Optionally create an instance of `Glob` with the given `options`.

**Params**

* `options` **{Object}**

**Example**

```js

var Glob = require('glob-fs').Glob;

var glob = new Glob();

```

### [.use](index.js#L178)

Add a middleware to be called in the order defined.

**Params**

* `fn` **{Function}**

* `returns` **{Object}**: Returns the `Glob` instance, for chaining.

**Example**

```js

var gitignore = require('glob-fs-gitignore');

var dotfiles = require('glob-fs-dotfiles');

var glob = require('glob-fs')({ foo: true })

  .use(gitignore())

  .use(dotfiles());

var files = glob.readdirSync('**');

```

### [.exclude](index.js#L219)

Thin wrapper around `.use()` for easily excluding files or directories that match the given `pattern`.

**Params**

* `pattern` **{String}**

* `options` **{Object}**

**Example**

```js

var gitignore = require('glob-fs-gitignore');

var dotfiles = require('glob-fs-dotfiles');

var glob = require('glob-fs')()

  .exclude(/\.foo$/)

  .exclude('*.bar')

  .exclude('*.baz');

var files = glob.readdirSync('**');

//=> ['index.js', 'README.md', ...]

```

## Middleware

glob-fs uses middleware to add file matching and exclusion capabilities, or other features that may or may not eventually become core functionality.

**What is a middleware?**

A middleware is a function that "processes" files as they're read from the file system by glob-fs.

Additionally, middleware can:

* be chained

* `include` or `exclude` a file based on some condition, like whether or not one of its properties matches a regex or glob pattern.

* determine whether or not to continue recursing in a specific directory

* modifying an existing property to the `file` object

* add a new property to the `file` object

### Middleware examples

**Ignoring files**

In the following example, `notemp` is a complete and functional middleware for excluding any filepath that has the substring `temp`:

```js

var glob = require('glob-fs')();

function notemp(file) {

  if (/temp/.test(file.path)) {

    file.exclude = true;

  }

  return file;

}

glob.use(notemp)

  .readdirStream('**/*.js')

  .on('data', function(file) {

    console.log(file.relative);

  });

```

**Matching**

Pattern matching is done by default in glob-fs, but you get disable the built-in matchers or get more specific by adding a middleware that uses [micromatch][] or [minimatch](https://github.com/isaacs/minimatch#readme) for matching files.

```js

var glob = require('glob-fs')({ gitignore: true });

var mm = require('micromatch');

glob.use(function(file) {

    if (mm.isMatch(file.relative, 'vendor/**')) file.exclude = true;

    return file;

  })

  .readdirStream('**/*.js')

  .on('data', function(file) {

    console.log(file.relative);

  });

```

**recursion**

Here is how a middleware might determine whether or not to recurse based on a certain pattern:

```js

var glob = require('glob-fs')();

// this specific check is already done by glob-fs, it's just used here as an example 

function recurse(file) {

  // `file.pattern` is an object with a `glob` (string) property

  file.recurse = file.pattern.glob.indexOf('**') !== -1;

  return file;

}

// use the middleware

glob.use(recurse)

  .readdir('**/*.js', function(err, files) {

    console.log(files);

  });

```

**Built-in middleware**

Currently glob-fs includes and runs the following middleware automatically:

* [glob-fs-dotfiles](https://github.com/jonschlinkert/glob-fs-dotfiles): glob-fs middleware for automatically ignoring dotfiles.

* [glob-fs-gitignore](https://github.com/jonschlinkert/glob-fs-gitignore): glob-fs middleware for automatically ignoring files specified in `.gitignore`

**Disabling built-ins**

To disable built-in middleware and prevent them from running, pass `builtins: false` on the global options. This will disable **all built-in middleware**.

Example:

```js

var glob = require('glob-fs')({builtins: false});

```

To disable a specific middleware from running, you can usually pass the name of the middleware on the options, like `dotfiles: false`, but it's best to check the readme of that middleware for specifics.

### Middleware conventions

* **Naming**: any middleware published to npm should be prefixed with `glob-fs-`, as in: `glob-fs-dotfiles`.

* **Keywords**: please add `glob-fs` to the keywords array in package.json

* **Options**: all middleware should return a function that takes an `options` object, as in the [Middleware Example](#middleware-example)

* **Return `file`**: all middleware should return the `file` object after processing.

### Advice for middleware authors

* A middleware should only do one specific thing.

* Multiple middleware libs can be bundled together to create a single middleware.

* Pattern matching should be extremely specific. Don't force downstream middleware to reverse your mistakes.

* As mentioned in the [middleware conventions](#middleware-conventions) section, **always return the `file` object**.

* A single conditional should only set `file.exclude` to `true`, or `file.include` to `true`, never both.

* It's completely okay to check `this.options`

* Middleware modules should be fully documented.

## Globbing examples

Note that the `gitignore` option is already `true` by default, it's just shown here as a placeholder for how options may be defined.

### async

```js

var glob = require('glob-fs')({ gitignore: true });

glob.readdir('**/*.js', function(err, files) {

  console.log(files);

});

```

### promise

```js

var glob = require('glob-fs')({ gitignore: true });

glob.readdirPromise('**/*')

  .then(function (files) {

    console.log(files);

  });

```

### stream

```js

var glob = require('glob-fs')({ gitignore: true });

glob.readdirStream('**/*')

  .on('data', function (file) {

    console.log(file.path);

  })

```

### sync

```js

var glob = require('glob-fs')({ gitignore: true });

var files = glob.readdirSync('**/*.js');

console.log(files);

```

## Events

_(WIP)_

The following events are emitted with all "read" methods:

* `read`: emitted immediately before an iterator calls the first middleware.

* `include`: emits a `file` object when it's matched

* `exclude`: emits a `file` object when it's ignored/excluded

* `file`: emits a `file` object when the iterator pushes it into the results array. Only applies to `sync`, `async` and `promise`.

* `dir`: emits a `file` object when the iterator finds a directory

* `end` when the iterator is finished reading

* `error` on errors

### Event examples

**async**

```js

var glob = require('..')({ gitignore: true });

glob.on('dir', function (file) {

  console.log(file);

});

glob.readdir('**/*.js', function (err, files) {

  if (err) return console.error(err);

  console.log(files.length);

});

```

**promise**

```js

var glob = require('glob-fs')({ gitignore: true });

glob.on('include', function (file) {

  console.log('including:', file.path);

});

glob.on('exclude', function (file) {

  console.log('excluding:', file.path);

});

glob.readdirPromise('**/*');

```

**sync**

Also has an example of a custom event, emitted from a middleware:

```js

var glob = require('glob-fs')({ gitignore: true })

  .use(function (file) {

    if (/\.js$/.test(file.path)) {

      // custom event

      this.emit('js', file);

    }

    return file;

  });

glob.on('js', function (file) {

  console.log('js file:', file.path);

});

glob.on('exclude', function (file) {

  console.log('excluded:', i.excludes++);

});

glob.on('include', function (file) {

  console.log('included:', i.includes++)

});

glob.on('end', function () {

  console.log('total files:', this.files.length);

});

glob.readdirSync('**/*.js');

```

**stream**

```js

var glob = require('glob-fs')({ gitignore: true })

glob.readdirStream('**/*')

  .on('data', function (file) {

    console.log(file.path)

  })

  .on('error', console.error)

  .on('end', function () {

    console.log('end');

  });

```

## FAQ

* when files are read from the file system, an object is created to keep a record of the file's `path`, `dirname`, and fs `stat` object and other pertinent information that makes it easier to make decisions about inclusion and exclusion later on.

* `file` objects are decorated with a `parse` method that is used to calculate the `file.relative` and `file.absolute` properties.

* the `file.parse()` method is called in the iterator, right after the call to `  fs.stats` and just before the call to the middleware handler (`.handle()`). This ensures that all middleware have access to necessary path information.

* `file.relative` is the file path that's actually pushed into the `files` array that is ultimately returned.

* `file.relative` is calculated using `path.relative(file.path, cwd)`, where `cwd` is passed on the options (globally, or on a middleware), and `file.path` is typically the absolute, actual file path to the file being globbed.

## TODO

**middleware**

* [x] middleware

* [x] middleware handler

* [ ] externalize middleware to modules (started, [prs welcome!](#contributing))

**events**

* [x] events

**tests**

* [x] unit tests (need to be moved)

**iterators**

* [x] sync iterator

* [x] async iterator

* [x] stream iterator

* [x] promise iterator

**read methods**

* [x] glob.readdir (async)

* [x] glob.readdirSync

* [x] glob.readdirStream

* [x] glob.readdirPromise

**patterns**

* [ ] Multiple pattern support. will need to change pattern handling, middleware handling. this is POC currently

* [ ] Negation patterns (might not do this, since it can be handled in middleware)

* [ ] matching method, memoized/cached/bound to a glob pattern or patterns, so it can be reused without having to recompile the regex.

**other**

* [ ] clean up `./lib`

* [ ] comparsion to [node-glob][]

## Community middleware

_(Add your project to the [.verb.md](./.verb.md) template do a PR!)_

* [glob-fs-dotfiles](https://github.com/jonschlinkert/glob-fs-dotfiles): glob-fs middleware for automatically ignoring dotfiles.

* [glob-fs-gitignore](https://github.com/jonschlinkert/glob-fs-gitignore): glob-fs middleware for automatically ignoring files specified in `.gitignore`

## Related projects

* [braces](https://github.com/jonschlinkert/braces): Fastest brace expansion for node.js, with the most complete support for the Bash 4.3 braces… [more](https://github.com/jonschlinkert/braces)

* [fill-range](https://github.com/jonschlinkert/fill-range): Fill in a range of numbers or letters, optionally passing an increment or multiplier to… [more](https://github.com/jonschlinkert/fill-range)

* [is-glob](https://github.com/jonschlinkert/is-glob): Returns `true` if the given string looks like a glob pattern.

* [micromatch](https://github.com/jonschlinkert/micromatch): Glob matching for javascript/node.js. A drop-in replacement and faster alternative to minimatch and multimatch. Just… [more](https://github.com/jonschlinkert/micromatch)

## Running tests

Install dev dependencies:

```sh

$ npm i -d && npm test

```

## Contributing

Pull requests and stars are always welcome. For bugs and feature requests, [please create an issue](https://github.com/jonschlinkert/glob-fs/issues/new)

## Author

**Jon Schlinkert**

+ [github/jonschlinkert](https://github.com/jonschlinkert)

+ [twitter/jonschlinkert](http://twitter.com/jonschlinkert)

## License

Copyright © 2015 Jon Schlinkert

Released under the MIT license.

***

_This file was generated by [verb-cli](https://github.com/assemble/verb-cli) on July 11, 2015._
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/micromatch/glob-fs

Awesome Lists containing this project

README
