Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/nuxt-community/nuxt-generate-cluster
Multi-threaded generator command for nuxt.js
https://github.com/nuxt-community/nuxt-generate-cluster
cluster generator multi-threaded nuxt
Last synced: about 2 months ago
JSON representation
Multi-threaded generator command for nuxt.js
- Host: GitHub
- URL: https://github.com/nuxt-community/nuxt-generate-cluster
- Owner: nuxt-community
- License: mit
- Created: 2017-10-21T18:20:44.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2021-11-05T15:47:56.000Z (about 3 years ago)
- Last Synced: 2024-10-30T00:34:04.198Z (about 2 months ago)
- Topics: cluster, generator, multi-threaded, nuxt
- Language: JavaScript
- Homepage:
- Size: 849 KB
- Stars: 153
- Watchers: 4
- Forks: 17
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Multi-threaded generate command for Nuxt.js
[](https://www.npmjs.com/package/nuxt-generate-cluster)
[](https://www.npmjs.com/package/nuxt-generate-cluster)
> Use multiple workers to generate the static files for your Nuxt.js project
## Setup
Install the package
```
yarn add nuxt-generate-cluster
```
Add a generate script to your `package.json`
```js
"scripts": {
"generate": "nuxt-generate -w 4"
}
```
## Nuxt config options
Configure the generate options in `nuxt.config.js`
```js
generate: {
workers: 4,
workerConcurrency: 500,
concurrency: 500,
routes (callback, params) {
return axios.get('https://api.example.com/routes?since=' + params.lastFinished)
.then((res) => {
return res.data
})
},
done ({ duration, errors, workerInfo }) {
if (errors.length) {
axios.post('https://api.example.com/routes', { generate_errors: errors })
}
}
}
```
### `workers`
- Default: number of processors
The number of workers that should be started. It probably has no use to start more workers then number of processors in your system.
### `workerConcurrency`
- Default: `500`
To even the load between workers they are sent batches of routes to generate, otherwise a worker with 'easy' routes might finish long before others. Workers will also still use the concurrency option from Nuxt.
### `routes`
The default [Nuxt.js routes method](https://nuxtjs.org/api/configuration-generate#routes) has been extended so you can pass additional parameters to it, see params parameter in example config under Setup. By default
it will list 3 timestamps:
- `lastStarted`
The unix timestamp when the nuxt-generate command was last executed, should be just now
- `lastBuilt`
The unix timestamp when the nuxt project was last built by nuxt-generate
- `lastFinished`
The unix timestamp when nuxt-generate last finished succesfully (eg not interrupted by `ctrl+c`)
> Timestamps are locally stored in `~/.data-store/nuxt-generate-cluster.json`, see [data-store](https://github.com/jonschlinkert/data-store)
### `beforeWorkers`
This method is called on the master just before the workers are started/forked. It receives the Nuxt options as first argument.
Use this method if you experience serialization issues or when your Nuxt config is too big. The Nuxt options are stringified and then passed as environment variable to the workers, on Windows there seems to be a maximum size of 32KB for env variables.
Properties which should be safe to remove are (not a complete list):
- buildModules (and their options)
- serverMiddleware
### `done`
This method will be called when all workers are finished, it receives two arguments:
- The first argument is an object with statistics:
- `duration`
The total time in seconds that the command ran
- `errors`
An array of all the errors that were encountered. Errors can have type `handled` or `unhandled`, for the latter the error message will contain the stacktrace
```js
[ { type: 'handled',
route: '/non-existing-link',
error:
{ statusCode: 404,
message: 'The message from your 404 page' } }
]
```
- `workerInfo`
An object with detailed information about each worker. Data passed is from the watchdog object that we use internally to monitor the worker status.
```js
{{ '6707': // the process id of the worker
{ start: [ 1929158, 859524606 ], // process.hrtime the worker was started
duration: 109567, // how long the worker was active
signal: 0, // the signal by which the worker was killed (if any)
code: 0, // the exit status of the worker
routes: 73, // the number of routes generated by this worker
errors: [] }, // the errors this worker encountered, errors of all workers
// combined is the error argument above
}
```
- The second argument is the Nuxt instance
## Command-line options
> Please note that you need to explicitly indicate with `-b` that you want to (re-)build your project
```
$ ./node_modules/.bin/nuxt-generate -h
Multi-threaded generate for nuxt using cluster
Description
Generate a static web application (server-rendered)
Usage
$ nuxt-generate
Options
-b, --build Whether to (re-)build the nuxt project
-c, --config-file Path to Nuxt.js config file (default: nuxt.config.js)
-h, --help Displays this message
-p, --params Extra parameters which should be passed to routes method
(should be a JSON string or queryString)
-q, --quiet Decrease verbosity (repeat to decrease more)
-v, --verbose Increase verbosity (repeat to increase more)
--fail-on-page-error Immediately exit when a page throws an unhandled error
-w, --workers [NUM] How many workers should be started
(default: # cpus)
-wc [NUM], How many routes should be sent to
--worker-concurrency [NUM] a worker per iteration
```
If you need to have more control which routes should be generated, use the `-p` option to pass additional parameters to your routes method.
```
# nuxt.config.js
generate: {
routes (callback, params) {
console.log(params)
}
}
$ nuxt-generate -w 2 -p id=1&id=2
// will print =>
{ id: [ '1', '2' ],
lastStarted: 1508609323,
lastBuilt: 0,
lastFinished: 0 }
```
If you are using a npm script under bash use `--` to pass the parameters to nuxt-generate instead of npm:
```
$ npm run generate -- -p '{ "id": [1,2,3] }'
// will print =>
{ id: [ 1, 2, 3 ],
lastStarted: 1508786574,
lastBuilt: 0,
lastFinished: 0 }
```
## Logging
You can use multiple `-v` or `-q` on the command-line to increase or decrease verbosity.
We use consola for logging and set a default log level of 3 unless DEBUG is set, then its 5.
If you want to log debug messages without setting DEBUG you can use `-vv` on the command-line
The difference between log levels 2, 3 and 4 are:
- `Level 2` (-q)
Only print master messages like worker started / exited. No worker messages.
- `Level 3`
Also print which routes the workers generated
- `Level 4` (-v)
Also print how much time the route generation took and messages between master and workers