Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/davideicardi/live-plugin-manager

Plugin manager and installer for Node.JS
https://github.com/davideicardi/live-plugin-manager
extensions-manager node-js npm plugin-manager typescript
Last synced: 2 days ago
JSON representation
Plugin manager and installer for Node.JS
Host: GitHub
URL: https://github.com/davideicardi/live-plugin-manager
Owner: davideicardi
License: mit
Created: 2017-04-04T20:31:10.000Z (over 7 years ago)
Default Branch: master
Last Pushed: 2024-06-11T21:27:58.000Z (6 months ago)
Last Synced: 2024-12-13T09:03:09.602Z (9 days ago)
Topics: extensions-manager, node-js, npm, plugin-manager, typescript
Language: TypeScript
Homepage:
Size: 591 KB
Stars: 242
Watchers: 12
Forks: 44
Open Issues: 38
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project

README

        # live-plugin-manager

[![npm version](https://badge.fury.io/js/live-plugin-manager.svg)](https://www.npmjs.com/package/live-plugin-manager)

[![Build Status](https://travis-ci.org/davideicardi/live-plugin-manager.svg?branch=master)](https://travis-ci.org/davideicardi/live-plugin-manager)

`live-plugin-manager` is a Node.js module that allows you to 

install, uninstall and load any node package at runtime from npm registry.

My main goal is to allow any application to be easily extensible by 

installing and running any node package at runtime, without deployments or server customization. You can basically allow your users to decide what plugin/extension to install.

Main features are:

- Install and run Node packages at runtime (no deployment)

- Install from npm registry (private or public)

- Install from filesystem

- Install from github (branch, commit or tag)

- Update/uninstall packages

- Any Node.js packages can be installed

  - No special configuration is required

- Installed packages can have dependencies 

  - dependencies are automatically installed

  - when updating a dependencies all dependents are reloaded

- Support for concurrency operation on filesystem (cloud/web farm scenario where file system is shared)

  - A filesystem lock is used to prevent multiple instances to work on the same filesystem in the same moment

- Each package run in an semi isolated environment (VM sandbox)

- Set different environment variables for each package

- Implemented in TypeScript

- Fully tested (mocha tests)

There are some **known limitations**, see section at the end.

## Installation

```

npm install live-plugin-manager

```

## Usage

```js

import {PluginManager} from "live-plugin-manager";

const manager = new PluginManager();

async function run() {

  await manager.install("moment");

  const moment = manager.require("moment");

  console.log(moment().format());

  await manager.uninstall("moment");

}

run();  

```

In the above code I install `moment` package at runtime, load and execute it.

Plugins are installed inside the directory specified in the `PluginManager` constructor or in the `plugin_packages` directory if not specified.

Each time your application start you should reinstall any packages that you need; already downloaded packages are not automatically installed, but installation is faster because no new file is downloaded. Typically I suggest to put the list of the installed packages in a database or any other central repository.

Here another more complex scenario where I install `express` with all it's dependencies, just to demonstrate how many possibilities you can have:

```js

import {PluginManager} from "live-plugin-manager";

const manager = new PluginManager();

async function run() {

  await manager.install("express");

  const express = manager.require("express");

  const app = express();

  app.get("/", function(req: any, res: any) {

    res.send("Hello World!");

  });

  const server = app.listen(3000, function() {

  });

}

run();

```

## Load plugins

`live-plugin-manager` doesn't have any special code or convention to load plugins.

When you require a plugin it just load the main file (taken from `package.json`) and execute it, exactly like standard node.js `require`.

Often when working with plugins you need some extension point or convention to actually integrate your plugins inside your host application. Here are some possible solutions:

- [c9/architect](https://github.com/c9/architect)

- [easeway/js-plugins](https://github.com/easeway/js-plugins)

Another solution is to load your plugins inside a Dependency Injection container. 

I'm working also on [shelf-dependency](https://www.npmjs.com/package/shelf-dependency), a simple dependency injection/inversion of control container that can be used to load plugins.

## Samples

- See `./samples` and `./test` directories

- Sample of an extensible web application: [lpm-admin](https://github.com/davideicardi/lpm-admin)

## Reference

### PluginManager.constructor(options?: Partial\)

Create a new instance of `PluginManager`. Takes an optional `options` parameter with the following properties:

- `cwd`: current working directory (default to `process.cwd()`)

- `pluginsPath`: plugins installation directory (default to `.\plugin_packages`, see `cwd`). Directory is created if not exists

- `versionsPath`: directory containing the individual versions of the plugins (default to `./plugin_packages/.versions`). Directory is created if not exists

- `npmRegistryUrl`: npm registry to use (default to https://registry.npmjs.org)

- `npmRegistryConfig`: npm registry configuration see [npm-registry-client config](https://github.com/npm/npm-registry-client)

- `ignoredDependencies`: array of string or RegExp with the list of dependencies to ignore, default to `@types/*`

- `staticDependencies`: object with an optional list of static dependencies that can be used to force a dependencies to be ignored (not installed when a plugin depend on it) and loaded from this list

- `githubAuthentication`: Github authentication configuration (optional). See `installFromGithub` or [Github api authentication](https://github.com/octokit/node-github#authentication) for more info.

- `lockWait`: A number of milliseconds to wait for locks when installing modules to expire before giving up. (default 2 min)

- `lockStale`: A number of milliseconds before installations locks are considered to have expired. (default 3 min)

### pluginManager.install(name: string, version?: string): Promise\

Install the specified package from npm registry or directly from github repository. 

`version` parameter can be a version like `1.0.3` or a github repository in the format `owner/repository_name#ref` (like `expressjs/express#351396f`).

If a version is specified package is downloaded from NPM.

Dependencies are automatically installed (devDependencies are ignored).

### pluginManager.installFromNpm(name: string, version = "latest"): Promise\

Install the specified package from npm registry. Dependencies are automatically installed (devDependencies are ignored).

By default is the package is already available in the download folder then and version is compatible then it is not requested again from npm. Change this behavior by setting `npmInstallMode: "noCache"` options.

To setup authentication for private npm registry use:

```js

const manager = new PluginManager({

  npmRegistryUrl: "http://your-private-registry",

  npmRegistryConfig: {

    auth: {

      token: "your-token"

    }

  }

});

```

### pluginManager.installFromGithub(repository: string): Promise\

Install the specified package from github. `repository` is specified in the format `owner/repository_name#ref` (like `expressjs/express#351396f`).

Dependencies are automatically installed (devDependencies are ignored).

Note: Github has an API rate limit of 60 calls if not authenticated. To authenticate calls just set the `githubAuthentication` property in the options:

```js

manager = new PluginManager({

  githubAuthentication: {

    "type": "basic",

    "username": "YOUR_USER",

    "password": "YOUR_PERSONAL_TOKEN"

  }

});

```

See [Github api authentication](https://github.com/octokit/node-github#authentication) for more info.

### installFromPath(location: string, options: {force: boolean} = {}): Promise\

Install the specified package from a filesystem location. Dependencies are automatically installed from npm. `node_modules` folder is excluded from source location.

### installFromCode(name: string, code: string, version?: string): Promise\

Install a package by specifying code directly. If no version is specified it will be always reinstalled.

### uninstall(name: string): Promise\

Uninstall the package. Dependencies are not uninstalled automatically.

### uninstallAll(): Promise\

Uninstall all installed packages.

### list(): Promise\

Get the list of installed packages.

### require(name: string): any

Load and get the instance of the plugin. Node.js `require` rules are used to load modules.

Calling require multiple times gives always the same instance until plugins changes.

### getInfo(name: string): IPluginInfo | undefined

Get information about an installed package.

### runScript(code: string): any

Run the specified Node.js javascript code with the same context of plugins. Script are executed using `vm` as with each plugin.

### async queryPackage(name: string, version?: string): Promise

Get package/module info from npm registry or github depending of the version format.

### async queryPackageFromNpm(name: string, version = "latest"): Promise

Get package/module info from npm registry.

### async queryPackageFromGithub(repository: string): Promise

Get package/module info from github registry.

## Security

Often is a bad idea for security to allow installation and execution of any node.js package inside your application.

When installing a package it's code is executed with the same permissions of your host application and can potentially damage your entire server.

I suggest usually to allow to install only a limited sets of plugins or only allow trusted administrator to install plugins.

## Sandbox

For advanced scenarios you can customize the sandbox where each plugin run. Sandbox defined NodeJS `global` object.

A typical scenario is to customize the environment variables `global.process.env`. To customize the sandbox you can set a single

sandbox for all plugins using `options.sandbox` in the `PluginManager` constructor or set a different sandbox for each plugin, see

`PluginManager.setSandboxTemplate` and `PluginManager.getSandboxTemplate` functions.

## Under to hood

This project use the following dependencies to do it's job:

- [vm](https://nodejs.org/api/vm.html): compiling and running plugin code within V8 Virtual Machine contexts

- [lockfile](https://github.com/npm/lockfile): file system locking to prevent concurrent operations (see below)

- [tar.gz](https://github.com/alanhoff/node-tar.gz): extract package file

- [fs-extra](https://github.com/jprichardson/node-fs-extra): file system operations

- [debug](https://github.com/visionmedia/debug): debug information

- (removed for now because I have problems with getArchiveLink api, 302 ...) [github](https://www.npmjs.com/package/github)

- (removed for now because for some dependencies issues) [npm-registry-client](https://github.com/npm/npm-registry-client): npm registry handling

While I have tried to mimic the standard Node.js module and package architecture there are some differences.

First of all is the fact that plugins by definition are installed at runtime in contrast with a standard Node.js application where modules are installed before executing the node.js process.

Modules can be loaded one or more times, instead in standard Node.js they are loaded only the first time that you `require` it.

Only one version of a plugin can be installed, also for dependencies, while in Node.js multiple version can be installed (each module can have it's own `node_modules`).

### Locking

A file system locking is implemented using [lockfile](https://github.com/npm/lockfile) library to allow multiple app instances to share the file system. 

This is common in some cloud scenario (for example Azure) where file system is shared between instances.

Locking is used to ensure that only one instance is installing/uninstalling plugins in a given moment.

You can configure `lockWait` inside constructor to configure lock timeout, and `lockStale` to consider a file lock no more valid after that period.

## Git integration

Remember to git ignore the directory where packages are installed, see `pluginsPath`.

Usually you should add this to your `.gitignore` file:

```

plugin_packages

```

## Known limitations

There are some known limitations when installing a package:

- `process.on` has been stubbed, so plugins that use these events may not work as expected. 

- No `pre/post-install` scripts are executed (some packages use this scripts to build assets or for platform specific installation, so for this reason some packages are not supported)

- C/C++ packages (`.node`) are not supported

- Plugin dependencies can be specified only as NPM dependencies (version number) or github dependencies (owner/repo), url or other kind of dependencies are not supported

If you find other problems please open an issue.

## Development

Compile typescript using:

```

npm run src-build

```

Run tests using:

```

npm run test

```

Due to some github rate limits, github related tests can fail if you don't specify a github token. To specify it use:

```

github_auth_username=YOURUSER github_auth_token=YOUR_TOKEN npm test

```

The token must only have public access permission.

## License (MIT)

MIT License

Copyright (c) 2021 Davide Icardi

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all

copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.