Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/plotdb/registry


https://github.com/plotdb/registry

Last synced: about 2 months ago
JSON representation

Awesome Lists containing this project

README 

        
# @plotdb/registry



`@plotdb/registry` is a local package cache service for dynamic package loading. Similar to CDN but it's meant to provide package access control directly via the backend access point.



Basic idea:



 - reverse proxy (nginx) check if a file exists. return it directly if found.

 - if the file is not found, request is passed to registry backend.

 - registry backend looks up the given package based on the file's path, and download the package if found.

 - registry backend instruct nginx to redirect for the downloaded file.



You can config the backend router to allow only certain packages to be downloaded, or provide additional information to download private packages.



`@plotdb/registry` provides:



 - a package provider interface for fetching, gatekeeping and chaining other package providers.

 - an express router for accepting fetching requests from users.

 - a nginx config generator for accessing requested files based on availability of file.



## Usage



`@plotdb/registry` is expected to be used along with expressjs and nginx.



First, include `@plotdb/registry`:



    registry = require("@plotdb/registry")



prepare a provider:



    # our own provider, only fetch modules in @plotdb scope

    myprovider = new registry do

      check: ({name, version, path}) ->

        if /@plotdb/.exec(name) => return Promise.resolve!

        return lderror.reject 403

    # chain a default jsdelivr provider

    myprovider.chain(registry.provider.jsdelivr);



add a route:



    app.get("/mylib/*", registry.route({

      provide: myprovider

      root:

        pub: "/lib"               # root path in URL users access.

        fs: "/var/lib/cdn/cache"  # root folder for `@plotdb/registry` to store file.

        internal: "/ilib"         # internal redirect URL from server to nginx.

    });



generate customied nginx config:



    npx registry-nginx -c myconfig.yaml > reg.ngx



include the generated file in the server block of your main nginx config:



    server {

      include reg.ngx

    }



where the config file ( `myconfig.yaml` above ) should contains following fields:



 - `internal`: internal path for `X-Accel-Redirect`. should be a dummy path which is not used.

 - `pub`: `pub` in the `root` parameter in `registry.route` call.

 - `fs`: `fs` in the `root` parameter in `registry.route` call.

 - `upstream`: upstream url for your backend server defined in your nginx config file. e.g., `backend_api`.

 - `cache`: with two subfields:

   - `name`: cache name.

     - for caching to work, you should manually add a cache path directive using the name configed here, e.g.,



           proxy_cache_path /tmp/cache keys_zone=mycache:10m levels=1:2 inactive=600s max_size=100m; 



   - `period`: how long the nginx proxy cache should last. e.g., `10m`.



Check `sample/config.ngx` and `sample/config.yaml` for a reference of nginx config file and corresponding config yaml.



## Registry Provider Specification



Registry providers are used to access a requested resource which is defined by its `namespace`, `name`, `version`, `path` and optionally `type`.



A registry provider can be either following format:

 - object: described below

 - (TBD) string: indicate a root path for a requested resource

 - (TBD) function: return an URL for a requested resource when called



A registry provider object should contain following fields:



 - `name`: provider name

 - `check({name, version})`: access control for the given package `{name, version}`.

   - when omitted, no check will be done.

   - return:

     - if the given package is not allowed:

       - a Promise rejects with `{id: 403, name: 'lderror'}` (or, `lderror.reject(403)`)

     - otherwise:

       - a Promise resolves with nothing.



 - `fetchRealVersion(opt)`: fetch version and tarball informations of a given package.

   - return a promise resolves with an object.

     - resolved object should contains following fields:

       - `version`: actual version for the given package information.

       - `url`: tarball url

   - `opt` is an object with following fields:

     - `root`: root directory of registry cache. 

     - `path`: an object with following fields:

       - `base`: an object with following fields:

         - `pkg`: package root directory ( exclude version name )

         - `version`: package directory of specific version.

       - `version`: path of the internal file storing return object of `fetchRealVersion` call.

       - `404`: path of the internal file that if it exists then the given version of this package is not found.

     - `name`: package name.

     - `version`: package version. should be semver. could be version range, `latest` or `main`.

     - `cachetime`: expected cache time in seconds. default 3600 if omitted.`

     - `versionType`: either `latest`, `specifc` or `range`.

     - `force`: when true, should always try to fetch, ignoring cache or current status.

 - `fetchBundleFile(opt)`: fetch the given package and ectract it.

   - return a promise resolves when bundle file is extracted completely.

   - check `fetchRealVersion` for the definition of `opt`.



A provider provides following APIs:



 - `opt(opt)`: provide additional options for this provider.

   - once provided, this can be accessed via `this._opt` internally.

 - `fetch(opt)`: download the indicated released packages and create a local copy at specified location.

   - `opt` is an object with following fields:

      - `root`: root directory for keeping cached files.

      - `name`: package name. scope is possible, such as `@plotdb/block`.

      - `version`: package version. should always in semver format (e.g., `1.0.0`) or one of `latest` or `main`.

        - in `@plotdb/block` `main` is the locked version, however there is no locked version defined here,

          so `main` is equivalent to `latest`.

        - in github, tags is used for fetching release. tags should be in format `vx.y.z`. e.g., `v1.0.0`.

      - `force`: default false. when true, ignore cache / 404 status and always try fetching package again.

      - `cachetime`: default 3600 seconds. cache for how long (in seconds) since the last fetch attempts.

    - return value: a Promise, resolves if package is found and downloaded. reject `e` in following situation:

      - lderror.id(e) is not 0, but following:

        - 404: package not found.

        - 998: package either found or not found. result cached so no fetch is performed.

          - by our design, router send a `X-Accel-Redirect` to nginx if 998.

            if it's actually 404, nginx will then look up the file and report 404 after found not found.

          - the result will then cached by nginx and won't hit registry backend until cache expires.

      - otherwise, it's an internal exception and should be logged and tracked.

 - `check({name, version})`: call the `check()` function provided in constructor.

 - `chain(providers)`: chain given `providers` in this provider.

   - `providers`: either another provider, or a list of other providers.

   - chained providers will be called if current provider return 404.



## License



MIT License