https://github.com/faust64/keengreeper
Keep NodeJS modules up-to-date
https://github.com/faust64/keengreeper
cronjob greenkeeper-shrinkwrap nodejs-development update-script
Last synced: 8 months ago
JSON representation
Keep NodeJS modules up-to-date
- Host: GitHub
- URL: https://github.com/faust64/keengreeper
- Owner: faust64
- License: bsd-2-clause
- Created: 2017-05-07T00:22:24.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2018-04-21T11:26:11.000Z (about 8 years ago)
- Last Synced: 2025-02-17T15:51:48.669Z (over 1 year ago)
- Topics: cronjob, greenkeeper-shrinkwrap, nodejs-development, update-script
- Language: Shell
- Homepage:
- Size: 61.5 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Keengreeper
Keep git-hosted NodeJS projects up-to-date - shrinkwrap-capable.
AKA: GreenKeeper replacement. Following on the forced migration to GreenKeeper2,
and the late annouce that we would now be charged: here is a rudimentary drop-in
replacement, that actually handles shrinkwrap, and may also be used raising
alerts based on Snyk security advisories.
## Dependencies
* bash (installing & running NVM)
* curl (querying NPMJS & Snyk, installing NVM, querying CircleCI, notifying
slack)
* git, runtime user should have username & email configured
* ssh key pair, setting up passwordless access to your repositories
## Introducing KeenGreeper
The `configure` script would allow you to create and manage a
leightweight database listing the repositories we would track, a cache of the
NodeJS dependencies involved and their last known versions, as well as a list of
modules whose updates we should ignore.
Running this pseudo-shell, type `h` or `help` to display the help menu.
The `updateModules` script should then iterate on previously registered
repositories, making sure they're working on the staging branch (if such exists,
master otherwise), pulling the lastest changes from git upstream, looking for
dependencies listed in the corresponding package.json, checking for known
updates. Whenever some new versions are detected, a branch is created,
your package.json gets updated. If a `node_shrinkwrap` folder exists, its
content would be updated as well. Eventually these new branches are pushed to
upstream for further verification (hopefully: CI).
Then, the `updateVulnerabilities` script may be used to iterate on previously
cached NodeJS modules, looking for potential vulnerabilities.
The `expireCaches` drops references to dependencies from your NodeJS modules &
Snyk caches, when these are no longer relevant to your repositories.
The `checkVulnerabilities` checks your dependencies against Snyk advisories,
and eventually sends Slack and/or mail notifications.
The `mergeBranches` checks for previously-pushed patch branches, queries
CircleCI and may merge successfully-tested patches, eventually cleaning up
previously-pushed branches.
## Setup
We first need to set some variables. Look at `env.sample` for an example, you
would have to define:
* `DBDIR`, the folder we would keep our internal configurations & caches into
* `CACHETTL`, the amount of seconds a record from npmjs.org should be kept
* `CIRCLE_TOKEN`, optional token querying CircleCI for tests results, assuming
some of your repositories could be private
* `LOGDIR`, the folder we would store npm install and shrinkwrap logs into
* `MAIL_RCPT`, the recipients of mail alerts upon matching a vulnerable modules
* `SLACK_HOOK_URL`, upon matching vulnerable modules, send Slack notification
* `SNYKTTL`, the amount of seconds a record from Snyk should be kept
* `TMPDIR`, the folder we will be preparing our new commits into
* `WORKDIR`, the folder we would keep cloned repositories into
Having set the proper values, make sure to install that file as
`/etc/keengreeper.conf`, to have it loaded automatically - otherwise, make sure
to source that environment file before running any of the enclosed scripts. Also
make sure the directories involved do exist and are writeable by the user that
will run our scripts.
Then, we can start registering repositories:
```
git/keengreeper$ ./configure
WARNING: nodejs-update-db absent, creating empty dataset
<$ type h or help for usage, q or quit to exit
$> a PeerioTechnologies/peerio-inferno
Cloning into 'peerio-inferno'...
remote: Counting objects: 3806, done.
remote: Compressing objects: 100% (72/72), done.
remote: Total 3806 (delta 18), reused 0 (delta 0), pack-reused 3734
Receiving objects: 100% (3806/3806), 56.82 MiB | 11.56 MiB/s, done.
Resolving deltas: 100% (1758/1758), done.
Checking connectivity... done.
add git@mygitlab.example.com:DevOps/awesome-project
Cloning into 'peerio-shark'...
remote: Counting objects: 6732, done.
remote: Total 6732 (delta 0), reused 0 (delta 0), pack-reused 6731
Receiving objects: 100% (6732/6732), 66.28 MiB | 11.15 MiB/s, done.
Resolving deltas: 100% (3696/3696), done.
Checking connectivity... done.
quit
```
Note that we would be assuming the user running these scripts can already
passwordlessly clone and write to the repositories you would `add`. Using an SSH
key is recommended. Running these scripts from some un-attended station:
creating a separate git user with limited privileges is encourraged.
You may configure a couple modules that shouldn't be taken into consideration
by our update process, whatever the reasons:
```
git/keengreeper$ ./configure
$> i mailparser
ign basho-riak-client
ignore sinon
di mailparser
li
basho-riak-client
sinon
$> quit
```
Finally, you can use the main script updating your projects dependencies:
```
git/keengreeper$ ./updateModules
refreshing npmjs local cache
processing peerio-server
skipping peerio-server::basho-riak-client - dependency registered to exclude file
skipping peerio-server::sinon - dependency registered to exclude file
processing desktop-update-proxy
staging q@desktop-update-proxy 1.4.1 -> 1.5.0
building staged changes for desktop-update-proxy
[chaos-q1.5.0 5c6edf6] chore(package): bump q to 1.5.0
1 file changed, 1 insertion(+), 1 deletion(-)
done
```
Being sure our magic works, you may consider setting up some cron job:
```
(
crontab -l
echo "30 */2 * * * `pwd`/updateModules >>log/cron.log 2>&1"
echo "45 */6 * * * `pwd`/updateVulnerabilities >>log/snyk-cron.log 2>&1"
echo "15 0 * * * `pwd`/expireCaches >>log/cache-cron.log 2>&1"
echo " 0 22 * * * `pwd`/checkVulnerabilities >>log/vuln-cron.log 2>&1"
echo "20 */2 * * * `pwd`/mergeBranches >>log/merge-cron.log 2>&1"
) | crontab -
```