Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/polaris64/web_exploit_detector

The Web Exploit Detector is a Node.js application used to detect possible infections, malicious code and suspicious files in web hosting environments
https://github.com/polaris64/web_exploit_detector

cms detection-rules exploits infection nodejs php scanner security-audit suspicious-files web wordpress wso-webshell

Last synced: 3 months ago
JSON representation

The Web Exploit Detector is a Node.js application used to detect possible infections, malicious code and suspicious files in web hosting environments

Awesome Lists containing this project

README 

        
# Web Exploit Detector



![Web Exploit Detector logo](https://www.polaris64.net/i/blogs/online-security/web-exploit-detector.png?w=250&h=)



## Introduction

The Web Exploit Detector is a Node.js application (and NPM module) used to detect possible infections, malicious code and suspicious files in web hosting environments. This application is intended to be run on web servers hosting one or more websites. Running the application will generate a list of files that are potentially infected together with a description of the infection and references to online resources relating to it.



As of version 1.1.0 the application also includes utilities to generate and compare snapshots of a directory structure, allowing users to see if any files have been modified, added or removed.



The application is hosted here on GitHub so that others can benefit from it, as well as allowing others to contribute their own detection rules.



## Links



 - My website: [https://www.polaris64.net/](https://www.polaris64.net/)

 - My cybersecurity blog, which contains articles describing some of these exploits and how to remove them: [https://www.polaris64.net/blog/cyber-security](https://www.polaris64.net/blog/cyber-security)

 - [Contact me](https://www.polaris64.net/contact_me.php)

 - [NPM module](https://www.npmjs.com/package/web_exploit_detector)



## Installation



### Regular users

The simplest way to install Web Exploit Detector is as a global NPM module: -



`npm install -g web_exploit_detector`



If you are running Linux or another Unix-based OS you might need to run this command as root (e.g. `sudo npm install -g web_exploit_detector`).



### Updating

The module should be updated regularly to make sure that all of the latest detection rules are present. Running the above command will always download the latest stable (tested) version. To update a version that has already been installed, simply run the following: -



`npm update -g web_exploit_detector`



Again, you may have to use the `sudo` command as above.



### Technical users

You can also clone the Git repository and run the script directly like so: -



 1. `git clone https://github.com/polaris64/web_exploit_detector`

 2. `cd web_exploit_detector`

 3. `npm install`



## Running



### From NPM module

If you have installed Web Exploit Detector as an NPM module (see above) then running the scanner is as simple as running the following command, passing in the path to your webroot (location of your website files): -



`wed-scanner --webroot=/var/www/html`



Other command-line options are available, simply run `wed-scanner --help` to see a help message describing them.



Running the script in this way will produce human-readable output to the console. This is very useful when running the script with `cron` for example as the output can be sent as an e-mail whenever the script runs.



The script also supports the writing of results to a more computer-friendly JSON format for later processing. To enable this output, see the `--output` command line argument.



### From cloned Git repository

Simply call the script via node and pass the path to your webroot as follows: -



`node index.js --webroot=/var/www/html`



## Recursive directory snapshots

The Web Exploit Detector also comes with two utilities to help to identify files that might have changed unexpectedly. A successful attack on a site usually involves deleting files, adding new files or changing existing files in some way.



### Snapshots

A snapshot (as used by these utilities) is a JSON file which lists all files as well as a description of their contents at the point in which the snapshot was created. If a snapshot was generated on Monday, for example, and then the site was attacked on Tuesday, then running a comparison between this snapshot and the current site files afterwards will show that one or more files were added, deleted or changed. The goal of these utilities therefore is to allow these snapshots to be created and for the comparisons to be performed when required.



The snapshot stores each file path together with a SHA-256 hash of the file contents. A hash, or digest, is a small summary of a message, which in this case is the file's contents. If the file contents change, even in a very small way, the hash will become completely different. This provides a good way of detecting any changes to file contents.



### Usage

The following two utilities are also installed as part of Web Exploit Detector: -



 - `wed-generate-snapshot`: this utility allows a snapshot to be generated for all files (recursively) in a directory specified by "--webroot". The snapshot will be saved to a file specified in the "--output" option.

 - `wed-compare-snapshot`: once a snapshot has been generated it can be compared against the current contents of the same directory. The snapshot to check is specified using the "--snapshot" option. The base directory to check against is stored within the snapshot, but if the base directory has changed since the snapshot was generated then the --webroot option can be used.



### Workflow

Snapshots can be generated as frequently as required, but as a general rule of thumb they should be generated whenever a site is in a clean (non-infected) state and whenever a legitimate change has been made. For CMS-based sites like WordPress, snapshots should be created regularly as new uploads will cause the new state to change from the stored snapshot. For sites whose files should never change, a single snapshot can be generated and then used indefinitely ensure nothing actually does change.



## Usage as a module

The `src/web-exploit-detector.js` script is an ES6 module that exports the set of rules as `rules` as well as a number of functions: -



 - `executeTests(settings)`: runs the exploit checker based on the passed `settings` object. For usage, please consult the `index.js` script.

 - `formatResult(result)`: takes a single test `result` from the array returned from `executeTests()` and generates a string of results ready for output for that test.

 - `getFileList(path)`: returns an array of files from the base `path` using `readDirRecursive()`.

 - `processRulesOnFile(file, rules)`: processes all rules from the array `rules` on a single `file` (string path).

 - `readDirRecursive(path)`: recursive function which returns a Promise which will be resolved with an array of all files in `path` and sub-directories.



The `src/cli.js` script is a simple command-line interface (CLI) to this module as used by the `wed-scanner` script, so reading this script shows one way in which this module can be used.



The project uses Babel to compile the ES6 modules in "src" to plain JavaScript modules in "lib". If you are running an older version of Node.js then modules can be `require()'d` from the "lib" directory instead.



### Building

The package contains Babel as a dev-dependency and the "build" and "watch:build" scripts. When running the "build" script (`npm run build`), the ES6 modules in "./src" will be compiled and saved to "./lib", where they are included by the CLI scripts.



The "./lib" directory is included in the repository so that any user can clone the repository and run the application directly without having to install dev-dependencies and build the application.



## Excluding results per rule

Sometimes rules, especially those tagged with `suspicion`, will identify a clean file as a potential exploit. Because of this, a system to allow files to be excluded from being checked for a rule is also included.



The `wed-results-to-exceptions` script takes an output file from the main detector script (see the `--output` option) and gives you the choice to exclude each file in turn for each specific rule. All excluded files are stored in a file called `wed-exceptions.json` (in the user's home directory) which is read by the main script before running the scan. If a file is listed in this file then all attached rules (by ID) will be skipped when checking this file.



For usage instructions, simply run `wed-results-to-exceptions`. You will need to have a valid output JSON from a previous run of the main detector first using the `--output` option.



For users working directly with the Git repository, run `node results_to_exceptions.js` in the project root directory.



## Rule engine

The application operates using a collection of "rules" which are loaded when the application is run. Each rule consists of an ID, name, description, list of URLs, tags, deprecation flag and most importantly a set of tests.



Each individual test must be one of the following: -



 - A regular expression: the simplest type of test, any value matching the regex will pass the test.

 - A Boolean callback: the callback function must return a Boolean value indicating if the value passes the test. The callback is free to perform any synchronous operations.

 - A Promise callback: the callback function must return a Promise which is resolved with a Boolean value indicating if the value passes the test. This type of callback is free to perform any asynchronous operations.



The following test types are supported: -



 - "path": used to check the file path. This test must exist and should evaluate to true if the file path is considered to match the rule.

 - "content": used to check the contents of a file. This test is optional and file contents will only be read and sent to rules that implement this test type. When this test is a function, the content (string) will be passed as the first argument and the file path will be passed as the second argument, allowing the test to perform additional file operations.



## Expanding on the rules

As web-based exploits are constantly evolving and new exploits are being created, the set of rules need to be updated too. As I host a number of websites I am constantly observing new kinds of exploits, so I will be adding to the set of rules whenever I can. I run this tool on my own servers, so of course I want it to be as functional as possible!



This brings me onto the reasons why I have made this application available as an open-source project: firstly so that you and others can benefit from it and secondly so that we can all collaborate to contribute detection rules so that the application is always up to date.



### Contributing rules

If you have discovered an exploit that is not detected by this tool then please either contact me to let me know or even better, write your own rule and add it to the third-party rule-set (rules/third-party/index.js), then send me a pull request.



Don't worry if you don't know how to write your own rules; the most important thing is that the rule gets added, so feel free to send me as much information as you can about the exploit and I will try to create my own rule for it.



Rules are categorised, but the simplest way to add your own rule is to add it to the third-party rule-set mentioned above. Rule IDs are written in the following format: "author:type:sub-type(s):rule-id". For example, one of my own rules is "P64:php:cms:wordpress:wso\_webshell". "P64" is me (the author), "php:cms:wordpress" is the grouping (a PHP-specific rule, for the Content Management System (CMS) called WordPress) and "wso\_webshell" is the specific rule ID. When writing your own rules, try to follow this format, and replace "P64" with your own GitHub username or other unique ID.



## Unit tests and linting

The project contains a set of Jasmine tests which can be run using `npm test`. It also contains an ESLint configuration, and ESLint can be run using `npm run lint`.



When developing, tests can also be run whenever a source file changes by running `npm run watch:test`. To run tests and ESLint, the `npm run watch:all` script can be used.



Please note that unless you already have Jasmine and/or nodemon installed, you should run `npm install` in non-production mode to ensure that the dev-dependencies are installed.



## Credits

Thanks to the Reddit user mayupvoterandomly for suggesting the directory snapshot functionality that was added in 1.1.0 and for suggesting new rules that will be added soon.



## License

ISC License



Copyright (c) 2017, Simon Pugnet



Permission to use, copy, modify, and/or distribute this software for any

purpose with or without fee is hereby granted, provided that the above

copyright notice and this permission notice appear in all copies.



THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH

REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY

AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,

INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM

LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE

OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR

PERFORMANCE OF THIS SOFTWARE.