https://github.com/alertot/detectem

detectem - detect software and its version on websites.
https://github.com/alertot/detectem

detection detector python security

Last synced: 6 months ago
JSON representation

detectem - detect software and its version on websites.

• Host: GitHub
• URL: https://github.com/alertot/detectem
• Owner: alertot
• License: mit
• Created: 2016-11-15T18:27:09.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2021-03-25T23:12:55.000Z (about 5 years ago)
• Last Synced: 2025-12-09T11:05:14.132Z (7 months ago)
• Topics: detection, detector, python, security
• Language: Julia
• Homepage:
• Size: 1.3 MB
• Stars: 157
• Watchers: 4
• Forks: 32
• Open Issues: 2
• Metadata Files:
  • Readme: README.rst
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
detectem

========

**Python 3.6 or greater is required.**

.. image:: https://img.shields.io/pypi/v/detectem.svg

        :target: https://pypi.python.org/pypi/detectem

.. image:: https://img.shields.io/travis/alertot/detectem.svg

        :target: https://travis-ci.org/alertot/detectem

detectem is a specialized software detector.

Let's see it in action.

.. code-block:: bash

  $ det http://domain.tld

  [{'name': 'phusion-passenger', 'version': '4.0.10'},

   {'name': 'apache-mod_bwlimited', 'version': '1.4'},

   {'name': 'apache-mod_fcgid', 'version': '2.3.9'},

   {'name': 'jquery', 'version': '1.11.3'},

   {'name': 'crayon-syntax-highlighter', 'version': '2.7.2_beta'}]

Using a serie of indicators, it's able to detect software running on a site

and extract accurately its version information.

It uses Splash_ API

to render the website and start the detection routine.

It does full analysis on requests, responses and even on the DOM_!

There are two important articles to read:

* `Reasons to create detectem `_

* `Introduction to detectem `_

Features

--------

* Detect software in modern web technologies.

* Browser support provided by Splash_.

* Analysis on requests made and responses received by the browser.

* Get software information from the DOM.

* Great performance (less than 10 seconds to get a fingerprint).

* Plugin system to add new software easily.

* Test suite to ensure plugin result integrity.

* Continuous development to support new features.

Installation

------------

1. Install Docker_ and add your user to the docker group, then you avoid to use sudo.

2. Pull the image::

    $ docker pull scrapinghub/splash

3. Create a virtual environment with Python >= 3.6 .

4. Install detectem::

    $ pip install detectem

5. Run it against some URL::

    $ det http://domain.tld

Other installation method

-------------------------

detectem as Docker Container

------------------------------

Let's see it in action.

.. code-block:: bash

  $ docker-compose run --rm detectem http://domain.tld

  [{'name': 'phusion-passenger', 'version': '4.0.10'},

   {'name': 'apache-mod_bwlimited', 'version': '1.4'},

   {'name': 'apache-mod_fcgid', 'version': '2.3.9'},

   {'name': 'jquery', 'version': '1.11.3'},

   {'name': 'crayon-syntax-highlighter', 'version': '2.7.2_beta'}]

But first that all we must do:

Installation

------------

1. Install the last `Docker CE Stable version`_.

2. Add your user to the docker group and logout::

    $ sudo usermod -aG docker $USER

3. Make sure you have logout to apply changes, then log in again.

4. Install `Docker Compose`_

5. Download to your workspace the docker-compose building files.

    `Dockerfile-alternate`_

    `docker-compose.yml`_

6. Build the required docker images for detectem at the same directory as the

   previous point::

    $ docker-compose up -d

7. Run detectem against some URL::

    $ docker-compose run --rm detectem http://domain.tld

Documentation

-------------

The documentation is at `ReadTheDocs `_.

.. _Docker: http://docker.io

.. _Splash: https://github.com/scrapinghub/splash

.. _DOM: https://en.wikipedia.org/wiki/Document_Object_Model

.. _`Docker CE Stable version`: https://www.docker.com/community-edition

.. _`Docker compose`: https://docs.docker.com/compose/install/

.. _Dockerfile-alternate: extras/docker/Dockerfile-alternate

.. _docker-compose.yml: extras/docker/docker-compose.yml