https://github.com/rfc-st/humble
A humble, and 𝗳𝗮𝘀𝘁, security-oriented HTTP headers analyzer.
https://github.com/rfc-st/humble
analysis checklist cybersecurity header-parser headers http infosec kali-linux owasp python3 security security-audit security-scanner security-tools
Last synced: 3 months ago
JSON representation
A humble, and 𝗳𝗮𝘀𝘁, security-oriented HTTP headers analyzer.
- Host: GitHub
- URL: https://github.com/rfc-st/humble
- Owner: rfc-st
- License: mit
- Created: 2020-06-12T08:25:50.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-10-23T21:56:36.000Z (12 months ago)
- Last Synced: 2024-10-25T12:12:45.318Z (12 months ago)
- Topics: analysis, checklist, cybersecurity, header-parser, headers, http, infosec, kali-linux, owasp, python3, security, security-audit, security-scanner, security-tools
- Language: Python
- Homepage: https://github.com/rfc-st/humble
- Size: 16.7 MB
- Stars: 257
- Watchers: 5
- Forks: 20
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
- awesome-hacking-lists - rfc-st/humble - A humble, and 𝗳𝗮𝘀𝘁, security-oriented HTTP headers analyzer. (Python)
README
humble
A humble, and fast, security-oriented HTTP headers analyzer
"千里之行，始於足下 - 老子"
("A journey of a thousand miles begins with a single step. - Lao Tzu")
"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"
### Table of contents
[Features](#features)
[Screenshots](#screenshots)
[Installation & Update (Source code)](#installation--update-source-code)
[Installation & Maintenance (Docker)](#installation--maintenance-docker)
[Installation & Update (Kali Linux)](#installation--update-kali-linux)
[Usage](#usage)
[Advanced Usage (Linux)](#advanced-usage-linux)
[Checks: Missing Headers](#checks-missing-headers)
[Checks: Fingerprint Headers](#checks-fingerprint-headers)
[Checks: Deprecated Headers and Insecure Values](#checks-deprecated-headersprotocols-and-insecure-values)
[Checks: Empty Values](#checks-empty-values)
[Guidelines included](#guidelines-included-to-enable-security-http-headers)
[To-Do](#to-do)
[Further Reading](#further-reading)
[Contribute](#contribute)
[Acknowledgements](#acknowledgements)
[License](#license)
## Features
:heavy_check_mark: 58 [checks](#checks-enabled-headers) for enabled security-related HTTP response headers.
:heavy_check_mark: 14 [checks](#checks-missing-headers) for missing security-related HTTP response headers (the ones I consider essential).
:heavy_check_mark: 1234 [checks](#checks-fingerprint-headers) for fingerprinting through HTTP response headers.
:heavy_check_mark: 152 [checks](#checks-deprecated-headersprotocols-and-insecure-values) for deprecated HTTP response headers/protocols or with insecure/wrong values.
:heavy_check_mark: 28 [checks](https://github.com/rfc-st/humble/blob/master/additional/insecure.txt#L46-L73) related to Content Security Policy [Level 3](https://www.w3.org/TR/CSP3/).
:heavy_check_mark: Can check for compliance with the OWASP Secure Headers Project Best Practices.
:heavy_check_mark: Can exclude specific HTTP response headers from the analysis.
:heavy_check_mark: Can analyze _raw response files_: text files with HTTP response headers and values. Ex: curl option '--dump-header'.
:heavy_check_mark: Can export each analysis to CSV, HTML5, JSON, PDF 1.4, TXT and XML (and in a filename and path of your choice).
:heavy_check_mark: Can check for outdated SSL/TLS protocols and vulnerabilities: requires the **amazing** https://testssl.sh/.
:heavy_check_mark: Can provide brief and detailed analysis along with HTTP response headers.
:heavy_check_mark: Includes browser support references for enabled HTTP security headers: provided by https://caniuse.com/.
:heavy_check_mark: Highlights experimental headers in each analysis.
:heavy_check_mark: Each detailed analysis may include up to dozens of official links, references and technical articles.
:heavy_check_mark: Can display analysis, messages, and most errors in either English or Spanish.
:heavy_check_mark: Saves each analysis and highlights improvements or deficiencies compared to the previous one.
:heavy_check_mark: Can display analysis statistics for a specific URL or across all of them.
:heavy_check_mark: Can display fingerprint statistics for a specific term or the Top 20.
:heavy_check_mark: Can display guidelines for enabling security HTTP response headers on popular frameworks, servers, and services.
:heavy_check_mark: Code reviewed via Bandit, Flake8, pyinstrument, SonarQube for IDE and Sourcery.
:heavy_check_mark: Tested, one by one, on thousands of URLs.
:heavy_check_mark: Tested on Docker 26.1, Kali Linux 2021.1, macOS 14.2.1 and Windows 10 20H2.
:heavy_check_mark: Almost all the code available under one of the most permissive licenses: MIT.
:heavy_check_mark: Regularly updated.
:heavy_check_mark: Minimal dependencies required.
:heavy_check_mark: Developed entirely in my spare time, no strings attached: feel free to try it out and integrate it into your projects!.
:heavy_check_mark: And with the approval of several AI :smile:!.
## Screenshots
.: (Windows) - Brief analysis.
.: (Linux) - Brief analysis along with HTTP response headers.
.: (Linux) - Detailed analysis, in Spanish.
.: (Linux) - Analysis of a "raw response file". Example.
.: (Linux) - SSL/TLS checks.
```bash
Options used: -f -g -p -U -s --hints
```
.: (Linux) - Compliance with OWASP 'Secure Headers Project' best practices.
.: (Linux) - List of HTTP fingerprint headers based on a specific term.
.: (Linux) - Brief analysis saved as CSV. Example.
.: (Windows) - Detailed analysis saved as PDF. Example.
.: (Linux) - Detailed analysis saved as HTML. Example.
.: (Linux) - Brief analysis saved as JSON. Example.
.: (Linux) - Brief analysis saved as XML. Example.
.: (Linux) - Analysis history file: Date, URL, Enabled, Missing, Fingerprint, Deprecated/Insecure, Empty headers & Total warnings (the four previous totals).
.: (Linux) - Statistics of the analysis performed against a specific URL.
.: (Linux) - Statistics of the analysis performed against all URLs, in Spanish.
.: (Windows) - Checking for updates
## Installation & update (Source code)
> [!NOTE]
> Python 3.11 or higher is required.
```bash
# Install python3 and python3-pip:
# (Windows) https://www.python.org/downloads/windows/
# (Linux) if not available, install them: e.g. Synaptic, apt, dnf, yum ...
# (macOS) https://www.python.org/downloads/macos/
# Install Git:
# (Windows) https://git-scm.com/download/win
# (Linux) https://git-scm.com/download/linux
# (macOS) https://git-scm.com/download/mac
# Set up a virtual environment (pending how to do it in Windows), download 'humble' and its dependencies
# '/home/bluesman/humble_venv' is a example path for the virtual environment
$ python3 -m venv /home/bluesman/humble_venv
$ source /home/bluesman/humble_venv/bin/activate
$ cd /home/bluesman/humble_venv/
$ git clone https://github.com/rfc-st/humble.git
$ cd humble
$ pip3 install -r requirements.txt
# Analyze! :). Linux and Windows examples
$ python3 humble.py -u https://google.com
$ py humble.py -u https://google.com
# Good practice: deactivate the virtual environment after you have finished using 'humble'
$ deactivate
# Activate the virtual environment to analyze URLs again with 'humble'
$ cd /home/bluesman/humble_venv/
$ source /home/bluesman/humble_venv/bin/activate
$ cd humble
# Updating (weekly): activate the virtual environment and from 'humble' folder
$ git pull
# Updating (Release): activate the virtual environment, download the latest source code file
# and decompress it in the 'humble' folder, overwriting files.
https://github.com/rfc-st/humble/releases
```
## Installation & maintenance (Docker)
> [!NOTE]
> Python 3.11 will be used to [build](https://github.com/rfc-st/humble/blob/master/Dockerfile) the image.
```bash
# Install Docker, and make sure it's running:
# E.g. (Linux): https://www.kali.org/docs/containers/installing-docker-on-kali/
# E.g. (macOs): https://docs.docker.com/desktop/install/mac-install/
# E.g. (Windows): https://docs.docker.com/desktop/install/windows-install/
# Clone the repository *or* download & decompress the latest release:
$ git clone https://github.com/rfc-st/humble.git
https://github.com/rfc-st/humble/releases
# Build the image inside the 'humble' folder: providing the TAG as the latest Release of 'humble' (e.g. 1.48).
# https://github.com/rfc-st/humble/releases (Windows may require elevated console privileges)
$ docker build -t humble:1.48 .
# Run the analysis specifying the above TAG, along with the specific options for 'humble':
# '-it', required: allocate a pseudo-TTY and keep the input interactive.
# '-rm', required: automatically remove the container and associated anonymous volumes when it exits.
# (Linux/macOS)
# E.g. Analyze https://facebook with a brief analysis:
$ docker run -it --rm --name humble humble:1.48 /bin/bash -c "python3 humble.py -u https://facebook.com -b"
# (Windows)
# E.g. Analyze https://facebook with a brief analysis:
$ docker run -it --rm --name humble humble:1.48 python3 humble.py -u https://facebook.com -b
# Removing (and untagging) previous images of 'humble' after upgrading to the latest release.
$ docker rmi humble:1.48
```
## Installation & update (Kali Linux)
> [!NOTE]
> Python 3.11 or higher is required.
```bash
# Verify that the 'humble' package contains 'Homepage: https://github.com/rfc-st/humble'
$ apt show humble
# Install it and grant permissions (e.g. to enable analysis history and export analysis)
$ sudo apt install humble
$ sudo chmod -R a+rwx /usr/share/humble
# Analyze! :)
$ cd /usr/share/humble
$ python3 humble.py -u https://google.com
# Updating (monthly):
$ sudo apt update
$ sudo apt install --only-upgrade humble
```
## Usage
```console
(Windows) $ py humble.py
(Linux) $ python3 humble.py
(macOS) $ python3 humble.py
usage: humble.py [-h] [-a] [-b] [-c] [-df] [-e [TESTSSL_PATH]] [-f [FINGERPRINT_TERM]] [-g] [-grd] [-if INPUT_FILE] [-l {es}] [-lic] [-o {csv,html,json,pdf,txt,xml}]
[-of OUTPUT_FILE] [-op OUTPUT_PATH] [-r] [-s [SKIP_HEADERS ...]] [-u URL] [-ua USER_AGENT] [-v]
'humble' (HTTP Headers Analyzer) | https://github.com/rfc-st/humble | v.2025-03-28
options:
-h, --help show this help message and exit
-a Shows statistics of the performed analysis; if the '-u' parameter is ommited they will be global
-b Shows overall findings; if omitted detailed ones will be shown
-c Checks URL response HTTP headers for compliance with OWASP 'Secure Headers Project' best practices
-df Do not follow redirects; if omitted the last redirection will be the one analyzed
-e [TESTSSL_PATH] Shows only TLS/SSL checks; requires the PATH of testssl (https://testssl.sh/)
-f [FINGERPRINT_TERM] Shows fingerprint statistics; if 'FINGERPRINT_TERM' (e.g., 'Google') is omitted the top 20 results will be shown
-g Shows guidelines for enabling security HTTP response headers on popular frameworks, servers and services
-grd Shows the checks to grade an analysis, along with advice for improvement
-if INPUT_FILE Analyzes 'INPUT_FILE': must contain HTTP response headers and values separated by ': '; E.g. 'server: nginx'
-l {es} Defines the language for displaying analysis, errors and messages; if omitted, will be shown in English
-lic Shows the license for 'humble', along with permissions, limitations and conditions.
-o {csv,html,json,pdf,txt,xml} Exports analysis to 'humble_scheme_URL_port_yyyymmdd_hhmmss_language.ext' file; json will have a brief analysis
-of OUTPUT_FILE Exports analysis to 'OUTPUT_FILE'; if omitted the default filename of the parameter '-o' will be used
-op OUTPUT_PATH Exports analysis to 'OUTPUT_PATH'; must be absolute. If omitted the PATH of 'humble.py' will be used
-r Shows HTTP response headers and a detailed analysis; '-b' parameter will take priority
-s [SKIP_HEADERS ...] Skips 'deprecated/insecure' and 'missing' checks for the indicated 'SKIP_HEADERS' (separated by spaces)
-u URL Scheme, host and port to analyze. E.g. https://google.com
-ua USER_AGENT User-Agent ID from 'additional/user_agents.txt' file to use. '0' will show all and '1' is the default
-v, --version Checks for updates at https://github.com/rfc-st/humble
examples:
-u URL -a Shows statistics of the analysis performed against the URL
-u URL -b Analyzes URL and reports overall findings
-u URL -b -o csv Analyzes URL and exports overall findings to CSV format
-u URL -l es Analyzes URL and reports (in Spanish) detailed findings
-u URL -o pdf Analyzes URL and exports detailed findings to PDF format
-u URL -o html -of test Analyzes URL and exports detailed findings to HTML format and 'test' filename
-u URL -o pdf -op D:/Tests Analyzes URL and exports detailed findings to PDF format and 'D:/Tests' path
-u URL -r Analyzes URL and reports detailed findings along with HTTP response headers
-u URL -s ETag NEL Analyzes URL and skips 'deprecated/insecure' and 'missing' checks for 'ETag' and 'NEL' headers
-u URL -ua 4 Analyzes URL using the fourth User-Agent of 'additional/user_agents.txt' file
-a -l es Shows statistics (in Spanish) of the analysis performed against all URLs
-f Google Shows HTTP fingerprint headers related to the term 'Google'
want to contribute?:
How to https://github.com/rfc-st/humble/#contribute
```
## Advanced usage (Linux)
.: Show only the analysis summary.
```
$ python3 humble.py -u https://www.spacex.com | grep -A 8 "\!." | sed $'1i \n'
```
.: Show only the URL, date and analysis summary.
```
$ python3 humble.py -u https://www.spacex.com | grep -A8 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed '5,6d' | sed '1i\'
```
.: Show only the deprecated headers/protocols and insecure values.
```
$ python3 humble.py -u https://www.spacex.com | sed -n '/\[4/,/^\[5/ { /^\[5/!p }' | sed '$d' | sed $'1i \n'
```
.: Check for HTTP client errors (4XX).
```
$ python3 humble.py -u https://my.prelude.software/demo/index.pl | grep -A1 -B5 'Note : \|Nota : ' --color=never
```
.: Analyze multiple URLs and save the results as PDFs; thanks Eduardo for this example!.
```
$ datasets=('https://facebook.com' 'https://github.com' 'https://www.spacex.com'); for dataset in "${datasets[@]}"; do python3 humble.py -u "$dataset" -o pdf; done
```
## Checks: enabled headers
Check this file.
## Checks: missing headers
Check this file.
## Checks: fingerprint headers
Check this file.
## Checks: deprecated headers/protocols and insecure values
Check this file.
> [!NOTE]
> _humble_ tries to be **strict**: both in checking HTTP response headers and their values; some of these headers may be experimental and you may not agree with all the results after analysis.
>
> And that's **OK**! :smiley:; you should **never** blindly trust the results of security tools: there should be further work to decide whether the risk is non-existent, potential or real depending on the analyzed URL (its exposure, environment, etc).
## Checks: empty values
Any HTTP response header.
## Guidelines included to enable security HTTP headers
* Amazon Web Services
* Angular
* Apache HTTP Server
* Cloudflare
* LiteSpeed Web Server
* Microsoft Internet Information Services
* Nginx
* Node.js
* Spring
* WordPress
## To-Do
- [ ] Add more Header/Value checks (only security-oriented)
- [ ] Google Style Python Docstrings and documentation via Sphinx
## Further reading
* Web browsers' experimental features, roadmaps, technology previews and trials:
Google Chrome
Microsoft Edge
Mozilla Firefox
Opera
Safari
* Similar tools on GitHub:
'HTTP Headers Analyze'
'HTTP Headers Secure'
'HTTP Headers Security'
OWASP Secure Headers Project
* References and standards:
Can I use?
Mozilla Developer Network
World Wide Web Consortium
* Additional information:
Common response headers
Security Headers (HTTP response header analyzer)
Scott Helme (Security Researcher)
## Contribute
* Read this first!.
* Report a Bug.
* Create a Feature request.
* Report a Security Vulnerability.
* Send me your suggestions: rafael.fcucalon@gmail.com
* Or use that email to tell me about integrations of this tool in others!
* And to recommend me a good Blues! :sunglasses:
Thanks for downloading _'humble'_, for trying it and for your time!.
## Acknowledgements
* Bandit, colorama, Flake8, fpdf2, pyinstrument, requests, SonarQube for IDE, Sourcery, testssl.sh and tldextract authors/teams: you rock :metal:!.
* 1nabillion for this.
* Aniket Navlur for this gem.
* Azathothas for reporting this bug.
* bulaktm for this suggestion.
* confuciussayuhm for this suggestion.
* cr4zyfish for some of these suggestions.
* danterolle for this.
* David for believing in the usefulness of this tool.
* Eduardo for the first Demo and the example "(Linux) - Analyze multiple URLs and save the results as PDFs".
* gl4nce for this suggestion.
* İDRİS BUDAK for reporting the need to this check.
* javelinsoft for this.
* Julio for testing on macOS.
* kazet for this suggestion.
* manuel-sommer for this, this and this!.
* mfabbri for this.
* mgrottenthaler for this.
* MikeAnast for several suggestions.
* n3bojs4, ehlewis and dkadev for this and this.
* Sophie Brun for keeping 'humble' updated in Kali Linux and for this.
* stanley101music for this, this and this!.
* vincentcox for this and this.
## License
MIT © 2020-2025 Rafa 'Bluesman' Faura (rafael.fcucalon@gmail.com)
Original Creator - Rafa 'Bluesman' Faura (rafael.fcucalon@gmail.com)
