{"id":13840730,"url":"https://github.com/regilero/HTTPWookiee","last_synced_at":"2025-07-11T09:33:03.519Z","repository":{"id":41380876,"uuid":"64682879","full_name":"regilero/HTTPWookiee","owner":"regilero","description":"HTTPWookiee is an HTTP server and proxy stress tool (respect of RFC, HTTP Smuggling issues, etc). If you run an HTTP server project contact me for private repository access with more tests.","archived":false,"fork":false,"pushed_at":"2017-12-02T12:47:26.000Z","size":189,"stargazers_count":49,"open_issues_count":1,"forks_count":11,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-08-05T17:25:35.878Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/regilero.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-08-01T16:21:33.000Z","updated_at":"2024-03-21T03:15:16.000Z","dependencies_parsed_at":"2022-09-12T04:01:37.017Z","dependency_job_id":null,"html_url":"https://github.com/regilero/HTTPWookiee","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/regilero%2FHTTPWookiee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/regilero%2FHTTPWookiee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/regilero%2FHTTPWookiee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/regilero%2FHTTPWookiee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/regilero","download_url":"https://codeload.github.com/regilero/HTTPWookiee/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225712670,"owners_count":17512455,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T17:00:54.401Z","updated_at":"2024-11-21T10:30:42.273Z","avatar_url":"https://github.com/regilero.png","language":"Python","funding_links":[],"categories":["Tools"],"sub_categories":[],"readme":"HTTP Wookiee\n============\n\n.. image:: https://travis-ci.org/regilero/HTTPWookiee.svg?branch=master\n    :target: https://travis-ci.org/regilero/HTTPWookiee\n\nThis project is an HTTP 1.1 (rfc 7230) stress tool. The goal is to test web\nservers and reverse proxies RFC compliance on various points of the protocol.\n\nThe tests are mostly security oriented. And failures may express potential\nHTTP Smuggling issues (hence the wookiees).\n\nSchema\n=======\n\nThis tool can stress directly an HTTP server (steps 1 and 4) **or** can be used\nto stress an HTTP Reverse Proxy (where this tool is both the client and the\nbackend server of this proxy, steps 1 to 5).\n\n.. image:: docs/schema.png\n\nWarning\n********\n\n\n**Tests may report false positives and wrong levels of criticity. Hard to catch\nall the existing behaviors of all http servers. But some tests are really exact\n(like double content-lenght issues or content-length+chunked, read the code for\ncomments and ask me if you wonder about a specific problem).**\n\n**Never try to run theses tests on a public network or on public websites.\nEven testing your own website may be dangerous if an unknown reverse proxy is\npresent between your website and the stress tool.**\n\n**This project contains several safeguards against this usage, but this is open\nsource code, and you could certainly remove many safeguards.**\n\n**Any usage in the wild is at your own risk and may lead you to a courthouse.\nNot because bad HTTP syntax issues are very dangerous, they are usually not. But\nyou should really understand how HTTP Smuggling issues may break more than only\nyour own communication. It is very hard to control the real effects of bad\nrequests in a complex chain of servers, and you may not be able to control\nthese effects. This is not a reflected XSS test tool, you play with\nlow-level protocol issues here, stress your own servers, on your own local network, or\neven better on your own computer. Help us fix issues on HTTP agents, and do not\ntry to be a moron. By the way if you want to contribute, please act with caution\non reporting new tests if the major HTTP players are failing the tests.\nThere is no hurry, send me an email, store code on private repositories, etc.**\n\nIf you really understand theses points, and still really want to use this code,\non your own servers, set a local boolean variable\nI_HAVE_READ_AND_UNDERSTAND_THE_FAQ_AND_I_AM_RESPONSIBLE_OF_MY_ACTS in your\napplication configuration file.\n\nInstall\n*******\n\nVirtualenv \u0026 pip\n----------------\n\nYou may want to create virtualenvs before installing this script (pip install virtualenv)::\n\n    mkdir ~/venvs/\n    cd ~/venvs/\n    # for python 2\n    virtualenv HTTPWookieep2 -p python2\n    # for python3\n    virtualenv HTTPWookieep3 -p python3\n\nThen you can activate the environment in python3 (recommended) or python2 mode::\n\n    # python 2\n    source ~/venvs/HTTPWookieep2/bin/activate\n    # python 3\n    source ~/venvs/HTTPWookieep3/bin/activate\n\nOnce the virtualenv is activated you need to install the dependencies and the current application.\nOnce this application will be available on pypi a simple pip install would work::\n\n   # not yet\n   #pip install HTTPWookiee\n\nBut currently, or if you want to work from the source code (and maybe alter this code), you need to\nuse the `-e` option of pip, to install dependencies but keep the current code on the right path (and\navoid having a copy in the sites-packages of this virtualenv)::\n\n    cd /path/to/HTTPWookiee\n    # this will install all dependencies in your virtualenv\n    # the '-e' is a dev-mode option to keep the current\n    # directory as the source of the code for HTTPWookiee\n    pip install -e .\n\nFrom Source\n------------\n\nIf you work from source you can instead go to the source directory and run::\n\n    pip install -e .\n    # or\n    pip install -e .[dev,test]\n\nYou will certainly want to install :code:`tox`. **tox** is used to run the internal\ntests (with python2.7, 3.4 and 3.5)::\n\n    tox -r\n\nSettings\n---------\n\nDefine your personnal settings in a :code:`${HOME}/.httpwookieerc` file.\n\nYou can also define a :code:`HTTPWOOKIEE_CONF` env variable with the config file path.\n\nCheck class Config in config.py for available settings.\n\nCheck the :code:`conf_sample.conf` file for examples and comments on the various\nmain options.\n\nHere is a very minimal config file::\n\n    [main]\n    DEBUG: true\n    SERVER_HOST: dummy-host2.example.com\n    SERVER_SSL: false\n    SERVER_IP: 127.0.0.1\n    SERVER_PORT: 8080\n    BACKEND_PORT: 8282\n\n\nRun\n****\n\n**If you use virtualenv, do not forget to activate your virtualenv first.**::\n\n    source ~/venvs/HTTPWookieep3/bin/activate\n\nThere are two main modes (both running by default):\n\n**1) -** 'client': in this mode, HTTPWookiee is an HTTP client, simply\nsending requests against the tested HTTP server::\n\n    HTTPWookiee            Tested HTTP\n      Client                Server\n        |                      |\n        |-----Request---------\u003e|\n        |\u003c----Response---------|\n        |\n    [analysis]\n\n**2) -** 'server': in this mode HTTPWookiee will also run a server, used\nas a backend for the tested HTTP Reverse Proxy::\n\n    HTTPWookiee          Tested HTTP         HTTPWookiee\n      Client            Reverse Proxy       Server Thread\n        |                    |                   |\n        |-----Request-------\u003e|                   |\n        |                    |-----Request------\u003e|\n        |                    |\u003c----Response------|\n        |\u003c----Response-------|                   |\n        |                    |                   |\n    [analysis]\u003c- - - - - - - - - - - [ internal transmission ]\n\nNote that the port, IP address and url used in tests can be defined in the\nconfiguration file.\nTo use HTTPWookiee in this mode you will need to alter the external\nReverse Proxy configuration, to set the right IP and port of the backend,\ntargeting the IP and port of the server thread in HTTPWookiee.\n\nFor each mode, a big variety of tests exist. You can run the tests by\ncalling the python test file directly::\n\n    ./httpwookiee/client/tests_chunks.py\n\nIf you are working from sources (and not from a pip\nversion of this package, and you did not use pip install -e, as you should)\nyou may have to alter your PYTHONPATH to run the program\nthis way (to get the httpwookiee directory managed as a valid module, but\nas I said using pip install with `-e` option is better)::\n\n    export PYTHONPATH=\".\"\n    ./httpwookiee/client/tests_chunks.py\n\n\nThe easiest way is to used the root `httpwookiee.py` script with the :code:`-m` or\n:code:`--match` option::\n\n    ./httpwookiee.py -m client -m chunks\n\nTo run the reverse proxy server tests alter the mode::\n\n    ./httpwookiee.py -m server -m chunks\n\nA second level of match (:code:`-M`) can be made on the tests class names (first level was\nfor the test files names). Use it with :code:`-l` to control the tests list before\nrunning the real thing. And then you have the :code:`-e` and :code:`-E` to add exclusions\nin the matching policy.\n\nAnother filter exists, matching the test numbers on the final individual tests\nwhere the number is :code:`XXX` in :code:`test_XXX_foo_bar_something`.\nSay you identified a test named :code:`test_3010_method_separator` that you\nwant to run with also the test :code:`test_3011_location_separator`, you can use::\n\n    ./httpwookiee.py -m client -t 3010 -t 3011\n\nBut **be carefull**, running only individual tests i hazardous, as currently\nthere is no way to ensure the *preflight* tests associated with theses tests\nwould run before, so some tests may lack some informations (like choosing\nbetween GET and POST).\n\nYou can control the output with options :code:`-V` and :code:`-n`.\n\nThis is an example of output without any of theses options::\n\n    #### letters for each test result (theses are the type of responses\n    #### received, not the status of the test)\n    444444aaF4444444aFaF4444444444444(...)4444444aa444444aaF4444444444\n    #### Then the list of all failed tests details\n    #### starting with gravity (great potential of false positive here)\n    FAIL gravity: Minor\n    #### then name of the test and class of the test\n    [test_location_separator_09_and_extra_proto](httpwookiee.staging.client.tests_first_line.TestURLEncodedTabFirstLineSpaceSeparators)\n    #### here the traceback\n     (...)\n    #### and here the main assertion error\n    AssertionError: Bad response status \"Accepted\"\n\n    Stdout:\n    #### here details on request sent\n    --\u003e =None=\n    --\u003e # Connecting to Host: localhost IP: 127.0.0.1 PORT: 19080\n    --\u003e # socket ok\n    --\u003e # client connection established.\n    --\u003e # SENDING (159) =====\u003e\n    --\u003e GET /?djjd5a=kjdej7%09HTTP/0.9 HTTP/1.1[CR][LF]\n    Foo: Bar[CR][LF]\n    User-Agent: script-httpwookiee @ spambot_irc + select union[CR][LF]\n    X-Wookiee: 140044686096720[CR][LF]\n    Host: localhost[CR][LF]\n    [CR][LF]\n\n    #### here details on things received\n    --\u003e # ====================\u003e\n    --\u003e # ...\n    \u003c-- # \u003c==== READING \u003c===========\n    \u003c-- # ...\n    \u003c-- # read timeout(0.2), nothing more is coming\n    \u003c-- # \u003c====FINAL RESPONSE===============\n    \u003c-- HTTP/1.1 200 OK\n    Request-Number: 111\n    Date: Thu, 18 Aug 2016 14:43:39 GMT\n    Connection: keep-alive\n    Content-Length: 0\n\n    #### followed by internal analysis of the response(s)\n    (...)\n    ++++++++++++++++++++++++++++++++++++++\n    (...) and then the next failing test...\n\nWith :code:`-v` or :code:`--verbose` the first line of responses status will send more\ninformations::\n\n    #### you have the name of the test, the class, then a comment on the test.\n    #### at the end of the second line you have the status (same as the small\n    #### letter without the Verbose option, but more readable) and between '[]'\n    #### the status of the test.\n    (...)\n    [test_line_suffix_with_double_HTTP11] (httpwookiee.client.tests_first_line.TestTabFirstLineSpaceSeparators)\n      Ending first line with two times the protocol................................. ... --err400--    [ok]\n    [test_location_separator] (httpwookiee.client.tests_first_line.TestTabFirstLineSpaceSeparators)\n      After the query string, valid separator or a forbidden char?.................. ... -accepted-    [ok]\n    [test_location_separator_09_and_extra_proto] (httpwookiee.client.tests_first_line.TestTabFirstLineSpaceSeparators)\n      After the query, valid separator or a forbidden char? Proto repeated.......... ... -accepted-[Minor]-  [FAIL]\n    [test_method_separator] (httpwookiee.client.tests_first_line.TestTabFirstLineSpaceSeparators)\n      Test various characters after the METHOD...................................... ... --err400--    [ok]\n    [test_line_prefix] (httpwookiee.client.tests_first_line.TestVerticalTabFirstLineSpaceSeparators)\n      Some characters before the query.............................................. ... --err400--    [ok]\n    [test_line_suffix] (httpwookiee.client.tests_first_line.TestVerticalTabFirstLineSpaceSeparators)\n      Let's add some garbage after the protocol..................................... ... --err400--    [ok]\n    [test_line_suffix_with_char] (httpwookiee.client.tests_first_line.TestVerticalTabFirstLineSpaceSeparators)\n      Let's add some garbage after the protocol. With a letter after................ ... --err400--    [ok]\n    [test_line_suffix_with_char_H] (httpwookiee.client.tests_first_line.TestVerticalTabFirstLineSpaceSeparators)\n    (...)\n\nwith :code:`-n`, or :code:`--no-buffer`, you have the output of the tests directly while they\nare running, this is especially useful to check that non-failing tests are\nreally behaving well.\n\nDo not hesitate to use :code:` 2\u003e\u00261 | less -R` after the command to get an output\nwith both stderr and stdout, and to paginate this output (with less).  The :code:`-R`\noption for less will manage the colorization of this output.\n\n**Finally** you can always **run a wookiee test file directly**.\nSay you want to run the first line tests on an http server (not a proxy),\nyou need the matching test file in the client subdirectory::\n\n    ./httpwookiee/client/tests_first_line.py 2\u003e\u00261|less -R\n\nAnd if you want to test the server (reverse proxy) mode only::\n\n    ./httpwookiee/server/tests_first_line.py 2\u003e\u00261|less -R\n\n\nRun tests against embedded docker images\n*****************************************\n\nIn the :code:`tests` subdirectory you can find some Dockerfiles that can be used\nto run httpwookiee against some well known HTTP servers and proxies.\n\nLet's try for example 2 different Nginx versions::\n\n    # let's go to the nginx subdirectory\n    cd tests/dockernginx/\n    # build the various docker images and start these images\n    # each image will contain a reverse proxy listening for\n    # an HTTPWookiee backend server on the docker host\n    sudo ./build.sh\n    # get back to the project main dir\n    cd ../..\n    # load the right configuration for HttpWookiee\n    # especially because each docker reverse proxy has his own listening port\n    export HTTPWOOKIEE_CONF=/path/to/HTTPWookiee/tests/dockernginx/latest/config.ini\n    # run some tests\n    ./httpwookiee.py -m server -m chunks -V 2\u003e\u00261|less -r\n    # load another conf for the next docker\n    export HTTPWOOKIEE_CONF=/path/to/HTTPWookiee/tests/dockernginx/stable/config.ini\n    # run some tests again, etc\n    ./httpwookiee.py -m server -m chunks -V 2\u003e\u00261|less -r\n    # etc.\n\n\nInternal tests\n***************\n\nHTTPWookiee uses test tools to perform tests against HTTP servers. But it also\nuses test tools for internal tests. Like testing the internal HTTP parser,\nensuring python 2.7 and python 3.4 support, or checking pep8 syntax.\n\nTo run theses tests check the 'From source' part on the 'Install' section.\n\nHTTP Smuggling\n**************\n\nThe tests performed by this tool *could* be used to perform HTTP smuggling\nattacks.\nUsually an attack needs to combine several flaws (like, at least, a transmission\nissue, where a bad syntax is transmitted by a reverse proxy, and a request or\nresponse splitting issue).\n\nUsage of smuggling issues are:\n\n- **security filters bypass** (we can expand that in a reverse proxy suffering from a splitting issue to hiding queries with bad syntax which are usually not transmitted by this proxy -- encapsulation of smuggling attacks)\n- **HTTP credentials hijacking**, using unterminated queries and keep-alive connections\n- **cache poisoning**, which can be expanded to tcp connections poisoning\n- **DOS**, if you get the wrong content, or if the server crash\n\nHere are some examples of combined flaws.\n\nThis is a **splitting** issue, where the hidden Transfer-Encoding header may be read\nby a bad parser as a real header (if :code:`\\rZ==\\r\\n` for this parser, like in old\nnodejs versions). Transfer-Encoding has priority on Content-Length.\n\n.. image:: docs/demo1-A.png\n\nThe second query is unterminated and this can be used to capture HTTP credentials\nif a reverse proxy is reusing the same connection with another user (here the\n:code:`GET /foo` query).\n\n.. image:: docs/demo1-B.png\n\nHere is another example of smuggling attack, combining 5 flaws:\n\n- \"Transfer Encoding\" header transmitted (wrong, space is not a valid character)\n- \"Transfer Encoding\" header magically fixed to \"Transfer-Encoding\" (wrong, no magic fix should apply), now a **splitting** issue.\n- HTTP/0.9 considered as a valid HTTP 0.9 marker (it is not)\n- Range header applied on a 0.9 query (no headers should apply), used to extract content from an image (targeting exif headers).\n- the last flaw is invisible in this image, it's a tcp poisoning, where the 0.9 response (which is the HTTP response hidden in the image) is stored on the connection and reused for the next user response.\n\n.. image:: docs/demo2.png\n\nDemonstrations of theses 2 attacks can be seen in this video:\n\n..raw::html\n\n  \u003ciframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/lY_Mf2Fv7kI\" frameborder=\"0\" allowfullscreen\u003e\u003c/iframe\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fregilero%2FHTTPWookiee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fregilero%2FHTTPWookiee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fregilero%2FHTTPWookiee/lists"}