{"id":13586332,"url":"https://github.com/bitsofinfo/testssl.sh-alerts","last_synced_at":"2025-04-13T07:56:06.599Z","repository":{"id":41298273,"uuid":"155570044","full_name":"bitsofinfo/testssl.sh-alerts","owner":"bitsofinfo","description":"Alerting engine (slack etc) for testssl.sh JSON result output files","archived":false,"fork":false,"pushed_at":"2019-07-10T15:49:34.000Z","size":1737,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-13T07:56:02.163Z","etag":null,"topics":["alerting","alerts","certificates","continuous-integration","security","slack","ssl","testssl"],"latest_commit_sha":null,"homepage":"https://bitsofinfo.wordpress.com/2018/12/21/slack-alerts-for-testssl-sh/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bitsofinfo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-10-31T14:21:50.000Z","updated_at":"2025-03-18T08:52:21.000Z","dependencies_parsed_at":"2022-09-11T13:30:58.842Z","dependency_job_id":null,"html_url":"https://github.com/bitsofinfo/testssl.sh-alerts","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ftestssl.sh-alerts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ftestssl.sh-alerts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ftestssl.sh-alerts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bitsofinfo%2Ftestssl.sh-alerts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bitsofinfo","download_url":"https://codeload.github.com/bitsofinfo/testssl.sh-alerts/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248681490,"owners_count":21144700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alerting","alerts","certificates","continuous-integration","security","slack","ssl","testssl"],"created_at":"2024-08-01T15:05:29.191Z","updated_at":"2025-04-13T07:56:06.574Z","avatar_url":"https://github.com/bitsofinfo.png","language":"Python","readme":"# testssl.sh-alerts\n\nProvides a daemon that monitors for new [testssl.sh JSON result output files](https://github.com/drwetter/testssl.sh), evaluates them using [ObjectPath expressions](http://objectpath.org/) to trigger events which can be reacted to in different ways such as sending Slack alerts, copying JSON result files to a new location or anything else you with do do. You can implement a simple python `Reactor` with a `handleTriggers` method and do whatever you want to extend the available functionality.\n\n![](docs/diag1.png)\n\n# Requirements\n\n**Python 3.6**\n\nDependencies:\n```\npip install objectpath pyyaml python-dateutil watchdog slackclient pygrok jinja2 twisted\n```\n\n# Overview and Configuration\n\nThe handler engine is configured by one or more YAML configuration documents\n([see example here](example-config.yaml)) that can be dropped in the `--config-dir`.\nEach `testssl.sh` JSON result file detected will be processed against every YAML config\nfile loaded into the system and there is no limit to the number of config files.\n\nEach config file configures the handler engine for each JSON result file to be processed.\nThe handler engine makes heavy use of [ObjectPath expressions](http://objectpath.org/)\nto craft an `evaluation_doc` which is comprised of the original `testssl.sh` JSON result\ndocument/object contents plus addition meta-data that can be extracted from the result file path\nvia Grok ([via PyGrok](https://github.com/garyelephant/pygrok)) expressions and permitting full customization of the resulting meta-data and `evaluation_doc` structure and property names.\n\nOnce this `evaluation_doc` is constructed the config file's `trigger_on` triggers are\nevaluated one by one against the `evaluation_doc`. Each trigger is comprised of a `title`,\nan [ObjectPath](http://objectpath.org/) expression, and the names of one or more `reactors`\nthat should be invoked if the [ObjectPath](http://objectpath.org/) expression returns a value.\nIf the value is a `Boolean` it must be `True` for the for trigger to be considered \"fired\". Any\nother value returned by an expression other than `None` also qualifies as the trigger being fired.\n\nFor each fired trigger it will be passed to its configured `reactor`. There are two provided\nreactors:\n\n* **SlackReactor**: [slackreactor.py](reactors/slackreactor.py) - Sends Slack alerts using a jinja2 template, see [example-config.yaml](example-config.yaml) for details\n* **CopyFileReactor**: [copyfilereactor.py](reactors/copyfilereactor.py) - Copies files. For example to copy the testssl.sh JSON files that container certain vulnerabilities, see [example-config.yaml](example-config.yaml) for details\n\n\n# Configuration details\n\nFor full configuration details its best to see the ([see example here](example-config.yaml))\nin [example-config.yaml](example-config.yaml)\n\n\n## Usage\n\n```\n./testssl_result_handler.py --help\n\nusage: testssl_result_handler.py [-h] [-i INPUT_DIR]\n                                 [-f INPUT_FILENAME_FILTER] [-I CONFIG_DIR]\n                                 [-l LOG_FILE] [-x LOG_LEVEL]\n                                 [-w INPUT_DIR_WATCHDOG_THREADS]\n                                 [-s INPUT_DIR_SLEEP_SECONDS]\n                                 [-d DEBUG_OBJECTPATH_EXPR] [-D] [-E]\n                                 [-p HTTPSERVER_PORT] [-r HTTPSERVER_ROOT_DIR]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -i INPUT_DIR, --input-dir INPUT_DIR\n                        Directory path to recursively monitor for new `*.json`\n                        testssl.sh result files. Default './input'\n  -f INPUT_FILENAME_FILTER, --input-filename-filter INPUT_FILENAME_FILTER\n                        Regex for filter --input-dir files from triggering the\n                        watchdog. Default '.*testssloutput.+.json'\n  -I CONFIG_DIR, --config-dir CONFIG_DIR\n                        Directory path to recursively monitor for new `*.yaml`\n                        result handler config files. Default './configs'\n  -l LOG_FILE, --log-file LOG_FILE\n                        Path to log file, default None, STDOUT\n  -x LOG_LEVEL, --log-level LOG_LEVEL\n                        log level, default DEBUG\n  -w INPUT_DIR_WATCHDOG_THREADS, --input-dir-watchdog-threads INPUT_DIR_WATCHDOG_THREADS\n                        max threads for watchdog input-dir file processing,\n                        default 10\n  -s INPUT_DIR_SLEEP_SECONDS, --input-dir-sleep-seconds INPUT_DIR_SLEEP_SECONDS\n                        When a new *.json file is detected in --input-dir, how\n                        many seconds to wait before processing to allow\n                        testssl.sh to finish writing. Default 5\n  -d DEBUG_OBJECTPATH_EXPR, --debug-object-path-expr DEBUG_OBJECTPATH_EXPR\n                        Default False. When True, adds more details on\n                        ObjectPath expression parsing to logs\n  -D, --debug-dump-evaldoc\n                        Flag to enable dumping the 'evaluation_doc' to STDOUT\n                        after it is constructed for evaluations (WARNING: this\n                        is large \u0026 json pretty printed)\n  -E, --dump-evaldoc-on-error\n                        Flag to enable dumping the 'evaluation_doc' to STDOUT\n                        (json pretty printed) on any error (WARNING: this is\n                        large \u0026 json pretty printed)\n  -p HTTPSERVER_PORT, --httpserver-port HTTPSERVER_PORT\n                        Default None, if a numeric port is specified, this\n                        will startup a simple twisted http server who's\n                        document root is the --httpserver-root-dir\n  -r HTTPSERVER_ROOT_DIR, --httpserver-root-dir HTTPSERVER_ROOT_DIR\n                        Default None, if specified the embedded http server\n                        will serve up content from this directory, has no\n                        effect if --httpserver-port is not specified\n```\n\n# Example\n\nNote the example below is just that; an example. The reactor engine is completely\ncustomizable and you can fully customize any of the available reactors with regards\nto the message format and or any other actions and even make your own reactors.\n\nThat said: Lets startup the result handler:\n\n```\nmkdir input/\nmkdir configs/\nmkdir output/\n\n./testssl_result_handler.py \\\n  --debug-object-path-expr True \\\n  --input-dir ./input \\\n  --config-dir ./configs \\\n  --input-filename-filter '.*_testssl_.+.json' \\\n  --httpserver-port 7777 \\\n  --httpserver-root-dir output/\n\n```\n\nAt this point the handler is up and running.... the `--httpserver-root-dir` is serving\nup the files copied via the `CopyFileReactor` config in the `example-config.yaml`\n\n```\n2018-11-15 14:17:34,583 - root - INFO - Monitoring for new result handler config YAML files at: ./configs\n2018-11-15 14:17:34,584 - root - INFO - Monitoring for new testssl.sh result JSON files at: ./input\n2018-11-15 14:17:34,585 - root - INFO - Starting HTTP server listening on: 7777 and serving up: output/\n...\n```\n\nLets copy a sample config to `configs/`\n\n```\ncp example-config.yaml configs/\n```\n\nWhich is then consumed by the handler...\n\n```\n...\n2018-11-15 14:17:39,713 - root - INFO - Responding to creation of result handler config file: ./configs/example-config.yaml\n...\n```\n\nAt this point the handler watchdog is monitoring `input/` for any `.*testssl.+.json` files.\nLets copy some sample output generated by the [testssl.sh-processor](https://github.com/bitsofinfo/testssl.sh-processor)\nproject's example output found in the `sample/` directory.\n\n```\ncp -R sample/* input/\n```\n\nOnce detected by the watchdog the result handler begins to react, since the `cert_expiration_gte30days` trigger fires, we send both a slack alert and copy the JSON result file to the given path as specified in the config's `reactor_engine` settings contained within `example-config.yaml`. The files matching the triggers are available under the the copy to dir under `output/`.\nThis functionality could be used to automatically move `testssl.sh` JSON result files that flag vulnerabilities or upcoming expirations to a target location for further action or review.\n\n```\n...\n2018-11-15 14:17:46,447 - root - INFO - Responding to parsable testssl.sh JSON result: ./input/20181113_194917-www.google.com-testssl_cmds/www.google.com/20181108120000/public/search/20181108120000_testssl_www.google.com.json\n2018-11-15 14:17:46,448 - root - INFO - Received event for create of new testssl.sh JSON result file: './input/20181113_194917-www.google.com-testssl_cmds/www.google.com/20181108120000/public/search/20181108120000_testssl_www.google.com.json'\n2018-11-15 14:17:46,449 - root - INFO - testssl.sh JSON result file loaded OK: './input/20181113_194917-www.google.com-testssl_cmds/www.google.com/20181108120000/public/search/20181108120000_testssl_www.google.com.json'\n2018-11-15 14:17:46,449 - root - INFO - Evaluating ./input/20181113_194917-www.google.com-testssl_cmds/www.google.com/20181108120000/public/search/20181108120000_testssl_www.google.com.json against config 'example-config.yaml' ...\n2018-11-15 14:17:46,458 - root - DEBUG - exec_objectpath: query: $.testssl_result.scanResult[0].serverDefaults[split(@.id,' ')[0] is 'cert_notAfter'][@.finding]\n2018-11-15 14:17:46,465 - root - DEBUG - exec_objectpath: query: $.testssl_result.scanResult[0].serverDefaults[split(@.id,' ')[0] is 'cert_notAfter'][@.finding] raw result type(): \u003cclass 'generator'\u003e\n2018-11-15 14:17:46,465 - root - DEBUG - exec_objectpath: query: $.testssl_result.scanResult[0].serverDefaults[split(@.id,' ')[0] is 'cert_notAfter'][@.finding] next() returned val: 2019-01-22 06:14\n...\n2018-11-15 14:17:46,665 - root - DEBUG - Invoking reactor: slack for 1 fired triggers\n2018-11-15 14:17:46,683 - root - DEBUG - SlackReactor: Sending to slack....\n2018-11-15 14:17:46,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): hooks.slack.com\n2018-11-15 14:17:47,589 - urllib3.connectionpool - DEBUG - https://hooks.slack.com:443 \"POST /services/TE2KJDF4L/BE22XTKGQ/4UKdwVZQ54U1NW8p7mtdowfN HTTP/1.1\" 200 22\n...\n2018-11-15 14:17:47,597 - root - DEBUG - Invoking reactor: copy_json_result for 1 fired triggers\n2018-11-15 14:17:47,597 - root - DEBUG - CopyFileReactor: attempting cleanup of output/ older than 0.0001 days...\n2018-11-15 14:17:47,601 - root - INFO - CopyFileReactor: Copied OK /home/bitsofinfo/code/github.com/bitsofinfo/testssl.sh-alerts/input/20181113_194917-www.google.com-testssl_cmds/www.google.com/20181108120000/public/search/20181108120000_testssl_www.google.com.json TO output/testssl.sh-issues/cert_expiration_gte30days/\n...\n2018-11-15 14:17:47,601 - root - DEBUG - Invoking reactor: copy_html_result for 1 fired triggers\n2018-11-15 14:17:47,604 - root - INFO - CopyFileReactor: Copied OK /home/bitsofinfo/code/github.com/bitsofinfo/testssl.sh-alerts/input/20181113_194917-www.google.com-testssl_cmds/www.google.com/20181108120000/public/search/20181108120000_testssl_www.google.com.html TO output/testssl.sh-issues/cert_expiration_gte30days/\n```\n\n## Result of `SlackReactor` alert:\n![](docs/alert.png)\n\n## Result of `CopyFileReactor`:\n![](docs/output.png)\n\n## Contents of http://localhost:7777:\n![](docs/http.png)\n\nAgain; note the example above is just that; an example. The reactor engine is completely\ncustomizable and you can fully customize any of the available reactors with regards\nto the message format and or any other actions and even make your own reactors.\n\n\n# Docker\n\nThe dockerfile contained in this project can easily be built locally and run easily\nvia the command line just like via the shell directly.\n\n```\ndocker build -t testssl-alerts .\n\ndocker run \\\n  -v /path/to/configs:/configs \\\n  -v /path/to/output:/input \\\n  testssl-alerts \\\n  testssl_result_handler.py \\\n    --input-dir /input \\\n    --config-dir /configs\n```\n\n## Creating your own Reactors\n\nIts pretty easy to create your own custom reactor.\n\n1. Create a new class at `reactors/myreactor.py` and declare a class in it called `MyReactor`\n\n2. Declare it and its configuration under `reactor_engines`. You can have different reactors that all\nleverage the same reactor `class_name` but behave differently. Triggers reference the arbitrary `reactor-name`\nwhich can be any valid yaml key name.\n\n```\nreactor_engines:\n  [reactor-name]:\n    class_name: \"MyReactor\"\n    ...\n    ...\n```\n\n2. It should support a constructor and declared the named method as follows:\n```\nclass [MyCustom]Reactor():\n  ...\n\n  # Constructor\n  # passed the raw reactor_config object\n  def __init__(self, reactor_config):\n      self.my_prop = reactor_config['my_prop']\n\n\n  # When invoked this is passed\n  #\n  # - 'triggers_fired' - array of trigger_result objects.\n  #                      Where each trigge_result is defined as:\n  # {\n  #    'tag': [short name of the trigger]\n  #    'title':[see above, title of the trigger name],\n  #    'reactors':[see above, array of configured reactor names],\n  #    'objectpath':[see above, the objectpath],\n  #    'results':[array of raw object path result values],\n  #    'config_filename':[name of the YAML config the trigger was defined in],\n  #    'testssl_json_result_abs_file_path':[absolute path to the testssl.sh JSON result file],\n  #    'testssl_json_result_filename':[filename only of JSON result file],\n  #    'evaluation_doc':[the evalution_doc object that the trigger evaluated]\n  #  }\n  #\n  #\n  # - 'objectpath_ctx' - a reference to the ObjectPathContext object used\n  #                      when processing the trigger evaluations. The following\n  #                      ObjectPathContext properties can be used in the reactor\n  #                      for further ObjectPath based functionality if the reactor\n  #                      plugin wishes to take advantage of it.\n  # {\n  #    exec_objectpath: ObjectPathContext function reference\n  #    exec_objectpath_specific_match: ObjectPathContext function reference\n  #    exec_objectpath_first_match: ObjectPathContext function reference\n  #    evaluation_doc: the raw evalution_doc object that the trigger evaluated\n  #  }\n  #\n  #\n  def handleTriggers(self, triggers_fired, objectpath_ctx):\n    ...\n    DO YOUR WORK HERE!\n    ...\n```\n\n\n## Related\n\n* This tool was originally developed as the 3rd stage in a pipeline of `testssl.sh` automation, consuming the `testssl.sh` JSON output files produced by the [testssl.sh-processor daemon](https://github.com/bitsofinfo/testssl.sh-processor) available at https://github.com/bitsofinfo/testssl.sh-processor which is a tool to bulk test `testssl.sh` command files generated by another process, using a file monitoring watchdog.\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitsofinfo%2Ftestssl.sh-alerts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitsofinfo%2Ftestssl.sh-alerts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitsofinfo%2Ftestssl.sh-alerts/lists"}