{"id":16725316,"url":"https://github.com/christiangalsterer/httpbeat","last_synced_at":"2025-07-18T01:14:53.217Z","repository":{"id":49443631,"uuid":"46674589","full_name":"christiangalsterer/httpbeat","owner":"christiangalsterer","description":"Elastic Beat to call HTTP endpoints","archived":false,"fork":false,"pushed_at":"2018-03-23T12:25:23.000Z","size":29599,"stargazers_count":107,"open_issues_count":6,"forks_count":39,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-02T02:21:43.619Z","etag":null,"topics":["beat-frameworks","beats","elastic-beats","elasticsearch","golang","logstash","metricbeat"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/christiangalsterer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-11-22T18:49:53.000Z","updated_at":"2024-07-24T21:15:24.000Z","dependencies_parsed_at":"2022-09-06T01:30:19.120Z","dependency_job_id":null,"html_url":"https://github.com/christiangalsterer/httpbeat","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/christiangalsterer/httpbeat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/christiangalsterer%2Fhttpbeat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/christiangalsterer%2Fhttpbeat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/christiangalsterer%2Fhttpbeat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/christiangalsterer%2Fhttpbeat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/christiangalsterer","download_url":"https://codeload.github.com/christiangalsterer/httpbeat/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/christiangalsterer%2Fhttpbeat/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265687133,"owners_count":23811219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beat-frameworks","beats","elastic-beats","elasticsearch","golang","logstash","metricbeat"],"created_at":"2024-10-12T22:48:47.878Z","updated_at":"2025-07-18T01:14:53.193Z","avatar_url":"https://github.com/christiangalsterer.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/christiangalsterer/httpbeat.svg?branch=master)](https://travis-ci.org/christiangalsterer/httpbeat)\n[![codecov.io](http://codecov.io/github/christiangalsterer/httpbeat/coverage.svg?branch=master)](http://codecov.io/github/christiangalsterer/httpbeat?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/github.com/christiangalsterer/httpbeat)](https://goreportcard.com/report/github.com/christiangalsterer/httpbeat)\n[![license](https://img.shields.io/github/license/christiangalsterer/httpbeat.svg)](https://github.com/christiangalsterer/httpbeat)\n[![Github All Releases](https://img.shields.io/github/downloads/christiangalsterer/httpbeat/total.svg)](https://github.com/christiangalsterer/httpbeat)\n\n![Elastic Beats 5.6.3](https://img.shields.io/badge/Elastic%20Beats-v5.6.3-blue.svg)\n![Golang 1.9](https://img.shields.io/badge/Golang-v1.9-blue.svg)\n\n# Httpbeat\n\n\u003e I'm happy to announce that Httpbeat will become a core feature of the Beats framework 6.0 with the introduction of the [Http module](https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-http.html) of [Metricbeat](https://www.elastic.co/guide/en/beats/metricbeat/master/index.html).\n\n\u003e I would like to give special thanks to [Nicolas Ruflin](https://github.com/ruflin) for his support during the migration of the Httpbeat into the new [Http module](https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-http.html).\n\n\u003e I will continue the support for Httpbeat for Beats 5.x, but all users of Beats 6.0 and newer versions are encouraged to migrate to the [Http module](https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-http.html) of [Metricbeat](https://www.elastic.co/guide/en/beats/metricbeat/master/index.html).\n\n\nWelcome to Httpbeat.\n\nHttpbeat is a [Beat](https://www.elastic.co/products/beats) used to call HTTP endpoints.\nMultiple endpoints can be configured which are polled in a regular interval and the result is shipped to the configured output channel.\n\nHttpbeat is inspired by the Logstash [http_poller](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html) input filter but doesn't require that the endpoint is reachable by Logstash as Httpbeat pushes the data to Logstash or Elasticsearch.\nThis is often necessary in security restricted network setups, where Logstash is not able to reach all servers. Instead the server to be monitored itself has Httpbeat installed and can send the data or a collector server has Httpbeat installed which is deployed in the secured network environment and can reach all servers to be monitored.\n\nExample use cases are:\n* Monitor [Apache Stats](https://httpd.apache.org/docs/2.4/mod/mod_status.html)\n* Monitor Java application with [Jolokia](https://jolokia.org)\n* Monitor [Spring Boot Actuators](http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#production-ready)\n* Monitor [Docker Remote API](https://docs.docker.com/engine/reference/api/docker_remote_api/)\n\nEnsure that this folder is at the following location:\n`${GOPATH}/src/github.com/christiangalsterer`\n\n# Installation\n\n## Download\nPre-compiled binaries for different operating systems are available for [download](https://github.com/christiangalsterer/httpbeat/releases/tag/4.0.0).\n\n## Installation\nInstall the package for your operation system by running the respective package manager or unzipping the package.\n\n## Configuration\nAdjust the `httpbeat.yml` configuration file to your needs. You may take `httpbeat.full.yml` as an example containing all possible configuration values.\n\n## Running\nIn order to start Httpbeat please use the respective startup script, e.g. `/usr/bin/httpbeat.sh`.\n\n## Starting Httbeat as Service\nWhere supported Httpbeat can be started also using the respetive service scripts, e.g. `etc/init.d/httpbeat`.\n\n## Building and Releasing Httpbeat\n\n### Requirements\n\n* [Golang](https://golang.org/dl/) 1.9\n* [Glide](https://github.com/Masterminds/glide) \u003e= 0.13.0\n\n### Build\n\nTo build the binary for httpbeat run the command below. This will generate a binary\nin the same directory with the name httpbeat.\n\n```\nmake clean \u0026\u0026 make\n```\n\n### Run\n\nTo run httpbeat with debugging output enabled, run:\n\n```\n./httpbeat -c httpbeat.yml -e -d \"*\"\n```\n\nTo run httpbeat in normal mode, run:\n\n```\n./httpbeat -c httpbeat.yml\n```\n\n### Test\n\nTo test httpbeat, run the following command:\n\n```\nmake testsuite\n```\n\nalternatively:\n```\nmake unit-tests\nmake system-tests\nmake integration-tests\nmake coverage-report\n```\n\nThe test coverage is reported in the folder `./build/coverage/`\n\n### Update\n\nEach beat has a template for the mapping in elasticsearch and a documentation for the fields\nwhich is automatically generated based on `etc/fields.yml`.\nTo generate etc/httpbeat.template.json and etc/httpbeat.asciidoc\n\n```\nmake update\n```\n\n### Cleanup\n\nTo clean httpbeat source code, run the following commands:\n\n```\nmake check\nmake simplify\n```\n\nTo clean up the build directory and generated artifacts, run:\n\n```\nmake clean\n```\n\n\n### Clone\n\nTo clone httpbeat from the git repository, run the following commands:\n\n```\nmkdir -p ${GOPATH}/github.com/christiangalsterer\ncd ${GOPATH}/github.com/christiangalsterer\ngit clone https://github.com/christiangalsterer/httpbeat\n```\n\n\nFor further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html).\n\n\n## Packaging\n\nThe beat frameworks provides tools to crosscompile and package your beat for different platforms. This requires [docker](https://www.docker.com/) and vendoring as described above. To build packages of your beat, run the following command:\n\n```\nmake package\n```\n\nThis will fetch and create all images required for the build process. The hole process to finish can take several minutes.\n\n# Releases and Downloads\n\n## 4.1.0 (2017-11-01) [Download](https://github.com/christiangalsterer/httpbeat/releases/tag/4.1.0)\n[![Github Releases (by Release)](https://img.shields.io/github/downloads/christiangalsterer/httpbeat/4.1.0/total.svg)](https://github.com/christiangalsterer/httpbeat/releases/tag/4.1.0)\n\nFeature and Bugfix release containing the following changes:\n* Update to beats v5.6.3\n* [Support returning HTTP status phrase](https://github.com/christiangalsterer/httpbeat/issues/27)\n* Fix: Set correct version in package names and package metadata\n* Fix: [HTTP Status code in event doesn't match documentation](https://github.com/christiangalsterer/httpbeat/issues/26)\n* Fix: [Making the template work for kibana - Use `object` instead of `nested`](https://github.com/christiangalsterer/httpbeat/issues/25)\n\n## 4.0.0 (2017-02-19) [Download](https://github.com/christiangalsterer/httpbeat/releases/tag/4.0.0)\n[![Github Releases (by Release)](https://img.shields.io/github/downloads/christiangalsterer/httpbeat/4.0.0/total.svg)](https://github.com/christiangalsterer/httpbeat/releases/tag/4.0.0)\n\nFeature and bugfix release containing the following **breaking** changes:\n* Renamed configuration parameter `urls` to `hosts`. Please update your configuration accordingly.\n* Renamed configuration parameter `cron` to `schedule`. Please update your configuration accordingly.\n* Update to beats v5.2.1\n* Fix: Default schedule not working\n\n## 3.3.0 (2017-02-04) [Download](https://github.com/christiangalsterer/httpbeat/releases/tag/3.3.0)\n[![Github Releases (by Release)](https://img.shields.io/github/downloads/christiangalsterer/httpbeat/3.3.0/total.svg)](https://github.com/christiangalsterer/httpbeat/releases/tag/3.3.0)\n\n\nFeature release containing the following changes:\n* Update to beats v5.2.0\n* Allow to specify the replacement character for `json_dot_mode : replace`, default replacement character is `_`\n\n## 3.2.0 (2017-01-14) [Download](https://github.com/christiangalsterer/httpbeat/releases/tag/3.2.0)\n[![Github Releases (by Release)](https://img.shields.io/github/downloads/christiangalsterer/httpbeat/3.2.0/total.svg)](https://github.com/christiangalsterer/httpbeat/releases/tag/3.2.0)\n\n\nFeature and bugfix release containing the following changes:\n* Various documentation improvements\n* Update to beats v5.1.2\n* Update to Go 1.7.4\n* Fix: [Panic when using TLS configuration (Go 1.7.4)](https://github.com/christiangalsterer/httpbeat/pull/15)\n* Fix: [Proper support for ssl.verification_mode](https://github.com/christiangalsterer/httpbeat/issues/14)\n\n## 3.1.1 (2017-01-07) [Download](https://github.com/christiangalsterer/httpbeat/releases/tag/3.1.1)\n[![Github Releases (by Release)](https://img.shields.io/github/downloads/christiangalsterer/httpbeat/3.1.1/total.svg)](https://github.com/christiangalsterer/httpbeat/releases/tag/3.1.1)\n\nStarting with this release pre-compiled binaries for different operating systems are available under the respective tag in the github project.\n\nBugfix release containing the following changes:\n* Move files into correct place to allow correct bulding with `make package`\n* Move files into correct place to allow correct bulding with `make update`\n* Cleanup of documentation\n\n## 3.1.0 (2016-12-23)\n\nFeature release containing the following changes:\n* Update to beats v5.1.1\n\n## 3.0.0 (2016-12-18)\n\nFeature release containing the following changes:\n* [Output format of response body is now defined via output_format parameter](https://github.com/christiangalsterer/httpbeat/issues/8). Default is 'string'\n\nBugfix release containing the following changes:\n* [Missing es2x template](https://github.com/christiangalsterer/httpbeat/issues/13)\n* [Correct parsing of large numbers in JSON output](https://github.com/christiangalsterer/httpbeat/issues/12)\n\n## 2.0.0 (2016-11-26)\n\nFeature release containing the following changes:\n* Update to beats v5.0.1\n\nPlease note that this release contains the following breaking changes introduced by beats 5.0.X, see also [Beats Changelog](https://github.com/elastic/beats/blob/v5.0.0-beta1/CHANGELOG.asciidoc)\n* SSL Configuration\n    * rename tls configurations section to ssl\n    * rename certificate_key configuration to key.\n    * replace tls.insecure with ssl.verification_mode setting.\n    * replace tls.min/max_version with ssl.supported_protocols setting requiring full protocol name\n\n## 1.2.0 (2016-07-19)\n\nFeature release containing the following changes:\n* Update to Go 1.6\n* Update to libbeat 1.2.3\n* Use [Glide](https://github.com/Masterminds/glide) for dependency management\n\n## 1.1.0 (2016-04-06)\n\nFeature release containing the following changes:\n* [Provide output directly as JSON](https://github.com/christiangalsterer/httpbeat/issues/2)\n\n## 1.0.1 (2016-02-17)\n\nBugfix release containing the following changes:\n* Fix: [Infinite loop when using logstash output](https://github.com/christiangalsterer/httpbeat/issues/4)\n* Fix: [Hanging during shutdown](https://github.com/christiangalsterer/httpbeat/issues/5)\n\n## 1.0.0 (2015-12-29)\n* Initial release\n\n# Configuration\n\n## Configuration Options\n\nSee [here](docs/configuration.asciidoc) for more information.\n\n## Exported Document Types\n\nThere is exactly one document type exported:\n\n- `type: httpbeat` http request and response information\n\n## Exported Fields\n\nSee [here](docs/fields.asciidoc) for a detailed description of all exported fields.\n\n### httpbeat type\n\n\u003cpre\u003e\n{\n  \"_index\": \"httpbeat-2015.12.05\",\n  \"_type\": \"httpbeat\",\n  \"_source\": {\n    \"@timestamp\": \"2015-12-05T11:16:13.070Z\",\n    \"beat\": {\n      \"hostname\": \"mbp.box\",\n      \"name\": \"mbp.box\"\n    },\n    \"count\": 1,\n    \"fields\": {\n      \"host\": \"test\"\n    },\n    \"request\": {\n      \"url\": \"http://httpbin.org/headers\",\n      \"method\": \"get\",\n      \"headers\": {\n        \"Accept\": \"application/json\",\n        \"Foo\": \"bar\"\n      }\n    },\n    \"response\": {\n      \"statusCode\": 200,\n      \"headers\": {\n        \"Access-Control-Allow-Credentials\": \"true\",\n        \"Access-Control-Allow-Origin\": \"*\",\n        \"Connection\": \"keep-alive\",\n        \"Content-Length\": \"220\",\n        \"Content-Type\": \"application/json\",\n        \"Date\": \"Sat, 05 Dec 2015 11:16:13 GMT\",\n        \"Server\": \"nginx\"\n      },\n      \"body\": \"{\\n  \\\"headers\\\": {\\n    \\\"Accept\\\": \\\"application/json\\\", \\n    \\\"Accept-Encoding\\\": \\\"gzip\\\", \\n    \\\"Authorization\\\": \\\"Basic Zm9vOmJhcg==\\\", \\n    \\\"Foo\\\": \\\"bar\\\", \\n    \\\"Host\\\": \\\"httpbin.org\\\", \\n    \\\"User-Agent\\\": \\\"Go-http-client/1.1\\\"\\n  }\\n}\\n\"\n    },\n    \"type\": \"httpbeat\"\n  },\n  \"fields\": {\n    \"timestamp\": [\n      1449314173\n    ]\n  },\n  \"sort\": [\n    1449314173\n  ]\n}\n\u003c/pre\u003e\n\n\n## Elasticsearch Template\n\nTo apply the Httpbeat template:\n```\ncurl -XPUT 'http://localhost:9200/_template/httpbeat' -d@httpbeat.template.json\n```\n\n# Contribution\nAll sorts of contributions are welcome. Please create a pull request and/or issue.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchristiangalsterer%2Fhttpbeat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchristiangalsterer%2Fhttpbeat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchristiangalsterer%2Fhttpbeat/lists"}