{"id":18974138,"url":"https://github.com/eht16/pylogbeat","last_synced_at":"2025-11-24T03:04:50.299Z","repository":{"id":60722583,"uuid":"163671150","full_name":"eht16/pylogbeat","owner":"eht16","description":"Simple, incomplete implementation of the Beats protocol used by Elastic Beats and Logstash","archived":false,"fork":false,"pushed_at":"2023-07-30T14:28:00.000Z","size":51,"stargazers_count":17,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-09T00:38:13.897Z","etag":null,"topics":["beats","logging","logstash","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eht16.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-31T13:26:46.000Z","updated_at":"2024-04-16T18:05:40.000Z","dependencies_parsed_at":"2024-06-19T02:56:50.113Z","dependency_job_id":null,"html_url":"https://github.com/eht16/pylogbeat","commit_stats":{"total_commits":30,"total_committers":1,"mean_commits":30.0,"dds":0.0,"last_synced_commit":"60d89e47e9af62269d7acba9ffffa8b7d13f7ecc"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eht16%2Fpylogbeat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eht16%2Fpylogbeat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eht16%2Fpylogbeat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eht16%2Fpylogbeat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eht16","download_url":"https://codeload.github.com/eht16/pylogbeat/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249011703,"owners_count":21198077,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beats","logging","logstash","python"],"created_at":"2024-11-08T15:14:13.680Z","updated_at":"2025-11-24T03:04:50.289Z","avatar_url":"https://github.com/eht16.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"PyLogBeat\n=========\n\n[![CI Tests](https://github.com/eht16/pylogbeat/actions/workflows/tests.yml/badge.svg)](https://github.com/eht16/pylogbeat/actions/workflows/tests.yml)\n[![PyPI](https://img.shields.io/pypi/v/pylogbeat.svg)](https://pypi.org/project/pylogbeat/)\n[![Python Versions](https://img.shields.io/pypi/pyversions/pylogbeat.svg)](https://pypi.org/project/pylogbeat/)\n[![License](https://img.shields.io/pypi/l/pylogbeat.svg)](https://pypi.org/project/pylogbeat/)\n\nPyLogBeat is a simple, incomplete implementation of the Beats protocol\nused by Elastic Beats and Logstash. For more information about Beats see\nhttps://www.elastic.co/products/beats and\nhttps://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html.\n\nWith this library it is possible to send log messages or any data to\nLogstash' beats input plugin or any other service which implements\nthe Beats protocol.\n\nThe main difference to other transport mechanisms like direct TCP\nor UDP transfer is that with the Beats protocol there is a higher\nreliability of the data transfer, especially since the server\nacknowledges the data it received so the client knows whether and\nwhat to resend.\n\n\nInstallation\n------------\n\nThe easiest method is to install directly from pypi using pip:\n\n    pip install pylogbeat\n\n\nIf you prefer, you can download PyLogBeat from\nhttps://github.com/eht16/pylogbeat and install it directly from source:\n\n    python setup.py install\n\n\nDue to a bug (\u003chttps://github.com/logstash-plugins/logstash-input-beats/pull/342\u003e)\nin some Logstash versions, you will need Logstash \u003e= 5.6.12 or Logstash \u003e= 6.4.0.\n\n\nGet the Source\n--------------\n\nThe source code is available at https://github.com/eht16/pylogbeat/.\n\n\nUsage\n-----\n\n### Simple use\n\n```python\n    message = {'@timestamp': '2018-01-02T01:02:03',  '@version': '1', 'message': 'hello world'}\n    client = PyLogBeatClient('localhost', 5959, ssl_enable=False)\n    client.connect()\n    client.send([message])\n    client.close()\n```\n\n### Using a context manager\n\n```python\n    with PyLogBeatClient('localhost', 5959, ssl_enable=False) as client:\n        client.send([message])\n```\n\n### Using a SSL connection\n\n```python\n    with PyLogBeatClient('localhost', 5959, ssl_enable=True, ssl_verify=True,\n            keyfile='certificate.key', certfile='certificate.crt', ca_certs='ca.crt') as client:\n        client.send([message])\n```\n\nFor details regarding the SSL certificates and how to configure the\nLogstash input for SSL, see\nhttps://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html.\n\n\nMessage Format\n--------------\n\n`PyLogBeatClient.send()` accepts a sequence (list, tuple, set) of \"messages\".\nThe messages itself can either be a `dict` object representing the final\nmessage to be sent to Logstash or a `bytes` or `string` object which must\ncontain properly formatted `JSON`.\nIf a `dict` is passed as element, it is converted to `JSON` using\n`json.dumps()`.\n\n### Example message\n\nThe following example is a message as `JSON`:\n\n```python\n    {\n        \"@timestamp\": \"2018-01-02T01:02:03\",\n        \"@version\": \"1\",\n        \"extra\": {\n            \"application\": \"django_example\",\n            \"django_version\": \"2.1.0\",\n            \"environment\": \"production\"\n        },\n        \"host\": \"my-local-host\",\n        \"level\": \"INFO\",\n        \"logsource\": \"my-local-host\",\n        \"message\": \"foo bar\",\n        \"pid\": 65534,\n        \"program\": \"example.py\",\n        \"type\": \"python-logstash\"\n    }\n```\n\nThis is the standard Logstash message format in JSON.\n\n\nLogging\n-------\n\nPyLogBeat uses a logger named \"pylogbeat\" to log some debug messages\nand warnings in case of errors. By default, the logger's log level\nis set to `Warning` so you will not see any debug log messages.\nIf necessary simply change the log level of the logger to see the debug\nmessages. For example:\n\n```python\n    import logging\n    logging.getLogger('pylogbeat').setLevel(logging.DEBUG)\n```\n\nIt is important to make this change *after* you imported\nthe `pylogbeat` module.\n\nFurthermore, PyLogBeatClient's constructor method takes a `use_logging`\nargument which should be a boolean indicating whether the logging\nsubsystem should be used at all. The argument defaults to `False`,\ni.e. if you want any logging, you need to pass `True`.\nIf PyLogBeat is used itself as part of the logging system (e.g.\nas the transport of a handler), it is important to not emit any new\nlog messages once the logging subsystem has been shutdown or is in the\nprocess of shutting down. In this case, `use_logging` must be `False`\nin order to suppress generating log messages.\n\n\nProtocol Support\n----------------\n\nThe implemented Beats protocol is not yet officially specified and\ndocumented, unfortunately. Hopefully the Beats developers will\nprovide a specification in the future.\nSo far, sending the data and waiting for the ACK from the server is\nimplemented. But there might some details from the protocol missing\nin the implementation.\n\n\nFuture Maintenance\n------------------\n\nIf you are interested in the code, want to improve it and/or\ncomplete the protocol support, please feel free to send PRs.\nI would be happy if someone likes to continue developing this library\nand would also take full maintainership for future development and\nreleases.\n\n\nContributing\n------------\n\nFound a bug or got a feature request? Please report it at\nhttps://github.com/eht16/pylogbeat/issues.\n\n\nCredits\n-------\n\nThis code is based on https://github.com/brxie/PyLumberjack and\nadopted to support version 2 of the protocol.\nThanks to brxie for the initial code.\n\n\nChangeLog\n---------\n\n### 2.1.0 / 2025-11-23\n\n- Shutdown the TCP socket before closing it\n- Support setting \"ssl_verify_flags\" to override Python's default\n- Update test matrix to support Python 3.11 to 3.14, drop support\n  for older Python versions and require at least Python 3.11\n\n\n### 2.0.1 / 2023-07-30\n\n- Load certificate chain only if a certificate was specified\n\n\n### 2.0.0 / 2020-10-04\n\n- Remove \"six\" dependency\n- Require Python \u003e= 3.6\n\n\n### 1.0.5 / 2020-10-04\n\n- Set `python_requires` to Python \u003e=2.7 or Python \u003e= 3.6 for\n  smooth upgrade to upcoming Python3 only.\n  This way Python2 only users will stay at this release.\n\n\n### 1.0.4 / 2020-05-06\n\n- Validate input data and fix documentation about accepted values (#2)\n\n\n### 1.0.3 / 2020-04-22\n\n- Add note about required Logstash versions (#1)\n- Improve unit tests\n\n\n### 1.0.2 / 2018-12-31\n\n- Add badges to README\n\n\n### 1.0.1 / 2018-12-31\n\n- Fix typo in setup.py\n- Use distribution \"trusty\" for Travis builds\n\n\n### 1.0.0 / 2018-12-31\n\n- Initial release\n\n\nLicense\n-------\nPyLogBeat is licensed under the Apache License 2.0.\n\n\nAuthor\n------\n\nEnrico Tröger \u003cenrico.troeger@uvena.de\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feht16%2Fpylogbeat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feht16%2Fpylogbeat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feht16%2Fpylogbeat/lists"}