Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/getsentry/sentlog
CLI util for parsing arbitrary text data and sending error-like events to Sentry
https://github.com/getsentry/sentlog
go golang grok logging sentry tag-non-production
Last synced: about 5 hours ago
JSON representation
CLI util for parsing arbitrary text data and sending error-like events to Sentry
- Host: GitHub
- URL: https://github.com/getsentry/sentlog
- Owner: getsentry
- License: mit
- Created: 2019-06-25T08:35:25.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-08-27T02:57:08.000Z (about 1 year ago)
- Last Synced: 2024-08-02T12:23:52.500Z (3 months ago)
- Topics: go, golang, grok, logging, sentry, tag-non-production
- Language: Go
- Homepage:
- Size: 37.1 KB
- Stars: 17
- Watchers: 4
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# sentlog
## This is a Sentry Hackweek project! Development may stop anytime. You've been warned.
`sentlog` is a command-line tool that can read arbitrary text files (e.g., webserver or database logs), search for specific user-defined patterns, and report the findings to Sentry.
## Table of Contents
- [Introduction](#Introduction)
- [Downloads](#Downloads)
- [Command Line Arguments](#Command-Line-Arguments)
- [Example](#Example)
- [Configuration File](#Configuration-File)
- [About Patterns](#About-Patterns)
## Introduction
Sentry provides SDKs for a lot of different [platforms and frameworks](https://docs.sentry.io/). However, you might also want to use Sentry for parts of your infrastructure that were not developed by you, or don't have an integration with Sentry (yet): databases, web servers, and even operating system kernels. What do these tools have in common? They normally have some sort of output (i.e. logs), where both regular events and errors are usually logged. So why not parsing those logs and look for entries that look like errors? We can do that. And what platform do we usually use for error management? Sentry, of course!
And this is when `sentlog` steps in.
## Downloads
`sentlog` binaries can be downloaded from [GitHub releases](https://github.com/getsentry/sentlog/releases).
## Command Line Arguments
```sh
usage: sentlog [] []
Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
-p, --pattern=PATTERN Pattern to look for
--dry-run Dry-run mode
--no-follow Do not wait for the new data
--from-line=-1 Start reading from this line number
-c, --config=CONFIG Path to the configuration
-v, --verbose Print every match
Args:
[] File to parse
```
`sentlog` can operate in two modes:
1. Basic: filename and pattern are specified on the command line
2. Advanced: using the configuration file provided by `--config` argument
## Example
The following example shows how you can run `sentlog` in Basic mode.
```sh
export SENTLOG_SENTRY_DSN="https://[email protected]/YYY" # Your Sentry DSN
sentlog /var/log/postgresql/postgresql-9.6.log \
-p '^%{DATESTAMP:timestamp}.*FATAL:.*host "%{IP:host}", user "%{USERNAME:user}", database "%{WORD:database}"'
```
...will watch the PostgreSQL log (`/var/log/postgresql/postgresql-9.6.log`) for events that look like this:
```
2019-05-21 08:51:09 GMT [11212]: FATAL: no pg_hba.conf entry for host "123.123.123.123", user "postgres", database "testdb"
```
`sentlog` will extract the timestamp, IP address, username, and database from the entry, and will add them as tags to the Sentry event.
## Configuration File
```yaml
---
# Sentry DSN (also can be configured via environment)
sentry_dsn: https://[email protected]/YYY
# Additional Grok pattern files
pattern_files:
- ./patterns1.txt
- ../patterns2.txt
# List of files that we want to watch
inputs:
- file: /var/log/nginx/error.log
# Patterns to find and report
patterns:
- "%{NGINX_ERROR_LOG}"
# Additional tags that will be added to the Sentry event
tags:
pattern: nginx_error
custom: tag
```
## About Patterns
`sentlog` uses [Grok patterns](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_grok_basics) to match your data. A cool thing about Grok expressions is that they can be nested, which lets you to define complex matching expressions based on smaller building blocks ([example](https://github.com/vjeantet/grok/blob/5a86c829f3c347ec23dbd632af2db0d3508c11ce/patterns/grok-patterns)).
This Grok debugger can be quite helpful when preparing your Grok expressions: https://grokdebug.herokuapp.com/