https://github.com/xrstf/logstats

primitive tool to gather statistics from nginx logfiles
https://github.com/xrstf/logstats

go log monitoring nginx statistics

Last synced: 2 months ago
JSON representation

primitive tool to gather statistics from nginx logfiles

• Host: GitHub
• URL: https://github.com/xrstf/logstats
• Owner: xrstf
• Created: 2018-04-16T06:40:26.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2018-04-21T18:14:51.000Z (about 8 years ago)
• Last Synced: 2026-03-12T04:56:02.054Z (3 months ago)
• Topics: go, log, monitoring, nginx, statistics
• Language: Go
• Size: 14.6 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# logstats - A Simple Logfile Analyzer

This project implements a very basic parser for nginx access logs, which counts

the number of requests and sent bytes and things like that.

## Why?

I'm using this to monitor some of my web projects. I have a cronjob running this

program once every minute, letting it count how many requests hit my project and

what HTTP status codes I generated. The result is then consumed by a script that

pumps the data into InfluxDB.

It's useful for projects that have no telemetry built into them, but are proxied

behind a reverse proxy.

## How?

1. Clone the repository

2. Run `make deps` to fetch the required dependencies.

3. To build the app, run `make app`. For a statically compiled version, run

   `make static`.

3. Run it.

To run, copy the `config.dist.yaml`, adjust to your needs and fire away:

    $ ./logstats myconfig.yaml access.log

    2018/04/16 15:14:23 Range start: 2018-04-16 05:14:23

    2018/04/16 15:14:23 Range end:   2018-04-16 15:14:23

    {

      "hits": {

        "fonts": 0,

        "images": 0,

        "media": 0,

        "robots": 9450,

        "styles": 0,

        "total": 669375

      },

      "traffic": {

        "fonts": 0,

        "images": 0,

        "media": 0,

        "robots": 982800,

        "styles": 0,

        "total": 7976381175

      },

      "status": {

        "200": 612675,

        "301": 44100,

        "304": 12600

      },

      "methods": {

        "GET": 669375

      },

      "protocols": {

        "HTTP/1.0": 582750,

        "HTTP/1.1": 81900,

        "HTTP/2.0": 4725

      },

      "uniqueIPs": 28

    }

Make sure to pipe the log output (stderr) away to only process the resulting

JSON on stdout.

## Caveats

If you're using a custom nginx log format, this will not work for you without

adjusting the code base.

The code is probably not the most efficient. Make sure to use the `read` parameter

in the config file accordingly. Reading a 500MB log file is somewhere around

20 seconds, but is relatively memory efficient (using less than 10 MB).

## License

WTFPL