https://github.com/tinybirdco/nginx-logs-analytics

https://github.com/tinybirdco/nginx-logs-analytics

Last synced: 10 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/tinybirdco/nginx-logs-analytics
• Owner: tinybirdco
• License: mit
• Created: 2021-05-06T13:10:02.000Z (about 5 years ago)
• Default Branch: main
• Last Pushed: 2021-05-07T17:33:28.000Z (about 5 years ago)
• Last Synced: 2025-02-27T05:25:24.231Z (over 1 year ago)
• Language: Shell
• Size: 82 KB
• Stars: 2
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
This project should be used along with tbtail to send and process Nginx logs on Tinybird.

Events are ingested to a Null Data Source (`nginx_raw`), then the info from the JSON is extracted to columns in `nginx_log`, where all the logs are saved with a TTL of 3 days. From there, materialized views with aggregations on 4 different time frames (monthly, daily, hourly and by 5 mins intervals) are created and saved in other Data Sources

![Diagram of what this project creates](/nginx_logs_diagram.png)

To replicate the project in your Tintybird account, run

```bash

git clone git@github.com:tinybirdco/nginx-logs-analytics.git # clones the data project

cd nginx-logs-analytics/tb_project sets the working directory to the root dir of the data project

tb auth # will ask for your admin token. You can get it from https://ui.tinybird.co/tokens. Will create a .tinyb file in the current folder

sh push_resources.sh # Creates the full data project on your Tinybird account (Data Sources, Materialized Views, pipes and endpoints)

```