https://github.com/centminmod/centminmod-nginx-json-logs
https://github.com/centminmod/centminmod-nginx-json-logs
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/centminmod/centminmod-nginx-json-logs
- Owner: centminmod
- Created: 2023-09-24T04:05:47.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2023-10-11T21:43:13.000Z (over 2 years ago)
- Last Synced: 2024-12-31T01:41:54.244Z (over 1 year ago)
- Language: Python
- Size: 13.7 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
Tools for parsing Centmin Mod Nginx JSON log format based logging as per https://community.centminmod.com/threads/how-to-configure-nginx-for-json-based-access-logging.19641/
* [bot-rate.py parser](#bot-ratepy-parser)
* [Centmin Mod Nginx JSON Logs](#centmin-mod-nginx-json-logs)
* [Nginx JSON Format Logs with Cloudflare Proxy](#nginx-json-format-logs-with-cloudflare-proxy)
# bot-rate.py parser
The `bot-rate.py` script will parse Centmin Mod Nginx JSON log fields for specific case-sensitive keyword i.e. `botname` and calculate the request rate for per second, minute, hour and day.
## Requirements
`bot-rate.py` by default is set to support Python 2.7 with script using
```
#!/usr/bin/env python2.7
```
If you're using Python 3, you can modify script to use
```
#!/usr/bin/env python3
```
`bot-rate.py` supports gzip and zstd compressed JSON logs too.
```
pip install zstandard
```
```
./bot-rate.py /home/nginx/domains/domain.com/log/access_log.json.gz botname
./bot-rate.py /home/nginx/domains/domain.com/log/access_log.json.zst botname
```
## Examples
```
./bot-rate.py
Usage: ./bot-rate.py [keyword] [field_name]
```
```
./bot-rate.py /home/nginx/domains/domain.com/log/access_log.json botname
```
Example filtering for `nmap` requests in `/home/nginx/domains/domain.com/log/access.json` Nginx JSON log.
```
./bot-rate.py /home/nginx/domains/domain.com/log/access.json nmap
Requests with keyword 'nmap':
Requests per second:
2023-09-30T22:16:45 (2) 2023-09-30T22:16:46 (2) 2023-09-30T23:59:59 (2)
2023-10-01T00:00:00 (2) 2023-10-01T06:18:43 (1) 2023-10-01T06:18:45 (1)
2023-10-01T10:43:56 (1) 2023-10-01T10:43:57 (2) 2023-10-01T12:09:02 (2)
2023-10-01T12:09:03 (2) 2023-10-01T16:33:04 (2) 2023-10-01T16:33:05 (2)
2023-10-01T18:12:36 (4) 2023-10-01T23:07:46 (4) 2023-10-01T23:14:57 (4)
2023-10-02T13:15:02 (3) 2023-10-02T13:15:03 (1) 2023-10-02T14:12:44 (2)
2023-10-02T14:12:45 (2) 2023-10-02T19:24:12 (2) 2023-10-02T19:24:13 (2)
2023-10-02T19:24:17 (3) 2023-10-02T19:24:18 (1) 2023-10-03T01:06:57 (1)
2023-10-03T01:06:58 (2) 2023-10-03T01:06:59 (1) 2023-10-03T02:04:14 (1)
2023-10-03T02:04:15 (3) 2023-10-03T12:49:30 (1) 2023-10-03T12:49:31 (3)
2023-10-03T19:52:17 (3) 2023-10-03T19:52:18 (1) 2023-10-03T19:56:34 (4)
2023-10-03T22:00:12 (4) 2023-10-03T22:14:23 (2) 2023-10-03T22:14:24 (1)
2023-10-03T22:14:25 (1) 2023-10-04T05:43:46 (2) 2023-10-04T05:43:47 (2)
2023-10-04T17:20:10 (2) 2023-10-04T17:20:11 (2) 2023-10-04T19:41:31 (4)
2023-10-04T21:18:55 (1) 2023-10-04T21:18:56 (3) 2023-10-04T21:26:16 (4)
2023-10-04T21:31:35 (4) 2023-10-05T06:19:56 (2) 2023-10-05T06:19:57 (2)
2023-10-05T18:10:58 (4) 2023-10-06T00:01:32 (4) 2023-10-06T00:02:28 (1)
2023-10-06T00:02:29 (3) 2023-10-06T00:03:54 (2) 2023-10-06T00:03:55 (2)
2023-10-06T00:09:39 (2) 2023-10-06T00:09:40 (2) 2023-10-06T09:46:45 (4)
2023-10-06T18:26:09 (2) 2023-10-06T18:26:10 (2) 2023-10-06T23:32:02 (3)
2023-10-06T23:32:03 (1) 2023-10-06T23:36:28 (2) 2023-10-06T23:36:29 (2)
2023-10-07T03:46:15 (1) 2023-10-07T03:46:16 (3) 2023-10-07T03:51:05 (1)
2023-10-07T03:51:06 (1) 2023-10-07T03:51:07 (2) 2023-10-07T12:04:00 (4)
2023-10-08T00:33:49 (2) 2023-10-08T00:33:50 (2) 2023-10-08T02:57:46 (3)
2023-10-08T02:57:47 (1) 2023-10-08T03:00:50 (3) 2023-10-08T03:00:51 (1)
2023-10-08T09:06:37 (4) 2023-10-08T09:07:29 (2) 2023-10-08T09:07:30 (2)
2023-10-08T13:25:04 (1) 2023-10-08T13:25:05 (3) 2023-10-09T01:29:29 (4)
2023-10-09T06:58:03 (4) 2023-10-09T08:02:11 (4) 2023-10-09T08:06:37 (2)
2023-10-09T08:06:38 (2) 2023-10-09T08:07:03 (1) 2023-10-09T08:07:04 (3)
2023-10-09T14:36:06 (2) 2023-10-09T14:36:07 (2) 2023-10-09T20:36:25 (1)
2023-10-10T00:30:12 (4) 2023-10-10T04:08:13 (4) 2023-10-10T09:28:15 (1)
2023-10-10T09:28:16 (3) 2023-10-10T10:42:52 (4) 2023-10-10T10:47:06 (3)
2023-10-10T10:47:07 (1) 2023-10-10T13:18:19 (2) 2023-10-10T13:18:20 (2)
2023-10-11T00:15:12 (2) 2023-10-11T00:15:13 (2) 2023-10-11T03:49:26 (1)
2023-10-11T03:49:27 (3) 2023-10-11T10:03:11 (4) 2023-10-11T12:59:05 (1)
2023-10-11T19:53:34 (1) 2023-10-11T19:53:35 (3) 2023-10-11T19:58:21 (4)
2023-10-11T20:11:13 (4) 2023-10-11T20:16:58 (4)
Requests per minute:
2023-09-30T22:16 (4) 2023-09-30T23:59 (2) 2023-10-01T00:00 (2)
2023-10-01T06:18 (2) 2023-10-01T10:43 (3) 2023-10-01T12:09 (4)
2023-10-01T16:33 (4) 2023-10-01T18:12 (4) 2023-10-01T23:07 (4)
2023-10-01T23:14 (4) 2023-10-02T13:15 (4) 2023-10-02T14:12 (4)
2023-10-02T19:24 (8) 2023-10-03T01:06 (4) 2023-10-03T02:04 (4)
2023-10-03T12:49 (4) 2023-10-03T19:52 (4) 2023-10-03T19:56 (4)
2023-10-03T22:00 (4) 2023-10-03T22:14 (4) 2023-10-04T05:43 (4)
2023-10-04T17:20 (4) 2023-10-04T19:41 (4) 2023-10-04T21:18 (4)
2023-10-04T21:26 (4) 2023-10-04T21:31 (4) 2023-10-05T06:19 (4)
2023-10-05T18:10 (4) 2023-10-06T00:01 (4) 2023-10-06T00:02 (4)
2023-10-06T00:03 (4) 2023-10-06T00:09 (4) 2023-10-06T09:46 (4)
2023-10-06T18:26 (4) 2023-10-06T23:32 (4) 2023-10-06T23:36 (4)
2023-10-07T03:46 (4) 2023-10-07T03:51 (4) 2023-10-07T12:04 (4)
2023-10-08T00:33 (4) 2023-10-08T02:57 (4) 2023-10-08T03:00 (4)
2023-10-08T09:06 (4) 2023-10-08T09:07 (4) 2023-10-08T13:25 (4)
2023-10-09T01:29 (4) 2023-10-09T06:58 (4) 2023-10-09T08:02 (4)
2023-10-09T08:06 (4) 2023-10-09T08:07 (4) 2023-10-09T14:36 (4)
2023-10-09T20:36 (1) 2023-10-10T00:30 (4) 2023-10-10T04:08 (4)
2023-10-10T09:28 (4) 2023-10-10T10:42 (4) 2023-10-10T10:47 (4)
2023-10-10T13:18 (4) 2023-10-11T00:15 (4) 2023-10-11T03:49 (4)
2023-10-11T10:03 (4) 2023-10-11T12:59 (1) 2023-10-11T19:53 (4)
2023-10-11T19:58 (4) 2023-10-11T20:11 (4) 2023-10-11T20:16 (4)
Requests per hour:
2023-09-30T22 (4) 2023-09-30T23 (2) 2023-10-01T00 (2)
2023-10-01T06 (2) 2023-10-01T10 (3) 2023-10-01T12 (4)
2023-10-01T16 (4) 2023-10-01T18 (4) 2023-10-01T23 (8)
2023-10-02T13 (4) 2023-10-02T14 (4) 2023-10-02T19 (8)
2023-10-03T01 (4) 2023-10-03T02 (4) 2023-10-03T12 (4)
2023-10-03T19 (8) 2023-10-03T22 (8) 2023-10-04T05 (4)
2023-10-04T17 (4) 2023-10-04T19 (4) 2023-10-04T21 (12)
2023-10-05T06 (4) 2023-10-05T18 (4) 2023-10-06T00 (16)
2023-10-06T09 (4) 2023-10-06T18 (4) 2023-10-06T23 (8)
2023-10-07T03 (8) 2023-10-07T12 (4) 2023-10-08T00 (4)
2023-10-08T02 (4) 2023-10-08T03 (4) 2023-10-08T09 (8)
2023-10-08T13 (4) 2023-10-09T01 (4) 2023-10-09T06 (4)
2023-10-09T08 (12) 2023-10-09T14 (4) 2023-10-09T20 (1)
2023-10-10T00 (4) 2023-10-10T04 (4) 2023-10-10T09 (4)
2023-10-10T10 (8) 2023-10-10T13 (4) 2023-10-11T00 (4)
2023-10-11T03 (4) 2023-10-11T10 (4) 2023-10-11T12 (1)
2023-10-11T19 (8) 2023-10-11T20 (8)
Requests per day:
2023-09-30 (6) 2023-10-01 (27) 2023-10-02 (16)
2023-10-03 (28) 2023-10-04 (24) 2023-10-05 (8)
2023-10-06 (32) 2023-10-07 (12) 2023-10-08 (24)
2023-10-09 (25) 2023-10-10 (24) 2023-10-11 (29)
```
Example filtering for `Mozilla` requests in `/home/nginx/domains/domain.com/log/access.json` Nginx JSON log.
```
./bot-rate.py /home/nginx/domains/domain.com/log/access.json Mozilla
Requests with keyword 'Mozilla':
Requests per second:
2023-09-30T21:54:53 (1) 2023-09-30T22:02:05 (1) 2023-09-30T22:16:45 (2)
2023-09-30T22:16:46 (2) 2023-09-30T23:29:31 (2) 2023-09-30T23:55:04 (2)
2023-09-30T23:59:59 (2) 2023-10-01T00:00:00 (2) 2023-10-01T00:53:41 (1)
2023-10-01T01:09:39 (1) 2023-10-01T04:12:17 (1) 2023-10-01T04:18:18 (1)
2023-10-01T06:18:43 (6) 2023-10-01T06:18:44 (6) 2023-10-01T06:18:45 (4)
2023-10-01T06:18:46 (4) 2023-10-01T06:18:47 (2) 2023-10-01T06:18:48 (2)
2023-10-01T06:18:49 (2) 2023-10-01T06:18:50 (1) 2023-10-01T06:18:51 (1)
2023-10-01T06:18:52 (1) 2023-10-01T06:20:15 (2) 2023-10-01T06:20:16 (1)
2023-10-01T09:59:31 (1) 2023-10-01T10:15:26 (1) 2023-10-01T10:43:56 (1)
2023-10-01T10:43:57 (2) 2023-10-01T11:27:37 (1) 2023-10-01T12:09:02 (2)
2023-10-01T12:09:03 (2) 2023-10-01T12:14:59 (1) 2023-10-01T12:18:19 (1)
2023-10-01T15:03:55 (1) 2023-10-01T15:15:15 (1) 2023-10-01T16:11:28 (1)
2023-10-01T16:33:04 (2) 2023-10-01T16:33:05 (2) 2023-10-01T17:17:30 (2)
2023-10-01T18:12:36 (4) 2023-10-01T22:52:19 (1) 2023-10-01T23:07:46 (4)
2023-10-01T23:08:15 (1) 2023-10-01T23:14:57 (4) 2023-10-01T23:31:04 (2)
2023-10-01T23:31:05 (1) 2023-10-01T23:35:01 (1) 2023-10-01T23:39:17 (1)
2023-10-01T23:40:24 (1) 2023-10-01T23:45:31 (1) 2023-10-01T23:45:33 (1)
2023-10-01T23:45:35 (1) 2023-10-01T23:49:40 (1) 2023-10-01T23:50:48 (1)
2023-10-01T23:53:53 (1) 2023-10-01T23:55:26 (1) 2023-10-01T23:59:51 (1)
2023-10-02T00:05:15 (1) 2023-10-02T00:08:29 (1) 2023-10-02T00:17:27 (1)
2023-10-02T03:00:36 (1) 2023-10-02T03:01:06 (1) 2023-10-02T03:01:12 (1)
2023-10-02T03:08:31 (1) 2023-10-02T04:23:01 (1) 2023-10-02T04:41:58 (1)
2023-10-02T05:17:26 (1) 2023-10-02T08:18:48 (1) 2023-10-02T08:18:52 (1)
2023-10-02T11:49:00 (1) 2023-10-02T11:53:58 (1) 2023-10-02T13:15:02 (3)
2023-10-02T13:15:03 (1) 2023-10-02T14:12:44 (2) 2023-10-02T14:12:45 (2)
2023-10-02T17:18:51 (1) 2023-10-02T18:29:16 (1) 2023-10-02T18:33:28 (1)
2023-10-02T18:45:11 (1) 2023-10-02T18:45:12 (1) 2023-10-02T19:14:53 (1)
2023-10-02T19:14:55 (1) 2023-10-02T19:24:12 (2) 2023-10-02T19:24:13 (2)
2023-10-02T19:24:17 (3) 2023-10-02T19:24:18 (1) 2023-10-02T20:17:01 (2)
2023-10-02T20:17:02 (1) 2023-10-02T20:43:01 (1) 2023-10-02T20:44:26 (1)
2023-10-02T20:44:28 (1) 2023-10-02T21:34:38 (1) 2023-10-02T21:35:20 (1)
2023-10-02T21:35:56 (1) 2023-10-02T22:08:32 (1) 2023-10-02T22:09:23 (1)
2023-10-02T22:09:24 (1) 2023-10-02T22:40:55 (1) 2023-10-02T23:23:45 (1)
2023-10-02T23:33:21 (1) 2023-10-02T23:33:22 (3) 2023-10-02T23:36:36 (1)
2023-10-03T00:11:08 (1) 2023-10-03T00:39:48 (1) 2023-10-03T00:40:50 (1)
2023-10-03T00:57:51 (1) 2023-10-03T01:03:49 (1) 2023-10-03T01:03:51 (1)
2023-10-03T01:06:57 (1) 2023-10-03T01:06:58 (2) 2023-10-03T01:06:59 (1)
2023-10-03T01:20:29 (1) 2023-10-03T01:32:24 (1) 2023-10-03T01:37:52 (1)
2023-10-03T02:04:14 (1) 2023-10-03T02:04:15 (3) 2023-10-03T02:40:47 (1)
2023-10-03T02:40:48 (2) 2023-10-03T03:42:21 (1) 2023-10-03T04:40:21 (1)
2023-10-03T07:40:27 (1) 2023-10-03T12:49:30 (1) 2023-10-03T12:49:31 (3)
2023-10-03T14:13:46 (1) 2023-10-03T15:47:40 (1) 2023-10-03T17:08:40 (1)
2023-10-03T18:24:27 (1) 2023-10-03T18:28:49 (2) 2023-10-03T19:09:30 (1)
2023-10-03T19:52:17 (3) 2023-10-03T19:52:18 (1) 2023-10-03T19:56:34 (4)
2023-10-03T21:29:44 (1) 2023-10-03T21:29:45 (1) 2023-10-03T22:00:12 (4)
2023-10-03T22:05:15 (1) 2023-10-03T22:14:23 (2) 2023-10-03T22:14:24 (1)
2023-10-03T22:14:25 (1) 2023-10-04T03:04:13 (1) 2023-10-04T05:43:46 (2)
2023-10-04T05:43:47 (2) 2023-10-04T06:08:21 (1) 2023-10-04T06:45:01 (1)
2023-10-04T06:50:48 (1) 2023-10-04T09:58:30 (1) 2023-10-04T11:27:13 (1)
2023-10-04T11:45:24 (1) 2023-10-04T11:48:14 (2) 2023-10-04T12:38:37 (1)
2023-10-04T12:38:38 (1) 2023-10-04T16:46:57 (1) 2023-10-04T16:47:00 (1)
2023-10-04T16:47:02 (1) 2023-10-04T17:20:10 (2) 2023-10-04T17:20:11 (2)
2023-10-04T18:50:23 (1) 2023-10-04T19:41:31 (4) 2023-10-04T20:07:36 (1)
2023-10-04T21:18:55 (1) 2023-10-04T21:18:56 (3) 2023-10-04T21:26:16 (4)
2023-10-04T21:31:35 (4) 2023-10-04T22:03:51 (1) 2023-10-05T01:09:03 (1)
2023-10-05T02:02:52 (1) 2023-10-05T06:19:56 (2) 2023-10-05T06:19:57 (2)
2023-10-05T06:43:34 (1) 2023-10-05T06:50:13 (1) 2023-10-05T08:38:54 (1)
2023-10-05T09:16:30 (1) 2023-10-05T10:05:11 (1) 2023-10-05T11:04:24 (1)
2023-10-05T11:20:29 (4) 2023-10-05T13:43:16 (1) 2023-10-05T15:06:08 (3)
2023-10-05T15:15:14 (2) 2023-10-05T15:15:53 (1) 2023-10-05T18:10:58 (4)
2023-10-05T18:38:59 (1) 2023-10-05T19:43:24 (1) 2023-10-05T19:50:16 (1)
2023-10-05T20:38:51 (1) 2023-10-06T00:01:32 (4) 2023-10-06T00:02:28 (1)
2023-10-06T00:02:29 (3) 2023-10-06T00:03:54 (2) 2023-10-06T00:03:55 (2)
2023-10-06T00:09:39 (2) 2023-10-06T00:09:40 (2) 2023-10-06T05:47:46 (1)
2023-10-06T06:43:53 (1) 2023-10-06T06:49:34 (1) 2023-10-06T07:35:55 (1)
2023-10-06T07:51:34 (1) 2023-10-06T07:57:52 (1) 2023-10-06T07:57:54 (2)
2023-10-06T09:46:45 (4) 2023-10-06T10:51:12 (1) 2023-10-06T11:59:49 (1)
2023-10-06T11:59:51 (1) 2023-10-06T12:47:42 (1) 2023-10-06T13:33:29 (1)
2023-10-06T13:42:36 (1) 2023-10-06T18:26:09 (2) 2023-10-06T18:26:10 (2)
2023-10-06T19:55:36 (1) 2023-10-06T21:19:54 (1) 2023-10-06T22:34:49 (1)
2023-10-06T22:51:47 (1) 2023-10-06T23:32:02 (3) 2023-10-06T23:32:03 (1)
2023-10-06T23:36:28 (2) 2023-10-06T23:36:29 (2) 2023-10-07T00:15:41 (1)
2023-10-07T00:40:19 (1) 2023-10-07T00:40:52 (1) 2023-10-07T01:16:43 (1)
2023-10-07T03:46:15 (1) 2023-10-07T03:46:16 (3) 2023-10-07T03:51:05 (1)
2023-10-07T03:51:06 (1) 2023-10-07T03:51:07 (2) 2023-10-07T04:13:39 (1)
2023-10-07T04:13:41 (1) 2023-10-07T06:44:05 (1) 2023-10-07T06:49:47 (1)
2023-10-07T09:03:39 (1) 2023-10-07T12:04:00 (4) 2023-10-07T12:42:04 (1)
2023-10-07T12:42:08 (1) 2023-10-07T12:42:15 (1) 2023-10-07T21:30:16 (1)
2023-10-07T21:30:17 (1) 2023-10-07T21:30:49 (1) 2023-10-07T21:46:47 (1)
2023-10-07T23:02:46 (1) 2023-10-08T00:27:27 (1) 2023-10-08T00:33:49 (2)
2023-10-08T00:33:50 (2) 2023-10-08T02:57:46 (3) 2023-10-08T02:57:47 (1)
2023-10-08T03:00:50 (3) 2023-10-08T03:00:51 (1) 2023-10-08T06:33:17 (1)
2023-10-08T06:51:37 (1) 2023-10-08T06:58:17 (1) 2023-10-08T09:06:37 (4)
2023-10-08T09:07:29 (2) 2023-10-08T09:07:30 (2) 2023-10-08T10:19:38 (1)
2023-10-08T10:19:39 (1) 2023-10-08T10:57:40 (1) 2023-10-08T13:25:04 (1)
2023-10-08T13:25:05 (3) 2023-10-08T13:36:02 (1) 2023-10-08T13:36:03 (2)
2023-10-08T13:36:05 (2) 2023-10-08T14:46:27 (9) 2023-10-08T14:46:28 (7)
2023-10-08T14:46:30 (1) 2023-10-08T14:47:26 (1) 2023-10-08T14:48:40 (1)
2023-10-08T15:15:10 (1) 2023-10-08T18:12:36 (1) 2023-10-08T20:58:11 (1)
2023-10-08T20:58:13 (1) 2023-10-09T01:29:29 (4) 2023-10-09T01:37:58 (1)
2023-10-09T01:54:08 (1) 2023-10-09T02:35:23 (1) 2023-10-09T04:26:06 (1)
2023-10-09T06:58:03 (4) 2023-10-09T08:02:11 (4) 2023-10-09T08:06:37 (2)
2023-10-09T08:06:38 (2) 2023-10-09T08:07:03 (1) 2023-10-09T08:07:04 (3)
2023-10-09T11:46:33 (1) 2023-10-09T11:53:25 (1) 2023-10-09T14:36:06 (2)
2023-10-09T14:36:07 (2) 2023-10-09T15:18:15 (2) 2023-10-09T15:35:28 (1)
2023-10-09T23:15:50 (2) 2023-10-10T00:30:12 (4) 2023-10-10T01:35:38 (1)
2023-10-10T03:16:34 (1) 2023-10-10T04:08:13 (4) 2023-10-10T04:08:26 (1)
2023-10-10T04:15:38 (1) 2023-10-10T06:27:32 (1) 2023-10-10T09:28:15 (1)
2023-10-10T09:28:16 (3) 2023-10-10T10:32:41 (1) 2023-10-10T10:32:46 (1)
2023-10-10T10:36:26 (1) 2023-10-10T10:36:27 (1) 2023-10-10T10:42:52 (4)
2023-10-10T10:47:06 (3) 2023-10-10T10:47:07 (1) 2023-10-10T13:18:19 (2)
2023-10-10T13:18:20 (2) 2023-10-10T14:09:20 (1) 2023-10-10T14:09:26 (1)
2023-10-10T14:09:33 (1) 2023-10-10T17:28:27 (1) 2023-10-10T18:06:01 (1)
2023-10-11T00:15:12 (2) 2023-10-11T00:15:13 (2) 2023-10-11T02:22:45 (1)
2023-10-11T02:22:51 (2) 2023-10-11T03:49:26 (1) 2023-10-11T03:49:27 (3)
2023-10-11T03:55:01 (1) 2023-10-11T04:48:10 (1) 2023-10-11T05:03:50 (1)
2023-10-11T05:03:52 (1) 2023-10-11T05:50:12 (1) 2023-10-11T06:46:22 (1)
2023-10-11T06:46:25 (1) 2023-10-11T09:25:05 (1) 2023-10-11T09:32:11 (1)
2023-10-11T09:59:03 (1) 2023-10-11T10:03:11 (4) 2023-10-11T11:01:07 (1)
2023-10-11T11:44:52 (1) 2023-10-11T12:55:39 (1) 2023-10-11T12:59:05 (8)
2023-10-11T12:59:06 (10) 2023-10-11T12:59:07 (4) 2023-10-11T12:59:08 (7)
2023-10-11T12:59:09 (4) 2023-10-11T12:59:10 (3) 2023-10-11T12:59:11 (3)
2023-10-11T12:59:13 (3) 2023-10-11T12:59:14 (2) 2023-10-11T12:59:15 (2)
2023-10-11T12:59:16 (2) 2023-10-11T12:59:17 (3) 2023-10-11T12:59:18 (2)
2023-10-11T12:59:19 (3) 2023-10-11T12:59:21 (3) 2023-10-11T12:59:22 (1)
2023-10-11T12:59:23 (2) 2023-10-11T12:59:24 (1) 2023-10-11T12:59:25 (3)
2023-10-11T12:59:27 (2) 2023-10-11T12:59:28 (3) 2023-10-11T12:59:29 (1)
2023-10-11T12:59:31 (1) 2023-10-11T12:59:32 (3) 2023-10-11T12:59:33 (2)
2023-10-11T12:59:34 (1) 2023-10-11T12:59:35 (2) 2023-10-11T12:59:36 (1)
2023-10-11T12:59:40 (1) 2023-10-11T12:59:41 (1) 2023-10-11T12:59:42 (2)
2023-10-11T12:59:43 (2) 2023-10-11T12:59:44 (3) 2023-10-11T12:59:45 (2)
2023-10-11T12:59:46 (3) 2023-10-11T12:59:48 (1) 2023-10-11T12:59:49 (2)
2023-10-11T12:59:50 (2) 2023-10-11T12:59:51 (2) 2023-10-11T12:59:52 (1)
2023-10-11T12:59:53 (1) 2023-10-11T12:59:56 (1) 2023-10-11T12:59:58 (1)
2023-10-11T12:59:59 (1) 2023-10-11T13:00:03 (1) 2023-10-11T13:00:04 (1)
2023-10-11T13:00:09 (2) 2023-10-11T13:00:10 (1) 2023-10-11T13:00:11 (2)
2023-10-11T13:00:12 (1) 2023-10-11T13:00:13 (1) 2023-10-11T13:00:14 (1)
2023-10-11T13:00:15 (2) 2023-10-11T13:10:53 (1) 2023-10-11T13:11:38 (1)
2023-10-11T13:14:13 (1) 2023-10-11T13:14:14 (1) 2023-10-11T16:09:49 (1)
2023-10-11T17:57:39 (1) 2023-10-11T17:57:43 (1) 2023-10-11T19:53:34 (1)
2023-10-11T19:53:35 (3) 2023-10-11T19:58:21 (4) 2023-10-11T20:11:13 (4)
2023-10-11T20:16:58 (4)
Requests per minute:
2023-09-30T21:54 (1) 2023-09-30T22:02 (1) 2023-09-30T22:16 (4)
2023-09-30T23:29 (2) 2023-09-30T23:55 (2) 2023-09-30T23:59 (2)
2023-10-01T00:00 (2) 2023-10-01T00:53 (1) 2023-10-01T01:09 (1)
2023-10-01T04:12 (1) 2023-10-01T04:18 (1) 2023-10-01T06:18 (29)
2023-10-01T06:20 (3) 2023-10-01T09:59 (1) 2023-10-01T10:15 (1)
2023-10-01T10:43 (3) 2023-10-01T11:27 (1) 2023-10-01T12:09 (4)
2023-10-01T12:14 (1) 2023-10-01T12:18 (1) 2023-10-01T15:03 (1)
2023-10-01T15:15 (1) 2023-10-01T16:11 (1) 2023-10-01T16:33 (4)
2023-10-01T17:17 (2) 2023-10-01T18:12 (4) 2023-10-01T22:52 (1)
2023-10-01T23:07 (4) 2023-10-01T23:08 (1) 2023-10-01T23:14 (4)
2023-10-01T23:31 (3) 2023-10-01T23:35 (1) 2023-10-01T23:39 (1)
2023-10-01T23:40 (1) 2023-10-01T23:45 (3) 2023-10-01T23:49 (1)
2023-10-01T23:50 (1) 2023-10-01T23:53 (1) 2023-10-01T23:55 (1)
2023-10-01T23:59 (1) 2023-10-02T00:05 (1) 2023-10-02T00:08 (1)
2023-10-02T00:17 (1) 2023-10-02T03:00 (1) 2023-10-02T03:01 (2)
2023-10-02T03:08 (1) 2023-10-02T04:23 (1) 2023-10-02T04:41 (1)
2023-10-02T05:17 (1) 2023-10-02T08:18 (2) 2023-10-02T11:49 (1)
2023-10-02T11:53 (1) 2023-10-02T13:15 (4) 2023-10-02T14:12 (4)
2023-10-02T17:18 (1) 2023-10-02T18:29 (1) 2023-10-02T18:33 (1)
2023-10-02T18:45 (2) 2023-10-02T19:14 (2) 2023-10-02T19:24 (8)
2023-10-02T20:17 (3) 2023-10-02T20:43 (1) 2023-10-02T20:44 (2)
2023-10-02T21:34 (1) 2023-10-02T21:35 (2) 2023-10-02T22:08 (1)
2023-10-02T22:09 (2) 2023-10-02T22:40 (1) 2023-10-02T23:23 (1)
2023-10-02T23:33 (4) 2023-10-02T23:36 (1) 2023-10-03T00:11 (1)
2023-10-03T00:39 (1) 2023-10-03T00:40 (1) 2023-10-03T00:57 (1)
2023-10-03T01:03 (2) 2023-10-03T01:06 (4) 2023-10-03T01:20 (1)
2023-10-03T01:32 (1) 2023-10-03T01:37 (1) 2023-10-03T02:04 (4)
2023-10-03T02:40 (3) 2023-10-03T03:42 (1) 2023-10-03T04:40 (1)
2023-10-03T07:40 (1) 2023-10-03T12:49 (4) 2023-10-03T14:13 (1)
2023-10-03T15:47 (1) 2023-10-03T17:08 (1) 2023-10-03T18:24 (1)
2023-10-03T18:28 (2) 2023-10-03T19:09 (1) 2023-10-03T19:52 (4)
2023-10-03T19:56 (4) 2023-10-03T21:29 (2) 2023-10-03T22:00 (4)
2023-10-03T22:05 (1) 2023-10-03T22:14 (4) 2023-10-04T03:04 (1)
2023-10-04T05:43 (4) 2023-10-04T06:08 (1) 2023-10-04T06:45 (1)
2023-10-04T06:50 (1) 2023-10-04T09:58 (1) 2023-10-04T11:27 (1)
2023-10-04T11:45 (1) 2023-10-04T11:48 (2) 2023-10-04T12:38 (2)
2023-10-04T16:46 (1) 2023-10-04T16:47 (2) 2023-10-04T17:20 (4)
2023-10-04T18:50 (1) 2023-10-04T19:41 (4) 2023-10-04T20:07 (1)
2023-10-04T21:18 (4) 2023-10-04T21:26 (4) 2023-10-04T21:31 (4)
2023-10-04T22:03 (1) 2023-10-05T01:09 (1) 2023-10-05T02:02 (1)
2023-10-05T06:19 (4) 2023-10-05T06:43 (1) 2023-10-05T06:50 (1)
2023-10-05T08:38 (1) 2023-10-05T09:16 (1) 2023-10-05T10:05 (1)
2023-10-05T11:04 (1) 2023-10-05T11:20 (4) 2023-10-05T13:43 (1)
2023-10-05T15:06 (3) 2023-10-05T15:15 (3) 2023-10-05T18:10 (4)
2023-10-05T18:38 (1) 2023-10-05T19:43 (1) 2023-10-05T19:50 (1)
2023-10-05T20:38 (1) 2023-10-06T00:01 (4) 2023-10-06T00:02 (4)
2023-10-06T00:03 (4) 2023-10-06T00:09 (4) 2023-10-06T05:47 (1)
2023-10-06T06:43 (1) 2023-10-06T06:49 (1) 2023-10-06T07:35 (1)
2023-10-06T07:51 (1) 2023-10-06T07:57 (3) 2023-10-06T09:46 (4)
2023-10-06T10:51 (1) 2023-10-06T11:59 (2) 2023-10-06T12:47 (1)
2023-10-06T13:33 (1) 2023-10-06T13:42 (1) 2023-10-06T18:26 (4)
2023-10-06T19:55 (1) 2023-10-06T21:19 (1) 2023-10-06T22:34 (1)
2023-10-06T22:51 (1) 2023-10-06T23:32 (4) 2023-10-06T23:36 (4)
2023-10-07T00:15 (1) 2023-10-07T00:40 (2) 2023-10-07T01:16 (1)
2023-10-07T03:46 (4) 2023-10-07T03:51 (4) 2023-10-07T04:13 (2)
2023-10-07T06:44 (1) 2023-10-07T06:49 (1) 2023-10-07T09:03 (1)
2023-10-07T12:04 (4) 2023-10-07T12:42 (3) 2023-10-07T21:30 (3)
2023-10-07T21:46 (1) 2023-10-07T23:02 (1) 2023-10-08T00:27 (1)
2023-10-08T00:33 (4) 2023-10-08T02:57 (4) 2023-10-08T03:00 (4)
2023-10-08T06:33 (1) 2023-10-08T06:51 (1) 2023-10-08T06:58 (1)
2023-10-08T09:06 (4) 2023-10-08T09:07 (4) 2023-10-08T10:19 (2)
2023-10-08T10:57 (1) 2023-10-08T13:25 (4) 2023-10-08T13:36 (5)
2023-10-08T14:46 (17) 2023-10-08T14:47 (1) 2023-10-08T14:48 (1)
2023-10-08T15:15 (1) 2023-10-08T18:12 (1) 2023-10-08T20:58 (2)
2023-10-09T01:29 (4) 2023-10-09T01:37 (1) 2023-10-09T01:54 (1)
2023-10-09T02:35 (1) 2023-10-09T04:26 (1) 2023-10-09T06:58 (4)
2023-10-09T08:02 (4) 2023-10-09T08:06 (4) 2023-10-09T08:07 (4)
2023-10-09T11:46 (1) 2023-10-09T11:53 (1) 2023-10-09T14:36 (4)
2023-10-09T15:18 (2) 2023-10-09T15:35 (1) 2023-10-09T23:15 (2)
2023-10-10T00:30 (4) 2023-10-10T01:35 (1) 2023-10-10T03:16 (1)
2023-10-10T04:08 (5) 2023-10-10T04:15 (1) 2023-10-10T06:27 (1)
2023-10-10T09:28 (4) 2023-10-10T10:32 (2) 2023-10-10T10:36 (2)
2023-10-10T10:42 (4) 2023-10-10T10:47 (4) 2023-10-10T13:18 (4)
2023-10-10T14:09 (3) 2023-10-10T17:28 (1) 2023-10-10T18:06 (1)
2023-10-11T00:15 (4) 2023-10-11T02:22 (3) 2023-10-11T03:49 (4)
2023-10-11T03:55 (1) 2023-10-11T04:48 (1) 2023-10-11T05:03 (2)
2023-10-11T05:50 (1) 2023-10-11T06:46 (2) 2023-10-11T09:25 (1)
2023-10-11T09:32 (1) 2023-10-11T09:59 (1) 2023-10-11T10:03 (4)
2023-10-11T11:01 (1) 2023-10-11T11:44 (1) 2023-10-11T12:55 (1)
2023-10-11T12:59 (108) 2023-10-11T13:00 (12) 2023-10-11T13:10 (1)
2023-10-11T13:11 (1) 2023-10-11T13:14 (2) 2023-10-11T16:09 (1)
2023-10-11T17:57 (2) 2023-10-11T19:53 (4) 2023-10-11T19:58 (4)
2023-10-11T20:11 (4) 2023-10-11T20:16 (4)
Requests per hour:
2023-09-30T21 (1) 2023-09-30T22 (5) 2023-09-30T23 (6)
2023-10-01T00 (3) 2023-10-01T01 (1) 2023-10-01T04 (2)
2023-10-01T06 (32) 2023-10-01T09 (1) 2023-10-01T10 (4)
2023-10-01T11 (1) 2023-10-01T12 (6) 2023-10-01T15 (2)
2023-10-01T16 (5) 2023-10-01T17 (2) 2023-10-01T18 (4)
2023-10-01T22 (1) 2023-10-01T23 (23) 2023-10-02T00 (3)
2023-10-02T03 (4) 2023-10-02T04 (2) 2023-10-02T05 (1)
2023-10-02T08 (2) 2023-10-02T11 (2) 2023-10-02T13 (4)
2023-10-02T14 (4) 2023-10-02T17 (1) 2023-10-02T18 (4)
2023-10-02T19 (10) 2023-10-02T20 (6) 2023-10-02T21 (3)
2023-10-02T22 (4) 2023-10-02T23 (6) 2023-10-03T00 (4)
2023-10-03T01 (9) 2023-10-03T02 (7) 2023-10-03T03 (1)
2023-10-03T04 (1) 2023-10-03T07 (1) 2023-10-03T12 (4)
2023-10-03T14 (1) 2023-10-03T15 (1) 2023-10-03T17 (1)
2023-10-03T18 (3) 2023-10-03T19 (9) 2023-10-03T21 (2)
2023-10-03T22 (9) 2023-10-04T03 (1) 2023-10-04T05 (4)
2023-10-04T06 (3) 2023-10-04T09 (1) 2023-10-04T11 (4)
2023-10-04T12 (2) 2023-10-04T16 (3) 2023-10-04T17 (4)
2023-10-04T18 (1) 2023-10-04T19 (4) 2023-10-04T20 (1)
2023-10-04T21 (12) 2023-10-04T22 (1) 2023-10-05T01 (1)
2023-10-05T02 (1) 2023-10-05T06 (6) 2023-10-05T08 (1)
2023-10-05T09 (1) 2023-10-05T10 (1) 2023-10-05T11 (5)
2023-10-05T13 (1) 2023-10-05T15 (6) 2023-10-05T18 (5)
2023-10-05T19 (2) 2023-10-05T20 (1) 2023-10-06T00 (16)
2023-10-06T05 (1) 2023-10-06T06 (2) 2023-10-06T07 (5)
2023-10-06T09 (4) 2023-10-06T10 (1) 2023-10-06T11 (2)
2023-10-06T12 (1) 2023-10-06T13 (2) 2023-10-06T18 (4)
2023-10-06T19 (1) 2023-10-06T21 (1) 2023-10-06T22 (2)
2023-10-06T23 (8) 2023-10-07T00 (3) 2023-10-07T01 (1)
2023-10-07T03 (8) 2023-10-07T04 (2) 2023-10-07T06 (2)
2023-10-07T09 (1) 2023-10-07T12 (7) 2023-10-07T21 (4)
2023-10-07T23 (1) 2023-10-08T00 (5) 2023-10-08T02 (4)
2023-10-08T03 (4) 2023-10-08T06 (3) 2023-10-08T09 (8)
2023-10-08T10 (3) 2023-10-08T13 (9) 2023-10-08T14 (19)
2023-10-08T15 (1) 2023-10-08T18 (1) 2023-10-08T20 (2)
2023-10-09T01 (6) 2023-10-09T02 (1) 2023-10-09T04 (1)
2023-10-09T06 (4) 2023-10-09T08 (12) 2023-10-09T11 (2)
2023-10-09T14 (4) 2023-10-09T15 (3) 2023-10-09T23 (2)
2023-10-10T00 (4) 2023-10-10T01 (1) 2023-10-10T03 (1)
2023-10-10T04 (6) 2023-10-10T06 (1) 2023-10-10T09 (4)
2023-10-10T10 (12) 2023-10-10T13 (4) 2023-10-10T14 (3)
2023-10-10T17 (1) 2023-10-10T18 (1) 2023-10-11T00 (4)
2023-10-11T02 (3) 2023-10-11T03 (5) 2023-10-11T04 (1)
2023-10-11T05 (3) 2023-10-11T06 (2) 2023-10-11T09 (3)
2023-10-11T10 (4) 2023-10-11T11 (2) 2023-10-11T12 (109)
2023-10-11T13 (16) 2023-10-11T16 (1) 2023-10-11T17 (2)
2023-10-11T19 (8) 2023-10-11T20 (8)
Requests per day:
2023-09-30 (12) 2023-10-01 (87) 2023-10-02 (56)
2023-10-03 (53) 2023-10-04 (41) 2023-10-05 (31)
2023-10-06 (50) 2023-10-07 (29) 2023-10-08 (59)
2023-10-09 (35) 2023-10-10 (38) 2023-10-11 (171)
```
Example filtering for `185.170.144.3` requests in `/home/nginx/domains/domain.com/log/access.json` Nginx JSON log.
```
./bot-rate.py /home/nginx/domains/domain.com/log/access.json 185.170.144.3
Requests with keyword '185.170.144.3':
Requests per second:
2023-10-02T15:21:04 (1) 2023-10-02T15:21:05 (1) 2023-10-02T15:21:06 (1)
2023-10-06T01:26:26 (1) 2023-10-06T01:26:28 (1) 2023-10-07T02:33:57 (1)
2023-10-07T02:33:58 (1) 2023-10-09T11:10:45 (1) 2023-10-09T11:10:47 (1)
2023-10-11T20:59:35 (1) 2023-10-11T20:59:37 (1) 2023-10-11T20:59:39 (1)
Requests per minute:
2023-10-02T15:21 (3) 2023-10-06T01:26 (2) 2023-10-07T02:33 (2)
2023-10-09T11:10 (2) 2023-10-11T20:59 (3)
Requests per hour:
2023-10-02T15 (3) 2023-10-06T01 (2) 2023-10-07T02 (2)
2023-10-09T11 (2) 2023-10-11T20 (3)
Requests per day:
2023-10-02 (3) 2023-10-06 (2) 2023-10-07 (2)
2023-10-09 (2) 2023-10-11 (3)
```
# Centmin Mod Nginx JSON Logs
To setup Nginx JSON access logging, you will need to manually edit your main `/usr/local/nginx/conf/nginx.conf` config file and add a 4th `log_format` to existing listing which is listed directly below where log_format named = `main_json`:
```
log_format main_json escape=json '{'
'"msec": "$msec", ' # request unixtime in seconds with a milliseconds resolution
'"connection": "$connection", ' # connection serial number
'"connection_requests": "$connection_requests", ' # number of requests made in connection
'"pid": "$pid", ' # process pid
'"request_id": "$request_id", ' # the unique request id
'"request_length": "$request_length", ' # request length (including headers and body)
'"remote_addr": "$remote_addr", ' # client IP
'"remote_user": "$remote_user", ' # client HTTP username
'"remote_port": "$remote_port", ' # client port
'"time_local": "$time_local", '
'"time_iso8601": "$time_iso8601", ' # local time in the ISO 8601 standard format
'"request": "$request", ' # full path no arguments if the request
'"request_uri": "$request_uri", ' # full path and arguments if the request
'"args": "$args", ' # args
'"status": "$status", ' # response status code
'"body_bytes_sent": "$body_bytes_sent", ' # the number of body bytes exclude headers sent to a client
'"bytes_sent": "$bytes_sent", ' # the number of bytes sent to a client
'"http_referer": "$http_referer", ' # HTTP referer
'"http_user_agent": "$http_user_agent", ' # user agent
'"http_x_forwarded_for": "$http_x_forwarded_for", ' # http_x_forwarded_for
'"http_host": "$http_host", ' # the request Host: header
'"server_name": "$server_name", ' # the name of the vhost serving the request
'"request_time": "$request_time", ' # request processing time in seconds with msec resolution
'"upstream": "$upstream_addr", ' # upstream backend server for proxied requests
'"upstream_connect_time": "$upstream_connect_time", ' # upstream handshake time incl. TLS
'"upstream_header_time": "$upstream_header_time", ' # time spent receiving upstream headers
'"upstream_response_time": "$upstream_response_time", ' # time spend receiving upstream body
'"upstream_response_length": "$upstream_response_length", ' # upstream response length
'"upstream_cache_status": "$upstream_cache_status", ' # cache HIT/MISS where applicable
'"ssl_protocol": "$ssl_protocol", ' # TLS protocol
'"ssl_session_reused": "$ssl_session_reused", ' # TLS session reused
'"ssl_cipher": "$ssl_cipher", ' # TLS cipher only OpenSSL 3.0 supported
'"ssl_curve": "$ssl_curve", ' # TLS curve OpenSSL 1.0.2+ supported
'"ssl_curves": "$ssl_curves", ' # TLS curves
'"scheme": "$scheme", ' # http or https
'"request_method": "$request_method", ' # request method
'"server_protocol": "$server_protocol", ' # request protocol, like HTTP/1.1 or HTTP/2.0
'"pipe": "$pipe", ' # “p” if request was pipelined, “.” otherwise
'"gzip_ratio": "$gzip_ratio", '
'"http_cf_ray": "$http_cf_ray"'
'}';
```
Next in your Nginx site vhost config file i.e. HTTPS vhost would be at `/usr/local/nginx/conf/conf.d/domain.com.ssl.conf`, you will need to add a new access_log line referencing the `log_format named = main_json`. The buffer and flush directives tell Nginx to use a memory buffer to write to access log every 256KB size or after 5 minutes for better performance. So you will need to do Nginx reload/restart to flush any memory buffered access logs to disk if you want to inspect latest log info.
```
access_log /home/nginx/domains/domain.com/log/access_log.json main_json buffer=256k flush=5m;
```
Normally, you would want to place it grouped with your existing access_log in your Nginx site vhost config file like.
```
access_log /home/nginx/domains/domain.com/log/access.log combined buffer=256k flush=5m;
access_log /home/nginx/domains/domain.com/log/access_log.json main_json buffer=256k flush=5m;
error_log /home/nginx/domains/domain.com/log/error.log;
```
Then restart Nginx server for changes to take affect
```
service nginx restart
```
or cmd shortcut
```
ngxrestart
```
You can use jq to filter just the Nginx metrics you want. Example
```
cat access_log.json | tail -1 | jq -r '"\(.time_local) \(.remote_addr) \(.http_x_forwarded_for) \(.request_uri) \(.status) \(.request_method) \(.http_host) \(.scheme) \(.server_protocol) \(.ssl_protocol) \(.ssl_cipher) \(.http_cf_ray) \(.http_user_agent)"'
03/May/2020:08:59:16 +0000 66.249.73.203 66.249.73.203 /threads/ovh-new-infrastructure-line-severs-intel-xeon-e-22xx-xeon-silver-amd-epyc-7371.18392/ 200 GET community.centminmod.com https HTTP/1.1 TLSv1.3 TLS_AES_256_GCM_SHA384 58d8c8f393bbe049-DFW Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
```
Or find out the proportion of HTTPS SSL ciphers served to visitors
```
cat access_log.json | jq -r '.ssl_cipher' | sort | uniq -c | sort -rn
3099 TLS_AES_256_GCM_SHA384
171 ECDHE-ECDSA-AES256-GCM-SHA384
22 ECDHE-ECDSA-CHACHA20-POLY1305
2 ECDHE-ECDSA-AES128-SHA
```
HTTPS SSL protocols and ciphers served to visitors
```
cat access_log.json | jq -r '"\(.ssl_protocol) \(.ssl_cipher)"' | sort | uniq -c | sort -rn
3099 TLSv1.3 TLS_AES_256_GCM_SHA384
171 TLSv1.2 ECDHE-ECDSA-AES256-GCM-SHA384
22 TLSv1.2 ECDHE-ECDSA-CHACHA20-POLY1305
1 TLSv1 ECDHE-ECDSA-AES128-SHA
1 TLSv1.1 ECDHE-ECDSA-AES128-SHA
```
HTTPS SSL protocols and HTTP status codes served to visitors
```
cat access_log.json | jq -r '"\(.ssl_protocol) \(.status)"' | sort | uniq -c | sort -rn
2115 TLSv1.3 200
478 TLSv1.3 301
226 TLSv1.3 307
150 TLSv1.2 200
135 TLSv1.3 304
76 TLSv1.3 303
62 TLSv1.3 403
25 TLSv1.2 304
6 TLSv1.2 301
5 TLSv1.3 404
4 TLSv1.2 307
3 TLSv1.2 403
3 TLSv1.2 303
2 TLSv1.2 400
1 TLSv1 400
1 TLSv1.3 405
1 TLSv1.3 400
1 TLSv1.1 400
```
HTTPS protocol and HTTP status codes for only urls = /login/login
```
cat access_log.json | jq -r 'select(.request_uri == "/login/login")| "\(.ssl_protocol) \(.status)"' | sort | uniq -c | sort -rn
25 TLSv1.3 200
```
# Nginx JSON Format Logs with Cloudflare Proxy
If you have Cloudflare proxy in front of Nginx, you can alter the Nginx json log format added to /usr/local/nginx/conf/nginx.conf to below version for additional logged fields
`log_format named = cf_json`
```
log_format cf_json escape=json '{'
'"msec": "$msec", ' # request unixtime in seconds with a milliseconds resolution
'"connection": "$connection", ' # connection serial number
'"connection_requests": "$connection_requests", ' # number of requests made in connection
'"pid": "$pid", ' # process pid
'"request_id": "$request_id", ' # the unique request id
'"request_length": "$request_length", ' # request length (including headers and body)
'"remote_addr": "$remote_addr", ' # client IP
'"remote_user": "$remote_user", ' # client HTTP username
'"remote_port": "$remote_port", ' # client port
'"time_local": "$time_local", '
'"time_iso8601": "$time_iso8601", ' # local time in the ISO 8601 standard format
'"request": "$request", ' # full path no arguments if the request
'"request_uri": "$request_uri", ' # full path and arguments if the request
'"args": "$args", ' # args
'"status": "$status", ' # response status code
'"body_bytes_sent": "$body_bytes_sent", ' # the number of body bytes exclude headers sent to a client
'"bytes_sent": "$bytes_sent", ' # the number of bytes sent to a client
'"http_referer": "$http_referer", ' # HTTP referer
'"http_user_agent": "$http_user_agent", ' # user agent
'"http_x_forwarded_for": "$http_x_forwarded_for", ' # http_x_forwarded_for
'"http_host": "$http_host", ' # the request Host: header
'"server_name": "$server_name", ' # the name of the vhost serving the request
'"request_time": "$request_time", ' # request processing time in seconds with msec resolution
'"upstream": "$upstream_addr", ' # upstream backend server for proxied requests
'"upstream_connect_time": "$upstream_connect_time", ' # upstream handshake time incl. TLS
'"upstream_header_time": "$upstream_header_time", ' # time spent receiving upstream headers
'"upstream_response_time": "$upstream_response_time", ' # time spend receiving upstream body
'"upstream_response_length": "$upstream_response_length", ' # upstream response length
'"upstream_cache_status": "$upstream_cache_status", ' # cache HIT/MISS where applicable
'"ssl_protocol": "$ssl_protocol", ' # TLS protocol
'"ssl_session_reused": "$ssl_session_reused", ' # TLS session reused
'"ssl_cipher": "$ssl_cipher", ' # TLS cipher only OpenSSL 3.0 supported
'"ssl_curve": "$ssl_curve", ' # TLS curve OpenSSL 1.0.2+ supported
'"ssl_curves": "$ssl_curves", ' # TLS curves
'"scheme": "$scheme", ' # http or https
'"request_method": "$request_method", ' # request method
'"server_protocol": "$server_protocol", ' # request protocol, like HTTP/1.1 or HTTP/2.0
'"pipe": "$pipe", ' # “p” if request was pipelined, “.” otherwise
'"gzip_ratio": "$gzip_ratio", '
'"http_cf_ray": "$http_cf_ray", '
'"http_cf_worker": "$http_cf_worker", '
'"http_cf_request_id": "$http_cf_request_id", '
'"http_cf_railgun": "$http_cf_railgun", '
'"http_accept": "$http_accept"'
'}';
```
and use this in nginx vhost adding a 2nd log for
`/home/nginx/domains/domain.com/log/access_log.json` using nginx `log format = cf_json`
```
access_log /home/nginx/domains/domain.com/log/access.log combined buffer=256k flush=5m;
access_log /home/nginx/domains/domain.com/log/access_log.json cf_json buffer=256k flush=5m;
error_log /home/nginx/domains/domain.com/log/error.log;
```