{"id":13467480,"url":"https://github.com/GilbN/geoip2influx","last_synced_at":"2025-03-26T02:31:34.337Z","repository":{"id":46513433,"uuid":"274749855","full_name":"GilbN/geoip2influx","owner":"GilbN","description":"A python script that will parse the nginx access.log and send geolocation metrics and log metrics to InfluxDB","archived":false,"fork":false,"pushed_at":"2024-09-10T19:39:00.000Z","size":24288,"stargazers_count":78,"open_issues_count":2,"forks_count":16,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-22T14:41:24.332Z","etag":null,"topics":["docker","geoip2","geolocation-metrics","grafana","grafana-dashboard","hacktoberfest","influxdb","libmaxminddb","log-metrics","maxmind","maxminddb","nginx","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GilbN.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"GilbN"}},"created_at":"2020-06-24T19:13:24.000Z","updated_at":"2025-02-05T22:27:29.000Z","dependencies_parsed_at":"2024-01-14T16:08:43.750Z","dependency_job_id":"1a67bb99-bdb9-46dc-855f-26f0dee58ed3","html_url":"https://github.com/GilbN/geoip2influx","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GilbN%2Fgeoip2influx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GilbN%2Fgeoip2influx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GilbN%2Fgeoip2influx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GilbN%2Fgeoip2influx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GilbN","download_url":"https://codeload.github.com/GilbN/geoip2influx/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245576529,"owners_count":20638125,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","geoip2","geolocation-metrics","grafana","grafana-dashboard","hacktoberfest","influxdb","libmaxminddb","log-metrics","maxmind","maxminddb","nginx","python"],"created_at":"2024-07-31T15:00:57.091Z","updated_at":"2025-03-26T02:31:29.318Z","avatar_url":"https://github.com/GilbN.png","language":"Python","funding_links":["https://github.com/sponsors/GilbN"],"categories":["Python"],"sub_categories":[],"readme":"# [Geoip2Influx](https://github.com/gilbN/geoip2influx)\n\n[![Docker Cloud Build Status](https://img.shields.io/docker/cloud/build/gilbn/geoip2influx?style=for-the-badge)](https://hub.docker.com/r/gilbn/geoip2influx/builds)\n[![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/gilbn/geoip2influx?color=blue\u0026style=for-the-badge)](https://hub.docker.com/r/gilbn/geoip2influx)\n[![Docker Pulls](https://img.shields.io/docker/pulls/gilbn/geoip2influx?color=blue\u0026style=for-the-badge)](https://hub.docker.com/r/gilbn/geoip2influx)\n[![GitHub](https://img.shields.io/github/license/gilbn/geoip2influx?color=blue\u0026style=for-the-badge)](https://github.com/gilbN/geoip2influx/blob/master/LICENSE)\n[![Discord](https://img.shields.io/discord/591352397830553601?color=blue\u0026style=for-the-badge)](https://discord.gg/HSPa4cz)\n[![](https://img.shields.io/badge/Blog-technicalramblings.com-blue?style=for-the-badge)](https://technicalramblings.com/)\n***\n\nA python script that will parse the nginx access.log and send geolocation metrics and log metrics to InfluxDB\n\n![](https://i.imgur.com/mh0IhYA.jpg)\n\n### For the linuxserver/letsencrypt docker mod, click here : https://github.com/gilbN/lsio-docker-mods/tree/master/letsencrypt/geoip2-nginx-stats\n\n***\n\n## Usage\n\n### Enviroment variables:\n\nThese are the **default** values for all envs. \nAdd the ones that differ on your system. \n\n| Environment Variable | Example Value | Description |\n| -------------------- | ------------- | ----------- |\n| NGINX_LOG_PATH | /config/log/nginx/access.log | Container path for Nginx logfile , defaults to the example. |\n| INFLUX_HOST | localhost | Host running InfluxDB. |\n| INFLUX_HOST_PORT | 8086 | Optional, defaults to 8086. |\n| INFLUX_DATABASE | geoip2influx | Optional, defaults to geoip2influx. |\n| INFLUX_USER | root | Optional, defaults to root. |\n| INFLUX_PASS | root | Optional, defaults to root. |\n| GEO_MEASUREMENT | geoip2influx | InfluxDB measurement name for geohashes. Optional, defaults to the example. |\n| LOG_MEASUREMENT | nginx_access_logs | InfluxDB measurement name for nginx logs. Optional, defaults to the example. |\n| SEND_NGINX_LOGS | true | Set to `false` to disable nginx logs. Optional, defaults to `true`. |\n| GEOIP2INFLUX_LOG_LEVEL | info | Sets the log level in geoip2influx.log. Use `debug` for verbose logging Optional, defaults to info. |\n| GEOIP2INFLUX_LOG_PATH | /config/log/geoip2influx/geoip2influx.log | Optional. Defaults to example. |\n| INFLUX_RETENTION | 7d | Sets the retention for the database. Optional, defaults to example.|\n| INFLUX_SHARD | 1d | Set the shard for the database. Optional, defaults to example. |\n| MAXMINDDB_LICENSE_KEY | xxxxxxx | Add your Maxmind licence key |\n| MAXMINDDB_USER_ID | xxxxxxx| Add your Maxmind account id |\n\n\n### MaxMind Geolite2\n\nDefault download location is `/config/geoip2db/GeoLite2-City.mmdb`\n\nGet your licence key here: https://www.maxmind.com/en/geolite2/signup\n\n### InfluxDB \n\n#### InfluxDB v2.x is not supported. Use v1.8x. \n\nThe InfluxDB database will be created automatically with the name you choose.\n\n```\n-e INFLUX_DATABASE=geoip2influx \n```\n\n### Docker\n\n```bash\ndocker create \\\n  --name=geoip2influx \\\n  -e PUID=1000 \\\n  -e PGID=1000 \\\n  -e TZ=Europe/Oslo \\\n  -e INFLUX_HOST=\u003cinfluxdb host\u003e \\\n  -e INFLUX_HOST_PORT=\u003cinfluxdb port\u003e \\\n  -e MAXMINDDB_LICENSE_KEY=\u003clicense key\u003e\\\n  -e MAXMINDDB_USER_ID=\u003caccount id\u003e\\\n  -v /path/to/appdata/geoip2influx:/config \\\n  -v /path/to/nginx/accesslog/:/config/log/nginx/ \\\n  --restart unless-stopped \\\n  ghcr.io/gilbn/geoip2influx\n```\n\n### Docker compose\n\n```yaml\nversion: \"2.1\"\nservices:\n  geoip2influx:\n    image: ghcr.io/gilbn/geoip2influx\n    container_name: geoip2influx\n    environment:\n      - PUID=1000\n      - PGID=1000\n      - TZ=Europe/Oslo\n      - INFLUX_HOST=\u003cinfluxdb host\u003e\n      - INFLUX_HOST_PORT=\u003cinfluxdb port\u003e\n      - MAXMINDDB_LICENSE_KEY=\u003clicense key\u003e\n      - MAXMINDDB_USER_ID=\u003caccount id\u003e\n    volumes:\n      - /path/to/appdata/geoip2influx:/config\n      - /path/to/nginx/accesslog/:/config/log/nginx/\n    restart: unless-stopped\n```\n\n***\n\n## Grafana dashboard: \n### [Grafana Dashboard Link](https://grafana.com/grafana/dashboards/12268/)\n\nNeeds the [grafana-worldmap-panel](https://grafana.com/grafana/plugins/grafana-worldmap-panel/?tab=installation)\n***\n\n## Sending Nginx log metrics\n\nNginx needs to be compiled with the geoip2 module: https://github.com/leev/ngx_http_geoip2_module\n\n1. Add the following to the http block in your `nginx.conf` file:\n\n```nginx\ngeoip2 /config/geoip2db/GeoLite2-City.mmdb {\nauto_reload 5m;\n$geoip2_data_country_iso_code country iso_code;\n$geoip2_data_city_name city names en;\n}\n\nlog_format custom '$remote_addr - $remote_user [$time_local]'\n           '\"$request\" $status $body_bytes_sent'\n           '\"$http_referer\" $host \"$http_user_agent\"'\n           '\"$request_time\" \"$upstream_connect_time\"'\n           '\"$geoip2_data_city_name\" \"$geoip2_data_country_iso_code\"';\n ```\n \n 2. Set the access log use the `custom` log format. \n ```nginx\n access_log /config/log/nginx/access.log custom;\n ```\n\n### Multiple log files\n\nIf you separate your nginx log files but want this script to parse all of them you can do the following:\n\nAs nginx can have multiple `access log` directives in a block, just add another one in the server block. \n\n**Example**\n\n```nginx\n\taccess_log /config/log/nginx/technicalramblings/access.log custom;\n\taccess_log /config/log/nginx/access.log custom;\n```\nThis will log the same lines to both files.\n\nThen use the `/config/log/nginx/access.log` file in the `NGINX_LOG_PATH` variable. \n\n***\n\n## Updates \n\n**xx.xx.24** - Refactor to alpine 3.20. New env required. MAXMINDDB_USER_ID. \n\n**21.06.20** - Added $host(domain) to the nginx log metrics. This will break your nginx logs parsing, as you need to update the custom log format.\n\n**06.06.20** - Added influx retention policy to try and mitigate max-values-per-tag limit exceeded errors.\n\n  * `-e INFLUX_RETENTION` Default 30d\n  * `-e INFLUX_SHARD` Default 2d\n  * It will only add the retention policy if the database doesn't exist.\n\n**30.05.20** - Added logging. Use `-e GEOIP2INFLUX_LOG_LEVEL` to set the log level.\n\n**15.05.20** - Removed `GEOIP2_KEY` and `GEOIP_DB_PATH`variables. With commit https://github.com/linuxserver/docker-letsencrypt/commit/75b9685fdb3ec6edda590300f289b0e75dd9efd0 the letsencrypt container now natively supports downloading and updating(weekly) the GeoLite2-City database!\n\n***\n\nAdapted source: https://github.com/ratibor78/geostat\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGilbN%2Fgeoip2influx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FGilbN%2Fgeoip2influx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGilbN%2Fgeoip2influx/lists"}