Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/o-x-l/haproxy-geoip
HAProxy (community) GeoIP Lookups
https://github.com/o-x-l/haproxy-geoip
asn asn-lookup country country-codes geoip geoip-lookup haproxy haproxy-configuration ipinfo maxmind maxmind-geoip
Last synced: 3 days ago
JSON representation
HAProxy (community) GeoIP Lookups
- Host: GitHub
- URL: https://github.com/o-x-l/haproxy-geoip
- Owner: O-X-L
- License: mit
- Created: 2023-11-19T21:05:45.000Z (about 1 year ago)
- Default Branch: latest
- Last Pushed: 2024-09-18T16:28:05.000Z (4 months ago)
- Last Synced: 2025-01-04T10:09:42.186Z (3 days ago)
- Topics: asn, asn-lookup, country, country-codes, geoip, geoip-lookup, haproxy, haproxy-configuration, ipinfo, maxmind, maxmind-geoip
- Language: Lua
- Homepage: https://docs.o-x-l.com/proxy/reverse_haproxy.html
- Size: 66.4 KB
- Stars: 8
- Watchers: 1
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# HAProxy - GeoIP Lookups
This is an example on how to use GeoIP lookups in combination with HAProxy.
Data linking requests to its origin country and ASN/ISP can be very useful when dealing with application-level attacks.
This allows you also to handle requests from specific countries and ASNs (p.e. datacenters/hosting providers) differently than others.
NOTE: This functionality is covered by the [HAProxy Enterprise Maxmind-Module](https://www.haproxy.com/documentation/hapee/latest/load-balancing/geolocation/maxmind/)! Only use this implementation if you are limited to the community edition.
----
## Topology
You can implement this in two ways:
* Use a custom backend-service to do this lookups
[Go-based binary](https://github.com/O-X-L/geoip-lookup-service/releases/tag/1.0)
[Python3-based](https://github.com/O-X-L/haproxy-geoip/tree/latest/backend)
* UNTESTED: Use the [resty-maxminddb LUA library](https://raw.githubusercontent.com/anjia0532/lua-resty-maxminddb/master/lib/resty/maxminddb.lua) to query the MMDB databases directly from LUA
----
### Lookup via Backend
1. Request hits HAProxy
2. HAProxy calls LUA script to delegate GeoIP-database lookup
3. LUA calls a minimal web-service on localhost that queries the GeoIP-database(s)
You can either use a [Go-based](https://github.com/O-X-L/geoip-lookup-service) or [Python3-based](https://github.com/O-X-L/haproxy-geoip/tree/latest/backend) HTTP-Server as backend
### Lookup via Library
1. Request hits HAProxy
2. HAProxy calls LUA script for querying the GeoIP-database(s)
----
### GeoIP
You will have to download some MMDB GeoIP databases.
* **IPInfo**: [Information](https://ipinfo.io/products/free-ip-database), [CC4 License](https://creativecommons.org/licenses/by-sa/4.0/) (*allows for commercial usage - you need to add an attribution*)
**Attribution**: `
IP address data powered by IPinfo
`
* **MaxMind**: [Information](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data), [EULA](https://www.maxmind.com/en/geolite2/eula) (*allows for limited commercial usage - you need to add an attribution*)
**Attribution**: `This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com.`
----
## Setup
* Add the LUA script to your system
* Install and set up the GeoIP lookup-backend of your choice
## Config
* Load the LUA module by adding lua-load `/etc/haproxy/lua/geoip_lookup.lua` in the global section
* Execute the LUA script on HTTP requests:
* In HTTP mode
```
# country
http-request lua.lookup_geoip_country
# asn
http-request lua.lookup_geoip_asn
```
* In TCP mode
```
# country
tcp-request content lua.lookup_geoip_country
# asn
tcp-request content lua.lookup_geoip_asn
```
* Log the data:
* In HTTP mode
```
http-request capture var(txn.geoip_asn) len 10
http-request capture var(txn.geoip_country) len 2
```
* In TCP mode
```
tcp-request content capture var(txn.geoip_asn) len 10
tcp-request content capture var(txn.geoip_country) len 2
```
You can also check out the configuration example: `haproxy_example.cfg`
----
### Cache-Map
You will have to decide if you want to use [HAProxy Maps](https://www.haproxy.com/blog/introduction-to-haproxy-maps) to cache Lookup results.
This can speed up the lookup for IPs that have already connected to your server.
It will also use more memory.
See also: `haproxy_example.cfg - test_country_cachemap`
The speed-improvements as seen by running the test-script are: `first: 0.03, second: 0.00`
By utilizing [HAProxy's ipmask](https://www.haproxy.com/blog/ip-masking-in-haproxy) (`src,ipmask(24,48)`) feature we are able to reduce the needed entries inside the map to the minimal subnets that are announced on public BGP.
----
### Lookup
#### via Go-Backend
Download the binary for you system from [the releases](https://github.com/O-X-L/geoip-lookup-service/releases).
[Read the documentation](https://github.com/O-X-L/geoip-lookup-service) on how to use it.
You need to use the `lua/geoip_lookup_w_backend.lua` script.
It is recommended to start Go-Backend with `-plain` command line argument to get the variable in plain text format.
----
#### via Python-Backend
To query the MMDB databases, you will have to install the [maxminddb python-module](https://github.com/maxmind/MaxMind-DB-Reader-python):
```bash
python3 -m pip install maxminddb
```
You will have to update the paths to your database-files in the `backend/geoip_lookup_backend.py` file!
You need to use the `lua/geoip_lookup_w_backend.lua` script.
----
#### via Library
WARNING: UNTESTED
To query the MMDB databases, you will have to install the [resty-maxminddb LUA library](https://raw.githubusercontent.com/anjia0532/lua-resty-maxminddb/master/lib/resty/maxminddb.lua) and its dependencies.
You need to use the `lua/geoip_lookup_w_lib.lua` script.
----
## Run
### With Go-Backend
```bash
# start the web-service
./geoip-lookup &
# initialize the haproxy map(s)
touch /tmp/haproxy_geoip_country.map
# start haproxy
haproxy -W -f haproxy_example.cfg
```
### With Python-Backend
```bash
# start the web-service
python3 backend/geoip_lookup.py &
# initialize the haproxy map(s)
touch /tmp/haproxy_geoip_country.map
# start haproxy
haproxy -W -f haproxy_example.cfg
```
### With LUA-Library
```bash
# initialize the haproxy map(s)
touch /tmp/haproxy_geoip_country.map
# start haproxy
haproxy -W -f haproxy_example.cfg
```
----
## Testing
You will have to copy some GeoIP databases to `/tmp`:
* '/tmp/maxmind_country.mmdb'
* '/tmp/maxmind_asn.mmdb'
* '/tmp/ipinfo_country.mmdb'
* '/tmp/ipinfo_asn.mmdb'
At least IPInfo OR MaxMind databases need to exist!
```bash
cd test
bash test.sh
>
> CLEANUP
>
> WARN: UNABLE TO TEST MaxMind databases as they are missing!
>
> STARTING HAPROXY
>
> TESTING with PYTHON-BACKEND
> LINKING IPInfo databases
> 127.0.0.1 - - [22/Nov/2023 21:21:00] "GET /?lookup=country&ip=1.1.1.1 HTTP/1.1" 200 -
> 127.0.0.1 - - [22/Nov/2023 21:21:00] "GET /?lookup=continent&ip=1.1.1.1 HTTP/1.1" 200 -
> 127.0.0.1 - - [22/Nov/2023 21:21:00] "GET /?lookup=asn&ip=1.1.1.1 HTTP/1.1" 200 -
> 127.0.0.1 - - [22/Nov/2023 21:21:00] "GET /?lookup=asname&ip=1.1.1.1 HTTP/1.1" 200 -
> REQUEST TIMES: 0.01 => 0.00 (cached)
>
> STOPPING HAPROXY
>
> FINISHED - exiting
```
Feel free to [contribute more test-cases](https://github.com/O-X-L/haproxy-geoip/blob/latest/test/requests.sh)!