https://github.com/powerdns/redis-ipprefix
Experiments in storing IP prefix data in Redis
https://github.com/powerdns/redis-ipprefix
Last synced: 6 months ago
JSON representation
Experiments in storing IP prefix data in Redis
- Host: GitHub
- URL: https://github.com/powerdns/redis-ipprefix
- Owner: PowerDNS
- License: gpl-2.0
- Created: 2016-11-21T20:56:46.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2016-11-22T13:54:13.000Z (over 9 years ago)
- Last Synced: 2025-04-13T21:51:31.756Z (over 1 year ago)
- Language: Python
- Size: 18.6 KB
- Stars: 1
- Watchers: 8
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: COPYING
Awesome Lists containing this project
README
# Experiments in storing IP prefix data in Redis
Random ramblings below. This README currently lags behind the code.
Test with `diff -u expected_output <(./redis-ip.py )`
## IPv4
```
127.0.0.1:6379> zadd foo 8 vijf-acht
(integer) 1
127.0.0.1:6379> zadd foo 13 zeven-dertien
(integer) 1
127.0.0.1:6379> zrangebyscore foo 13 inf
1) "zeven-dertien"
127.0.0.1:6379> zrangebyscore foo 8 inf
1) "vijf-acht"
2) "zeven-dertien"
```
If we assume no overlapping subnets, we can add `LIMIT 0 1` to the zrangebyscore. If we assume overlap, we can limit it a small number (20-50) and evaluate them all, fetching more if necessary. (Or: we make sure the lexical ordering gives us the smallest one first (this is not the full story it turns out)). We can put all of this in Redis-side Lua.
### IPv6
ZSET scores are 64 bit floats, which gives us 53 bits of positive integer resolution. The idea is to split the v6 address into chunks that we can 'stack'. 128/53 = 2.4 so we could do this in 3 levels but for convenience I propose to do 4*32 bits.
# 3 hashes + 1 zset
We could hash the first 3 chunks, doing a BYSCORE lookup only on the fourth chunk. This would mean 4 lookups for any address, but the code would be simpler than for the next option. However, it is likely that most setups will have /64, /56 or /48 boundaries, in which case fewer lookups seems better - although I wonder how '3 hash lookups + 1 zset lookup' compares, in terms of performance, to '2 zset lookups'.
# 4 zsets
If instead we stack 4 zsets, any level could return a positive result, at least in a no-overlap situation. Even with overlaps we could probably provide a downward referral.
Both setups (4 zets or 3 hashes) do mean duplication for anything shorther than /97.