Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/carlosabalde/libvmod-redis
VMOD using the synchronous hiredis library API to access Redis servers from VCL.
https://github.com/carlosabalde/libvmod-redis
redis varnish vmod
Last synced: 2 months ago
JSON representation
VMOD using the synchronous hiredis library API to access Redis servers from VCL.
- Host: GitHub
- URL: https://github.com/carlosabalde/libvmod-redis
- Owner: carlosabalde
- License: other
- Created: 2014-08-22T15:52:54.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2024-09-17T12:30:03.000Z (3 months ago)
- Last Synced: 2024-09-17T15:19:11.489Z (3 months ago)
- Topics: redis, varnish, vmod
- Language: C
- Size: 1.2 MB
- Stars: 80
- Watchers: 8
- Forks: 21
- Open Issues: 2
-
Metadata Files:
- Readme: README.rst
- License: COPYING
Awesome Lists containing this project
README
.. image:: https://github.com/carlosabalde/libvmod-redis/workflows/CI/badge.svg?branch=master
:alt: GitHub Actions CI badge
:target: https://github.com/carlosabalde/libvmod-redis/actions
.. image:: https://codecov.io/gh/carlosabalde/libvmod-redis/branch/master/graph/badge.svg
:alt: Codecov badge
:target: https://codecov.io/gh/carlosabalde/libvmod-redis
VMOD using the `synchronous hiredis library API `_ to access Redis servers from VCL.
Highlights:
* **Full support for execution of LUA scripts** (i.e. ``EVAL`` command), including optimistic automatic execution of ``EVALSHA`` commands.
* **All Redis reply data types are supported**, including partial support to access to components of simple (i.e. not nested) array replies.
* **Redis pipelines are not (and won't be) supported**. LUA scripting, which is fully supported by the VMOD, it's a much more flexible alternative to pipelines for atomic execution and minimizing latency. Pipelines are hard to use and error prone, specially when using the ``WATCH`` command.
* **Support for classic Redis deployments** using multiple replicated Redis servers **and for clustered deployments based on Redis Cluster**.
* **Support for multiple databases and multiple Redis connections**, local to each Varnish worker thread, or shared using one or more pools.
* **Support for smart command execution**, selecting the destination server according with the preferred role (i.e. master or slave) and with distance and healthiness metrics collected during execution.
* **Support for Redis Sentinel**, allowing automatic discovery of sick / healthy servers and changes in their roles.
Please, check out `the project wiki `_ for some extra information and useful links.
Looking for official support for this VMOD? Please, contact `Allenta Consulting `_, a `Varnish Software Premium partner `_.
SYNOPSIS
========
import redis;
::
##
## Subnets.
##
Function subnets(STRING masks="")
##
## Sentinels.
##
Function sentinels(
STRING locations="",
INT period=60,
INT connection_timeout=500,
INT command_timeout=0,
ENUM { RESP2, RESP3, default } protocol="default",
BOOL tls=false,
STRING tls_cafile="",
STRING tls_capath="",
STRING tls_certfile="",
STRING tls_keyfile="",
STRING tls_sni="",
STRING password="")
##
## Proxy.
##
# Instance selection.
Function VOID use(STRING db)
# Proxied methods.
Method VOID .add_server(..., STRING db="")
Function VOID command(..., STRING db="")
Function VOID timeout(..., STRING db="")
Function VOID retries(..., STRING db="")
...
Method STRING .stats(..., STRING db="")
Method INT .counter(..., STRING db="")
##
## Databases.
##
# Constructor.
Object db(
STRING location="",
ENUM { master, slave, auto, cluster } type="auto",
INT connection_timeout=1000,
INT connection_ttl=0,
INT command_timeout=0,
INT max_command_retries=0,
BOOL shared_connections=true,
INT max_connections=128,
ENUM { RESP2, RESP3, default } protocol="default",
BOOL tls=false,
STRING tls_cafile="",
STRING tls_capath="",
STRING tls_certfile="",
STRING tls_keyfile="",
STRING tls_sni="",
STRING user="",
STRING password="",
INT sickness_ttl=60,
BOOL ignore_slaves=false,
INT max_cluster_hops=32)
Method VOID .add_server(
STRING location,
ENUM { master, slave, auto, cluster } type)
# Command execution.
Method VOID .command(STRING name)
Method VOID .timeout(INT command_timeout)
Method VOID .retries(INT max_command_retries)
Method VOID .push(STRING arg)
Method VOID .execute(BOOL master=true)
Method VOID .easy_execute(STRING command, [STRING command_args...], BOOL master=true, INT command_timeout, INT max_command_retries)
# Access to replies.
Method BOOL .replied()
Method BOOL .reply_is_error()
Method BOOL .reply_is_nil()
Method BOOL .reply_is_status()
Method BOOL .reply_is_integer()
Method BOOL .reply_is_boolean()
Method BOOL .reply_is_double()
Method BOOL .reply_is_string()
Method BOOL .reply_is_array()
Method STRING .get_reply()
Method STRING .get_error_reply()
Method STRING .get_status_reply()
Method INT .get_integer_reply()
Method BOOL .get_boolean_reply()
Method REAL .get_double_reply()
Method STRING .get_string_reply()
Method INT .get_array_reply_length()
Method BOOL .array_reply_is_error(INT index)
Method BOOL .array_reply_is_nil(INT index)
Method BOOL .array_reply_is_status(INT index)
Method BOOL .array_reply_is_integer(INT index)
Method BOOL .array_reply_is_boolean(INT index)
Method BOOL .array_reply_is_double(INT index)
Method BOOL .array_reply_is_string(INT index)
Method BOOL .array_reply_is_array(INT index)
Method STRING .get_array_reply_value(INT index)
# Other.
Method VOID .free()
Method STRING .stats(
ENUM { json, prometheus } format="json",
BOOL stream=0,
STRING prometheus_name_prefix="vmod_redis_",
BOOL prometheus_default_labels=1,
STRING prometheus_extra_labels="")
Method INT .counter(STRING name)
EXAMPLES
========
Single server
-------------
::
sub vcl_init {
# VMOD configuration: simple case, keeping up to one Redis connection
# per Varnish worker thread.
new db = redis.db(
location="192.168.1.100:6379",
type=master,
connection_timeout=500,
shared_connections=false,
max_connections=1);
}
sub vcl_deliver {
# Simple command execution.
db.command("SET");
db.push("foo");
db.push("Hello world!");
db.execute();
# Alternatively, the same can be achieved with one single command
db.easy_execute("SET", "foo", "Hello world!");
# LUA scripting.
db.command("EVAL");
db.push({"
redis.call('SET', KEYS[1], ARGV[1])
redis.call('SET', KEYS[2], ARGV[1])
"});
db.push("2");
db.push("foo");
db.push("bar");
db.push("Atomic hello world!");
db.execute();
# Array replies, checking & accessing to reply.
db.command("MGET");
db.push("foo");
db.push("bar");
db.execute();
if ((db.reply_is_array()) &&
(db.get_array_reply_length() == 2)) {
set resp.http.X-Foo = db.get_array_reply_value(0);
set resp.http.X-Bar = db.get_array_reply_value(1);
}
}
Multiple servers
----------------
::
sub vcl_init {
# VMOD configuration: master-slave replication, keeping up to two
# Redis connections per Varnish worker thread (up to one to the master
# server & up to one to the closest slave server).
redis.subnets(
masks={"
0 192.168.1.102/32,
1 192.168.1.103/32,
2 0.0.0.0/32
"});
new db = redis.db(
location="192.168.1.100:6379",
type=master,
connection_timeout=500,
shared_connections=false,
max_connections=2);
db.add_server("192.168.1.101:6379", slave);
db.add_server("192.168.1.102:6379", slave);
db.add_server("192.168.1.103:6379", slave);
}
sub vcl_deliver {
# SET submitted to the master server.
db.command("SET");
db.push("foo");
db.push("Hello world!");
db.execute();
# GET submitted to one of the slave servers.
db.command("GET");
db.push("foo");
db.execute(false);
set req.http.X-Foo = db.get_string_reply();
}
Clustered setup
---------------
::
sub vcl_init {
# VMOD configuration: clustered setup, keeping up to 100 Redis
# connections per server, all shared between all Varnish worker threads.
# Two initial cluster servers are provided; remaining servers are
# automatically discovered.
new db = redis.db(
location="192.168.1.100:6379",
type=cluster,
connection_timeout=500,
shared_connections=true,
max_connections=128,
max_cluster_hops=16);
db.add_server("192.168.1.101:6379", cluster);
}
sub vcl_deliver {
# SET internally routed to the destination server.
db.command("SET");
db.push("foo");
db.push("Hello world!");
db.execute();
# GET internally routed to the destination server.
db.command("GET");
db.push("foo");
db.execute(false);
set req.http.X-Foo = db.get_string_reply();
}
INSTALLATION
============
The source tree is based on autotools to configure the building, and does also have the necessary bits in place to do functional unit tests using the varnishtest tool.
**Beware this project contains multiples branches (master, 4.1, 4.0, etc.). Please, select the branch to be used depending on your Varnish Cache version (Varnish trunk → master, Varnish 4.1.x → 4.1, Varnish 4.0.x → 4.0, etc.).**
Dependencies:
* `hiredis `_ - minimalistic C Redis client library.
* `libev `_ - full-featured and high-performance event loop.
COPYRIGHT
=========
See LICENSE for details.
Public domain implementation of the SHA-1 cryptographic hash function by Steve Reid and embedded in this VMOD (required for the optimistic execution of ``EVALSHA`` commands) has been borrowed from `this project `_:
* https://github.com/clibs/sha1/blob/master/sha1.c
* https://github.com/clibs/sha1/blob/master/sha1.h
BSD's implementation of the CRC-16 cryptographic hash function by Georges Menie & Salvatore Sanfilippo and embedded in this VMOD (required for the Redis Cluster slot calculation) has been borrowed from the `Redis project `_:
* http://download.redis.io/redis-stable/src/crc16.c
Copyright (c) Carlos Abalde