Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/smithsonian/redisx
A light-weight Redis / Valkey client library for C/C++
https://github.com/smithsonian/redisx
c-language c-programming-language library open-source redis-client
Last synced: about 10 hours ago
JSON representation
A light-weight Redis / Valkey client library for C/C++
- Host: GitHub
- URL: https://github.com/smithsonian/redisx
- Owner: Smithsonian
- License: unlicense
- Created: 2024-05-05T08:46:36.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2024-09-22T20:23:54.000Z (3 days ago)
- Last Synced: 2024-09-24T21:16:59.402Z (about 15 hours ago)
- Topics: c-language, c-programming-language, library, open-source, redis-client
- Language: C
- Homepage: https://smithsonian.github.io/redisx/
- Size: 1 MB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# RedisX
A simple, light-weight C/C++ Redis client library.
- [API documentation](https://smithsonian.github.io/redisx/apidoc/html/files.html)
- [Project page](https://smithsonian.github.io/redisx) on github.io
Author: Attila Kovacs
Last Updated: 18 September 2024
## Table of Contents
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Building RedisX](#building-redisx)
- [Managing Redis server connections](#managing-redis-server-connections)
- [Simple Redis queries](#simple-redis-queries)
- [Accessing key / value data](#accessing-key-value-data)
- [Publish/subscribe (PUB/SUB) support](#publish-subscribe-support)
- [Atomic execution blocks and LUA scripts](#atomic-transaction-blocks-and-lua-scripts)
- [Advanced queries and pipelining](#advanced-queries)
- [Error handling](#error-handling)
- [Debug support](#debug-support)
- [Future plans](#future-plans)
__RedisX__ is a light-weight [Redis](https://redis.io) client library for C/C++. As such, it should also work with
Redis forks / clones like [Dragonfly](https://dragonfly.io) or [Valkey](https://valkey.io). It supports both
interactive and pipelined Redis queries, managing and processing subscriptions, atomic execution blocks, and LUA
scripts loading. It can be used with multiple Redis servers simultaneously also.
While there are other C/C++ Redis clients available, this one is C99 compatible, and hence can be used on older
platforms also. It is also small and fast, but still capable and versatile.
Rather than providing high-level support for every possible Redis command (which would probably be impossible given
the pace new commands are being introduced all the time), it provides a basic framework for synchronous and
asynchronous queries, with some higher-level functions for managing key/value storage types (including hash tables),
and PUB/SUB. Future releases may add further higher-level functionality based on demand for such features.
The __RedisX__ library was created, and is maintained, by Attila Kovács at the Center for Astrophysics \| Harvard
& Smithsonian, and it is available through the [Smithsonian/redisx](https://github.com/Smithsonian/redisx)
repository on GitHub.
There are no official releases of __RedisX__ yet. An initial 1.0.0 release is expected in late 2024 or early 2025.
Before then the API may undergo slight changes and tweaks. Use the repository as is at your own risk for now.
### Related links
- [Redis commands](https://redis.io/docs/latest/commands/) (reference documentation)
- [SMA eXchange (SMA-X)](https://docs.google.com/document/d/1eYbWDClKkV7JnJxv4MxuNBNV47dFXuUWu7C4Ve_YTf0/edit?usp=sharing)
-- A structured realtime database built on Redis.
* [Smithsonian/smax-server](https://github.com/Smithsonian/smax-server) -- SMA-X server configuration kit
* [Smithsonian/smax-clib](https://github.com/Smithsonian/smax-clib) -- A C/C++ client library and toolkit to SMA-X,
based on __RedisX__
* [Smithsonian/smax-python](https://github.com/Smithsonian/smax-python) -- A Python 3 client library to SMA-X
-----------------------------------------------------------------------------
The [Smithsonian/xchange](https://github.com/Smithsonian/xchange) library is both a build and a runtime dependency of
RedisX.
-----------------------------------------------------------------------------
The __RedisX__ library can be built either as a shared (`libredisx.so[.1]`) and as a static (`libredisx.a`) library,
depending on what suits your needs best.
You can configure the build, either by editing `config.mk` or else by defining the relevant environment variables
prior to invoking `make`. The following build variables can be configured:
- `XCHANGE`: the root of the location where the [Smithsonian/xchange](https://github.com/Smithsonian/xchange) library
is installed. It expects to find `xchange.h` under `$(XCHANGE)/include` and `libxchange.so` under `$(XCHANGE)/lib`
or else in the default `LD_LIBRARY_PATH`.
- `CC`: The C compiler to use (default: `gcc`).
- `CPPFLAGS`: C pre-processor flags, such as externally defined compiler constants.
- `CFLAGS`: Flags to pass onto the C compiler (default: `-Os -Wall -std=c99`). Note, `-Iinclude` will be added
automatically.
- `LDFLAGS`: Extra linker flags (default is _not set_). Note, `-lm -lxchange` will be added automatically.
- `BUILD_MODE`: You can set it to `debug` to enable debugging features: it will initialize the global `xDebug`
variable to `TRUE` and add `-g` to `CFLAGS`.
- `CHECKEXTRA`: Extra options to pass to `cppcheck` for the `make check` target
After configuring, you can simply run `make`, which will build the `shared` (`lib/libredisx.so[.1]`) and `static`
(`lib/libredisx.a`) libraries, local HTML documentation (provided `doxygen` is available), and performs static
analysis via the `check` target. Or, you may build just the components you are interested in, by specifying the
desired `make` target(s). (You can use `make help` to get a summary of the available `make` targets).
-----------------------------------------------------------------------------
## Managing Redis server connections
- [Initializing](#initializing)
- [Connecting](#connecting)
- [Disconnecting](#disconnecting)
- [Connection hooks](#connection-hooks)
The library maintains up to three separate connections (channels) for each separate Redis server instance used: (1) an
interactive client for sequential round-trip transactions, (2) a pipeline client for bulk queries and asynchronous
background processing, and (3) a subscription client for PUB/SUB requests and notifications. The interactive client is
always connected, the pipeline client is connected only if explicitly requested at the time of establishing the server
connection, while the subscription client is connected only as needed.
The first step is to create a `Redis` object, with the server name or IP address.
```c
// Configure the redis server to connect to "redis.mydomain.com".
Redis *redis = redisxInit("redis.mydomain.com");
if (redis == NULL) {
// Abort: something did not got to plan...
return;
}
```
Before connecting to the Redis server, you may configure optional settings, such as the TCP port number to use (if not
the default 6379), and the database authentication (if any):
```c
Redis *redis = ...
// (optional) configure a non-standard port number
redisxSetPort(&redis, 7089);
// (optional) Configure the database user (since Redis 6.0, using ACL)
redisxSetUser(&redis, "johndoe");
// (optional) Configure the database password...
redisxSetPassword(&redis, mySecretPasswordString);
```
You might also tweak the send/receive buffer sizes to use for clients, if you find the socket defaults sub-optimal for
your application:
```c
// (optional) Set the TCP send/rcv buffer sizes to use if not default values.
// This setting applies to all new connections after...
redisxSetTcpBuf(65536);
```
Optionally, you can select the database index to use now (or later, after connecting), if not the default (index
0):
```c
Redis *redis = ...
// Select the database index 2
redisxSelectDB(redis, 2);
```
Note, that you can switch the database index any time, with the caveat that it's not possible to change it for the
subscription client when there are active subscriptions.
Once configured, you can connect to the server as:
```c
// Connect to Redis, including a 2nd dedicated client for pipelined requests
int status = redisxConnect(&redis, TRUE);
if (status != X_SUCCESS) {
// Abort: we could not connect for some reason...
...
// Clean up...
redisxDestroy(&redis);
...
}
```
The above will establish both an interactive connection and a pipelined connection client, for processing both
synchronous and asynchronous requests (and responses).
When you are done with a specific Redis server, you should disconnect from it:
```c
Redis *redis = ...
redisxDisconnect(redis);
```
The user of the __RedisX__ library might want to know when connections to the server are established, or when
disconnections happen, and may want to perform some configuration or clean-up accordingly. For this reason, the
library provides support for connection 'hooks' -- that is custom functions that are called in the even of connecting
to or disconnecting from a Redis server.
Here is an example of a connection hook, which simply prints a message about the connection to the console.
```c
void my_connect_hook(Redis *redis) {
printf("Connected to Redis server: %s\n", redis->id);
}
```
And, it can be added to a Redis instance, between the `redisxInit()` and the `redisxConnect()` calls.
```c
Redis *redis = ...
redisxAddConnectHook(redis, my_connect_hook);
```
The same goes for disconnect hooks, using `redisxAddDisconnectHook()` instead.
-----------------------------------------------------------------------------
- [Interactive transactions](#interactive-transactions)
- [RESP data type](#resp-data-type)
Redis queries are sent as strings, according the the specification of the Redis protocol. All responses sent back by
the server using the RESP protocol. Specifically, Redis uses version 2.0 of the RESP protocol (a.k.a. RESP2) by
default, with optional support for the newer RESP3 introduced in Redis version 6.0. The RedisX library currently
processes the standard RESP2 replies only. RESP3 support to the library may be added in the future (stay tuned...)
The simplest way for running a few Redis queries is to do it in interactive mode:
```c
Redis *redis = ...
RESP *resp;
int status;
// Send "HGET my_table my_key" request
resp = redisxRequest(redis, "HGET", "my_table", "my_key", NULL, &status);
// Check return status...
if (status != X_SUCCESS) {
// Oops something went wrong...
...
}
...
```
The `redisxRequest()` sends a command with up to three arguments. If the command takes fewer than 3 parameters, then
the remaining ones must be set to `NULL`. This function thus offers a simple interface for running most basic
sequential queries. In cases where 3 parameters are nut sufficient, you may use `redisxArrayRequest()` instead, e.g.:
```c
...
char *args[] = { "my_table", "my_key" }; // parameters as an array...
// Send "HGET my_table my_key" request with an array of 2 parameters...
resp = redisxRequest(redis, "HGET", args, NULL, 2, &status);
...
```
The 4th argument in the list is an optional `int[]` array defining the individual string lengths of the parameters (if
need be, or else readily available). Here, we used `NULL` instead, which will use `strlen()` on each supplied
string-terminated parameter to determine its length automatically. Specifying the length may be necessary if the
individual parameters are not 0-terminated strings, or else substrings from a continuing string are to be used as
the parameter value.
In interactive mode, each request is sent to the Redis server, and the response is collected before the call returns
with that response (or `NULL` if there was an error).
### RESP data type
All responses coming from the Redis server are represented by a dynamically allocated `RESP` type (defined in
`redisx.h`) structure.
```c
typedef struct RESP {
char type; // RESP type: RESP_ARRAY, RESP_INT ...
int n; // Either the integer value of a RESP_INT response, or the
// dimension of the value field.
void *value; // Pointer to text (char *) content or to an array of components
// (RESP **)
} RESP;
```
whose contents are:
| RESP `type` | Redis ID | `n` |`value` cast in C |
|-------------------------|----------|-------------------------------|-----------------------|
| `RESP_ARRAY` | `*` | number of `RESP *` pointers | `(RESP **)` |
| `RESP_INT` | `:` | integer return value | `(void)` |
| `RESP_SIMPLE_STRING` | `+` | string length | `(char *)` |
| `RESP_ERROR` | `-` | string length | `(char *)` |
| `RESP_BULK_STRING` | `$` | string length or -1 if `NULL` | `(char *)` |
Each `RESP` has a type (e.g. `RESP_SIMPLE_STRING`), an integer value `n`, and a `value` pointer
to further data. If the type is `RESP_INT`, then `n` represents the actual return value (and the `value` pointer is
not used). For string type values `n` is the number of characters in the string `value` (not including termination),
while for `RESP_ARRAY` types the `value` is a pointer to an embedded `RESP` array and `n` is the number of elements
in that.
You may check the integrity of a `RESP` using `redisxCheckRESP()`. Since `RESP` data is dynamically allocated, the
user is responsible for discarding them once they are no longer needed, e.g. by calling `redisxDestroyRESP()`. The
two steps may be combined to automatically discard invalid or unexpected `RESP` data in a single step by calling
`redisxCheckDestroyRESP()`.
```c
RESP *r = ...
// Let's say we expect 'r' to contain of an embedded RESP array of 3 elements...
int status = redisxCheckDestroyRESP(r, RESP_ARRAY, 3);
if (status != X_SUCCESS) {
// Oops, 'r' was either NULL, or does not contain a RESP array with 3 elements...
...
}
else {
// Process the expected response...
...
redisxDestroyRESP(r);
}
```
Before destroying a RESP structure, the caller may want to de-reference values within it if they are to be used
as is (without making copies), e.g.:
```c
RESP *r = ...
char *stringValue = NULL; // to be extracted from 'r'
// Let's say we expect 'r' to contain of a simple string response (of whatever length)
int status = redisxCheckDestroyRESP(r, RESP_SIMPLE_STRING, 0);
if (status != X_SUCCESS) {
// Oops, 'r' was either NULL, or it was not a simple string type
...
}
else {
// Set 'stringValue' and dereference the value field in the RESP so it's not
// destroyed with the RESP itself.
stringValue = (char *) r->value;
r->value = NULL;
redisxDestroyRESP(r); // The 'stringValue' is still a valid pointer after!
}
```
-----------------------------------------------------------------------------
- [Getting and setting keyed values](#getting-and-setting-keyed-values)
- [Listing and scanning](#listing-and-scanning)
### Getting and setting keyed values
Key/value pairs are the bread and butter of Redis. They come in two variaties: (1) there are top-level key-value
pairs, and (2) there are key-value pairs organized into hash tables, where the table name is a top-level key, but the
fields in the table are not. The RedisX library offers a unified approach for dealing with key/value pairs, whether
they are top level or hash-tables. Simply, a table name `NULL` is used to refer to top-level keys.
Retrieving individual keyed values is simple:
```c
Redis *redis = ...;
int len; // Variable in which we return the length of the value or an error code
// Get the "property" field from the "system:subsystem" hash table
char *value = redisxGetStringValue(redis, "system:subsystem", "property", &len);
if (len < 0) {
// Oops something went wrong.
...
}
...
// Discard the value once it's no longer needed.
if(value) free(value);
```
The same goes for top-level keyed values, using `NULL` for the hash table name:
```c
// Get value for top-level key (not stored in hash table!)
char *value = redisxGetStringValue(redis, NULL, "my-key", &len);
```
Alternatively, you can get the value as an undigested `RESP`, using `redisxGetValue()` instead, which allows you to
check and inspect the response in more detail (e.g. to check for potential errors, or unexpected response types).
Setting values is straightforward also:
```c
Redis *redis = ...;
// Set the "property" field in the "system:subsystem" hash table to -2.5
// using the interactive client connection, without requiring confirmation.
int status = redisxSetValue(redis, "system:subsystem", "property", "-2.5", FALSE);
if (status != X_SUCCESS) {
// Oops something went wrong.
...
}
```
It's worth noting here, that values in Redis are always represented as 'strings', hence non-string data, such as
floating-point values, must be converted to strings first. Additionally, the `redisxSetValue()` function works with
0-terminated string values only, but Redis may also store unterminated byte sequences of known length also. If you
find that you need to store an unterminated string (such as a binary sequence) as a value, you may just use the
lower-level `redisxArrayRequest()` instead to process a Redis `SET` or `HSET` command with explicit byte-length
specifications.
In the above example we have set the value using the interactive client to Redis, which means that the call will
return only after confirmation or result is received back from the server. As such, a subsequent `redisxGetValue()` of
the same table/key will be guaranteed to return the updated value always.
However, we could have set the new value asynchronously over the pipeline connection (by using `TRUE` as the last
argument). In that case, the call will return as soon as the request was sent to Redis (but not confirmed, nor
possibly transmitted yet!). As such, a subsequent `redisxGetValue()` on the same key/value field may race the request
in transit, and may return the previous value on occasion. So, it's important to remember that while pipelining can
make setting multiple Redis fields very efficient, we have to be careful about retrieving the same values afterwards
from the same program thread. (Arguably, however, there should never be a need to query values we set ourselves, since
we readily know what they are.)
Finally, if you want to set values for multiple fields in a Redis hash table atomically, you may use
`redisxMultiSet()`, which provides a high-level interface to the Redis `HMSET` command.
The functions `redisxGetKeys()` and `redisxGetTable()` allow to return the set of Redis keywords or all key/value
pairs in a table atomically. However, these commands can be computationally expensive for large tables and/or many
top-level keywords, which means that the Redis server may block for undesirably long times while the result is
computed.
This is where scanning offers a less selfish (hence much preferred) alternative. Rather than returning all the keys
or key/value pairs contained in a table atomically at once, it allows to do it bit by bit with byte-sized individual
transactions that are guaranteed to not block the Redis server long, so it may remain responsive to other queries
also. For the caller the result is the same (minus the atomicity), except that the result is computed via a series of
quick Redis queries rather than with one potentially very expensive query.
For example, to retrieve all top-level Redis keys, sorted alphabetically, using the scanning approach, you may write
something like:
```c
Redis *redis = ...
int nMatches; // We'll return the number of matching Redis keys here...
int status; // We'll return the error status here...
// Return all redis keywords starting with "system:"
char **keys = redisxScanKeys(redis, "system:*", &nMatches, &status);
if (status != X_SUCCESS) {
// Oops something went wrong...
...
}
// Use 'keys' as appropriate, possibly de-referencing values we want to
// retain in other persistent data structures...
...
// Once done using the 'keys' array, we should destroy it
redisxDestroyKeys(keys, nMatches);
```
Or, to retrieve the values from a hash table for a set of keywords that match a glob pattern:
```c
...
// Scan all key/value pairs in hash table "system:subsystem"
RedisEntry *entries = redisxScanTable(redis, "system:subsystem", "*", &nMatches, &status);
if (status != X_SUCCESS) {
// Oops something went wrong.
...
}
// Use 'entries' as appropriate, possibly de-referencing values we want to
// retain in other persistent data structures...
...
// Once done using the 'keys' array, we should destroy it
redisxDestroyEntries(entries, nMatches);
```
Finally, you may use `redisxSetScanCount()` to tune just how many results should individial scan queries should return
(but only if you are really itching to tweak it). Please refer to the Redis documentation on the behavior of the
`SCAN` and `HSCAN` commands to learn more.
-----------------------------------------------------------------------------
## Publish/subscribe (PUB/SUB) support
- [Broadcasting messages](#broadcasting-messages)
- [Subscriptions](#subscriptions)
It is simple to send messages to subscribers of a given channel:
```c
Redis *redis = ...
// publish a message to the "hello_channel" subscribers.
int status = redisxPublish(redis, "hello_channel", "Hello world!", 0);
```
The last argument is an optional string length, if readily available, or if sending a substring only, or else a string
or byte sequence that is not null-terminated. If zero is used for the length, as in the example above, it will
automatically determine the length of the 0-terminated string message using `strlen()`.
Alternatively, you may use the `redisxPublishAsync()` instead if you want to publish on a subscription client to which
you have already have exlusive access (e.g. after an appropriate `redisxLockConnected()` call).
Subscriptions work conceptually similarly to pipelined requests. To process incoming messages you need to first
specify one or more `RedisSubscriberCall` functions, which will process PUB/SUB notifications automatically, in the
background, as soon as they are received. Each `RedisSubscriberCall` can pre-filter the channels for which it receives
notifications, by defining a channel stem. This way, the given processor function won't even be invoked if a
notification on a completely different channel arrives. Still, each `RedisSubscriberCall` implementation should
further check the notifying channel name as appropriate to ensure that it is in fact qualified to deal with a given
message.
Here is an example `RedisSubscriberCall` implementation to process messages:
```c
void my_event_processor(const char *pattern, const char *channel, const char *msg, long len) {
// We'll print the message onto the console
printf("Incoming message on channel %s: %s\n", channel, msg == NULL ? "" : msg);
}
```
There are some basic rules (best practices) for message processing. They should be fast, and never block for extended
periods. If extensive processing is required, or may need to wait extensively for some resource or mutex locking, then
its best that the processing function simply places the incoming message onto a queue, and let a separate background
thread to the heavy lifting without holding up the subsription processing of other callback routines.
Also, it is important that the call should never attempt to modify or call `free()` on the supplied string arguments,
since that would interfere with other subscriber calls.
Once the function is defined, you can activate it via:
```c
Redis *redis = ...
int status = redisxAddSubscriber(redis, "event:", my_event_processor);
if (status != X_SUCCESS) {
// Oops, something went wrong...
...
}
```
We should also start subscribing to specific channels and/or channel patterns.
```c
Redis *redis = ...
// We subscribe to all channels that beging with "event:"...
int status = redisxSubscribe(redis, "event:*");
if (status != X_SUCCESS) {
// Oops, something went wrong...
...
}
```
The `redisxSuibscribe()` function will translate to either a Redis `PSUBSCRIBE` or `SUBSCRIBE` command, depending
whether the `pattern` argument contains globbing patterns or not (respectively).
Now, we are capturing and processing all messages published to channels whose name begins with `"event:"`, using our
custom `my_event_processor` function.
To end the subscription, we trace back the same steps by calling `redisxUnsubscribe()` to stop receiving further
messages to the subscription channel or pattern, and by removing the `my_event_procesor` subscriber function as
appropriate (provided no other subscription needs it) via `redisxRemoveSubscriber()`.
-----------------------------------------------------------------------------
## Atomic execution blocks and LUA scripts
- [Execution blocks](#execution-blocks)
- [LUA script loading and execution](#lua-script-loading-and-execution)
- [Custom Redis functions](#custom-functions)
Sometimes you may want to execute a series of Redis command atomically, such that nothing else may alter the database
while the set of commands execute, so that related values are always in a coherent state. For example, you want to set
or query a collection of related variables so they change together and are reported together. You have two choices.
(1) you can execute the Redis commands in an execution block, or else (2) load a LUA script onto the Redis server and
call it with some parameters (possibly many times over).
Execution blocks offer a fairly simple way of bunching together a set of Redis commands that need to be executed
atomically. Such an execution block in RedisX may look something like:
```c
Redis *redis = ...;
RESP *result;
// Obtain a lock on the client on which to execute the block.
// e.g. the interactive client channel.
RedisClient *cl = redisxGetLockedConnectedClient(redis, REDISX_INTERACTIVE_CHANNEL);
if (cl == NULL) {
// Abort: we don't have exclusive access to the client
return;
}
// -------------------------------------------------------------------------
// Start an atomic execution block
redisxStartBlockAsync(cl);
// Send a number of Async requests
redisxSendRequestAsync(cl, ...);
...
// Execute the block of commands above atomically, and get the resulting RESP
result = redisxExecBlockAsync(cl);
// -------------------------------------------------------------------------
// Release exlusive access to the client
redisxUnlockClient(cl);
// Inspect the RESP, etc...
...
```
If at any point things don't go according to plan in the middle of the block, you can call `redisAbortBlockAsync()` to
abort and discard all prior commands submitted in the execution block already. It is important to remembet that every
time you call `redisxStartBlockAsync()`, you must call either `redisxExecBlockAsync()` to execute it or else
`redisxAbortBlockAsync()` to discard it. Failure to do so, will effectively end you up with a hung Redis client.
### LUA script loading and execution
[LUA](https://www.lua.org/) scripting offers a more capable version of executing complex routines on the Redis server.
LUA is a scripting language akin to python, and allows you to add extra logic, string manipulation etc. to your Redis
queries. Best of all, once you upload the script to the server, it can reduce network traffic significantly by not
having to repeatedly submit the same set of Redis commands every single time. LUA scipts also get executed very
efficiently on the server, and produce only the result you want/need.
Assuming you have prepared your LUA script appropriately, you can upload it to the Redis server as:
```c
Redis *redis = ...
char *script = ... // The LUA script as a 0-terminated string.
char *scriptSHA1 = NULL; // We'll store the SHA1 sum of the script here
// Load the script onto the Redis server
int status = redixLoadScript(redis, script, &scriptSHA1);
if(status != X_SUCCESS) {
// Oops, something went wrong...
...
}
```
Redis will refer to the script by its SHA1 sum, so it's important keep a record of it. You'll call the script with
its SHA1 sum, a set of redis keys the script may use, and a set of other parameters it might need.
```c
Redis *redis = ...
char *keyArgs[] = { "my-redis-key-argument", NULL }; // NULL-terminated array of keyword arguments
char *params[] = { "some-string", "-1.11", NULL }; // NULL-terminated array of extra parameters
// Execute the script, with the specified keyword arguments and parameters
RESP *reply = redisxRunScript(redis, scriptSHA1, keyArgs, params);
// Check and inspect the reply
...
```
Or, you can use `redisxRunScriptAsync()` instead to send the request to run the script, and then collect the response
later, as appropriate.
One thing to keep in mind about LUA scripts is that they are not fully persistent. They will be lost each time the
Redis server is restarted.
Functions, introduced in Redis 7, offer another evolutionary step over the LUA scripting described above. Unlike
scripts, functions are persistent and they can be called by name rather than a cryptic SHA1 sum. Otherwise, they offer
more or less the same functionality as scripts. __RedisX__ does not currently have a built-in high-level support
for managing and calling user-defined functions, but it is a feature that may be added in the not-too-distant future.
Stay tuned.
-----------------------------------------------------------------------------
## Advanced queries and pipelining
- [Asynchronous client processing](#asynchronous-client-processing)
- [Pipelined transactions](#pipelined-transactions)
### Asynchronous client processing
Sometimes you might want to micro manage how requests are sent and responses to them are received. __RedisX__ provides
a set of asynchronous client functions that do that. (You've seen these already further above in the
[Pipelined transaction](#pipelined-transactions) section.) These functions should be called with the specific client's
mutex locked, to ensure that other threads do not interfere with your sequence of requests and responses. E.g.:
```c
// Obtain the appropriate client with an exclusive lock on it.
RedisClient *cl = redisxGetLockedConnectedClient(...);
if (cl == NULL) {
// Abort: no such client is connected
return;
}
// Now send commands, and receive responses as you like using the redisx...Async() calls
...
// When done, release the lock
redisxUnlockClient(cl);
```
While you have the exclusive lock you may send any number of requests, e.g. via `redisxSendRequestAsync()` and/or
`redixSendArrayRequestAsync()`. Then collect replies either with `redisxReadReplyAsync()` or else
`redisxIgnoreReplyAsync()`. For example, the basic anatomy of sending a single request and then receiving a response,
while we have exclusive access to the client, might look something like this:
```c
...
// Send a command to Redis
int status = redisxSendRequestAsync(cl, ...);
if(status == X_SUCCESS) {
// Read the response
RESP *reply = redisxReadReplyAsync(cl);
// check and process the response
if(redisxCheckRESP(reply, ...) != X_SUCCESS) {
// Ooops, not the reply what we expected...
...
}
else {
// Process the response
...
}
// Destroy the reply
redisxDestroyRESP(reply);
}
...
```
For the best performance, you may want to leave the processing of the replies until after you unlock the client. I.e.,
you only block other threads from accessing the client while you send off the requests and collect the corresponding responses. You can then analyze the responses at your leisure outside of the mutexed section.
In some cases you may be OK with just firing off some Redis commands, without necessarily caring about responses.
Rather than ignoring the replies with `redisxIgnoreReplyAsync()` you might call `redisxSkipReplyAsync()` instead
__before__ `redisxSendRequestAsync()` to instruct Redis to not even bother about sending a response to your request
(it saves time and network bandwidth!):
```c
// We don't want to receive a response to our next command...
int status = redisxSkipReplyAsync(cl);
if (status == X_SUCCESS) {
// Now send the request...
status = redisxSendRequest(cl, ...);
}
if (status != X_SUCCESS) {
// Ooops, the request did not go through...
...
}
```
Of course you can build up arbitrarily complex set of queries and deal with a set of responses in different ways. Do
what works best for your application.
Depending on round-trip times over the network, interactive queries may be suitable for running up to a few thousand
queries per second. For higher throughput (up to ~1 million Redis transactions per second) you may need to access the
Redis database in pipelined mode. RedisX provides a dedicated pipeline client/channel to the Redis server (provided
the option to enable it has been used when `redixConnect()` was called).
In pipeline mode, requests are sent to the Redis server over the pipeline client in quick succession, without waiting
for responses to return for each one individually. Responses are processed in the background by a designated callback
function (or else discarded if no callback function has been set). This is what a callback function looks like:
```c
// Your own function to process responses to pipelined requests...
void my_resp_processor(RESP *r) {
// Do what you need to do with the asynchronous responses
// that come from Redis to bulk requests
...
}
```
Before sending the pipelined requests, the user first needs to specify the function to process the responses, e.g.:
```c
Redis *redis = ...
redisxSetPipelineConsumer(redis, my_resp_processor);
```
Request are sent via the `redisxSendRequestAsync()` and `redisxSendArrayRequestAsync()` functions. Note again, the
`Async` naming, which indicates the asynchronous nature of this calls -- and which suggests that these should be called
with the approrpiate mutex locked to prevent concurrency issues and to maintain a predictable order (very important!)
for processing the responses.
```c
Redis *redis = ...
// We'll use a dedicated pipeline connection for asynchronous requests
// This way, interactive requests can still be sent independently on the interactive
// channel independently, if need be.
RedisClient *pipe = redisxGetLockedConnectedClient(redis, REDISX_PIPELINE_CHANNEL);
// Check that the client is valid...
if (pipe == NULL) {
// Abort: we do not appear to have an active pipeline connection...
return;
}
// -------------------------------------------------------------------------
// Submit a whole bunch of asynchronous requests, e.g. from a loop...
for (...) {
int status = redisxSendRequestAsync(pipe, ...);
if (status != X_SUCCESS) {
// Oops, that did not go through...
...
}
else {
// We probably want to keep a record of what was requested and in what order
// so our processing function can make sense of the reponses as they arrive
// (in the same order...)
...
}
}
// -------------------------------------------------------------------------
// Release the exclusive lock on the pipeline channel, so
// other threads may use it now that we sent off our requests...
redisxUnlockClient(pipe);
```
It is important to remember that on the pipeline client you should never try to process responses directly from the
same function from which commands are being sent. That's what the interactive connection is for. Pipeline responses
are always processed by a background thread (or, if you don't specify your callback function they will be discarded).
The only thing your callback function can count on is that the same number of responses will be received as the number
of asynchronous requests that were sent out, and that the responses arrive in the same order as the order in which the
requests were sent.
It is up to you and your callback function to keep track of what responses are expected and in what order. Some best
practices to help deal with pipeline responses are summarized here:
- Use `redisxSkipReplyAsync()` prior to sending pipeline requests for which you do not need a response. (This way
your callback does not have to deal with unnecessary responses at all.
- For requests that return a value, keep a record (in a FIFO) of the expected types and your data that depends on
the content of the responses. For example, for pipelined `HGET` commands, your FIFO should have a record that
specifies that a bulk string response is expected, and a pointer to data which is used to store the returned value
-- so that you pipeline response processing callback function can check that the reponse is the expected type
(and size) and knows to assign/process the response appropriately to your application data.
- You may insert Redis `PING`/`ECHO` commands to section your responses, or to provide directives to your pipeline
response processor function. You can tag them uniquely so that the echoed responses can be parsed and interpreted
by your callback function. For example, you may send a `PING`/`ECHO` commands to Redis with the tag
`"@my_resp_processor: START sequence A"`, or something else meaningful that you can uniquely distinguish from all
other responses that you might receive.
__RedisX__ optimizes the pipeline client for high throughput (bandwidth), whereas the interactive and subscription
clients are optimized for low-latency, at the socket level.
-----------------------------------------------------------------------------
The principal error handling of RedisX is an extension of that of __xchange__, with further error codes defined in
`redisx.h`. The RedisX functions that return an error status (either directly, or into the integer designated by a
pointer argument), can be inspected by `redisxErrorDescription()`, e.g.:
```c
Redis *redis ...
int status = redisxSetValue(...);
if (status != X_SUCCESS) {
// Ooops, something went wrong...
fprintf(stderr, "WARNING! set value: %s", redisErrorDescription(status));
...
}
```
In addition you can define your own handler function to deal with transmission (send/receive) errors, by defining
your own `RedisErrorHandler` function, such as:
```c
void my_error_handler(Redis *redis, enum redisx_channel channel, const char *op) {
fprintf(stderr, "ERROR! %s: Redis at %s, channel %d\n", op, redis->id, channel);
}
```
Then activate it as:
```c
Redis *redis = ...
redisxSetTransmitErrorHandler(redis, my_error_handler);
```
After that, every time there is an error with sending or receiving packets over the network to any of the Redis
clients used, your handler will report it the way you want it.
-----------------------------------------------------------------------------
You can enable verbose output of the __RedisX__ library with `xSetVerbose(boolean)`. When enabled, it will produce
status messages to `stderr`so you can follow what's going on. In addition (or alternatively), you can enable debug
messages with `xSetDebug(boolean)`. When enabled, all errors encountered by the library (such as invalid arguments
passed) will be printed to `stderr`, including call traces, so you can walk back to see where the error may have
originated from. (You can also enable debug messages by default by defining the `DEBUG` constant for the compiler,
e.g. by adding `-DDEBUG` to `CFLAGS` prior to calling `make`).
In addition, you can use `redisxDebugTraffic(boolean)` to debug low-level traffic to/from the Redis server, which
prints excerpts of all incoming and outgoing messages from/to Redis to `stderr`.
For helping to debug your application, the __xchange__ library provides two macros: `xvprintf()` and `xdprintf()`,
for printing verbose and debug messages to `stderr`. Both work just like `printf()`, but they are conditional on
verbosity being enabled via `xSetVerbose(boolean)` and `xSetDebug(boolean)`, respectively. Applications using
__RedisX__ may use these macros to produce their own verbose and/or debugging outputs conditional on the same global
settings.
-----------------------------------------------------------------------------
Some obvious ways the library could evolve and grow in the not too distant future:
- Automated regression testing and coverage tracking.
- Keep track of subscription patterns, and automatically resubscribe to them on reconnecting.
- Support for the [RESP3](https://github.com/antirez/RESP3/blob/master/spec.md) standard and Redis `HELLO`.
- Support for [Redis Sentinel](https://redis.io/docs/latest/develop/reference/sentinel-clients/) clients, for high-availability server configurations.
- TLS support (perhaps...)
- Add high-level support for managing and calling custom Redis functions.
- Add support for `CLIENT TRACKING` / `CLIENT CACHING`.
- Add more high-level redis commands, e.g. for lists, streams, etc.
If you have an idea for a must have feature, please let me (Attila) know. Pull requests, for new features or fixes to
existing ones, are especially welcome!
-----------------------------------------------------------------------------
Copyright (C) 2024 Attila Kovács