https://github.com/nahratzah/hoard_cache

A cache library for c++
https://github.com/nahratzah/hoard_cache
Last synced: 22 days ago
JSON representation
A cache library for c++
Host: GitHub
URL: https://github.com/nahratzah/hoard_cache
Owner: nahratzah
License: bsd-2-clause
Created: 2022-01-07T09:05:55.000Z (over 3 years ago)
Default Branch: main
Last Pushed: 2023-03-13T16:54:03.000Z (about 2 years ago)
Last Synced: 2023-03-29T13:41:25.876Z (about 2 years ago)
Language: C++
Size: 326 KB
Stars: 4
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project

README

        # LibHoard

This is a library that allows for caching and memoidization.

It's named after a behaviour of dragons, keeping lots of shiny things.

This is a header-only library.

# Work in Progress

Still working on this, you could play with this.

The general shape of the code is pretty much stable, but some implementation is still shifting around.

# Quick Start

To include the cache in your code, you can simply add the `libhoard/include/` directory to your include-path.

Or if you use cmake, you can include the project (using `add_subdirectory(libhoard)` for example) and then link your project against the library: `target_link_library(your_target PUBLIC libhoard)`.

## The Most Trivial Cache

```

#include 

using key_type = int;

using mapped_type = std::string;

libhoard::cache c;

```

This cache:

- will not expire any elements (and thus grow without bounds).

- you can add elements to it, and look them up.

- elements not in the cache cannot be resolved.

Looking up a value: `std::optional = c.get(1);`

Adding/replacing a value:

- `c.emplace(17, "seventeen");`

- `c.emplace(std::piecewise_construct, std::make_tuple(17), std::make_tuple("seventeen"));`

Adding a value, if there isn't already one present:

- `std::string v = c.get_or_emplace(18, "eighteen");`

- `std::string v = c.get_or_emplace(std::piecewise_construct, std::make_tuple(18), std::make_tuple("eighteen"));`

Removing a value from the cache:

- `c.erase(17);`

Emptying the cache:

- `c.clear();`

# Policies

The cache accepts policies: `class cache`.

## Limiting the Size of the Cache

```

#include 

#include 

using key_type = int;

using mapped_type = std::string;

libhoard::cache<

    key_type, mapped_type,

    libhoard::max_size_policy

    > c(libhoard::max_size_policy(100));

```

Constructs a cache which holds up to 100 elements.

## Limiting the Age of Items in the Cache

```

#include 

#include 

using namespace std::literals::chrono_literals;

using key_type = int;

using mapped_type = std::string;

libhoard::cache<

    key_type, mapped_type,

    libhoard::max_age_policy

    > c(libhoard::max_age_policy(5m));

```

Constructs a cache where elements will be removed after 5 minutes.

## Thread-safe

Usually, when the cache is declared, it is thread-safe.

To make the cache not thread-safe, you'll want to add the `libhoard::thread_unsafe_policy`.

If you want to be explicit, you can add the `libhoard::thread_safe_policy`.

The refresh policy requires the `libhoard::thread_safe_policy` and introduces it as a dependency.

This ensures you can't accidentally introduce the `libhoard::thread_unsafe_policy` on a cache that requires thread safety to function correctly.

```

#include 

#include 

using key_type = int;

using mapped_type = std::string;

libhoard::cache<

    key_type, mapped_type,

    libhoard::thread_safe_policy // this is the default, you can omit this

    > c;

libhoard::cache<

    key_type, mapped_type,

    libhoard::thread_unsafe_policy

    > d;

```

## Smart Pointers and Singletons

When using smart pointers (such as [`std::shared_ptr`](https://en.cppreference.com/w/cpp/memory/shared_ptr)) you can set up the cache to use weak pointers when shrinking the cache.

The advantage is that, as long as your weak-pointer element is still strongly referenced elsewhere in the code, the cache will keep returning it.

For example, you could do something like:

```

using file_contents_ptr = std::shared_ptr;

struct file_contents_resolver {

  auto operator()(std::string filename) -> file_contents_ptr {

    std::ifstream f(filename);

    std::ostringstream oss;

    oss << f.rdbuf();

    return std::make_shared(oss.str());

  }

};

using file_cache = libhoard::cache<

    std::string, file_contents_ptr,

    libhoard::pointer_policy<>, // tells the cache that we're using shared pointers

    libhoard::weaken_policy<>,  // tells the cache to use weak_ptr, when shrinking the cache

    libhoard::max_size_policy,  // tells the cache we'll have a size limit

    libhoard::resolver_policy // use the above resolver

    >;

int main() {

  file_cache fc(libhoard::max_size_policy(2));

  file_contents_ptr a_txt = fc.get("a.txt");

  // cause a_txt to expire because the cache grows too large

  fc.get("b.txt");

  fc.get("b.txt");

  fc.get("c.txt");

  fc.get("c.txt");

  // despite having expired a_txt, because it was still referenced in our code,

  // the entry was preserved in the cache

  assert(a_txt == fc.get("a.txt");

}

```

This behaviour can be used to create singletons.

Suppose we have a monitoring tool, that needs to keep track of metrics across a large group of machines.

Each machine has 1000 metrics, and they're all named the same.

We have 10k machines.

And each metric name takes up, say, 60 bytes.

So in total, we would need `60 bytes * 1k * 10k = 600 MB` just to store the metric names.

But if we use a cache instead, we can deduplicate duplicate names:

```

using metric_name_cache = libhoard::cache<

    std::string, std::shared_ptr,

    libhoard::pointer_policy<>,

    libhoard::weaken_policy<>,

    libhoard::max_size_policy,

    libhoard::resolver_policy (*)(std::string)>>;

metric_name_cache metric_names(

    libhoard::max_size_policy(100),

    libhoard::resolver_policy (*)(std::string)>>([](std::string s) { return std::make_shared(s); }));

metric_names.get("my_metric_name"); // returns the same std::shared_ptr of "my_metric_name" each time

```

We changed the `std::string` to a `std::shared_ptr`, and now we only need `60 bytes * 1k = 60 kB` of memory.

## Installing a Resolver

Instead of using `cache::emplace` to insert elements into the cache, you can equip the cache with a resolver function.

For each lookup in the cache, that it doesn't hold a value for, it'll invoke the resolver function to perform a lookup.

```

#include 

#include 

using key_type = int;

using mapped_type = std::string;

libhoard::cache<

    key_type, mapped_type,

    libhoard::resolver_policy>

    > c(libhoard::resolver_policy>(

        [](int key) -> {

          std::ostringstream s;

          s << key;

          return s.str();

        }));

```

Now, if a value isn't present, it'll invoke the function. :)

```

c.get(5); // std::string("5")

c.get(100); // std::string("100")

```

## Installing an Async Resolver

Instead of using a synchronous resolver, you can install an asynchronous resolver.

An asynchronous resolver is one that completes using a callback.

```

#include 

#include  // Yes, the regular and asynchronous resolver share a header file.

using key_type = int;

using mapped_type = std::string;

struct resolver {

  template

  void operator()(std::shared_ptr callback, int key) {

    std::async(std::launch_async,

        [callback, key]() {

          std::ostringstream s;

          s << key;

          callback->assign(s.str());

        });

  }

};

libhoard::cache<

    key_type, mapped_type,

    libhoard::async_resolver_policy

    > c(libhoard::resolver_policy(resolver()));

```

An async resolver allows other calls into the cache while the lookup is progressing.

If another lookup for the same key arrives, it'll share the existing lookup.

When using an async resolver, you'll also need to perform an asyc lookup:

```

using error_type = std::exception_ptr; // The error type is controlled using the `libhoard::error_policy` policy.

std::future> future_value = cache.get(3);

future_value.get(); // std::variant(std::in_place_index<0>, "3")

```

## Refreshing Values

You can set up the cache to periodically refresh values in the cache.

```

#include 

#include 

#include 

using namespace std::literals::chrono_literals;

using key_type = int;

using mapped_type = std::string;

libhoard::cache<

    key_type, mapped_type,

    libhoard::resolver_policy>,

    libhoard::refresh_policy>

    > c(

        libhoard::resolver_policy>(

            [](int key) -> {

              std::ostringstream s;

              s << key;

              return s.str();

            }),

        libhoard::refresh_policy(

            1m, 15m

        ));

```

The refresh-delay (`1m` in this example) causes the cache to refresh the value every `1m`.

The idle-delay (`15m` in this example) makes it so the cache will stop refreshing and remove the value, if it has not been accessed in `15m`.

If the idle-delay is `0s`, then the cache will keep refreshing indefinitely (or until the value is removed for another reason).

The refresh policy is implemented by running a single thread that initiates resolution.

It'll work with both synchronous and asynchronous resolution.

Because the refresh policy requires that the cache is thread-safe, it'll pull in the thread-safe-policy automatically.

## Negative Cache (Caching Errors)

To get the cache to cache errors, we use the error policy.

```

#include 

#include 

libhoard::cache<

    key_type, mapped_type,

    libhoard::error_policy // std::exception_ptr is the default error type and can be omitted

    > c;

```

Adding this policy will cache errors, until any of the cache rules decides to remove or expire the value.

(In this example, since the cache doesn't have any rules regarding expiry or size, the error would be cached forever.)

It is recommended to use the `libhoard::error_max_age_policy` to get errors to only be cached for a limited time.

```

#include 

#include 

#include 

libhoard::cache<

    key_type, mapped_type,

    libhoard::error_policy // std::exception_ptr is the default error type and can be omitted

    libhoard::error_max_age_policy

    > c(libhoard::error_max_age_policy(5m));

```

This would cause errors to be cached for at most 5 minutes.

# In Combination with Asio

You can use this in combination with [asio](https://think-async.com/Asio/).

## Asio Resolver

The asio resolver is always async, and resembles the regular async resolver.

Except that it has an asio executor associated with it.

```

#include 

#include 

#include 

using key_type = int;

using mapped_type = std::string;

struct resolver {

  template

  void operator()(std::shared_ptr callback, int key) {

    executor.post(

        [callback, key]() {

          std::ostringstream s;

          s << key;

          callback->assign(s.str()); // fills in the cache.

        });

  }

  asio::system_executor executor;

};

asio::io_context ioctx;

libhoard::cache<

    key_type, mapped_type,

    libhoard::thread_safe_policy,

    libhoard::async_resolver_policy

    > c(libhoard::resolver_policy(resolver(), ioctx.get_executor()));

```

## Asio Refresh Policy

When using asio, we can also install a refresh policy that uses asio.

```

#include 

#include 

#include 

using key_type = int;

using mapped_type = std::string;

struct resolver {

  template

  void operator()(std::shared_ptr callback, int key) {

    std::async(std::launch_async,

        [callback, key]() {

          std::ostringstream s;

          s << key;

          callback->assign(s.str());

        });

  }

};

asio::io_context ioctx;

libhoard::cache<

    key_type, mapped_type,

    libhoard::thread_safe_policy,

    libhoard::async_resolver_policy,

    libhoard::asio_refresh_policy

    > c(libhoard::asio_resolver_policy(resolver(), ioctx.get_executor()),

        libhoard::asio_refresh_policy(ioctx.get_executor(), 1m, 15m));

```

Please refer to the regular resolver for the meaning of `1m` and `15m`.

The refresh-policy will run its timers on the associated executor.

But the resolver will run on its own executor.

You can mix-and-match regular and asio resolver-policy and refresh-policy.

This is very useful if you want to use a synchronous resolver-policy.

## Asio Dynamic Refresh Policy

Instead of using a fixed refresh timer, you can also use a function to declare an expiry dependent on the lookup value.

```

#include 

#include 

#include 

using key_type = int;

using mapped_type = std::string;

struct resolver {

  template

  void operator()(std::shared_ptr callback, int key) {

    std::async(std::launch_async,

        [callback, key]() {

          std::ostringstream s;

          s << key;

          callback->assign(s.str());

        });

  }

};

struct expire_functor;

asio::io_context ioctx;

libhoard::cache<

    key_type, mapped_type,

    libhoard::thread_safe_policy,

    libhoard::async_resolver_policy,

    libhoard::asio_refresh_fn_policy

    > c(libhoard::asio_resolver_policy(resolver(), ioctx.get_executor()),

        libhoard::asio_refresh_fn_policy(ioctx.get_executor(), expire_functor(), 15m));

```

The expire function must be a functor that takes a mapped-type as argument (`std::string` in our example) and returns either a `std::chrono::time_point` or a `std::chrono::duration`.

For example

```

struct expire_functor {

  auto operator()(const std::string& v) const -> std::chrono::steady_clock::duration {

    return 3m;

  }

};

```

would be the same as:

```

struct expire_functor {

  auto operator()(const std::string& v) const -> std::chrono::steady_clock::time_point {

    return std::chrono::steady_clock::now() + 3m;

  }

};

```

The code will figure out if a `time_point` or `duration` is used and automatically adapt accordingly.

# Extending

You can write your own policies to extend functionality.

For example you could use the asio code as a guide to adapt to your own favourite event-library.

Or for example create your own cost function to keep the cache size under control.

A lot of the documentation required to extend the code I'll need to write.

Please accept my apologies for the lack of documentation. <3
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/nahratzah/hoard_cache

Awesome Lists containing this project

README
