Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/collective/collective.purgebyid

Last synced: 3 months ago
JSON representation

Host: GitHub
URL: https://github.com/collective/collective.purgebyid
Owner: collective
License: other
Created: 2013-08-14T09:15:03.000Z (almost 11 years ago)
Default Branch: master
Last Pushed: 2024-02-29T14:14:12.000Z (4 months ago)
Last Synced: 2024-04-09T22:09:20.051Z (3 months ago)
Language: Python
Homepage:
Size: 120 KB
Stars: 5
Watchers: 118
Forks: 3
Open Issues: 0
Metadata Files:
- Readme: README.rst
- Changelog: CHANGES.rst
- License: LICENSE.GPL

Lists

awesome-plone - collective.purgebyid - Use tag-based cache invalidation in Plone (e.g. with Varnish's xkey module). (Sysadmin)

README

collective.purgebyid
====================

.. image:: https://img.shields.io/pypi/v/collective.purgebyid.svg
:target: https://pypi.python.org/pypi/collective.purgebyid/
:alt: Latest Version

.. image:: https://img.shields.io/pypi/pyversions/collective.purgebyid.svg?style=plastic
:target: https://pypi.python.org/pypi/collective.purgebyid/
:alt: Supported - Python Versions

.. image:: https://img.shields.io/pypi/dm/collective.purgebyid.svg
:target: https://pypi.python.org/pypi/collective.purgebyid/
:alt: Number of PyPI downloads

.. image:: https://img.shields.io/pypi/l/collective.purgebyid.svg
:target: https://pypi.python.org/pypi/collective.purgebyid/
:alt: License

.. image:: https://github.com/collective/collective.purgebyid/actions/workflows/tests.yml/badge.svg
:target: https://github.com/collective/collective.purgebyid/actions
:alt: Tests

.. image:: https://coveralls.io/repos/github/collective/collective.purgebyid/badge.svg?branch=master
:target: https://coveralls.io/github/collective/collective.purgebyid?branch=master
:alt: Coverage

collective.purgbyid is a new method for cache invalidation of Plone
based web sites. It uses the idea of adding an extra header, called
X-Ids-Involved, which contains the uuids of the objects involved in the
construction of the resources. For example, an image contains just its
uuid::

% wget -S http://localhost:8080/Plone/image01
...
X-Ids-Involved: #c8d7c0bc2b794325b916d990de91d7ee#

Other pages may be more complicated. Then a new purge rewrite rule adds
a custom URL to the set of URLs to purge: the "purge by id" custom URL
is in the form /@@purgebyid/ where UUID is the object's uuid to be
purged.

Last, Varnish is configured so that, when an URL /@@purgebyid/ is
purged, it will ban all the objects that match an X-Ids-Involved header
of the right type (i.e. containing the uuid of the resource to purge).
This means that when a resources is purged, it is enough to purge also
it /@@purgebyid/ URL because it will be Varnish responsibily to
also catch all of the occurrencies of the resources whenever the URL
which is used to access it.

Varnish without xkey varnish module
-----------------------------------

Without the xkey module, the way to purge a resource is to ban all objects
which have the X-Ids-Involved header with the id of the resource to be purged.

For a better understanding of the differences between the two approaches (ban vs. purge), please read:
https://varnish-cache.org/docs/trunk/users-guide/purging.html

Config example::

sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send PURGE requests."));
}
if (req.url ~ "^/@@purgebyid/") {
ban("obj.http.x-ids-involved ~ #" + regsub(req.url, "^/@@purgebyid/", "") + "#");
return(synth(200, "Ban added"));
}
return(purge);
}
}

sub vcl_deliver {
unset resp.http.x-ids-involved;
}

Varnish with xkey varnish module
--------------------------------

By default, Varnish uses the URL as the hash key for purging, but with
the xkey module (https://github.com/varnish/varnish-modules/blob/master/src/vmod_xkey.vcc)
there comes a secondary hash for doing so. Cached objects
being tagged can be specifically purged for a more targeted cache control.

To have xkey working, it is mandatory to provide a special HTTP header called
"Xkey" which contains all the tags (separated by white-space). Few additional codes in
the `vcl_backend_response` transforms the X-Ids-Involved header into an XKey.

Config example::

import xkey;

sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send PURGE requests."));
}
if (req.url ~ "^/@@purgebyid/") {
set req.http.n-gone = xkey.purge(regsub(req.url, "^/@@purgebyid/", ""));
# or: set req.http.n-gone = xkey.softpurge(regsub(req.url, "^/@@purgebyid/", ""));
return (synth(200, "Invalidated "+req.http.n-gone+" objects"));
}
}
return(purge);
}

sub vcl_backend_response {
if (beresp.http.x-ids-involved) {
set beresp.http.xkey = regsuball(beresp.http.x-ids-involved, "#", " ");
}
}

sub vcl_deliver {
unset resp.http.x-ids-involved;
unset resp.http.xkey;
}

A softpurge differs from a regular purge in that it resets an
object's TTL but keeps it available for grace mode and conditional
requests for the remainder of its configured grace and keep time.

How does it work? How to extend it?
-----------------------------------

During the publishing process all involved IDs (UUIDs and custom IDs) are collected
(by subscribing to IPubAfterTraversal).

Important are the adapters for IInvolvedID, which are responsible for collecting IDs for their given context.
The base implementation looks for the UUIDs, but may be specialized for your custom content types.

Apart from the adapter approach, there is the inline approach. You may call the following methods
from collective.purgebyid.api:

* mark_involved_objects(request, objs, stoponfirst=False)
* mark_involved(request, single_id)

Whereas the first method uses internally the adapters for IInvolvedIDs for the given objects,
the second one allows setting any arbitrary IDs.
These methods combined might be used in your views, whenever a certain object or part is being rendered.

Additionally, there is a utility browser view "purgebyid", that can be used in your template as follows:

.. code-block:: xml

...

Alternatively, you can again set arbitrary IDs:

.. code-block:: xml

After having collected all IDs a ITransform adapter puts the expected `X-Ids-Involved` header in
the HTTP response header.

When Plone sends a purge request to the configured Cache Proxy, it sends additionally a specialized
request for handling objects with tags.

References
----------

* Blog post https://www.biodec.com/it/blog/migliorare-la-gestione-del-purge-caching-in-plone-collective-purgebyid (italian language)