{"id":24407813,"url":"https://github.com/carlosabalde/libvmod-cfg","last_synced_at":"2025-07-14T02:32:53.255Z","repository":{"id":78611900,"uuid":"59377453","full_name":"carlosabalde/libvmod-cfg","owner":"carlosabalde","description":"VMOD useful to access to contents of environment variables and local or remote files from VCL, usually for configuration purposes, including execution of Lua and JavaScript programs","archived":false,"fork":false,"pushed_at":"2025-03-18T18:05:29.000Z","size":3161,"stargazers_count":25,"open_issues_count":2,"forks_count":3,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-19T05:24:57.579Z","etag":null,"topics":["duktape","javascript","lua","luajit","varnish-cache","vmods"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/carlosabalde.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-05-21T19:16:59.000Z","updated_at":"2025-02-21T13:11:27.000Z","dependencies_parsed_at":"2024-01-05T16:28:35.940Z","dependency_job_id":"263e9920-6383-4953-b258-cc7284e4de49","html_url":"https://github.com/carlosabalde/libvmod-cfg","commit_stats":null,"previous_names":[],"tags_count":76,"template":false,"template_full_name":null,"purl":"pkg:github/carlosabalde/libvmod-cfg","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carlosabalde%2Flibvmod-cfg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carlosabalde%2Flibvmod-cfg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carlosabalde%2Flibvmod-cfg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carlosabalde%2Flibvmod-cfg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/carlosabalde","download_url":"https://codeload.github.com/carlosabalde/libvmod-cfg/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carlosabalde%2Flibvmod-cfg/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265233753,"owners_count":23731825,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["duktape","javascript","lua","luajit","varnish-cache","vmods"],"created_at":"2025-01-20T05:18:07.262Z","updated_at":"2025-07-14T02:32:53.176Z","avatar_url":"https://github.com/carlosabalde.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n.. image:: https://github.com/carlosabalde/libvmod-cfg/actions/workflows/main.yml/badge.svg?branch=master\n   :alt: GitHub Actions CI badge\n   :target: https://github.com/carlosabalde/libvmod-cfg/actions\n.. image:: https://codecov.io/gh/carlosabalde/libvmod-cfg/branch/master/graph/badge.svg\n   :alt: Codecov badge\n   :target: https://codecov.io/gh/carlosabalde/libvmod-cfg\n\nVMOD useful to access to contents of environment variables and local or remote files from VCL, usually for configuration purposes.\n\nCurrently (1) JSON files; (2) Python's ConfigParser .INI-like files; (3) files containing collections of pattern matching rules; (4) Lua 5.1 scripts; and (5) ECMAScript (i.e. JavaScript) scripts are supported. Remote files can be accessed via HTTP or HTTPS.\n\nWondering why I created this VMOD? How it could make your life easier? I wrote a blog post with some answers: `Moving logic to the caching edge (and back) \u003chttps://www.carlosabalde.com/blog/2018/06/27/moving-logic-to-the-caching-edge-and-back\u003e`_.\n\nLooking for official support for this VMOD? Please, contact `Allenta Consulting \u003chttps://www.allenta.com\u003e`_, a `Varnish Software Premier Partner \u003chttps://www.varnish-software.com/partner/allenta-consulting\u003e`_.\n\nSYNOPSIS\n========\n\nimport cfg;\n\n::\n\n    ##\n    ## Environment variables.\n    ##\n\n    Object env()\n    Method STRING .dump(BOOL stream=0, STRING prefix=\"\")\n\n    Method BOOL .is_set(STRING name)\n    Method STRING .get(STRING name, STRING fallback=\"\")\n\n    ##\n    ## JSON \u0026 INI files.\n    ##\n\n    Object file(\n        STRING location,\n        STRING backup=\"\",\n        BOOL automated_backups=1,\n        INT period=60,\n        BOOL ignore_load_failures=1,\n        INT curl_connection_timeout=0,\n        INT curl_transfer_timeout=0,\n        BOOL curl_ssl_verify_peer=0,\n        BOOL curl_ssl_verify_host=0,\n        STRING curl_ssl_cafile=\"\",\n        STRING curl_ssl_capath=\"\",\n        STRING curl_proxy=\"\",\n        ENUM { ini, json } format=\"ini\",\n        STRING name_delimiter=\":\",\n        STRING value_delimiter=\";\")\n    Method BOOL .reload(BOOL force_backup=0)\n    Method STRING .dump(BOOL stream=0, STRING prefix=\"\")\n    Method VOID .inspect()\n\n    Method BOOL .is_set(STRING name)\n    Method STRING .get(STRING name, STRING fallback=\"\")\n\n    ##\n    ## Pattern matching rules.\n    ##\n\n    Object rules(\n        STRING location,\n        STRING backup=\"\",\n        BOOL automated_backups=1,\n        INT period=60,\n        BOOL ignore_load_failures=1,\n        INT curl_connection_timeout=0,\n        INT curl_transfer_timeout=0,\n        BOOL curl_ssl_verify_peer=0,\n        BOOL curl_ssl_verify_host=0,\n        STRING curl_ssl_cafile=\"\",\n        STRING curl_ssl_capath=\"\",\n        STRING curl_proxy=\"\")\n    Method BOOL .reload(BOOL force_backup=0)\n    Method VOID .inspect()\n\n    Method STRING .get(STRING value, STRING fallback=\"\")\n\n    ##\n    ## Lua \u0026 JavaScript scripts.\n    ##\n\n    Object script(\n        STRING location=\"\",\n        STRING backup=\"\",\n        BOOL automated_backups=1,\n        INT period=60,\n        BOOL ignore_load_failures=1,\n        ENUM { lua, javascript } type=\"lua\",\n        INT max_engines=128,\n        INT max_cycles=0,\n        INT min_gc_cycles=100,\n        BOOL enable_sandboxing=1,\n        INT lua_gc_step_size=100,\n        BOOL lua_remove_loadfile_function=1,\n        BOOL lua_remove_dotfile_function=1,\n        BOOL lua_load_package_lib=0,\n        BOOL lua_load_io_lib=0,\n        BOOL lua_load_os_lib=0,\n        INT curl_connection_timeout=0,\n        INT curl_transfer_timeout=0,\n        BOOL curl_ssl_verify_peer=0,\n        BOOL curl_ssl_verify_host=0,\n        STRING curl_ssl_cafile=\"\",\n        STRING curl_ssl_capath=\"\",\n        STRING curl_proxy=\"\")\n    Method BOOL .reload(BOOL force_backup=0)\n    Method VOID .inspect()\n\n    Method VOID .init(STRING code=\"\")\n    Method VOID .push(STRING arg)\n    Method VOID .execute(BOOL gc_collect=0, BOOL flush_jemalloc_tcache=1)\n\n    Method BOOL .result_is_error()\n    Method BOOL .result_is_{nil,null}()\n    Method BOOL .result_is_boolean()\n    Method BOOL .result_is_number()\n    Method BOOL .result_is_string()\n    Method BOOL .result_is_{table,array}()\n\n    Method STRING .get_result()\n\n    Method BOOL .get_boolean_result()\n    Method REAL .get_decimal_result()\n    Method INT .get_integer_result()\n    Method STRING .get_string_result()\n\n    Method INT .get_{table,array}_result_length()\n    Method BOOL .{table,array}_result_is_error(INT index)\n    Method BOOL .{table,array}_result_is_{nil/null}(INT index)\n    Method BOOL .{table,array}_result_is_boolean(INT index)\n    Method BOOL .{table,array}_result_is_number(INT index)\n    Method BOOL .{table,array}_result_is_string(INT index)\n    Method BOOL .{table,array}_result_is_{table/array}(INT index)\n    Method STRING .get_{table,array}_result_value(INT index)\n\n    Method VOID .free_result()\n\n    Method STRING .stats()\n    Method INT .counter(STRING name)\n\nEXAMPLE\n=======\n\nEnvironment variables\n---------------------\n\n::\n\n    export VCL_SETTINGS=file:///etc/varnish/vcl.ini\n\n/etc/varnish/vcl.ini\n--------------------\n\n::\n\n    server: ACME\n\n    [joke]\n    start: 1459468800\n    stop: 1459555200\n\nhttps://www.example.com/ttls.rules\n----------------------------------\n\n::\n\n    (?i)\\.(?:jpg|png|svg)(?:\\?.*)?$      -\u003e 7d\n    (?i)^www\\.(?:foo|bar)\\.com(?::\\d+)?/ -\u003e 1h\n\nhttps://www.example.com/backends.lua\n------------------------------------\n\n::\n\n    local host = string.gsub(string.lower(ARGV[0]), ':%d+$', '')\n    local url = string.lower(ARGV[1])\n\n    varnish.log('Running Lua backend selection logic')\n\n    -- Remember Lua's pattern matching is not equivalent to POSIX regular\n    -- expressions. Check https://www.lua.org/pil/20.2.html and\n    -- http://lua-users.org/wiki/PatternsTutorial for details.\n    -- Keep in mind varnish.regmatch(), varnish.regsub() and\n    -- varnish.regsuball() are available in order to circumvent this\n    -- inconvenience.\n    if host == 'www.foo.com' or host == 'www.bar.com' then\n        if string.match(url, '^/admin/') then\n            return 'new'\n        elseif varnish.regmatch(url, '^/(?:new|old)/') then\n            return varnish.regsub(url, '^/(new|old)/.*$', '\\1')\n        end\n    end\n\n    return 'old'\n\n/etc/varnish/default.vcl\n------------------------\n\n::\n\n    vcl 4.0;\n\n    import cfg;\n    import std;\n\n    backend old_be {\n        .host = \"127.0.0.1\";\n        .port = \"8080\";\n    }\n\n    backend new_be {\n        .host = \"127.0.0.1\";\n        .port = \"8888\";\n    }\n\n    acl internal {\n        \"localhost\";\n    }\n\n    sub vcl_init {\n        new env = cfg.env();\n\n        if (env.is_set(\"VCL_SETTINGS\")) {\n            new settings = cfg.file(env.get(\"VCL_SETTINGS\"));\n        } else {\n            return (fail);\n        }\n\n        new ttls = cfg.rules(\n            \"https://www.example.com/ttls.rules\",\n            period=300);\n\n        new backends = cfg.script(\n            \"https://www.example.com/backends.lua\",\n            period=60,\n            type=lua);\n    }\n\n    sub vcl_recv {\n        if (req.url ~ \"^/(?:settings|ttls|backends)/(?:reload|dump)/$\") {\n            if (client.ip ~ internal) {\n                if (req.url == \"/settings/reload/\") {\n                    if (settings.reload()) {\n                        return (synth(200, \"Settings reloaded.\"));\n                    } else {\n                        return (synth(500, \"Failed to reload settings.\"));\n                    }\n                } elsif (req.url == \"/ttls/reload/\") {\n                    if (ttls.reload()) {\n                        return (synth(200, \"TTLs rules reloaded.\"));\n                    } else {\n                        return (synth(500, \"Failed to reload TTLs rules.\"));\n                    }\n                } elsif (req.url == \"/backends/reload/\") {\n                    if (backends.reload()) {\n                        return (synth(200, \"Backends script reloaded.\"));\n                    } else {\n                        return (synth(500, \"Failed to reload backends script.\"));\n                    }\n                } elsif (req.url == \"/settings/dump/\") {\n                    return (synth(700, \"OK\"));\n                } else {\n                    return (synth(404, \"Not found.\"));\n                }\n            } else {\n                return (synth(405, \"Not allowed.\"));\n            }\n        }\n\n        if (std.time(settings.get(\"joke:start\"), now) \u003c now \u0026\u0026\n            std.time(settings.get(\"joke:stop\"), now) \u003e now) {\n            return (synth(418, \"I'm a teapot (RFC 2324)\"));\n        }\n    }\n\n    sub vcl_deliver {\n        call set_server;\n    }\n\n    sub vcl_synth {\n        call set_server;\n        if (resp.status == 418) {\n            return (deliver);\n        } elsif (resp.status == 700) {\n            set resp.status = 200;\n            set resp.http.Content-Type = \"application/json\";\n            settings.dump(stream=true);\n            return (deliver);\n        }\n    }\n\n    sub vcl_backend_fetch {\n        backends.init();\n        backends.push(bereq.http.Host);\n        backends.push(bereq.url);\n        backends.execute();\n        if (backends.get_result() == \"new\") {\n            set bereq.backend = new_be;\n        } else {\n            set bereq.backend = old_be;\n        }\n        backends.free_result();\n    }\n\n    sub vcl_backend_response {\n        set beresp.ttl = std.duration(\n            ttls.get(bereq.http.Host + bereq.url),\n            60s);\n    }\n\n    sub set_server {\n        if (settings.is_set(\"server\")) {\n            set resp.http.Server = settings.get(\"server\");\n        }\n    }\n\nAccess to variables\n-------------------\n\n::\n\n    $ curl http://127.0.0.1/settings/dump/ | python -m json.tool\n    {\n        \"joke:start\": \"1459468800\",\n        \"joke:stop\": \"1459555200\",\n        \"server\": \"ACME\"\n    }\n\nADVANCED SCRIPTING\n==================\n\nThe original goal of this VMOD was offering efficient strategies to parametrize\nVCL behavior based on information provided by external local or remote data\nsources. That evolved from environment variables and configuration JSON / INI\nfiles, to simple Lua / JavaScript programs executed in local interpreters\nembedded in the Varnish Cache core. All these strategies, specially the one based on\nINI files and the one based on Lua scripts interpreted by LuaJIT, have been\nsuccessfully and extensively tested in several highly trafficked environments.\n\nAt some point the VMOD evolved towards a more general framework useful to\nexecute arbitrarily complex Lua and JavaScript programs. Somehow something\nsimilar to OpenResty in the Nginx arena. For example, using the cfg VMOD you\ncan write crazy Lua-flavoured VCL. That includes loading any rocks\nyou might need, facilities to safely share state among execution engines or among\nVarnish threads, etc. Used with caution, this allows you to go beyond the\nlimits of VCL as a language and help you to model complex logic in the\ncaching layer. Of course, you can also use the VMOD to shoot yourself in\nthe foot.\n\nNext you can see a simple useless example showing the power of the VMOD.\nBeware it assumes a local Redis Server running and it depends on the\n``http``, ``redis-lua`` and ``lua-cjson`` rocks. As well, beware Varnish\nshould be started with the right environment variables properly configured\n(i.e. ``eval `luarocks path```).\n\n::\n\n    ...\n\n    sub vcl_init {\n        ...\n\n        new script = cfg.script(\n            \"/dev/null\",\n            period=0,\n            type=lua,\n            lua_remove_loadfile_function=false,\n            lua_load_package_lib=true,\n            lua_load_io_lib=true,\n            lua_load_os_lib=true);\n    }\n\n    sub vcl_deliver {\n        ...\n\n        script.init({\"\n            local http = require 'http.request'\n            local redis = require 'redis'\n            local json = require 'cjson'\n\n            if varnish.engine.client == nil then\n                varnish.engine.client = redis.connect('127.0.0.1', 6379)\n                assert(varnish.engine.client ~= nil)\n            end\n\n            local status, city = pcall(\n                varnish.engine.client.get, varnish.engine.client, ARGV[0])\n            if not status then\n                varnish.engine.client = nil\n                error(city)\n            end\n\n            local hit = city ~= nil\n\n            if not hit then\n                varnish.shared.incr('api-requests', 1, 'global')\n                local url = 'https://ipapi.co/' .. ARGV[0] .. '/json/'\n                local headers, stream = http.new_from_uri(url):go()\n                if headers:get(':status') == '200' then\n                    local info = json.decode(stream:get_body_as_string())\n                    city = info.city or '?'\n                else\n                    city = '?'\n                end\n                varnish.engine.client:set(ARGV[0], city, 'EX', 600)\n            end\n\n            varnish.set_header(\n                'X-Script-Redis-Hit',\n                hit and 'true' or 'false',\n                'resp')\n\n            varnish.set_header(\n                'X-Script-City',\n                city,\n                'resp')\n\n            varnish.set_header(\n                'X-Script-Executions-Counter',\n                varnish.shared.incr('executions', 1, 'global'),\n                'resp')\n\n            varnish.set_header(\n                'X-Script-API-Requests-Counter',\n                varnish.shared.get('api-requests', 'global'),\n                'resp')\n        \"});\n        script.push(client.ip);\n        script.execute();\n        script.free_result();\n    }\n\nINSTALLATION\n============\n\nThe source tree is based on autotools to configure the building, and does also have the necessary bits in place to do functional unit tests using the varnishtest tool.\n\n**Beware this project contains multiples branches (master, 4.1, etc.). Please, select the branch to be used depending on your Varnish Cache version (Varnish trunk → master, Varnish 4.1.x → 4.1, etc.).**\n\nDependencies:\n\n* `libcurl \u003chttps://curl.haxx.se/libcurl/\u003e`_ - multi-protocol file transfer library.\n* `luajit \u003chttp://luajit.org\u003e`_ (recommended; disabled with ``--disable-luajit``) or `lua 5.1 \u003chttps://www.lua.org\u003e`_ - powerful, efficient, lightweight, embeddable scripting language.\n\nBeware using LuaJIT GC64 mode is recommended is order to avoid ``not enough memory`` errors due to the 2 GiB (os much less) limitation. See `this excellent post by OpenResty \u003chttps://blog.openresty.com/en/luajit-gc64-mode/\u003e`_ for details.\n\nCOPYRIGHT\n=========\n\nSee LICENSE for details.\n\nBSD's implementation of the .INI file parser by Ben Hoyt has been borrowed from the `inih project \u003chttps://github.com/benhoyt/inih/\u003e`_:\n\n* https://github.com/benhoyt/inih/blob/master/ini.c\n* https://github.com/benhoyt/inih/blob/master/ini.h\n\nMIT's implementation of the JSON parser by Max Bruckner has been borrowed from the `cJSON project \u003chttps://github.com/DaveGamble/cJSON/\u003e`_:\n\n* https://github.com/DaveGamble/cJSON/blob/master/cJSON.c\n* https://github.com/DaveGamble/cJSON/blob/master/cJSON.h\n\nMIT's implementation of the JavaScript engine by Sami Vaarala has been built using the `Duktape project \u003chttps://github.com/svaarala/duktape/\u003e`_:\n\n::\n\n    $ python tools/configure.py \\\n          --output-directory /tmp/duktape \\\n          --source-directory src-input \\\n          --config-metadata config\n\nBSD's implementation of the red–black tree and the splay tree data structures by Niels Provos has been borrowed from the `Varnish Cache project \u003chttps://github.com/varnishcache/varnish-cache\u003e`_:\n\n* https://github.com/varnishcache/varnish-cache/blob/master/include/vtree.h\n\nCopyright (c) Carlos Abalde \u003ccarlos.abalde@gmail.com\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarlosabalde%2Flibvmod-cfg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcarlosabalde%2Flibvmod-cfg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarlosabalde%2Flibvmod-cfg/lists"}