{"id":13697525,"url":"https://github.com/openresty/replace-filter-nginx-module","last_synced_at":"2025-05-03T20:30:31.761Z","repository":{"id":6156073,"uuid":"7385528","full_name":"openresty/replace-filter-nginx-module","owner":"openresty","description":"Streaming regular expression replacement in response bodies","archived":false,"fork":false,"pushed_at":"2023-08-12T07:08:58.000Z","size":123,"stargazers_count":260,"open_issues_count":12,"forks_count":68,"subscribers_count":26,"default_branch":"master","last_synced_at":"2024-11-13T01:34:12.062Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openresty.png","metadata":{"files":{"readme":"README.markdown","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2012-12-31T09:28:27.000Z","updated_at":"2024-10-23T05:28:55.000Z","dependencies_parsed_at":"2022-08-24T10:20:52.736Z","dependency_job_id":"94a5c568-5e43-480e-a789-84b62f2fa87b","html_url":"https://github.com/openresty/replace-filter-nginx-module","commit_stats":{"total_commits":105,"total_committers":7,"mean_commits":15.0,"dds":0.09523809523809523,"last_synced_commit":"a93c6653a61ca226a14d84d66fc34ffe1ac247d8"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openresty%2Freplace-filter-nginx-module","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openresty%2Freplace-filter-nginx-module/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openresty%2Freplace-filter-nginx-module/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openresty%2Freplace-filter-nginx-module/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openresty","download_url":"https://codeload.github.com/openresty/replace-filter-nginx-module/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252252486,"owners_count":21718749,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T18:00:59.759Z","updated_at":"2025-05-03T20:30:30.420Z","avatar_url":"https://github.com/openresty.png","language":"C","funding_links":[],"categories":["Third Modules","Third Party Modules"],"sub_categories":["C Modules"],"readme":"Name\n====\n\nngx_replace_filter - Streaming regular expression replacement in response bodies.\n\n*This module is not distributed with the Nginx source.* See [the installation instructions](#installation).\n\nTable of Contents\n=================\n\n* [Name](#name)\n* [Status](#status)\n* [Synopsis](#synopsis)\n* [Description](#description)\n* [Directives](#directives)\n    * [replace_filter](#replace_filter)\n    * [replace_filter_types](#replace_filter_types)\n    * [replace_filter_max_buffered_size](#replace_filter_max_buffered_size)\n    * [replace_filter_last_modified](#replace_filter_last_modified)\n    * [replace_filter_skip](#replace_filter_skip)\n* [Installation](#installation)\n* [Trouble Shooting](#trouble-shooting)\n* [TODO](#todo)\n* [Community](#community)\n    * [English Mailing List](#english-mailing-list)\n    * [Chinese Mailing List](#chinese-mailing-list)\n* [Bugs and Patches](#bugs-and-patches)\n* [Author](#author)\n* [Copyright and License](#copyright-and-license)\n* [See Also](#see-also)\n\nStatus\n======\n\nThis module is already quite usable though still at the early phase of development\nand is considered experimental.\n\nSynopsis\n========\n\n```nginx\n    location /t {\n        default_type text/html;\n        echo abc;\n        replace_filter 'ab|abc' X;\n    }\n\n    location / {\n        # proxy_pass/fastcgi_pass/...\n\n        # caseless global substitution:\n        replace_filter '\\d+' 'blah blah' 'ig';\n        replace_filter_types text/plain text/css;\n    }\n\n    location /a {\n        # proxy_pass/fastcgi_pass/root/...\n\n        # remove line-leading spaces and line-trailing spaces,\n        # as well as blank lines:\n        replace_filter '^\\s+|\\s+$' '' g;\n    }\n\n    location /b {\n        # proxy_pass/fastcgi_pass/root/...\n\n        # only remove line-leading spaces and line-trailing spaces:\n        replace_filter '^[ \\f\\t]+|[ \\f\\t]+$' '' g;\n    }\n\n    location ~ '\\.cpp$' {\n        # proxy_pass/fastcgi_pass/root/...\n\n        replace_filter_types text/plain;\n\n        # skip C/C++ string literals:\n        replace_filter \"'(?:\\\\\\\\[^\\n]|[^'\\n])*'\" $\u0026 g;\n        replace_filter '\"(?:\\\\\\\\[^\\n]|[^\"\\n])*\"' $\u0026 g;\n\n        # remove all those ugly C/C++ comments:\n        replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n    }\n```\n\nDescription\n===========\n\nThis Nginx output filter module tries to do regular expression substitutions in\na non-buffered manner wherever possible.\n\nThis module does *not* use traditional backtracking regular expression engines like PCRE, rather,\nit uses the new [sregex](https://github.com/agentzh/sregex) library implemented by the author himself, which was designed with streaming processing in mind from the very beginning:\n\nA good common subset of Perl 5 regular expressions is supported by `sregex`. For the complete\nfeature list, check out sregex's documentation:\n\nhttps://github.com/agentzh/sregex#syntax-supported\n\nResponse body data is only buffered when absolutely necessary, like facing an incomplete capture that belongs to a possible match near the data chunk boundaries.\n\n[Back to TOC](#table-of-contents)\n\nDirectives\n==========\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter\n--------------\n**syntax:** *replace_filter \u0026lt;regex\u0026gt; \u0026lt;replace\u0026gt;*\n\n**syntax:** *replace_filter \u0026lt;regex\u0026gt; \u0026lt;replace\u0026gt; \u0026lt;options\u0026gt;*\n\n**default:** *no*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nSpecifies the regex pattern and text to be replaced, with optional regex flags.\n\nBy default, the filter stops matching after the first match is found. This behavior can be changed by specifying the `g` regex option.\n\nThe following regex options are supported:\n\n* `g`\n\n    for global search and substitution (default off)\n* `i`\n\n    for case-insensitive matching (default off)\n\nMultiple options can be combined in a single string argument, for example:\n\n```nginx\n    replace_filter hello hiya ig;\n```\n\nNginx variables can be interpolated into the text to be replaced, for example:\n\n```nginx\n    replace_filter \\w+ \"[$foo,$bar]\";\n```\n\nIf you want to use the literal dollar sign character (`$`), use the `$$` sequence for that,\nfor instance:\n\n```nginx\n    replace_filter \\w \"$$\";\n```\n\nUse of submatch capturing variables like `$\u0026`, `$1`, `$2`, and etc are also supported, for example,\n\n```nginx\n    replace_filter [bc]|d [$\u0026-$1-$2] g;\n```\n\nThe semantics of the submatch capturing variables is exactly the same as in the Perl 5 language.\n\nMultiple `replace_filter` directives in the same scope is also supported.\nAll the patterns will be applied at the same time as in a tokenizer.\nWe will *not* use the longest token match semantics, but rather, patterns will be prioritized according to their order in\nthe configure file.\n\nHere is an example for removing all the C/C++ comments from a C/C++ source code file:\n\n```nginx\n    replace_filter \"'(?:\\\\\\\\[^\\n]|[^'\\n])*'\" $\u0026 g;\n    replace_filter '\"(?:\\\\\\\\[^\\n]|[^\"\\n])*\"' $\u0026 g;\n    replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n```\n\nWhen the `Content-Encoding` response header is not empty (like `gzip`), the response\nbody will always remain intact. So usually you want to disable the gzip compression\nin your backend servers' responses by adding the following line to your `nginx.conf`\nif you are the ngx_proxy module:\n\n```nginx\n    proxy_set_header Accept-Encoding '';\n```\n\nYour responses can still be gzip compressed on the Nginx server level though.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_types\n--------------------\n\n**syntax:** *replace_filter_types \u0026lt;mime-type\u0026gt; ...*\n\n**default:** *replace_filter_types text/html*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nSpecify one or more MIME types (in the `Content-Type` response header) to be processed.\n\nBy default, only `text/html` typed responses are processed.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_max_buffered_size\n---------------------------------\n**syntax:** *replace_filter_max_buffered_size \u0026lt;size\u0026gt;*\n\n**default:** *replace_filter_max_buffered_size 8k*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nLimits the total size of the data buffered by the module at runtime. Default to `8k`.\n\nWhen the limit is reached, `replace_filter` will immediately stop processing and\nleave all the remaining response body data intact.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_last_modified\n----------------------------\n\n**syntax:** *replace_filter_last_modifiled keep | clear*\n\n**default:** *replace_filter_last_modified clear*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nControls how to deal with the existing Last-Modified response header.\n\nBy default, this module will clear the `Last-Modified` response header if there is any. You can specify\n\n```nginx\n    replace_filter_last_modified keep;\n```\n\nto always keep the original `Last-Modified` response header.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_skip\n-------------------\n\n**syntax:** *replace_filter_skip $var*\n\n**default:** *no*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output header filter*\n\nThis directive controls whether to skip all the `replace_filter` rules on a per-request basis.\n\nBoth constant values or strings containing NGINX variables are supported.\n\nWhen the value is evaluated to an empty value (\"\") or the value \"0\" in the request output header phase, no `replace_filter` rules will be skipped for the current request. Otherwise all the `replace_filter` rules will be skipped for the current request.\n\nBelow is a trivial example for this:\n\n```nginx\nset $skip '';\nlocation /t {\n    content_by_lua '\n        ngx.var.skip = 1\n        ngx.say(\"abcabd\")\n    ';\n    replace_filter_skip $skip;\n    replace_filter abcabd X;\n}\n```\n\n[Back to TOC](#table-of-contents)\n\nInstallation\n============\n\nYou need to install the sregex library first:\n\nhttps://github.com/agentzh/sregex\n\nAnd then rebuild your Nginx like this:\n\n```bash\n    ./configure --add-module=/path/to/replace-filter-nginx-module\n```\n\nIf sregex is not installed to the default prefix (i.e., `/usr/local`), then\nyou should specify the locations of your sregex installation via\nthe `SREGEX_INC` and `SREGEX_LIB` environments before running the\n`./configure` script, as in\n\n```bash\n    export SREGEX_INC=/opt/sregex/include\n    export SREGEX_LIB=/opt/sregex/lib\n```\n\nassuming that your sregex is installed to the prefix `/opt/sregex`.\n\nStarting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the `--add-dynamic-module=PATH` option instead of `--add-module=PATH` on the\n`./configure` command line above. And then you can explicitly load the module in your `nginx.conf` via the [load_module](http://nginx.org/en/docs/ngx_core_module.html#load_module)\ndirective, for example,\n\n```nginx\nload_module /path/to/modules/ngx_http_replace_filter_module.so;\n```\n\n[Back to TOC](#table-of-contents)\n\nTrouble Shooting\n================\n\n* If you are seeing the error \"error while loading shared libraries: libsregex.so.0: cannot open shared object file: No such file or directory\"\nwhile starting nginx, then it means that the installation path of your libsregex library\nis not in your system's default library search path. You can solve this issue by passing the option `--with-ld-opt='-Wl,-rpath,/usr/local/lib'` to nginx's `./configure` command. Alternatively, you can just add the path of your libsregex.so.0 to the `LD_LIBRARY_PATH` environment value before starting your nginx server.\n\n[Back to TOC](#table-of-contents)\n\nTODO\n====\n\n* optimize the special case for verbatim substitutions, i.e., `replace_filter \u003cregex\u003e $\u0026;`.\n* implement the `replace_filter_skip $var` directive to control whether to enable the filter on the fly.\n* reduce the amount of data that has to be buffered for when an partial match is already found.\n* recycle the memory blocks used to buffer the pending capture data and \"complex values\" for replacement.\n* allow use of inlined Lua code as the `replacement` argument of the `replace_filter` directive to generate the text to be replaced on-the-fly.\n\n[Back to TOC](#table-of-contents)\n\nCommunity\n=========\n\n[Back to TOC](#table-of-contents)\n\nEnglish Mailing List\n--------------------\n\nThe [openresty-en](https://groups.google.com/group/openresty-en) mailing list is for English speakers.\n\n[Back to TOC](#table-of-contents)\n\nChinese Mailing List\n--------------------\n\nThe [openresty](https://groups.google.com/group/openresty) mailing list is for Chinese speakers.\n\n[Back to TOC](#table-of-contents)\n\nBugs and Patches\n================\n\nPlease submit bug reports, wishlists, or patches by\n\n1. creating a ticket on the [GitHub Issue Tracker](http://github.com/agentzh/replace-filter-nginx-module/issues),\n1. or posting to the [OpenResty community](http://wiki.nginx.org/HttpLuaModule#Community).\n\n[Back to TOC](#table-of-contents)\n\nAuthor\n======\n\nYichun \"agentzh\" Zhang (章亦春) \u003cagentzh@gmail.com\u003e, OpenResty Inc.\n\n[Back to TOC](#table-of-contents)\n\nCopyright and License\n=====================\n\nThis module is licensed under the BSD license.\n\nCopyright (C) 2012-2017, by Yichun \"agentzh\" Zhang (章亦春), OpenResty Inc.\n\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n\n* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\n\n* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n[Back to TOC](#table-of-contents)\n\nSee Also\n========\n\n* agentzh's sregex library: https://github.com/agentzh/sregex\n* The standard ngx_sub_filter module: http://nginx.org/en/docs/http/ngx_http_sub_module.html\n* Slides for my talk \"sregex: matching Perl 5 regexes on data streams\": http://agentzh.org/misc/slides/yapc-na-2013-sregex.pdf\n[Back to TOC](#table-of-contents)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenresty%2Freplace-filter-nginx-module","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenresty%2Freplace-filter-nginx-module","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenresty%2Freplace-filter-nginx-module/lists"}