{"id":13436705,"url":"https://github.com/lyokha/nginx-haskell-module","last_synced_at":"2025-04-05T08:03:46.544Z","repository":{"id":49869323,"uuid":"48541221","full_name":"lyokha/nginx-haskell-module","owner":"lyokha","description":"A comprehensive web framework aimed at building custom Haskell handlers for the Nginx Web Server","archived":false,"fork":false,"pushed_at":"2024-04-13T23:05:20.000Z","size":7855,"stargazers_count":159,"open_issues_count":0,"forks_count":11,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-04-14T13:08:02.945Z","etag":null,"topics":["async","binding","haskell","nginx","service","web"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lyokha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2015-12-24T12:08:27.000Z","updated_at":"2024-08-10T12:00:14.584Z","dependencies_parsed_at":"2023-01-31T09:46:01.144Z","dependency_job_id":"d3a003d8-6706-4956-9cd9-c79e29ec12cf","html_url":"https://github.com/lyokha/nginx-haskell-module","commit_stats":{"total_commits":1018,"total_committers":2,"mean_commits":509.0,"dds":"0.0019646365422396617","last_synced_commit":"18b780b99ed6c3aee680b2d5a0c978ab6412927a"},"previous_names":[],"tags_count":85,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyokha%2Fnginx-haskell-module","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyokha%2Fnginx-haskell-module/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyokha%2Fnginx-haskell-module/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyokha%2Fnginx-haskell-module/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lyokha","download_url":"https://codeload.github.com/lyokha/nginx-haskell-module/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305930,"owners_count":20917207,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","binding","haskell","nginx","service","web"],"created_at":"2024-07-31T03:00:51.414Z","updated_at":"2025-04-05T08:03:46.520Z","avatar_url":"https://github.com/lyokha.png","language":"C","readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/images/nginx-haskell-module-social.png\"\u003e\n\u003c/div\u003e\n\n\u003c!--[![Build Status](https://travis-ci.com/lyokha/nginx-haskell-module.svg?branch=master)](https://travis-ci.com/lyokha/nginx-haskell-module)--\u003e\n[![Build Status](https://github.com/lyokha/nginx-haskell-module/workflows/CI/badge.svg)](https://github.com/lyokha/nginx-haskell-module/actions?query=workflow%3ACI)\n[![Hackage](https://img.shields.io/hackage/v/ngx-export.svg?label=hackage%20%7C%20ngx-export\u0026logo=haskell\u0026logoColor=%239580D1)](https://hackage.haskell.org/package/ngx-export)\n[![Hackage](https://img.shields.io/hackage/v/ngx-export-tools.svg?label=hackage%20%7C%20ngx-export-tools\u0026logo=haskell\u0026logoColor=%239580D1)](https://hackage.haskell.org/package/ngx-export-tools)\n[![Hackage](https://img.shields.io/hackage/v/ngx-export-distribution.svg?label=hackage%20%7C%20ngx-export-distribution\u0026logo=haskell\u0026logoColor=%239580D1)](https://hackage.haskell.org/package/ngx-export-distribution)\n[![Docker](https://img.shields.io/docker/v/lyokha/nginx-haskell-module/latest?label=docker\u0026logo=docker)](https://hub.docker.com/r/lyokha/nginx-haskell-module)\n[![Read the Docs](https://img.shields.io/readthedocs/nginx-haskell-module/latest?logo=readthedocs)](https://nginx-haskell-module.readthedocs.io/en/latest/)\n[![Doc](https://img.shields.io/badge/pdf-yet_another_doc_with_examples-786D5F.svg?logo=jupyter)](https://nbviewer.jupyter.org/github/lyokha/nginx-haskell-module/blob/master/docs/yet-another-doc-with-examples/nginx-haskell-module-yadwe.pdf)\n\nUse this module to build custom Haskell handlers and run them in the Nginx Web\nServer. Supported types of custom handlers include\n\n- synchronous variable handlers,\n- asynchronous tasks,\n- services (i.e. asynchronous tasks that are not bound to requests),\n- shared services (i.e. services that work exclusively on a single Nginx worker\n  process all the time),\n- content handlers,\n- POST request handlers.\n\nBesides the module itself, there are a number of Haskell packages which help to\nsolve typical higher-level tasks. See the\n[*list of batteries included*](#list-of-batteries-included).\n\n# Table of contents\n\n- [Building and installation](#building-and-installation)\n- [Synchronous tasks](#synchronous-tasks)\n- [Synchronous content handlers](#synchronous-content-handlers)\n- [Asynchronous tasks and request body handlers](#asynchronous-tasks-and-request-body-handlers)\n- [Asynchronous content handlers](#asynchronous-content-handlers)\n- [Asynchronous services](#asynchronous-services)\n- [Shared services](#shared-services)\n- [Service hooks](#service-hooks)\n- [C plugins with low level access to Nginx objects](#c-plugins-with-low-level-access-to-nginx-objects)\n- [Efficiency of data exchange between Nginx and Haskell handlers](#efficiency-of-data-exchange-between-nginx-and-haskell-handlers)\n- [Exceptions in Haskell handlers](#exceptions-in-haskell-handlers)\n- [Summary table of all Nginx directives of the module](#summary-table-of-all-nginx-directives-of-the-module)\n- [Module NgxExport.Tools](#module-ngxexporttools)\n- [List of batteries included](#list-of-batteries-included)\n- [See also](#see-also)\n\n# Building and installation\n\nIn directory with the Nginx sources, run\n\n```ShellSession\n$ ./configure --add-module=/path/to/echo_module_sources --add-module=/path/to/this_module_sources\n$ make\n$ sudo make install\n```\n\nYou may also want to add `--add-module=/path/to/this_module_sources/aliases`\nand\n`--add-module=/path/to/this_module_sources/examples/dynamicUpstreams/nginx-upconf-module`\nto the configure options to enable modules [*aliases*](aliases) and\n[*upconf*](examples/dynamicUpstreams) respectively.\n\nTo build examples, we will use *ghc*. This is rather not practical in modern\nworld where dependencies get normally installed by *cabal* into directories not\nknown to *ghc*. Look [*here*](haskell/ngx-export-distribution/README.md) to\nlearn how to build custom Haskell handlers using *cabal* and\n[*ngx-export-distribution*](https://hackage.haskell.org/package/ngx-export-distribution).\n\nIn brief:\n\n1. Run\n\n    ```ShellSession\n   $ nhm-tool init my-project\n    ```\n\n   to bootstrap build environment. This command produces files *cabal.project*,\n   *Setup.hs*, *my-project.cabal*, *Makefile*, *my_project.hs*, and *hie.yaml*\n   in the current directory.\n2. Put whatever Haskell code and exporters into *my_project.hs*.\n3. Build and (optionally) install the target library.\n\n    ```ShellSession\n   $ make\n   $ sudo make install\n    ```\n\n4. Load the target library\n\n    ```nginx\n   haskell load /var/lib/nginx/my_project.so;\n    ```\n\n   and use the Haskell exporters in the Nginx configuration file.\n\n# Synchronous tasks\n\nSynchronous tasks are mostly *pure* Haskell functions of various types. To make\nthem available in Nginx configuration files, they must be exported with special\ndeclarations named *exporters*. Below is a table of *type/exporter*\ncorrespondence for all available synchronous handlers.\n\n| Type                                       | Exporter                             |\n| ------------------------------------------ | ------------------------------------ |\n| `String -\u003e String`                         | `ngxExportSS` (`NGX_EXPORT_S_S`)     |\n| `String -\u003e String -\u003e String`               | `ngxExportSSS` (`NGX_EXPORT_S_SS`)   |\n| `String -\u003e Bool`                           | `ngxExportBS` (`NGX_EXPORT_B_S`)     |\n| `String -\u003e String -\u003e Bool`                 | `ngxExportBSS` (`NGX_EXPORT_B_SS`)   |\n| `[String] -\u003e String`                       | `ngxExportSLS` (`NGX_EXPORT_S_LS`)   |\n| `[String] -\u003e Bool`                         | `ngxExportBLS` (`NGX_EXPORT_B_LS`)   |\n| `ByteString -\u003e L.ByteString`               | `ngxExportYY` (`NGX_EXPORT_Y_Y`)     |\n| `ByteString -\u003e Bool`                       | `ngxExportBY` (`NGX_EXPORT_B_Y`)     |\n| `ByteString -\u003e IO L.ByteString`            | `ngxExportIOYY` (`NGX_EXPORT_IOY_Y`) |\n\nAll synchronous handlers may accept *strings* (one or two), a *list of strings*,\nor a *strict bytestring*, and return a *string*, a *boolean* or a *lazy\nbytestring*. The last handler from the table is *impure* or *effectful*, and it\nreturns a *lazy bytestring* wrapped in *IO Monad*.\n\nThere are two kinds of exporters which differ only in their implementations.\nThe first kind \u0026mdash; *camel-cased* exporters \u0026mdash; is implemented by means\nof *Template Haskell*, the other kind \u0026mdash; exporters in braces, as they are\nshown in the table \u0026mdash; is implemented using *CPP macros*. Both of them\nprovide *FFI* declarations for functions they export, but the camel-cased\nexporters are available only from a separate Haskell module\n[*ngx-export*](https://hackage.haskell.org/package/ngx-export), which can be\ndownloaded and installed by *cabal*, whereas the CPP exporters are implemented\ninside the *nginx-haskell-module* in so-called *standalone* approach, where\ncustom Haskell declarations get wrapped inside common Haskell code.\n\n## Examples\n\nIn all examples in this section and later, we will use *modular* approach with\n*camel-cased* exporters and separate compilation of Haskell code.\n\n#### File test.hs\n\n```haskell\n{-# LANGUAGE TemplateHaskell #-}\n\nmodule NgxHaskellUserRuntime where\n\nimport           NgxExport\nimport qualified Data.Char as C\n\ntoUpper :: String -\u003e String\ntoUpper = map C.toUpper\nngxExportSS 'toUpper\n\nngxExportSS 'reverse\n\nisInList :: [String] -\u003e Bool\nisInList [] = False\nisInList (x : xs) = x `elem` xs\nngxExportBLS 'isInList\n```\n\nIn this module, we declared three synchronous handlers: *toUpper*, *reverse*,\nand *isInList*. Handler *reverse* exports existing and well-known Haskell\nfunction *reverse* which reverses lists. Let's compile *test.hs* and move the\nlibrary to a directory, from where we will load this.\n\n```ShellSession\n$ ghc -O2 -dynamic -shared -fPIC -flink-rts test.hs -o test.so\n[1 of 1] Compiling NgxHaskellUserRuntime ( test.hs, test.o )\nLinking test.so ...\n$ sudo cp test.so /var/lib/nginx/\n```\n\nNote that in *ghc* older than *9.0.1*, option *-flink-rts* must be replaced\nwith option *-lHSrts-ghc\u0026dollar;(ghc \u0026dash;\u0026dash;numeric-version)*.\n\n#### File test.conf\n\n```nginx\nuser                    nginx;\nworker_processes        4;\n\nevents {\n    worker_connections  1024;\n}\n\nhttp {\n    default_type        application/octet-stream;\n    sendfile            on;\n\n    haskell load /var/lib/nginx/test.so;\n\n    server {\n        listen          8010;\n        server_name     main;\n\n        location / {\n            haskell_run toUpper $hs_upper $arg_u;\n            haskell_run reverse $hs_reverse $arg_r;\n            haskell_run isInList $hs_isInList $arg_a $arg_b $arg_c $arg_d;\n            echo \"toUpper $arg_u = $hs_upper\";\n            echo \"reverse $arg_r = $hs_reverse\";\n            echo \"$arg_a `isInList` [$arg_b, $arg_c, $arg_d] = $hs_isInList\";\n        }\n    }\n}\n```\n\nLibrary *test.so* gets loaded by Nginx directive *haskell load*. All synchronous\nhandlers run from directive *haskell_run*. The first argument of the directive\nis a name of a Haskell handler exported from the loaded library *test.so*, the\nsecond argument is an Nginx variable where the handler will put the result of\nits computation, the rest arguments are passed to the Haskell handler as\nparameters. Directive *haskell_run* has *lazy* semantics in the sense that it\nruns its handler only when the result is needed in a content handler or rewrite\ndirectives.\n\nLet's test the configuration with *curl*.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/?u=hello\u0026r=world\u0026a=1\u0026b=10\u0026c=1'\ntoUpper hello = HELLO\nreverse world = dlrow\n1 `isInList` [10, 1, ] = 1\n```\n\n# Synchronous content handlers\n\nThere are three types of exporters for synchronous content handlers.\n\n| Type                                                                | Exporter                                               |\n| ------------------------------------------------------------------- | ------------------------------------------------------ |\n| `ByteString -\u003e ContentHandlerResult`                                | `ngxExportHandler` (`NGX_EXPORT_HANDLER`)              |\n| `ByteString -\u003e L.ByteString`                                        | `ngxExportDefHandler` (`NGX_EXPORT_DEF_HANDLER`)       |\n| `ByteString -\u003e UnsafeContentHandlerResult`                          | `ngxExportUnsafeHandler` (`NGX_EXPORT_UNSAFE_HANDLER`) |\n\nTypes *ContentHandlerResult* and *UnsafeContentHandlerResult* are declared as\ntype synonyms in module *NgxExport*.\n\n```haskell\ntype ContentHandlerResult = (L.ByteString, ByteString, Int, HTTPHeaders)\ntype UnsafeContentHandlerResult = (ByteString, ByteString, Int)\ntype HTTPHeaders = [(ByteString, ByteString)]\n```\n\nAll content handlers are *pure* Haskell functions, as well as the most of other\nsynchronous handlers. The *normal* content handler returns a *4-tuple*\n*(response-body, content-type, HTTP-status, response-headers)*. The response\nbody consists of a number of chunks packed in a *lazy bytestring*, the content\ntype is a *strict bytestring* such as *text/html*. The *default* handler\ndefaults the content type to *text/plain* and the HTTP status to *200*, thus\nreturning only chunks of the response body. The *unsafe* handler returns a\n*3-tuple* with a single-chunked response body, the content type and the status,\nbut the both bytestring parameters are supposed to be taken from static data,\nwhich must not be cleaned up after request termination.\n\n*Normal* and *default* content handlers can be declared with two directives:\n*haskell_content* and *haskell_static_content*. The second directive runs its\nhandler only once, when the first request comes, and returns the same response\non further requests. The *unsafe* handler is declared with directive\n*haskell_unsafe_content*.\n\n## An example\n\nLet's replace Nginx directive *echo* with our own default content handler\n*echo*. Add in *test.hs*,\n\n```haskell\nimport           Data.ByteString (ByteString)\nimport qualified Data.ByteString.Lazy as L\n\n-- ...\n\necho :: ByteString -\u003e L.ByteString\necho = L.fromStrict\nngxExportDefHandler 'echo\n```\n\ncompile it and put *test.so* into */var/lib/nginx/*. Add new location */ch* into\n*test.conf*,\n\n```nginx\n        location /ch {\n            haskell_run toUpper $hs_upper $arg_u;\n            haskell_run reverse $hs_reverse $arg_r;\n            haskell_run isInList $hs_isInList $arg_a $arg_b $arg_c $arg_d;\n            haskell_content echo\n\"toUpper $arg_u = $hs_upper\nreverse $arg_r = $hs_reverse\n$arg_a `isInList` [$arg_b, $arg_c, $arg_d] = $hs_isInList\n\";\n        }\n```\n\nand test again.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/ch?u=content\u0026r=handler\u0026a=needle\u0026b=needle\u0026c=in\u0026d=stack'\ntoUpper content = CONTENT\nreverse handler = reldnah\nneedle `isInList` [needle, in, stack] = 1\n```\n\n# Asynchronous tasks and request body handlers\n\nThere are two types of Haskell handlers for per-request asynchronous tasks: an\nasynchronous handler and an asynchronous request body handler.\n\n| Type                                                                        | Exporter                                                   |\n| --------------------------------------------------------------------------- | ---------------------------------------------------------- |\n| `ByteString -\u003e IO L.ByteString`                                             | `ngxExportAsyncIOYY` (`NGX_EXPORT_ASYNC_IOY_Y`)            |\n| `L.ByteString -\u003e ByteString -\u003e IO L.ByteString`                             | `ngxExportAsyncOnReqBody` (`NGX_EXPORT_ASYNC_ON_REQ_BODY`) |\n\nNormal asynchronous handler accepts a strict bytestring and returns a lazy\nbytestring. Its type exactly corresponds to that of the handlers exported with\n*ngxExportIOYY*. Request body handler additionally accepts request body chunks\nin its first parameter.\n\nUnlike synchronous handlers, asynchronous per-request handlers are *eager*. This\nmeans that they will always run when declared in a location, no matter whether\ntheir results are going to be used in the response and rewrite directives, or\nnot. The asynchronous handlers run in an early *rewrite phase* (before rewrite\ndirectives), and in a late rewrite phase (after rewrite directives, if in the\nfinal location there are more asynchronous tasks declared). It is possible to\ndeclare many asynchronous tasks in a single location: in this case they are\nspawned one by one in order of their declarations, which lets using results of\nearly tasks in inputs of later tasks. This ordering rule extends naturally\nbeyond hierarchical levels: tasks declared in *server* clause run before tasks\nfrom *location* clauses, while tasks from *location-if* clauses run latest.\n\nAsynchronous tasks are bound to the Nginx event loop by means of *eventfd* (or\nPOSIX *pipes* if eventfd was not available on the platform when Nginx was being\ncompiled). When the rewrite phase handler of this module spawns an asynchronous\ntask, it opens an eventfd, then registers it in the event loop, and passes it to\nthe Haskell handler. As soon as the Haskell handler finishes the task and pokes\nthe result into buffers, it writes into the eventfd, thus informing the Nginx\npart that the task has finished. Then Nginx gets back to the module's rewrite\nphase handler, and it spawns the next asynchronous task, or returns (when there\nare no more tasks left), moving request processing to the next stage.\n\n## An example\n\nLet's add two asynchronous handlers into *test.hs*: one for extracting a field\nfrom POST data, and the other for delaying response for a given number of\nseconds.\n\n#### File test.hs (additions)\n\n```haskell\nimport qualified Data.ByteString.Char8 as C8\nimport qualified Data.ByteString.Lazy.Char8 as C8L\nimport           Control.Concurrent\nimport           Safe\n\n-- ...\n\nreqFld :: L.ByteString -\u003e ByteString -\u003e IO L.ByteString\nreqFld a fld = return $ maybe C8L.empty C8L.tail $\n    lookup (C8L.fromStrict fld) $ map (C8L.break (== '=')) $ C8L.split '\u0026' a\nngxExportAsyncOnReqBody 'reqFld\n\ndelay :: ByteString -\u003e IO L.ByteString\ndelay v = do\n    let t = readDef 0 $ C8.unpack v\n    threadDelay $ t * 1000000\n    return $ C8L.pack $ show t\nngxExportAsyncIOYY 'delay\n```\n\nThis code must be linked with *threaded* Haskell RTS this time!\n\n```ShellSession\n$ ghc -O2 -dynamic -shared -fPIC -flink-rts -threaded test.hs -o test.so\n[1 of 1] Compiling NgxHaskellUserRuntime ( test.hs, test.o )\nLinking test.so ...\n$ sudo cp test.so /var/lib/nginx/\n```\n\nNote that in *ghc* older than *9.0.1*, options *-flink-rts -threaded* must be\nreplaced with option *-lHSrts_thr-ghc\u0026dollar;(ghc \u0026dash;\u0026dash;numeric-version)*.\n\nLet's make location */timer*, where we will read how many seconds to wait in\nPOST field *timer*, and then wait them until returning the response.\n\n#### File test.conf (additions)\n\n```nginx\n        location /timer {\n            haskell_run_async_on_request_body reqFld $hs_timeout timer;\n            haskell_run_async delay $hs_waited $hs_timeout;\n            echo \"Waited $hs_waited sec\";\n        }\n```\n\nRun curl tests.\n\n```ShellSession\n$ curl -d 'timer=3' 'http://127.0.0.1:8010/timer'\nWaited 3 sec\n$ curl -d 'timer=bad' 'http://127.0.0.1:8010/timer'\nWaited 0 sec\n```\n\n# Asynchronous content handlers\n\nThere are two types of *impure* content handlers that allow for effectful code.\nOne of them corresponds to that of the *normal* content handler, except the\nresult is wrapped in *IO Monad*. The other accepts request body chunks in its\nfirst argument like the handler exported with *ngxExportAsyncOnReqBody*.\n\n| Type                                                                                | Exporter                                                                  |\n| ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |\n| `ByteString -\u003e IO ContentHandlerResult`                                             | `ngxExportAsyncHandler` (`NGX_EXPORT_ASYNC_HANDLER`)                      |\n| `L.ByteString -\u003e ByteString -\u003e IO ContentHandlerResult`                             | `ngxExportAsyncHandlerOnReqBody` (`NGX_EXPORT_ASYNC_HANDLER_ON_REQ_BODY`) |\n\nThe first handler is declared with directive *haskell_async_content*, the\nhandler that accepts request body chunks is declared with directive\n*haskell_async_content_on_request_body*.\n\nIt's easy to emulate effects in a synchronous content handler by combining the\nlatter with an asynchronous task like in the following example.\n\n```nginx\n        location /async_content {\n            haskell_run_async getUrl $hs_async_httpbin \"http://httpbin.org\";\n            haskell_content echo $hs_async_httpbin;\n        }\n```\n\nHere *getUrl* is an asynchronous Haskell handler that returns content of an HTTP\npage. This approach has at least two deficiencies related to performance and\nmemory usage. The content may be huge and chunked, and its chunks could be\nnaturally reused in the content handler. But they won't, because here they get\ncollected by directive *haskell_run_async* into a single chunk, and then passed\nto the content handler *echo*. The other problem deals with *eagerness* of\nasynchronous tasks. Imagine that we put in the location a rewrite to another\nlocation: handler *getUrl* will run before redirection, but variable\n*hs_async_httpbin* will never be used because we'll get out from the current\nlocation.\n\nThe task starts from the content handler asynchronously, and the lazy\nbytestring \u0026mdash; the contents \u0026mdash; gets used in the task as is, with all\nof its originally computed chunks.\n\n## Examples (including online image converter)\n\nLet's rewrite our *timer* example using *haskell_async_content*.\n\n#### File test.hs (additions)\n\n```haskell\n{-# LANGUAGE TupleSections #-}\n{-# LANGUAGE MagicHash #-}\n\n-- ...\n\nimport           GHC.Prim\nimport           Data.ByteString.Unsafe\nimport           Data.ByteString.Internal (accursedUnutterablePerformIO)\n\n-- ...\n\npackLiteral :: Int -\u003e GHC.Prim.Addr# -\u003e ByteString\npackLiteral l s = accursedUnutterablePerformIO $ unsafePackAddressLen l s\n\ndelayContent :: ByteString -\u003e IO ContentHandlerResult\ndelayContent v = do\n    v' \u003c- delay v\n    return $ (, packLiteral 10 \"text/plain\"#, 200, []) $\n        L.concat [\"Waited \", v', \" sec\\n\"]\nngxExportAsyncHandler 'delayContent\n```\n\nFor the *content type* we used a static string *\"text/plain\"#* that ends with a\n*magic hash* merely to avoid any dynamic memory allocations.\n\n#### File test.conf (additions)\n\n```nginx\n        location /timer/ch {\n            haskell_run_async_on_request_body reqFld $hs_timeout timer;\n            haskell_async_content delayContent $hs_timeout;\n        }\n```\n\nRun curl tests.\n\n```ShellSession\n$ curl -d 'timer=3' 'http://127.0.0.1:8010/timer/ch'\nWaited 3 sec\n$ curl 'http://127.0.0.1:8010/timer/ch'\nWaited 0 sec\n```\n\nIn the next example we will create an *online image converter* to convert images\nof various formats into PNG using Haskell library *JuicyPixels*.\n\n#### File test.hs (additions)\n\n```haskell\nimport           Codec.Picture\n\n-- ...\n\nconvertToPng :: L.ByteString -\u003e ByteString -\u003e IO ContentHandlerResult\nconvertToPng t = const $ return $\n    case decodeImage $ L.toStrict t of\n        Left e -\u003e (C8L.pack e, packLiteral 10 \"text/plain\"#, 500, [])\n        Right image -\u003e case encodeDynamicPng image of\n                Left e -\u003e (C8L.pack e, packLiteral 10 \"text/plain\"#, 500, [])\n                Right png -\u003e (png, packLiteral 9 \"image/png\"#, 200, [])\nngxExportAsyncHandlerOnReqBody 'convertToPng\n```\n\nWe are going to run instances of *convertToPng* on multiple CPU cores, and\ntherefore it's better now to compile this with option *-feager-blackholing*.\n\n```ShellSession\n$ ghc -O2 -feager-blackholing -dynamic -shared -fPIC -flink-rts -threaded test.hs -o test.so\n[1 of 1] Compiling NgxHaskellUserRuntime ( test.hs, test.o )\nLinking test.so ...\n$ sudo cp test.so /var/lib/nginx/\n```\n\n#### File test.conf (additions)\n\n```nginx\n    haskell rts_options -N4 -A32m -qg;\n\n    limit_conn_zone all zone=all:10m;\n\n    # ...\n\n        location /convert/topng {\n            limit_conn all 4;\n            client_max_body_size 20m;\n            haskell_request_body_read_temp_file on;\n            haskell_async_content_on_request_body convertToPng;\n        }\n```\n\nDirective *haskell rts_options* declares that we are going to use 4 CPU cores\n(*-N4*) for image conversion tasks: this is a good choice on a quad-core\nprocessor when high CPU utilization is expected. For dealing with huge images,\nwe also increased Haskell GC allocation area up to *32Mb* (*-A32m*) to possibly\nminimize frequency of GC calls. We also forcibly switched to sequential GC\n(*-qg*), which is quite appropriate in our intrinsically single-threaded handler\n*convertToPng*. Directives *limit_conn_zone* and *limit_conn* must effectively\nlimit number of simultaneously processed client requests to the number of CPU\ncores (*4*) in order to protect the CPU from overloading.\n\nIn location */convert/topng*, directive *client_max_body_size* declares that all\nrequests whose bodies exceed *20Mb* will be rejected. Directive\n*haskell_request_body_read_temp_file on* makes the Haskell part able to read\nhuge request bodies that have been buffered in a temporary file by Nginx.\nNotice that we do not pass any value into directive\n*haskell_async_content_on_request_body*, therefore its second argument is simply\nomitted.\n\nFor running tests, an original file, say *sample.tif*, must be prepared. We will\npipe command *display* from *ImageMagick* to the output of curl for more fun.\n\n```ShellSession\n$ curl --data-binary @sample.tif 'http://127.0.0.1:8010/convert/topng' | display\n```\n\n# Asynchronous services\n\nAsynchronous tasks run in a request context, whereas asynchronous services run\nin a worker context. They start when the module gets initialized in a worker,\nand stop when a worker terminates. They are useful for gathering rarely changed\ndata shared in many requests.\n\nThere is only one type of asynchronous services exporters.\n\n| Type                                            | Exporter                                            |\n| ----------------------------------------------- | --------------------------------------------------- |\n| `ByteString -\u003e Bool -\u003e IO L.ByteString`         | `ngxExportServiceIOYY` (`NGX_EXPORT_SERVICE_IOY_Y`) |\n\nIt accepts a strict bytestring and a boolean value, and returns a lazy\nbytestring (chunks of data). If the boolean argument is *True* then this service\nhas never been called before in this worker process: this can be used to\ninitialize some global data needed by the service on the first call.\n\nServices are declared with Nginx directive *haskell_run_service*. As far as they\nare not bound to requests, the directive is only available on the *http*\nconfiguration level.\n\n```nginx\n    haskell_run_service getUrlService $hs_service_httpbin \"http://httpbin.org\";\n```\n\nThe first argument is, as ever, the name of a Haskell handler, the second\n\u0026mdash; a variable where the service result will be put, and the third argument\nis data passed to the handler *getUrlService* in its first parameter. Notice\nthat the third argument cannot contain variables because variable handlers in\nNginx are only available in a request context, hence this argument may only be\na static string.\n\nAsynchronous services are bound to the Nginx event loop in the same way as\nasynchronous tasks. When a service finishes its computation, it pokes data into\nbuffers and writes into eventfd (or a pipe's write end). Then the event handler\nimmediately restarts the service with the boolean argument equal to *False*.\nThis is responsibility of the author of a service handler to avoid dry runs and\nmake sure that it is called not so often in a row. For example, if a service\npolls periodically, then it must delay for this time itself like in the\nfollowing example.\n\n## An example\n\nLet's retrieve content of a specific URL, say *httpbin.org*, in background. Data\nwill update every 20 seconds.\n\n#### File test.hs (additions)\n\n```haskell\nimport           Network.HTTP.Client\nimport           Control.Exception\nimport           System.IO.Unsafe\nimport           Control.Monad\n\n-- ...\n\nhttpManager :: Manager\nhttpManager = unsafePerformIO $ newManager defaultManagerSettings\n{-# NOINLINE httpManager #-}\n\ngetUrl :: ByteString -\u003e IO C8L.ByteString\ngetUrl url = catchHttpException $ getResponse url $ flip httpLbs httpManager\n    where getResponse u = fmap responseBody . (parseRequest (C8.unpack u) \u003e\u003e=)\n\ncatchHttpException :: IO C8L.ByteString -\u003e IO C8L.ByteString\ncatchHttpException = (`catch` \\e -\u003e\n        return $ C8L.pack $ \"HTTP EXCEPTION: \" ++ show (e :: HttpException))\n\ngetUrlService :: ByteString -\u003e Bool -\u003e IO L.ByteString\ngetUrlService url firstRun = do\n    unless firstRun $ threadDelay $ 20 * 1000000\n    getUrl url\nngxExportServiceIOYY 'getUrlService\n```\n\nThe *httpManager* defines a global state, not to say a *variable*: this is an\nasynchronous HTTP client implemented in module *Network.HTTP.Client*. Pragma\n*NOINLINE* ensures that all functions will refer to the same client object, i.e.\nit will nowhere be inlined. Functions *getUrl* and *catchHttpException* are used\nin our service handler *getUrlService*. The handler waits 20 seconds on every\nrun except the first, and then runs the HTTP client. All HTTP exceptions are\ncaught by *catchHttpException*, others hit the handler on top of the custom\nHaskell code and get logged by Nginx.\n\n#### File test.conf (additions)\n\n```nginx\n    haskell_run_service getUrlService $hs_service_httpbin \"http://httpbin.org\";\n\n    # ...\n\n        location /httpbin {\n            echo $hs_service_httpbin;\n        }\n```\n\nRun curl tests.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin'\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n  \u003cmeta http-equiv='content-type' value='text/html;charset=utf8'\u003e\n  \u003cmeta name='generator' value='Ronn/v0.7.3 (http://github.com/rtomayko/ronn/tree/0.7.3)'\u003e\n  \u003ctitle\u003ehttpbin(1): HTTP Client Testing Service\u003c/title\u003e\n\n...\n```\n\nThis must run really fast because it shows data that has already been retrieved\nby the service, requests do not trigger any network activity with *httpbin.org*\nby themselves!\n\n## Termination of a service\n\nServices are killed on a worker's exit with an asynchronous exception\n*WorkerProcessIsExiting*. Then the worker waits *synchronously* until all of its\nservices' threads exit, and calls *hs_exit()*. This scenario has two important\nimplications.\n\n1. The Haskell service handler may catch *WorkerProcessIsExiting* on exit and\n   make persistency actions such as writing files if they are needed.\n2. *Unsafe* *blocking* FFI calls must be avoided in service handlers as they may\n   hang the Nginx worker, and it won't exit. Using *interruptible* FFI fixes\n   this problem.\n\n# Shared services\n\nAn asynchronous service may store its result in shared memory accessible from\nall worker processes. This is achieved with directive\n*haskell_service_var_in_shm*. For example, the following declaration (in *http*\nclause),\n\n```nginx\n    haskell_service_var_in_shm httpbin 512k /tmp $hs_service_httpbin;\n```\n\nmakes service *getUrlService*, that stores its result in variable\n*hs_service_httpbin*, shared. The first argument of the directive \u0026mdash;\n*httpbin* \u0026mdash; is an identifier of a shared memory segment, *512k* is its\nmaximum size, */tmp* is a directory where *file locks* will be put (see below),\nand *\u0026dollar;hs_service_httpbin* is the service variable.\n\nShared services are called *shared* not only because they store results in\nshared memory, but also because at any moment of the Nginx master lifetime there\nis only one worker that runs a specific service. When workers start, they race\nto acquire a *file lock* for a service, and if a worker wins the race, it holds\nthe lock until it exits or dies. Other workers' services of the same type wait\nuntil the lock is freed. The locks are implemented via POSIX *advisory* file\nlocks, and so require a directory where they will be put. The directory must be\n*writable* to worker processes, and */tmp* seems to be a good choice in general.\n\n## Update variables\n\nThe active shared service puts the value of the shared variable in a shared\nmemory, services on other workers wait and do nothing else. Requests may come to\nany worker (with active or inactive services), fortunately the service result is\nshared and they can return it as is. But what if the result must be somehow\ninterpreted by Haskell handlers before returning it in the response? Could the\nhandlers just peek into the shared memory and do what they want with the shared\ndata? Unfortunately, not: the shared memory is accessible for reading and\nwriting only from the Nginx part!\n\nDoes it mean that we have only one option to let the Haskell part update its\nglobal state unavailable in inactive workers: passing values of shared variables\ninto the Haskell part on every request? This would be extremely inefficient.\nUpdate variables is a trick to avoid this. They evaluate to the corresponding\nservice variable's value only when it changes in the shared memory since the\nlast check in the current worker, and to an empty string otherwise. Every\nservice variable has its own update variable counterpart whose name is built\nfrom the service variable's name prefixed by *\\_upd\\_\\_*.\n\n### An example\n\nLet's extend our example with loading a page in background. We are still going\nto load *httpbin.org*, but this time let's assume that we have another task, say\nextracting all links from the page and showing them in the response sorted. For\nthat we could add a Haskell handler, say *sortLinks*, and pass to it all the\npage content on every request. But the page may appear huge, let's extract all\nthe links from it and put them into a global state using update variable\n*\\_upd\\_\\_hs_service_httpbin*. In this case, function *sortLinks* must be\nimpure, as it must be able to read from the global state.\n\n#### File test.hs (additions)\n\n```haskell\n{-# LANGUAGE OverloadedStrings #-}\n\n-- ...\n\nimport           Data.IORef\nimport           Text.Regex.PCRE.ByteString\nimport           Text.Regex.Base.RegexLike\nimport qualified Data.Array as A\nimport           Data.List\nimport qualified Data.ByteString as B\n\n-- ...\n\ngHttpbinLinks :: IORef [ByteString]\ngHttpbinLinks = unsafePerformIO $ newIORef []\n{-# NOINLINE gHttpbinLinks #-}\n\ngrepLinks :: ByteString -\u003e [ByteString]\ngrepLinks =\n    map (fst . snd) . filter ((1 ==) . fst) . concatMap A.assocs .\n        filter (not . null) . concatMap (matchAllText regex) .\n            C8.lines\n    where regex = makeRegex $ C8.pack \"a href=\\\"([^\\\"]+)\\\"\" :: Regex\n\ngrepHttpbinLinks :: ByteString -\u003e IO L.ByteString\ngrepHttpbinLinks \"\" = return \"\"\ngrepHttpbinLinks v = do\n    writeIORef gHttpbinLinks $ grepLinks $ B.copy v\n    return \"\"\nngxExportIOYY 'grepHttpbinLinks\n\nsortLinks :: ByteString -\u003e IO L.ByteString\nsortLinks \"httpbin\" =\n    L.fromChunks . sort . map (`C8.snoc` '\\n') \u003c$\u003e readIORef gHttpbinLinks\nsortLinks _ = return \"\"\nngxExportIOYY 'sortLinks\n```\n\nHere *gHttpbinLinks* is the global state, *grepHttpbinLinks* is a handler for\nupdate variable *\\_upd\\_\\_hs_service_httpbin*, almost all the time it does\nnothing \u0026mdash; just returns an empty string, but when the update variable\nbecomes not empty, it updates the global state and returns an empty string\nagain. Notice that the original bytestring is copied with *B.copy* before its\nparts get collected as matches and put in the global state. This is an important\nstep because the original bytestring's lifetime does not extend beyond the\ncurrent request whereas the global state may last much longer! Sometimes copying\nis not necessary, for example when the bytestring gets deserialized into an\nobject in-place. Handler *sortLinks* is parameterized by data identifier: when\nthe identifier is equal to *httpbin*, it reads the global state and returns it\nsorted, otherwise it returns an empty string.\n\n#### File test.conf (additions)\n\n```nginx\n    haskell_service_var_in_shm httpbin 512k /tmp $hs_service_httpbin;\n\n    # ...\n\n        location /httpbin/sortlinks {\n            haskell_run grepHttpbinLinks $_upd_links_ $_upd__hs_service_httpbin;\n            haskell_run sortLinks $hs_links \"${_upd_links_}httpbin\";\n            echo $hs_links;\n        }\n```\n\nWe have to pass variable *\\_upd\\_links\\_* in *sortLinks* because this will\ntrigger update in the worker by *grepHttpbinLinks*, otherwise update won't run:\nremember that Nginx directives are lazy? On the other hand, *\\_upd\\_links\\_* is\nalways empty and won't mess up with the rest of the argument \u0026mdash; value\n*httpbin*.\n\nRun curl tests.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin/sortlinks'\n/\n/absolute-redirect/6\n/anything\n/basic-auth/user/passwd\n/brotli\n/bytes/1024\n\n...\n```\n\n## Shm stats variables\n\nEvery service variable in shared memory has another associated variable that\nprovides basic stats in format *timestamp | size | changes | failures | failed*,\nwhere *timestamp* is a number of seconds elapsed from the beginning of the *UNIX\nepoch* till the last change of the variable's value, *size* is the size of the\nvariable in bytes, *changes* is a number of changes, and *failures* is a number\nof memory allocation failures since the last Nginx reload, the value of flag\n*failed* (*0* or *1*) denotes if the last attempt of memory allocation from the\nshared memory pool for a new value of the variable has failed. The name of the\nshm stats variable is built from the service variable's name with prefix\n*\\_shm\\_\\_*.\n\n### An example\n\nLet's add a location to show shm stats about our *httpbin* service. This time\nonly configuration file *test.conf* is affected.\n\n#### File test.conf (additions)\n\n```nginx\n        location /httpbin/shmstats {\n            echo \"Httpbin service shm stats: $_shm__hs_service_httpbin\";\n        }\n```\n\nRun curl tests.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin/shmstats'\nHttpbin service shm stats: 1516274639 | 13011 | 1 | 0 | 0\n```\n\nFrom this output we can find that payload size of *httpbin.org* is *13011*\nbytes, the service variable was updated only once (less than 20 seconds elapsed\nfrom start of Nginx), and that there were no memory allocation failures.\n\n## Update callbacks\n\nThere is a special type of single-shot services called update callbacks. They\nare declared like\n\n```nginx\n    haskell_service_var_update_callback cbHttpbin $hs_service_httpbin optional_value;\n```\n\nHere *cbHttpbin* is a Haskell handler exported by *ngxExportServiceIOYY* as\nalways. Variable *hs_service_httpbin* must be declared in directive\n*haskell_service_var_in_shm*. Argument *optional_value* is a string, it can be\nomitted, in which case handler *cbHttpbin* gets the value of service variable\n*hs_service_httpbin* as its first argument.\n\nUpdate callbacks do not return results. They run from a worker that holds the\nactive service on every change of the service variable, and shall be supposedly\nused to integrate with other Nginx modules by signaling specific Nginx locations\nvia an HTTP client.\n\n### An example\n\nLet's count all changes of service variable *hs_service_httpbin* during Nginx\nlifetime (originally I supposed that its content won't change after the first\ninitialization because *httpbin.org* looks like a static page, but responses\nappeared to be able to vary from time to time). For this we will use counters\nfrom\n[*nginx-custom-counters-module*](https://github.com/lyokha/nginx-custom-counters-module).\n\n#### File test.hs (additions)\n\n```haskell\ncbHttpbin :: ByteString -\u003e Bool -\u003e IO L.ByteString\ncbHttpbin url firstRun = do\n    when firstRun $ threadDelay $ 5 * 1000000\n    getUrl url\nngxExportServiceIOYY 'cbHttpbin\n```\n\nHandler *cbHttpbin* is a simple HTTP client. On the first run it waits 5 seconds\nbefore sending request because the request is supposed to be destined to self,\nwhile Nginx workers may appear to be not ready to accept it.\n\n#### File test.conf (additions)\n\n```nginx\n    haskell_service_var_update_callback cbHttpbin $hs_service_httpbin\n                                        \"http://127.0.0.1:8010/httpbin/count\";\n\n    # ...\n\n        location /httpbin/count {\n            counter $cnt_httpbin inc;\n            return 200;\n        }\n\n        location /counters {\n            echo \"Httpbin service changes count: $cnt_httpbin\";\n        }\n```\n\nWait at least 5 seconds after Nginx start and run curl tests.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/counters'\nHttpbin service changes count: 1\n```\n\nFurther the count will probably be steadily increasing.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/counters'\nHttpbin service changes count: 3\n```\n\n# Service hooks\n\nService hooks allow for interaction with running services, both per-worker and\nshared. They are supposed to change global states that affect services behavior\nand can be thought of as service API handlers, thereto being run from dedicated\nNginx locations.\n\n| Type                                       | Exporter                                           |\n| ------------------------------------------ | -------------------------------------------------- |\n| `ByteString -\u003e IO L.ByteString`            | `ngxExportServiceHook` (`NGX_EXPORT_SERVICE_HOOK`) |\n\nService hooks install a content handler when declared. In the following example,\n\n```nginx\n        location /httpbin/url {\n            haskell_service_hook getUrlServiceHook $hs_service_httpbin $arg_v;\n        }\n```\n\nlocation */httpbin/url* derives the content handler which signals all workers\nvia an event channel upon receiving a request. Then the event handlers in all\nworkers run the hook (*getUrlServiceHook* in our case) *synchronously*, and\nfinally send an asynchronous exception *ServiceHookInterrupt* to the service\nto which the service variable from the service hook declaration\n(*hs_service_httpbin*) corresponds. Being run synchronously, service hooks are\nexpected to be fast, only writing data passed to them (the value of *arg_v* in\nour case) into a global state. In contrast to *update variables*, this data has\na longer lifetime being freed in the Haskell part when the original bytestring\ngets garbage collected.\n\n## An example\n\nLet's make it able to change the URL for the *httpbin* service in runtime. For\nthis we must enable *getUrlService* to read from a global state where the URL\nvalue will reside.\n\n#### File test.hs (additions, getUrlService reimplemented)\n\n```haskell\nimport           Data.Maybe\n\n-- ...\n\ngetUrlServiceLink :: IORef (Maybe ByteString)\ngetUrlServiceLink = unsafePerformIO $ newIORef Nothing\n{-# NOINLINE getUrlServiceLink #-}\n\ngetUrlServiceLinkUpdated :: IORef Bool\ngetUrlServiceLinkUpdated = unsafePerformIO $ newIORef True\n{-# NOINLINE getUrlServiceLinkUpdated #-}\n\ngetUrlService :: ByteString -\u003e Bool -\u003e IO L.ByteString\ngetUrlService url = const $ do\n    url' \u003c- fromMaybe url \u003c$\u003e readIORef getUrlServiceLink\n    updated \u003c- readIORef getUrlServiceLinkUpdated\n    atomicWriteIORef getUrlServiceLinkUpdated False\n    unless updated $ threadDelay $ 20 * 1000000\n    getUrl url'\nngxExportServiceIOYY 'getUrlService\n\ngetUrlServiceHook :: ByteString -\u003e IO L.ByteString\ngetUrlServiceHook url = do\n    writeIORef getUrlServiceLink $ if B.null url\n                                       then Nothing\n                                       else Just url\n    atomicWriteIORef getUrlServiceLinkUpdated True\n    return $ if B.null url\n                 then \"getUrlService reset URL\"\n                 else L.fromChunks [\"getUrlService set URL \", url]\nngxExportServiceHook 'getUrlServiceHook\n```\n\nService hook *getUrlServiceHook* writes into two global states:\n*getUrlServiceLink* where the URL is stored, and *getUrlServiceLinkUpdated*\nwhich will signal service *getUrlService* that the URL has been updated.\n\n#### File test.conf (additions)\n\n```nginx\n    haskell_service_hooks_zone hooks 32k;\n\n    # ...\n\n        location /httpbin/url {\n            allow 127.0.0.1;\n            deny all;\n            haskell_service_hook getUrlServiceHook $hs_service_httpbin $arg_v;\n        }\n```\n\nDirective *haskell_service_hooks_zone* declares a shm zone where Nginx will\ntemporarily store data for the hook (the value of *arg_v*). This directive is\nnot mandatory: shm zone is not really needed when service hooks pass nothing.\nLocation */httpbin/url* is protected from unauthorized access with Nginx\ndirectives *allow* and *deny*.\n\nRun curl tests.\n\nFirst, let's check that *httpbin.org* replies as expected.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin'\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n  \u003cmeta http-equiv='content-type' value='text/html;charset=utf8'\u003e\n  \u003cmeta name='generator' value='Ronn/v0.7.3 (http://github.com/rtomayko/ronn/tree/0.7.3)'\u003e\n  \u003ctitle\u003ehttpbin(1): HTTP Client Testing Service\u003c/title\u003e\n\n...\n$ curl 'http://127.0.0.1:8010/httpbin/sortlinks'\n/\n/absolute-redirect/6\n/anything\n/basic-auth/user/passwd\n/brotli\n/bytes/1024\n\n...\n```\n\nThen change URL to, say, *example.com*,\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin/url?v=http://example.com'\n```\n\nand peek, by the way, into the Nginx error log.\n\n```ShellSession\n2018/02/13 16:12:33 [notice] 28794#0: service hook reported \"getUrlService set URL http://example.com\"\n2018/02/13 16:12:33 [notice] 28795#0: service hook reported \"getUrlService set URL http://example.com\"\n2018/02/13 16:12:33 [notice] 28797#0: service hook reported \"getUrlService set URL http://example.com\"\n2018/02/13 16:12:33 [notice] 28798#0: service hook reported \"getUrlService set URL http://example.com\"\n2018/02/13 16:12:33 [notice] 28797#0: an exception was caught while getting value of service variable \"hs_service_httpbin\": \"Service was interrupted by a service hook\", using old value\n```\n\nAll 4 workers were signaled, and the only *active* service (remember that\n*getUrlService* was made *shared*) was interrupted. Do not be deceived by *using\nold value*: the new URL will be read in by the service from the global state\nimmediately after restart, and the service variable will be updated.\n\nLet's see what we are getting now.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin'\n\u003c!doctype html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n    \u003ctitle\u003eExample Domain\u003c/title\u003e\n\n    \u003cmeta charset=\"utf-8\" /\u003e\n\n...\n$ curl 'http://127.0.0.1:8010/httpbin/sortlinks'\nhttp://www.iana.org/domains/example\n```\n\nLet's reset the URL.\n\n```ShellSession\n$ curl 'http://127.0.0.1:8010/httpbin/url'\n$ curl 'http://127.0.0.1:8010/httpbin'\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n  \u003cmeta http-equiv='content-type' value='text/html;charset=utf8'\u003e\n  \u003cmeta name='generator' value='Ronn/v0.7.3 (http://github.com/rtomayko/ronn/tree/0.7.3)'\u003e\n  \u003ctitle\u003ehttpbin(1): HTTP Client Testing Service\u003c/title\u003e\n\n...\n$ curl 'http://127.0.0.1:8010/httpbin/sortlinks'\n/\n/absolute-redirect/6\n/anything\n/basic-auth/user/passwd\n/brotli\n/bytes/1024\n\n...\n```\n\nIn the log we'll find\n\n```ShellSession\n2018/02/13 16:24:12 [notice] 28795#0: service hook reported \"getUrlService reset URL\"\n2018/02/13 16:24:12 [notice] 28794#0: service hook reported \"getUrlService reset URL\"\n2018/02/13 16:24:12 [notice] 28797#0: service hook reported \"getUrlService reset URL\"\n2018/02/13 16:24:12 [notice] 28798#0: service hook reported \"getUrlService reset URL\"\n2018/02/13 16:24:12 [notice] 28797#0: an exception was caught while getting value of service variable \"hs_service_httpbin\": \"Service was interrupted by a service hook\", using old value\n```\n\n## Service update hooks\n\nThis is a reimplementation of *update variables* for shared services by means of\nservice hooks. Update hooks have a number of advantages over update variables.\n\n1. No need for obscure treatment of update variables in configuration files.\n2. No need for copying the original argument: its data is freed in the Haskell\n   part.\n3. Nginx don't need to access shared memory on every single request for checking\n   if the service data has been altered.\n\nThere is a subtle difference with update variables though. As soon as with\nupdate hooks new service variable data is propagated to worker processes\nasynchronously via an event channel, there always exists a very short transient\nperiod between the moments when the service variable gets altered in shared\nmemory and the global state gets updated in a worker, during which events\nrelated to client requests may occur.\n\nAn update hook is exported with exporter *ngxExportServiceHook*, and declared\nusing directive *haskell_service_update_hook* on the *http* configuration level.\n\n### An example\n\nLet's reimplement the example with update of service links using a service hook.\n\n#### File test.hs (additions)\n\n```haskell\ngrepHttpbinLinksHook :: ByteString -\u003e IO L.ByteString\ngrepHttpbinLinksHook v = do\n    let links = grepLinks v\n        linksList = let ls = B.intercalate \" \" links\n                    in if B.null ls\n                        then \"\u003cNULL\u003e\"\n                        else ls\n    writeIORef gHttpbinLinks links\n    return $ L.fromChunks [\"getUrlService set links \", linksList]\nngxExportServiceHook 'grepHttpbinLinksHook\n```\n\n#### File test.conf (additions)\n\n```nginx\n    haskell_service_update_hook grepHttpbinLinksHook $hs_service_httpbin;\n\n    # ...\n\n        location /httpbin/sortlinks/hook {\n            haskell_run sortLinks $hs_links httpbin;\n            echo $hs_links;\n        }\n```\n\nFor testing this, watch the Nginx error log and change the URL of the service\nwith requests to location */httpbin/url* like in the previous example.\n\n# C plugins with low level access to Nginx objects\n\nSerialized pointer to the Nginx *request object* is accessible via a special\nvariable *\\_r\\_ptr*. Haskell handlers have no benefit from this because they do\nnot know how the request object is built. However they may run C code having\nbeen compiled with this knowledge. The low level access to the Nginx request\nobject makes it possible to do things that are not feasible to do without this.\nAs soon as a C plugin can do whatever a usual Nginx module can, using it from a\nHaskell handler must be very cautious. All synchronous and asynchronous Haskell\nhandlers can access the Nginx request object and pass it to a C plugin. Using it\nin a C plugin which runs in asynchronous context has not been investigated and\nis probably dangerous in many aspects, with exception (probably) of read-only\naccess. After all, an Nginx worker is a single-threaded process, and the\nstandard Nginx tools and APIs were not designed for using in multi-threaded\nenvironments. As such, using C plugins in asynchronous Haskell handlers must be\nregarded strictly as experimental!\n\n## An example\n\nLet's write a plugin that will add an HTTP header to the response.\n\n#### File test_c_plugin.c\n\n```c\n#include \u003cngx_core.h\u003e\n#include \u003cngx_http.h\u003e\n\nstatic const ngx_str_t haskell_module = ngx_string(\"Nginx Haskell module\");\n\nngx_int_t\nngx_http_haskell_test_c_plugin(ngx_http_request_t *r)\n{\n    ngx_table_elt_t  *x_powered_by;\n\n    x_powered_by = ngx_list_push(\u0026r-\u003eheaders_out.headers);\n\n    if (!x_powered_by) {\n        ngx_log_error(NGX_LOG_CRIT, r-\u003econnection-\u003elog, 0,\n                      \"Unable to allocate memory to set X-Powered-By header\");\n        return NGX_ERROR;\n    }\n\n    x_powered_by-\u003ehash = 1;\n    ngx_str_set(\u0026x_powered_by-\u003ekey, \"X-Powered-By\");\n    x_powered_by-\u003evalue = haskell_module;\n\n    return NGX_OK;\n}\n```\n\nLet's compile the C code. For this we need a directory where Nginx sources were\nsometime compiled. Let's refer to it in an environment variable *NGX_HOME*.\n\n```ShellSession\n$ NGX_HOME=/path/to/nginx_sources\n```\n\nHere we are going to mimic the Nginx build process.\n\n```ShellSession\n$ gcc -O2 -fPIC -c -o test_c_plugin.o -I $NGX_HOME/src/core -I $NGX_HOME/src/http -I $NGX_HOME/src/http/modules -I $NGX_HOME/src/event -I $NGX_HOME/src/event/modules -I $NGX_HOME/src/os/unix -I $NGX_HOME/objs test_c_plugin.c\n```\n\nNow we have an object file *test_c_plugin.o* to link with the Haskell code.\nBelow is the Haskell code itself.\n\n#### File test.hs (additions)\n\n```haskell\nimport           Data.Binary.Get\nimport           Foreign.C.Types\nimport           Foreign.Ptr\n\n-- ...\n\nforeign import ccall unsafe \"ngx_http_haskell_test_c_plugin\"\n    test_c_plugin :: Ptr () -\u003e IO CIntPtr\n\ntoRequestPtr :: ByteString -\u003e Ptr ()\ntoRequestPtr = wordPtrToPtr . fromIntegral . runGet getWordhost . L.fromStrict\n\ntestCPlugin :: ByteString -\u003e IO L.ByteString\ntestCPlugin v = do\n    res \u003c- test_c_plugin $ toRequestPtr v\n    return $ if res == 0\n                 then \"Success!\"\n                 else \"Failure!\"\nngxExportIOYY 'testCPlugin\n```\n\nHandler *testCPlugin* runs function *ngx_http_haskell_test_c_plugin()* from the\nC plugin and returns *Success!* or *Failure!* in cases when the C function\nreturns *NGX_OK* or *NGX_ERROR* respectively. When compiled with *ghc*, this\ncode has to be linked with *test_c_plugin.o*.\n\n```ShellSession\n$ ghc -O2 -dynamic -shared -fPIC -flink-rts -threaded test_c_plugin.o test.hs -o test.so\n[1 of 1] Compiling NgxHaskellUserRuntime ( test.hs, test.o )\nLinking test.so ...\n$ sudo cp test.so /var/lib/nginx/\n```\n\n#### File test.conf (additions)\n\n```nginx\n        location /cplugin {\n            haskell_run testCPlugin $hs_test_c_plugin $_r_ptr;\n            echo \"Test C plugin returned $hs_test_c_plugin\";\n        }\n```\n\nRun curl tests.\n\n```ShellSession\n$ curl -D- 'http://localhost:8010/cplugin'\nHTTP/1.1 200 OK\nServer: nginx/1.12.1\nDate: Thu, 08 Mar 2018 12:09:52 GMT\nContent-Type: application/octet-stream\nTransfer-Encoding: chunked\nConnection: keep-alive\nX-Powered-By: Nginx Haskell module\n\nTest C plugin returned Success!\n```\n\nThe header *X-Powered-By* is in the response!\n\nNotice that the value of *\\_r\\_ptr* has a binary representation and therefore\nmust not be used in textual contexts such as Haskell *data* declarations or JSON\nobjects. It makes sense to put *\\_r\\_ptr* in the beginning of the handler's\nargument as it must be easy to extract it from the rest of the argument later.\nThis can be achieved explicitly, e.g. *\u0026dollar;{\\_r\\_ptr}my data*, or by adding\nsuffix *(r)* at the end of the handler's name.\n\n## C plugins in service update hooks\n\nService update hooks can be used to replace service *update callbacks*. Indeed,\nbeing run *synchronously* from an event handler, a service hook could safely\ncall a C function which would acquire related to Nginx context from Nginx global\nvariables such as *ngx_cycle* for doing a variety of low level actions.\n\nBelow is a table of functions exported from the Haskell module that return\nopaque pointers to Nginx global variables for using them in C plugins.\n\n| Function                                   | Returned value and its type                                                                                                                                                    |\n| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| `ngxCyclePtr`                              | value of argument `cycle` in the worker's initialization function (of type `ngx_cycle_t *`)                                                                                    |\n| `ngxUpstreamMainConfPtr`                   | value of expression `ngx_http_cycle_get_module_main_conf(cycle, ngx_http_upstream_module)` in the worker's initialization function (of type `ngx_http_upstream_main_conf_t *`) |\n| `ngxCachedTimePtr`                         | *address* of the Nginx global variable `ngx_cached_time` (of type `volatile ngx_time_t **`)                                                                                    |\n\nNotice that besides synchronous nature of service update hooks, there are other\nfeatures that distinguish them from service update callbacks.\n\n1. As soon as running C plugins can be useful not only in shared services, but\n   in normal per-worker services too, service update hooks are allowed in both\n   the types.\n2. Unlike update callbacks, service hooks get triggered in all worker processes.\n3. Unlike update callbacks, service hooks get triggered even when the value of\n   the service variable has not been actually changed.\n\n### An example\n\nSee implementation of\n[*nginx-healthcheck-plugin*](https://github.com/lyokha/nginx-healthcheck-plugin).\n\n# Efficiency of data exchange between Nginx and Haskell handlers\n\nHaskell handlers may accept strings (`String` or `[String]`) and *strict*\nbytestrings (`ByteString`), and return strings, *lazy* bytestrings and booleans.\nInput C-strings are marshaled into a *String* with *peekCStringLen* which has\nlinear complexity $O(n)$, output *Strings* are marshaled into C-strings with\n*newCStringLen* which is also $O(n)$. The new C-strings get freed upon the\nrequest termination in the Nginx part.\n\nThe bytestring counterparts are much faster. Both input and output are $O(1)$,\nusing *unsafePackCStringLen* and a Haskell *stable pointer* to lazy bytestring\nbuffers created inside Haskell handlers. If an output lazy bytestring has more\nthan one chunk, a new single-chunked C-string will be created in variable and\nservice handlers, but not in content handlers because the former use the chunks\ndirectly when constructing contents. Holding a stable pointer to a bytestring's\nchunks in the Nginx part ensures that they won't be garbage collected until the\npointer gets freed. Stable pointers get freed upon the request termination for\nvariable and content handlers, and before the next service iteration for service\nhandlers.\n\nComplex scenarios may require *typed exchange* between Haskell handlers and the\nNginx part using *serialized* data types such as Haskell records. In this case,\n*bytestring* flavors of the handlers would be the best choice. There are two\nwell-known serialization mechanisms: *packing Show* / *unpacking Read* and\n*ToJSON* / *FromJSON* from Haskell package *aeson*. In practice, *Show* is\nbasically faster than *ToJSON*, however in many cases *FromJSON* outperforms\n*Read*.\n\nA variable handler of a shared service makes a copy of the variable's value\nbecause shared data can be altered by any worker at any moment, and there is no\nsafe way to hold a reference to a shared data without locking. In contrast, a\nvariable handler of a normal per-worker service shares a reference to the value\nwith the service. Obviously, this is still not safe. Imagine that some request\ngets a reference to a service value from the variable handler, then lasts some\ntime and later uses this reference again: the reference could probably be freed\nby this time because the service could have altered its data since the beginning\nof the request. This catastrophic scenario could have been avoided by using a\ncopy of the service value in every request like in shared services, but this\nwould unnecessarily hit performance, therefore requests share *counted\nreferences* to service values, and as soon as the count reaches *0*, the service\nvalue gets freed.\n\n# Exceptions in Haskell handlers\n\nThere is no way to catch exceptions in *pure* handlers. However they can arise\nfrom using *partial* functions such as *head* and *tail*! Switching to their\n*total* counterparts from module *Safe* can mitigate this issue, but it is not\npossible to eliminate it completely.\n\nFortunately, all exceptions, synchronous and asynchronous, are caught on top of\nthe module's Haskell code. If a handler does not catch an exception itself, the\nexception gets caught higher and logged by Nginx. However, using exception\nhandlers in Haskell handlers, when it's possible, should be preferred.\n\n# Summary table of all Nginx directives of the module\n\n| Directive                                                                | Level                                         | Comment                                                                                                                                                                                      |\n| ------------------------------------------------------------------------ | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `haskell compile`                                                        | `http`                                        | Compile Haskell code found in the last argument. Accepts arguments *threaded* (use *threaded* RTS library), *debug* (use *debug* RTS library), and *standalone* (use *standalone* approach). |\n| `haskell load`                                                           | `http`                                        | Load the specified Haskell library.                                                                                                                                                          |\n| `haskell ghc_extra_options`                                              | `http`                                        | Specify extra options for GHC when the library compiles.                                                                                                                                     |\n| `haskell rts_options`                                                    | `http`                                        | Specify options for Haskell RTS.                                                                                                                                                             |\n| `haskell program_options`                                                | `http`                                        | Specify program options. This is just another way for passing data into Haskell handlers.                                                                                                    |\n| `haskell_run`                                                            | `server`, `location`, `location if`           | Run a synchronous Haskell task.                                                                                                                                                              |\n| `haskell_run_async`                                                      | `server`, `location`, `location if`           | Run an asynchronous Haskell task.                                                                                                                                                            |\n| `haskell_run_async_on_request_body`                                      | `server`, `location`, `location if`           | Run an asynchronous Haskell request body handler.                                                                                                                                            |\n| `haskell_run_service`                                                    | `http`                                        | Run a Haskell service.                                                                                                                                                                       |\n| `haskell_service_var_update_callback`                                    | `http`                                        | Run a callback on a service variable's update.                                                                                                                                               |\n| `haskell_content`                                                        | `location`, `location if`                     | Declare a Haskell content handler.                                                                                                                                                           |\n| `haskell_static_content`                                                 | `location`, `location if`                     | Declare a static Haskell content handler.                                                                                                                                                    |\n| `haskell_unsafe_content`                                                 | `location`, `location if`                     | Declare an unsafe Haskell content handler.                                                                                                                                                   |\n| `haskell_async_content`                                                  | `location`, `location if`                     | Declare an asynchronous Haskell content handler.                                                                                                                                             |\n| `haskell_async_content_on_request_body`                                  | `location`, `location if`                     | Declare an asynchronous Haskell content handler with access to request body.                                                                                                                 |\n| `haskell_service_hook`                                                   | `location`, `location if`                     | Declare a service hook and create a content handler for managing the corresponding service.                                                                                                  |\n| `haskell_service_update_hook`                                            | `http`                                        | Declare a service update hook.                                                                                                                                                               |\n| `haskell_request_body_read_temp_file`                                    | `server`, `location`, `location if`           | This flag (*on* or *off*) makes asynchronous tasks and content handlers read buffered in a *temporary file* POST data. If not set, then buffered data is not read.                           |\n| `haskell_var_nocacheable`                                                | `http`                                        | All variables in the list become no cacheable and safe for using in ad-hoc iterations over *error_page* cycles. Applicable to variables of any *get handler*.                                |\n| `haskell_var_nohash`                                                     | `http`                                        | Nginx won't build hashes for variables in the list. Applicable to variables of any *get handler*.                                                                                            |\n| `haskell_var_compensate_uri_changes`                                     | `http`                                        | All variables in the list allow to cheat *error_page* when used in its redirections and make the cycle infinite.                                                                             |\n| `haskell_var_empty_on_error`                                             | `http`                                        | All variables in the list return empty values on errors while the errors are still being logged by Nginx. Applicable to effectful synchronous and asynchronous variable handlers.            |\n| `haskell_service_var_ignore_empty`                                       | `http`                                        | All service variables in the list do not write the service result when its value is empty.                                                                                                   |\n| `haskell_service_var_in_shm`                                             | `http`                                        | All service variables in the list store the service result in a shared memory. Implicitly declares a shared service.                                                                         |\n| `haskell_service_hooks_zone`                                             | `http`                                        | Declare shm zone for a temporary storage of service hooks data.                                                                                                                              |\n| `haskell_request_variable_name`                                          | `http`                                        | Change the name of the request variable if default value *\\_r\\_ptr* is already used.                                                                                                         |\n| `single_listener`                                                        | `server`                                      | Make the virtual server accept client requests only from a single worker process.                                                                                                            |\n\n# Module NgxExport.Tools\n\nPackage\n[*ngx-export-tools*](https://hackage.haskell.org/package/ngx-export-tools)\nprovides module\n[*NgxExport.Tools*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools.html)\nthat exports various utility functions and data as well as specialized service\nexporters and adapters. As soon as the module is well documented, its features\nare only basically lined up below.\n\n- Utility functions *terminateWorkerProcess* and *restartWorkerProcess* make it\n  possible to terminate the worker process from within a Haskell service.\n  Function *finalizeHTTPRequest* finalizes the current HTTP request from an\n  asynchronous Haskell handler with the specified HTTP status and body. Function\n  *ngxRequestPtr* unmarshals the value of Nginx variable *\\_r\\_ptr*. Function\n  *ngxNow* returns the current time cached inside the Nginx core.\n- Data *TimeInterval* and utility functions *toSec* and *threadDelaySec* can be\n  used to specify time delays for services.\n- A number of converters from custom types deriving or implementing instances of\n  *Read* and *FromJSON* (*readFromBytestring* and friends).\n- Special service exporters (*simple services*) combine various *sleeping*\n  strategies and typing policies of services and can be used to avoid usual\n  boilerplate code needed in the vanilla service exporters from module\n  *NgxExport*.\n- Special service adapters (*split services*) allow for distinguishing between\n  *ignition* services (those that run when the service runs for the first time)\n  and *deferred* services (those that run when the service runs for the second\n  time and later).\n- A simple combinator function *voidHandler* helps to avoid printing the final\n  *return L.empty* or *return \"\"* in effectful handlers which return unused or\n  empty bytestrings.\n\n# List of batteries included\n\nNote: click on the links to read descriptions and documentation.\n\n* Haskell modules with docs and examples on Hackage\n    * Package [*ngx-export*](https://hackage.haskell.org/package/ngx-export)\n        * Module [*NgxExport*](https://hackage.haskell.org/package/ngx-export/docs/NgxExport.html)\n    * Package [*ngx-export-tools*](https://hackage.haskell.org/package/ngx-export-tools)\n        * Module [*NgxExport.Tools*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools.html)\n        * Module [*NgxExport.Tools.Combinators*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools-Combinators.html)\n        * Module [*NgxExport.Tools.Read*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools-Read.html)\n        * Module [*NgxExport.Tools.SimpleService*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools-SimpleService.html)\n        * Module [*NgxExport.Tools.SplitService*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools-SplitService.html)\n        * Module [*NgxExport.Tools.System*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools-System.html)\n        * Module [*NgxExport.Tools.TimeInterval*](https://hackage.haskell.org/package/ngx-export-tools/docs/NgxExport-Tools-TimeInterval.html)\n    * Package [*ngx-export-tools-extra*](https://hackage.haskell.org/package/ngx-export-tools-extra)\n        * Module [*NgxExport.Tools.Aggregate*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-Aggregate.html)\n        * Module [*NgxExport.Tools.EDE*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-EDE.html)\n        * Module [*NgxExport.Tools.PCRE*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-PCRE.html)\n        * Module [*NgxExport.Tools.Prometheus*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-Prometheus.html)\n        * Module [*NgxExport.Tools.Resolve*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-Resolve.html)\n        * Module [*NgxExport.Tools.ServiceHookAdaptor*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-ServiceHookAdaptor.html)\n        * Module [*NgxExport.Tools.Subrequest*](https://hackage.haskell.org/package/ngx-export-tools-extra/docs/NgxExport-Tools-Subrequest.html)\n    * Package [*ngx-export-distribution*](https://hackage.haskell.org/package/ngx-export-distribution)\n        * Module [*NgxExport.Distribution*](https://hackage.haskell.org/package/ngx-export-distribution/docs/NgxExport-Distribution.html)\n* Haskell modules implemented as C plugins\n    * Package [*ngx-export-healthcheck*](https://github.com/lyokha/nginx-healthcheck-plugin)\n    * Package [*ngx-export-log*](https://github.com/lyokha/nginx-log-plugin)\n\n# See also\n\n* The old [*README.md*](docs/old-readme/README.md) contains more details on the\nimplementation of the module and some topics not covered here.\n* Module [*aliases*](aliases) declares handy aliases for a few directives from\nthis module.\n* Module [*upconf*](examples/dynamicUpstreams) provides basic support for\ndynamic upstreams.\n\nThere are some articles about the module in my blog.\n\n* [*nginx module to enable haskell binding to nginx configuration\nfiles*](https://lin-techdet.blogspot.com/2015/12/nginx-module-to-enable-haskell-binding.html),\n* [*nginx-haskell-module: labeled media routing\nexample*](https://lin-techdet.blogspot.com/2017/01/nginx-haskell-module-labeled-media.html),\n* [*Passing ByteString contents reliably into C\ncode*](https://lin-techdet.blogspot.com/2017/08/passing-bytestring-contents-reliably.html),\n* [*Signaling all worker processes in Nginx via an event\nchannel*](https://lin-techdet.blogspot.com/2018/03/signaling-all-worker-processes-in-nginx.html).\n\n","funding_links":[],"categories":["C"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyokha%2Fnginx-haskell-module","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flyokha%2Fnginx-haskell-module","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyokha%2Fnginx-haskell-module/lists"}