{"id":24543681,"url":"https://github.com/jccampagne/ocaml_ppx_extension_simple_tutorial","last_synced_at":"2025-04-15T12:56:48.830Z","repository":{"id":48452910,"uuid":"78788959","full_name":"jccampagne/ocaml_ppx_extension_simple_tutorial","owner":"jccampagne","description":"Simple ppx tutorial for OCaml","archived":false,"fork":false,"pushed_at":"2021-07-25T16:22:41.000Z","size":8,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-15T12:55:58.948Z","etag":null,"topics":["ast-mapper","learning-by-doing","ocaml","ppx","ppx-extension","ppx-rewriter","ppx-tutorial","tutorial"],"latest_commit_sha":null,"homepage":"","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jccampagne.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-12T21:34:09.000Z","updated_at":"2025-03-04T13:26:50.000Z","dependencies_parsed_at":"2022-08-24T05:30:32.671Z","dependency_job_id":null,"html_url":"https://github.com/jccampagne/ocaml_ppx_extension_simple_tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jccampagne%2Focaml_ppx_extension_simple_tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jccampagne%2Focaml_ppx_extension_simple_tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jccampagne%2Focaml_ppx_extension_simple_tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jccampagne%2Focaml_ppx_extension_simple_tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jccampagne","download_url":"https://codeload.github.com/jccampagne/ocaml_ppx_extension_simple_tutorial/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249076545,"owners_count":21208811,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast-mapper","learning-by-doing","ocaml","ppx","ppx-extension","ppx-rewriter","ppx-tutorial","tutorial"],"created_at":"2025-01-22T20:14:42.327Z","updated_at":"2025-04-15T12:56:48.806Z","avatar_url":"https://github.com/jccampagne.png","language":"OCaml","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Simple ppx tutorial\n\nThis is really simple tutorial on [PPX extension node rewriters](https://caml.inria.fr/pub/docs/manual-ocaml/extn.html#sec248). In this example, the extension will have no payload in order to keep the example very simple.\n\n# Files\n\nThe files in the project:\n\n1. `ppx_test_simple.ml` defines an abstract syntax tree preprocessor that replaces `[%simple_tag]` extension with the integer `1234567890` in source code.\n1. `sample_input.ml` will be used to demonstrated the PPX rewriter\n1. `Makefile` contains the all the commands to run the example\n\nYou can compile the example by just typing:\n\n```\nmake\n```\t\n\nIt will:\n\n1. compile the PPX rewriter source file `ppx_test_simple.ml` and output the executable `ppx_test_simple`;\n1. show the original source, for comparison;\n1. run the PPX rewriter on `sample_input.ml` and print the modified source code to standard output;\n1. compile `sample_input.ml` with the rewriter and build `test` executable;\n1. finally, run the final program `test`.\n\nYou should see this ouput:\n\n```\n# Create the executable ppx_test_simple\nocamlc -I +compiler-libs ocamlcommon.cma ppx_test_simple.ml -o ppx_test_simple\n# Output the original source\ncat sample_input.ml\nlet _ = Printf.printf \"%d\" [%simple_tag]\n# Output the modified source\nocamlc -dsource -ppx ./ppx_test_simple sample_input.ml\nlet _ = Printf.printf \"%d\" 1234567890\n# Compile with ppx extension\nocamlc -ppx ./ppx_test_simple sample_input.ml -o test\n# Run the program\n./test\n1234567890\n```\n\n# Detailed explanation\n\n## PPX rewriter\n\nWriting a PPX rewriter consists in defining an [`AST mapper`](https://caml.inria.fr/pub/docs/manual-ocaml/libref/Ast_mapper.html) that will be applied to the abstract syntax tree of a source code.\n\nAn AST mapper is basically a [record containing a set of callbacks](https://ocaml.org/api/compilerlibref/Ast_mapper.html#TYPEmapper) specifying what to do for every node types. Instead of defining every callbacks - 40 at the time of writing - it is common to base the new mapper on [the default AST mapper called `Ast_mapper.default_mapper`](https://ocaml.org/api/compilerlibref/Ast_mapper.html#VALdefault_mapper) which is the identity, by default all callbacks return the same unmodified.\n\n## Extention\n\nIn our case, we want to recognize the syntax `[%simple_tag]` as an [extension node](https://caml.inria.fr/pub/docs/manual-ocaml/extn.html#sec248) of [expression type](https://ocaml.org/api/compilerlibref/Parsetree.html#TYPEexpression) in our code and replace it with an integer. This will be achieved with the use of [PPX extension](https://ocaml.org/api/compilerlibref/Parsetree.html#TYPEextension). In this simple case, the extension node identifier is `simple_tag` and has no payload.\n\nIn order to help us write the pattern matching code, we can use `dumpast` PPX tools on the string `[%simple_tag]` as such:\n\n```\n$ ocamlfind ppx_tools/dumpast -e \"[%simple_tag]\"\n```\n\nThe `-e` option means that the string argument is an *expression*. `ocamlfind` is just here to find the executable `dumpast` (it maybe somewhere like `~/.opam/system/lib/ppx_tools/dumpast` depending on your installation, which you could call directly).\n\nThe output is:\n\n```\n[%simple_tag]\n==\u003e\n{pexp_desc = Pexp_extension ({txt = \"simple_tag\"}, PStr [])}\n=========\n```\n\nThis tells us that the string `[%simple_tag]` is an expression containing an extension whose identifier is `simple_tag`, and no payload (`PStr []`). The type definition is:\n\n```\ntype expression_desc = \n | ...\n | Pexp_extension of extension\n```\n\nIn turn, an extension is defined as such:\n\n```\ntype extension = string Asttypes.loc * payload \n```\n\nwith\n\n```\ntype 'a loc = 'a Location.loc = {\n  \ttxt : 'a;\n  \tloc : Location.t;\n}\n```\n\nand \n\n```\ntype payload = \n |\tPStr of structure\n | ...\n```\n\nWe'll stop drilling in the types for now as `payload` is an empty structure `PStr []` in this case.\n\nThese types are defined in [ParseTree Module](https://ocaml.org/api/compilerlibref/Parsetree.html).\n\n\nSo in order to write our PPX rewriter, we will pattern match on this value:\n\n```\n                                   a 2-tuple\n                              ________|___________________\n                             /                            \\\n{pexp_desc = Pexp_extension ({txt = \"simple_tag\"}, PStr [])}\n                              \\________________/   \\_____/\n                                      |               |\n                                 1st element       2nd elt.\n                                  location         payload\n                                \"simple_tag\"        empty\n```\n\n\nThe function that will look something like this:\n\n```\n(* val my_expression_mapper : Ast_mapper.mapper -\u003e Parsetree.expression -\u003e Parsetree.expression *)\n\nlet my_expression_mapper mapper expr =\n  match expr with\n  | {pexp_desc = Pexp_extension ({txt = \"simple_tag\"}, PStr [])} -\u003e ...\n  | other -\u003e default_mapper.expr mapper other\n```\n\n\nNote the ellipsis. What do we replace it with? Again, we can use `dumpast` to find out how to write the integer `1234567890`:\n\n```\n$  ocamlfind ppx_tools/dumpast -e \"1234567890\"\n1234567890\n==\u003e\n{pexp_desc = Pexp_constant (Pconst_integer (\"1234567890\", None))}\n=========\n```\n\nSo our function will look like:\n\n```\nlet my_expression_mapper mapper expr =\n  match expr with\n  | {pexp_desc = Pexp_extension ({txt = \"simple_tag\"}, PStr [])} -\u003e\n     Ast_helper.Exp.constant (Pconst_integer (\"1234567890\", None))\n  | other -\u003e default_mapper.expr mapper other\n```\n\n\nIt will match only expressions we are looking for and replace them with the integer `1234567890`, other expressions will just be left untouched.\n\nThe actual definition of the mapper will look like this:\n\n```\n(* val mapper_test_simple : 'a -\u003e Ast_mapper.mapper *)\n\nlet mapper_test_simple argv =\n  { default_mapper with expr = my_expression_mapper }\n```\n\n\nIt is the default mapper [Ast\\_mapper.default_mapper](https://ocaml.org/api/compilerlibref/Ast_mapper.html#VALdefault_mapper) with just the `expr` attribute replaced by our function `my_expression_mapper`.\n\nFinally we register the PPX rewriter.\n\n\n## Compiling the PPX rewriter\n\nTo compile the PPX rewrite you can do:\n\n```\n$ ocamlc -I +compiler-libs ocamlcommon.cma ppx_test_simple.ml -o ppx_test_simple\n```\n\nLet's understand what it does:\n\n```\nocamlc -I +compiler-libs ocamlcommon.cma ppx_test_simple.ml -o ppx_test_simple\n       \\_______________/ \\_____________/ \\________________/    \\_____________/\n           search for      library file      input file            output \n          compiler-libs                                          executable \n          in standard\n             paths\n```\n\nYou need to link against `ocamlcommon.cma` which is in `.../ocaml/4.03.0/lib/ocaml/compiler-libs/ocamlcommon.cma`. The output is an executable indeed.\n\nWhen you run it:\n```\n$ ./ppx_test_simple\nUsage: ./ppx_test_simple [extra_args] \u003cinfile\u003e \u003coutfile\u003e\n```\n\nBut the common way to use it is with the compiler as shown in the next section.\n\n## Running the PPX rewriter\n\nIf you want to view the transformed code, you can type:\n\n```\nocamlc -dsource -ppx ./ppx_test_simple sample_input.ml\n```\n\n\nTo actually compile the code and produce the final executable:\n\n```\nocamlc -ppx ./ppx_test_simple sample_input.ml -o test\n```\n\nYou can then run the executable:\n\n```\n$ ./test\n1234567890\n```\n\n## Some errors\n\n### `Uninterpreted extension` error\n\n```\n$ ocamlc sample_input.ml\nFile \"sample_input.ml\", line 1, characters 29-39:\nUninterpreted extension 'simple_tag'.\n```\n\nThe PPX extension was not specified. Fix:\n\n```\n$ ocamlc -dsource -ppx ./ppx_test_simple sample_input.ml \n```\n\n### `Error: External preprocessor does not produce a valid file`\n\n```\n$ ocamlc -dsource -ppx ./ppx_test_simple sample_input.ml\nFile \"sample_input.ml\", line 1:\nError: External preprocessor does not produce a valid file\nCommand line: ./ppx_test_simple '/var/folders/41/w5q9lk3j6xn4krfpglf5b_l80000gp/T/camlppx75184a' '/var/folders/41/w5q9lk3j6xn4krfpglf5b_l80000gp/T/camlppxe10080'\n```\n\nYou may have forgotten to register the mapper, be sure you did so:\n\n```\nlet () =\n  register \"ppx_test_simple\" mapper_test_simple\n```\n\n## Whole AST tree\n\nIt is possible to dump the whole parse tree of a program with ocamlc:\n\n```\n$ ocamlc -dparsetree -ppx ./ppx_test_simple sample_input.ml\n[\n  structure_item (sample_input.ml[1,0+0]..[1,0+45])\n    Pstr_value Nonrec\n    [\n      \u003cdef\u003e\n        pattern (sample_input.ml[1,0+4]..[1,0+5])\n          Ppat_any\n        expression (sample_input.ml[1,0+8]..[1,0+45])\n          Pexp_apply\n          expression (sample_input.ml[1,0+8]..[1,0+21])\n            Pexp_ident \"Printf.printf\" (sample_input.ml[1,0+8]..[1,0+21])\n          [\n            \u003carg\u003e\n            Nolabel\n              expression (sample_input.ml[1,0+22]..[1,0+26])\n                Pexp_constant PConst_string(\"%d\",None)\n            \u003carg\u003e\n            Nolabel\n              expression (_none_[1,0+-1]..[1,0+-1]) ghost\n                Pexp_constant PConst_int (1234567890,None)\n          ]\n    ]\n]\n```\n\n# Next\n\nNow let's define a PPX extension with a payload, that will be more interesting.\n\n# References:\n\n[The compiler front-end](https://caml.inria.fr/pub/docs/manual-ocaml/parsing.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjccampagne%2Focaml_ppx_extension_simple_tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjccampagne%2Focaml_ppx_extension_simple_tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjccampagne%2Focaml_ppx_extension_simple_tutorial/lists"}