{"id":15724334,"url":"https://github.com/cedlemo/ctypes-stubs-generation-notes","last_synced_at":"2025-05-13T04:42:06.219Z","repository":{"id":144939644,"uuid":"172580095","full_name":"cedlemo/ctypes-stubs-generation-notes","owner":"cedlemo","description":"OCaml Ctypes stubs generation notes","archived":false,"fork":false,"pushed_at":"2019-05-01T08:31:30.000Z","size":1775,"stargazers_count":19,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-20T22:32:47.546Z","etag":null,"topics":["ctypes","ctypes-bindings","dune","ocaml","ocaml-bindings"],"latest_commit_sha":null,"homepage":"","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cedlemo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-25T20:32:28.000Z","updated_at":"2023-08-26T20:02:24.000Z","dependencies_parsed_at":"2023-04-04T21:02:50.573Z","dependency_job_id":null,"html_url":"https://github.com/cedlemo/ctypes-stubs-generation-notes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedlemo%2Fctypes-stubs-generation-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedlemo%2Fctypes-stubs-generation-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedlemo%2Fctypes-stubs-generation-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedlemo%2Fctypes-stubs-generation-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cedlemo","download_url":"https://codeload.github.com/cedlemo/ctypes-stubs-generation-notes/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253877394,"owners_count":21977635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ctypes","ctypes-bindings","dune","ocaml","ocaml-bindings"],"created_at":"2024-10-03T22:16:15.662Z","updated_at":"2025-05-13T04:42:06.189Z","avatar_url":"https://github.com/cedlemo.png","language":"OCaml","readme":"# Understanding Ctypes C stubs generation.\n\n* [Introduction](#introduction)\n* [1 The default example](#1-the-default-example)\n  * [1 a Write a stubs module that is a functor which defines the bindings](#1-a-write-a-stubs-module-that-is-a-functor-which-defines-the-bindings)\n  * [1 b Write a module that uses the bindings module and outputs a C file](#1-b-write-a-module-that-uses-the-bindings-module-and-outputs-a-c-file)\n  * [1 c Launch the different phases of compile and file generation](#1-c-launch-the-different-phases-of-compile-and-file-generation)\n  * [1 d Using Dune with the default example](#1-d-using-dune-with-the-default-example)\n    * [Description of the dune files](#description-of-the-dune-files)\n* [2 Cstubs Enums bindings from the GObject-Introspection library](#2-enums-bindings-from-the-gobject-introspection-library)\n  * [2 a Introduction](#2-a-introduction)\n  * [2 b Directory structure](#2-b-directory-strutcure)\n  * [2 c The config directory](#2-c-the-config-directory)\n  * [2 d The bindings directory](#2-d-the-bindings-directory)\n  * [2 e The stubgen directory](#2-e-the-stubgen-directory)\n  * [2 f The lib directory](#2-f-the-lib-directory)\n* [Conclusion](#conclusion)\n\n## Introduction\n\nAs mentionned in the [README.md of the Ctypes Cstubs example](https://github.com/ocamllabs/ocaml-ctypes/blob/master/examples/cstubs_structs/README.md),\n\n\u003e Ctypes is generally used to specify how to call C code using a DSL that is executed at runtime.\n\nBut this has some limitations with:\n- data types like:\n  * structures\n  * [enums](https://discuss.ocaml.org/t/ctypes-enum-how-to-make-it-work/456/4)\n- constants\n\nIn order to be able to circumvent thoses limitations, there is the Cstubs module of Ctypes.\n\nI first describe the steps in the default example [cstubs_structs](https://github.com/ocamllabs/ocaml-ctypes/blob/master/examples/cstubs_structs) from the git repository of Ctypes, then I explain how to build it with dune.\n\nIn the second part I explain how to use the Cstubs enum bindings generator for\nthe GObject-Introspection library and how to build those bindings with dune. This\nexample shows how to discover the compile flags and use them for all the involved build steps.\n\n## 1 The default example\n\n### 1 a Write a stubs module that is a functor which defines the bindings\n\n* bindings.ml\n```ocaml\nmodule Stubs = functor (S : Cstubs_structs.TYPE) -\u003e struct\n  module Tm = struct\n    type tm\n    type t = tm Ctypes.structure\n    let t : t S.typ = S.structure \"tm\"\n    let tm_hour = S.(field t \"tm_hour\" int)\n    let tm_year = S.(field t \"tm_year\" int)\n\n    let () = S.seal t\n  end\n\n  module Limits = struct\n    let shrt_max = S.(constant \"SHRT_MAX\" short)\n  end\nend\n```\n\n### 1 b Write a module that uses the bindings module and outputs a C file\n\n* bindings_c_gen.ml\n\n```ocaml\nlet c_headers = \"#include \u003ctime.h\u003e\\n#include \u003climits.h\u003e\"\n\nlet main () =\n  let stubs_out = open_out \"bindings_stubs_gen.c\" in\n  let stubs_fmt = Format.formatter_of_out_channel stubs_out in\n  Format.fprintf stubs_fmt \"%s@\\n\" c_headers;\n  Cstubs_structs.write_c stubs_fmt (module Bindings.Stubs);\n  Format.pp_print_flush stubs_fmt ();\n  close_out stubs_out\n\nlet () = main ()\n```\n\n### 1 c Launch the different phases of compile and file generation\n\nHere are the all the steps needed to use the Ctypes stubs:\n\n1. Write a stubs module that is a functor which defines the bindings.\n2. Write a module that uses the bindings module and outputs a C file.\n3. Compile the program from step 2 and execute it.\n4. Compile the C program generated in step 3.\n5. Run the C program from step 4, generating an ML module.\n6. Compile the module generated in step 5.\n\nThe following schema illustrates those steps:\n\n![Ctypes Stubs generation schema](https://github.com/cedlemo/ctypes-stubs-generation-notes/raw/master/Ctypes_Stubs_generation.png)\n\n### 1 d Using Dune with the default example\n\n* https://github.com/ocaml/dune/issues/135\n* https://github.com/janestreet/async_ssl\n\nThis is quite simple for the default example, we just need to create a new directory\ntree that looks like that:\n\n```\ncstubs_structs_dune\n├── bin\n├── bindings\n└── stubgen\n```\n\nThe bin directory will contain the `main.ml`, which is the main program. The\nbindings directory is dedicated to the Ctypes bindings we want to use and in the\nstubgen there will be all the code generators.\n\nHere is the directory tree with all the files of the default project:\n\n```\ncstubs_structs_dune\n├── bin\n│   ├── dune\n│   └── main.ml\n├── bindings\n│   ├── bindings.ml\n│   └── dune\n├── dune-project\n└── stubgen\n    ├── bindings_c_gen.ml\n    └── dune\n```\n\n#### Description of the dune files\n\n* In the `bindings/dune` file\n\n```\n(library\n (name bindings)\n (synopsis \"Ctypes bindings that describe the lib FFI\")\n (libraries ctypes.stubs ctypes))\n```\n\nHere I declare a library with the name \"bindings\" so that it can be included in\nthe `bin/dune` file. In the last line there are the dependencies of the `bindings` library.\n\n* In the `stubgen/dune` file\n\n```\n(executable\n (name bindings_c_gen)\n (modules bindings_c_gen)\n (libraries bindings ctypes.stubs ctypes))\n\n(rule\n (targets bindings_stubs_gen.c)\n (deps (:stubgen ../stubgen/bindings_c_gen.exe))\n (action (with-stdout-to %{targets} (run %{stubgen} -c))))\n\n(rule (targets bindings_stubs_gen.exe)\n (deps (:first_dep bindings_stubs_gen.c))\n (action\n  (bash\n   \"%{cc} %{first_dep} -I `dirname %{lib:ctypes:ctypes_cstubs_internals.h}` -I %{ocaml_where} -o %{targets}\"))\n)\n```\n\nIn the first part, there is the declaration of an OCaml executable : `bindings_c_gen.exe`,\ngenerated from the file `stubgen/bindings_c_gen.ml` and from the `bindings` library,\nthis is the first part of the step 3 described previously.\n\nThen a *rule* is declared, it describes how to generate the file `bindings_stubs_gen.c` from\nthe executable `bindings_c_gen.exe`, this is the second part of the step 3.\n\nThe last *rule* tells how the file `bindings_stubs_gen.c` is compiled into the\nexecutable `bindings_stubs_gen.exe`, this is the step 4.\n\n* In the `bin/dune` file\n\n```\n(executable (name main)\n (libraries bindings ctypes.stubs ctypes ctypes.foreign)\n)\n\n(rule\n (targets bindings_stubs.ml)\n (deps ../stubgen/bindings_stubs_gen.exe)\n (action (with-stdout-to %{targets} (run %{deps} -ml))))\n```\n\nIn the first part, there is the declaration of an OCaml executable `main.exe` and\n its dependencies.\n\nIn the last part, there is the rule for the generation of the file `bindings_stubs.ml`\nvia the executable `bindings_stubs_gen.exe`, this is the part 5 described previously.\n\n## 2 Cstubs Enums bindings from the GObject-Introspection library\n\n### 2 a Introduction\nIn this example I will describe how to use the Ctypes Stubs module to bind C enums\n with `Cstubs.Types.TYPE.enum`. The enum used comes from the `gobject-introspection`\n library and is called `GITypeInfo`. Here is it's declaration:\n\n```c\ntypedef enum\n{\n  GI_INFO_TYPE_INVALID,\n  GI_INFO_TYPE_FUNCTION,\n  GI_INFO_TYPE_CALLBACK,\n  GI_INFO_TYPE_STRUCT,\n  GI_INFO_TYPE_BOXED,\n  GI_INFO_TYPE_ENUM,         /*  5 */\n  GI_INFO_TYPE_FLAGS,\n  GI_INFO_TYPE_OBJECT,\n  GI_INFO_TYPE_INTERFACE,\n  GI_INFO_TYPE_CONSTANT,\n  GI_INFO_TYPE_INVALID_0,    /* 10 */\n  GI_INFO_TYPE_UNION,\n  GI_INFO_TYPE_VALUE,\n  GI_INFO_TYPE_SIGNAL,\n  GI_INFO_TYPE_VFUNC,\n  GI_INFO_TYPE_PROPERTY,     /* 15 */\n  GI_INFO_TYPE_FIELD,\n  GI_INFO_TYPE_ARG,\n  GI_INFO_TYPE_TYPE,\n  GI_INFO_TYPE_UNRESOLVED\n} GIInfoType;\n```\n\nIn order to test if the bindings works, I will need to create bindings for the\nfollowing functions:\n\n* `g_irepository_find_by_name`\n* `g_irepository_require`\n* `g_base_info_get_type`\n* `g_base_info_unref`\n\nand data structures:\n* `Base_info`\n* `GError`\n\nWithout going too much in the details because this is related to the basic usage\nof Ctypes, the idea to test the bindings is :\n* to load the GObject-Introspection repository of the `GObject` namespace (ie. we\nload the description of the library `GObject`)\n* then search the [`Value` structure in the current repository](https://developer.gnome.org/gobject/stable/gobject-Generic-values.html#GValue-struct) and get it as a `Base_info` type.\n* then test if our bindings match it as a `GI_INFO_TYPE_STRUCT`.\n\nSo the main executable will look like this:\n\n```ocaml\nlet namespace = \"GObject\"\nlet typelib = Gi.Repository.require namespace ()\nlet struct_name = \"Value\"\n\nlet test_baseinfo_get_type () =\n  match Gi.Repository.find_by_name namespace struct_name with\n  | None -\u003e prerr_endline \"No base info found\"; exit 1\n  | Some base_info -\u003e\n      match Gi.Base_info.get_type base_info with\n        | Struct -\u003e print_endline \"It works!\"\n      | _ -\u003e prerr_endline \"Bad type\"; exit 1\n\nlet () = test_baseinfo_get_type ()\nHere is the file hierarchy for this:\n```\n\n### 2 b Directory structure\n\n```\n/\n├── bin\n│   ├── dune\n│   └── main.ml\n├── bindings\n│   ├── bindings.ml\n│   └── dune\n├── config\n│   ├── discover.ml\n│   └── dune\n├── dune-project\n├── lib\n│   ├── dune\n│   └── gi.ml\n└── stubgen\n    ├── bindings_c_gen.ml\n    └── dune\n```\n\n* the **bin** directory contains the main executable used to test the bindings.\n* the **config** directory contains the `discover.exe` code that is used to discover\n the libs and flags needed to compile the intermediate code generator and library.\n* the **bindings** directory contains the code that defines the enum bindings and\n that will be used by the C Stubs generator `bindings_c_gen.ml`.\n* the **stubgen** directory contains the C Stubs generator `bindings_c_gen.ml`\n* the **lib** directory will contains all the bindings in a library called *gi*.\n\n### 2 c The config directory\n\nIn the config directory, there are 2 files, the *dune* file and the *discover.ml* file.\n\nThe *dune* file is really simple, it declares an executable called *discover.exe*\nthat depends on the libraries base, stdio and configurator.\n\n```dune\n(executable\n (name discover)\n (libraries base stdio configurator))\n```\n\nThe `discover.exe` binary creates different files that will be used to pass flags during\nthe compilation steps of both C and OCaml binaries. Those flags are of two types:\n* cflags: compiler parameters (flags) that can be for example [`-pthread`](https://stackoverflow.com/questions/2127797/significance-of-pthread-flag-when-compiling) and include directories information (`-I/usr/lib/libffi-3.2.1/include`).\nThose flags are used in the early stage of the C compilation process, the [*preprocessing* step](https://stackoverflow.com/questions/6264249/how-does-the-compilation-linking-process-work).\n* libs: those flags which look like this `-lgirepository-1.0` or `-L/usr/lib/../lib`\nare used during the C linking phases. The `-lgirepository-1.0` flag tells the compiler\nto link the girepository library to the program, while the `-L/usr/lib/../lib` tells\nthe compilater where to search for library.\n\nThe [OCaml compiler accepts the following options](https://caml.inria.fr/pub/docs/manual-ocaml/native.html):\n* -cclib -llibname Pass the -llibname option to the linker . This causes the given C library to be linked with the program.\nThose files are generated at build time and can be found in *_build/default/stubgen*.\n* -ccopt option Pass the given option to the C compiler and linker. For instance,-ccopt -Ldir causes the C linker to search for C libraries in directory dir.\n\n   - *c_flags.sexp*\n\n   ```\n   (-I/usr/lib/libffi-3.2.1/include -I/usr/include/gobject-introspection-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -pthread)\n   ```\n   - *c_library_flags.sexp*\n   ```\n   (-L/usr/lib/../lib -lffi -lgirepository-1.0 -lgobject-2.0 -lglib-2.0)\n   ```\n   - *ccopts.sexp*\n   ```\n   (-Wl,-no-as-needed)\n   ```\n   - *c_library_flags*\n   ```\n   -L/usr/lib/../lib -lffi -lgirepository-1.0 -lgobject-2.0 -lglib-2.0\n   ```\n   - *c_flags*\n   ```\n   -I/usr/lib/libffi-3.2.1/include -I/usr/include/gobject-introspection-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -pthread\n   ```\n\nWhen one of those files is needed, for a build step, one will just use those kind of\nrules in the *dune* file:\n\n```\n(rule\n  (targets c_flags.sexp c_library_flags.sexp ccopts.sexp)\n  (deps    (:x ../config/discover.exe))\n  (action  (run %{x} -ocamlc %{ocamlc}))\n)\n```\n\n### 2 d The bindings directory\n\nin a *bindings.ml* file, a variant type called *baseinfo_type* is defined:\n\n```ocaml\ntype baseinfo_type =\n  | Invalid (** invalid type *)\n  | Function (** function, see Function_info *)\n  | Callback (** callback, see Function_info *)\n  | Struct (** struct, see Struct_info *)\n  | Boxed (** boxed, see Struct_info or Union_info *)\n  | Enum (** enum, see Enum_info *)\n  | Flags (** flags, see Enum_info *)\n  | Object (** object, see Object_info *)\n  | Interface (** interface, see Interface_info *)\n  | Constant (** contant, see Constant_info *)\n  | Invalid_0 (** deleted, used to be GI_INFO_TYPE_ERROR_DOMAIN. *)\n  | Union (** union, see Union_info *)\n  | Value (** enum value, see Value_info *)\n  | Signal (** signal, see Signal_info *)\n  | Vfunc (** virtual function, see VFunc_info *)\n  | Property (** GObject property, see Property_info *)\n  | Field (** struct or union field, see Field_info *)\n  | Arg (** argument of a function or callback, see Arg_info *)\n  | Type (** type information, see Type_info *)\n  | Unresolved (** unresolved type, a type which is not present in the typelib, or any of its dependencies. *)\n```\n\nand in a functor called *Enums*, we define the bindings between the variant type\ntags and the corresponding enum values:\n\n```ocaml\nmodule Enums = functor (T : Cstubs.Types.TYPE) -\u003e struct\n  let gi_info_type_invalid = T.constant \"GI_INFO_TYPE_INVALID\" T.int64_t\n  let gi_info_type_function = T.constant \"GI_INFO_TYPE_FUNCTION\" T.int64_t\n  (*\n    ...\n  *)\n  let gi_info_type_type = T.constant \"GI_INFO_TYPE_TYPE\" T.int64_t\n  let gi_info_type_unresolved = T.constant \"GI_INFO_TYPE_UNRESOLVED\" T.int64_t\n\n  let baseinfo_type = T.enum \"GIInfoType\" ~typedef:true [\n      Invalid, gi_info_type_invalid;\n      Function, gi_info_type_function;\n      (*\n        ...\n      *)\n      Type, gi_info_type_type;\n      Unresolved, gi_info_type_unresolved;\n    ]\n      ~unexpected:(fun _x -\u003e assert false)\nend\n```\n\nThe dune file is really simple:\n\n```\n(library\n  (name bindings)\n  (libraries ctypes.stubs ctypes)\n)\n```\nIt just defines the library name and its dependencies.\n\n### 2 e The stubgen directory\n\nThere are two files, the *bindings_c_gen.ml* and its build file called *dune*.\nIn this directory, a *bindings_c_gen.exe* file is generated with the following\nrule:\n\n```\n(executable\n (name bindings_c_gen)\n (modules bindings_c_gen)\n (libraries bindings ctypes.stubs ctypes))\n```\n\nWhen executed, this program generates the *bindings_stubs_gen.c*, which is the C\nsource code for the bindings generator.\n\n```\n(rule\n (targets bindings_stubs_gen.c)\n (deps (:stubgen ../stubgen/bindings_c_gen.exe))\n (action (with-stdout-to %{targets} (run %{stubgen} -c))))\n```\n\nThe following dune rule is used to build the *bindings_stubs_gen.exe*:\n```\n(rule (targets bindings_stubs_gen.exe)\n (deps bindings_stubs_gen.c c_flags c_library_flags)\n (action\n  (bash\n   \"%{cc} bindings_stubs_gen.c -I `dirname %{lib:ctypes:ctypes_cstubs_internals.h}` -I %{ocaml_where} $(\u003c c_flags) $(\u003c c_library_flags) -o %{targets}\"))\n)\n```\nThe particularity of this rule is that it uses the **bash** stanza to call the C\ncompiler with `%{cc}`. [This is variable automatically expanded by dune](https://dune.readthedocs.io/en/latest/dune-files.html?expansion#variables-expansion).\n\nThe executable is then used to generate the *bindings_stubs.ml* file in the\n*bindings* directory.\n\n### 2 f The lib directory\n\nThis is the final directory in which the classical Ctypes bindings will be \"merged\"\nwith the automatically generated bindings for the enum in order to create a\nlibrary for the bindings.\n\nThe library depends on the *Bindings_stubs*, and the build step needs some flags to compile.\nThe following rules describe to dune how to generate those dependencies.\n\n```\n(rule\n  (targets c_flags.sexp c_library_flags.sexp ccopts.sexp)\n  (deps    (:x ../config/discover.exe))\n  (action  (run %{x} -ocamlc %{ocamlc}))\n)\n\n(rule\n  (targets bindings_stubs.ml)\n  (deps ../stubgen/bindings_stubs_gen.exe)\n  (action (with-stdout-to %{targets} (run %{deps} -ml))))\n```\n\nThe library is build with :\n\n```\n(library\n (name        gi)\n  (libraries ctypes ctypes.foreign str bindings )\n  (c_flags         (:include c_flags.sexp))\n  (c_library_flags (:include c_library_flags.sexp))\n  (ocamlopt_flags (-ccopt (:include ccopts.sexp)))\n)\n```\n\n# Conclusion\n\nIn the main application, the *gi* library with the bindings can then be used:\n\n```ocaml\nlet namespace = \"GObject\"\nlet typelib = Gi.Repository.require namespace ()\nlet struct_name = \"Value\"\n\nlet test_baseinfo_get_type () =\n  match Gi.Repository.find_by_name namespace struct_name with\n  | None -\u003e prerr_endline \"No base info found\"; exit 1\n  | Some base_info -\u003e\n      match Gi.Base_info.get_type base_info with\n        | Struct -\u003e print_endline \"It works!\"\n      | _ -\u003e prerr_endline \"Bad type\"; exit 1\n\nlet () = test_baseinfo_get_type ()\n```\n\nRun the build process and the executable with `dune exec bin/main.exe --profile=release`.\nThe following output is displayed:\n\n```\nIt works!\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedlemo%2Fctypes-stubs-generation-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcedlemo%2Fctypes-stubs-generation-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedlemo%2Fctypes-stubs-generation-notes/lists"}