{"id":23977454,"url":"https://github.com/thecomputekid/scrapper","last_synced_at":"2026-05-01T23:35:11.517Z","repository":{"id":217948047,"uuid":"745182156","full_name":"theComputeKid/scrapper","owner":"theComputeKid","description":"pretends to export c++ functions but with a c abi","archived":false,"fork":false,"pushed_at":"2024-09-15T01:27:06.000Z","size":44,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-07T07:37:49.118Z","etag":null,"topics":["c","cpp","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/theComputeKid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-18T19:56:36.000Z","updated_at":"2024-09-15T01:27:10.000Z","dependencies_parsed_at":"2024-01-28T09:47:55.897Z","dependency_job_id":null,"html_url":"https://github.com/theComputeKid/scrapper","commit_stats":null,"previous_names":["thecomputekid/scrapper"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theComputeKid%2Fscrapper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theComputeKid%2Fscrapper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theComputeKid%2Fscrapper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theComputeKid%2Fscrapper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/theComputeKid","download_url":"https://codeload.github.com/theComputeKid/scrapper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240525233,"owners_count":19815431,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","cpp","python"],"created_at":"2025-01-07T07:37:54.967Z","updated_at":"2026-05-01T23:35:06.493Z","avatar_url":"https://github.com/theComputeKid.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\t\u003ch1\u003e\u003cstrong\u003escrapper\u003c/strong\u003e\u003c/h1\u003e\n\t\u003cp\u003epretends to export c++ functions with a c abi\u003c/p\u003e\n    \u003cimg alt=\"GitHub License\" src=\"https://img.shields.io/github/license/thecomputekid/scrapper?style=for-the-badge\u0026color=blue\"\u003e\n    \u003cimg alt=\"GitHub Workflow Status\" src=\"https://img.shields.io/github/actions/workflow/status/theComputeKid/scrapper/test.yml?style=for-the-badge\"\u003e\n\t\u003cimg alt=\"Lines of Code\" src=\"https://tokei.rs/b1/github/thecomputekid/scrapper?category=code\u0026style=for-the-badge\"\u003e\n\u003c/div\u003e\n\n## Summary\n\nThis project pretends to export your shared library's C++ functions with a C ABI so that the end user can benefit from a clean C++ interface to your library's functionality, while also benefitting from a stable C ABI (e.g. with no name mangling etc).\n\nFor example, if our shared library contains the following functionality that is to be made available to the user:\n\n```\ntemplate \u003ctypename T\u003e\nT add(T left, T right){\n  return left + right;\n};\n```\n\nThe user of our library should be able to use this function with its native C++ interface without worrying about ABI compatiblity:\n\n```\n// Exported (native) signature.\ntemplate \u003ctypename T\u003e\nT add(T left, T right);\n\n#include \u003ciostream\u003e\n\nint main(){\n  // C++ API used natively by the end user in their code.\n  std::cout \u003c\u003c \"1 + 2 = \" \u003c\u003c add(1, 2) \u003c\u003c std::endl;\n}\n```\n\n## Methodology\n\nThe project performs 3 steps:\n1. It wraps our library's C++ functions inside wrapper functions with a C-friendly signature (i.e.: `C++-to-C`).\n2. It then exports those C-friendly wrapper functions with a C ABI.\n3. It then wraps those exported C functions inside C++ functions with the same interface as the library's original C++ functions (i.e.: `C-to-C++`).\n\n## Example\nWe wish to export the following function for `T=float` and `T=double`:\n\n```\n// library.cpp\ntemplate \u003ctypename T\u003e\nT add(T left, T right){\n  return left + right;\n};\n```\n\n### 1. Wrap C++ functions in a C wrapper (`C++-to-C`)\n\nThe project will first produce the following wrapper:\n\n```\n// wrapper.hpp\n#pragma once\n\n// Forward declaration.\ntemplate \u003ctypename T\u003e\nT add(T left, T right);\n\nfloat add_f32(float left, float right){\n  return add(left, right);\n}\n\ndouble add_f64(double left, double right){\n  return add(left, right);\n}\n```\n\nThis generated file is meant to be included with the implementation of our C++ function:\n\n```\n// library.cpp\n\n#include \"wrapper.hpp\"\n\ntemplate \u003ctypename T\u003e\nT add(T left, T right){\n  return left + right;\n};\n```\n\n### 2. Exporting with a C ABI\n\nThe project will then add C export specifiers to the C function signatures so that they are exported from the library with the C ABI:\n\n```\n// wrapper.hpp\n#pragma once\n\n#ifdef _WIN32\n#  define SCRAPPER_EXPORT extern \"C\" __declspec(dllexport)\n#else\n#  define SCRAPPER_EXPORT extern \"C\" __attribute__((visibility(\"default\")))\n#endif\n\n// Forward declaration.\ntemplate \u003ctypename T\u003e\nT add(T left, T right);\n\nSCRAPPER_EXPORT float add_f32(float left, float right){\n  return add(left, right);\n}\n\nSCRAPPER_EXPORT double add_f64(double left, double right){\n  return add(left, right);\n}\n```\n\nAt this point, a limitation of the project becomes obvious; the C++ functions can only have C-friendly arguments, as otherwise they cannot be exported with the C ABI.\n\nAn export header is generated (with the corresponding OS-specific symbol import macros) to allow the user to call these functions exported with the C ABI:\n\n```\n// export.h\n#pragma once\n\n#ifdef __cplusplus\n#  define SCRAPPER_EXTERN_C extern \"C\"\n#else\n#  define SCRAPPER_EXTERN_C\n#endif\n\n#ifdef _WIN32\n#  define SCRAPPER_IMPORT SCRAPPER_EXTERN_C __declspec(dllimport)\n#else\n#  define SCRAPPER_IMPORT SCRAPPER_EXTERN_C\n#endif\n\nSCRAPPER_IMPORT float add_f32(float left, float right);\nSCRAPPER_IMPORT double add_f64(double left, double right);\n```\n\nAt this point, we have two exported symbols with the C ABI: `add_f32` and `add_f64`.\n\n### 3. Wrapping C functions in C++ wrappers (`C-to-C++`)\n\nIn order for the user to be able to use the original C++ API, the program then creates a C++ function inline with the exported header that is meant to be compiled with the user's code:\n\n```\n// export.h\n#pragma once\n\n#ifdef __cplusplus\n#  define SCRAPPER_EXTERN_C extern \"C\"\n#else\n#  define SCRAPPER_EXTERN_C\n#endif\n\n#ifdef _WIN32\n#  define SCRAPPER_IMPORT SCRAPPER_EXTERN_C __declspec(dllimport)\n#else\n#  define SCRAPPER_IMPORT SCRAPPER_EXTERN_C\n#endif\n\nSCRAPPER_IMPORT float add_f32(float left, float right);\nSCRAPPER_IMPORT double add_f64(double left, double right);\n\n#ifdef __cplusplus\ntemplate \u003ctypename T\u003e T add(T left, T right);\n#endif\n\n#ifdef __cplusplus\n#include \u003ctype_traits\u003e\ntemplate \u003ctypename T\u003e T add(T left, T right){\n  if constexpr(std::is_same_v\u003cT, float\u003e){\n    return add_f32(left, right);\n  } else if constexpr (std::is_same_v\u003cT, double\u003e){\n    return add_f64(left, right);\n  } else {\n    // Static assert to prevent unsupported type from being provided.\n  }\n}\n#endif\n```\n\nThe declaration of the C++ wrapper function is kept separate to the implementation (an opinionated decision) so the user-facing interface is clean. Having the implementation at the bottom allows the user to not care about the bottom half of the exported interface. Because the implementation is `constexpr` (and inline), there is no user runtime cost for the switchyard contained within it.\n\nThe program can also provide an even cleaner interface, if requested. It can generate a separate implementation file that is automatically included in the main export header:\n\n```\n// export.h\n#pragma once\n\n#ifdef __cplusplus\n#  define SCRAPPER_EXTERN_C extern \"C\"\n#else\n#  define SCRAPPER_EXTERN_C\n#endif\n\n#ifdef _WIN32\n#  define SCRAPPER_IMPORT SCRAPPER_EXTERN_C __declspec(dllimport)\n#else\n#  define SCRAPPER_IMPORT SCRAPPER_EXTERN_C\n#endif\n\nSCRAPPER_IMPORT float add_f32(float left, float right);\nSCRAPPER_IMPORT double add_f64(double left, double right);\n\n#ifdef __cplusplus\ntemplate \u003ctypename T\u003e T add(T left, T right);\n#include \"exportImpl.hpp\"\n#endif\n```\n\nThis creates a clean interface with the minimum information needed by the user of the code. The implementation file is then generated as:\n\n```\n// exportImpl.hpp\n#pragma once\n#include \"export.h\"\n\n#ifdef __cplusplus\n#include \u003ctype_traits\u003e\ntemplate \u003ctypename T\u003e T add(T left, T right){\n  if constexpr(std::is_same_v\u003cT, float\u003e){\n    return add_f32(left, right);\n  } else if constexpr (std::is_same_v\u003cT, double\u003e){\n    return add_f64(left, right);\n  } else {\n    // Static assert to prevent unsupported type from being provided.\n  }\n}\n```\n\nThe user will only need to include the main `export.h` header file. Note that the export header implementation file `exportImpl.hpp` includes `export.h`, but this is not strictly required and is only done to satisy code analysis tools which will want to index where the C functions used in the `if constexpr` switchyard are taken from.\n\n### 4. User code\n\nThe user can then call the C++ API directly, as if calling the C++ interface of the original C++ function that was meant to be exported from the library:\n\n```\n// user.cpp\n#include \"export.h\"\n#include \u003ciostream\u003e\n\nint main(){\n  // C++ API used natively by the end user in their code.\n  std::cout \u003c\u003c \"1 + 2 = \" \u003c\u003c add(1, 2) \u003c\u003c std::endl;\n}\n```\n\nThey can also call the C ABI functions directly:\n\n```\n// user.c\n#include \"export.h\"\n#include \u003cstdio.h\u003e\n\nint main(){\n  float const left = 9.0f;\n  float const right = 1.0f;\n  float const result = add_f32(left, right);\n  printf(\"%f + %f = %f\", left, right, result);\n}\n```\n\n## Usage\n\nThe configuration of the original C++ function(s) to be exported must be defined in a JSON file, provided to the program. Two examples are provided:\n\n```\npython3 ./scrapper.py examples/simple.json --output-header out/simple.h --output-impl out/simple.hpp\npython3 ./scrapper.py examples/advanced.json -oh out/advanced.h -oi out/advanced.hpp -s\n```\n\nwhere:\n- `--output-header`, `-oh`: The exported interface of your library. Equivalent to `export.h` in the example.\n- `--output-implementation`, `-oi`: The implementation of the C functions, to be compiled as part of the library. Equivalent to `wrapper.h` in the example.\n- `--separate-header`, `-s`: Split the export header into a cleaner interface as shown in the example with `exportImpl.hpp`. The program will automatically create this extra implementation file in the same directory as the `--output-header`.\n\nThe `simple` example contains the minimum JSON config required for the program to work and is a good starting point. Additional customization is welcome via pull requests.\n\n## JSON schema\n\nThe JSON schema used for validation is provided as part of the project in `utils/schema.json`. The JSON is validated as part of the script. An example is:\n\n```\n{\n    \"$schema\": \"https://raw.githubusercontent.com/theComputeKid/scrapper/main/utils/schema.json\",\n    \"includes\": [\n        \"\u003cstdint.h\u003e\"\n    ],\n    \"macro\": \"MY_PROJECT\",\n    \"mapping\": {\n        \"float\": \"fp32\"\n    },\n    \"functions\": [\n        {\n            \"name\": \"myFunc\",\n            \"combination\": \"fixed\",\n            \"templates\": [\n                {\n                    \"name\": \"S\",\n                    \"types\": [\n                        \"float\",\n                        \"int8_t\"\n                    ]\n                },\n                {\n                    \"name\": \"T\",\n                    \"types\": [\n                        \"double\",\n                        \"int16_t\"\n                    ]\n                }\n            ],\n            \"parameters\": [\n                {\n                    \"name\": \"p1\",\n                    \"type\": \"S\",\n                    \"const\": false,\n                    \"pointer\": false\n                },\n                {\n                    \"name\": \"p2\",\n                    \"type\": \"T\",\n                    \"const\": true,\n                    \"pointer\": true\n                }\n            ],\n            \"return\": {\n                \"type\": \"T\",\n                \"const\": false,\n                \"pointer\": false\n            }\n        }\n    ]\n}\n\n```\n\nThe supplied JSON must have:\n- `\"includes\"`: Optional. Any additional includes that the C exported ABI requires.\n- `\"macro\"`: Optional. This is the base macro used to define the export/import linkage of the generated files. For example, if `\"macro\" : \"PROJ\"`, then the macro `\"PROJ_EXPORT\"` will be used to define OS-specific linkage decorations (e.g.: `__declspec(dllexport)`). Default is `\"SCRAPPER\"`.\n- `\"functions\"`: Array of C++ functions to be exported.\n- `\"mapping\"`: Optional. This is the suffix added to the base C++ function name for each type to be exported. By default, the mapping is specified in `utils/mapping.json`, but this can be overridden. If a type is used that is not in the map, then it is used as-is as the suffix. For example, if `\"float\": \"fp32\"`, then the add function above would be exported as `float add_fp32(float left, float right)` for the `float` version.\n\nThe `\"functions\"` field is divided into:\n- `\"name\"`: Name of the C++ function to be exported. This is also the base name that the generated functions will use.\n- `\"description\"`: Optional. Generate doxygen comments (also per parameter).\n- `\"combination\"`: Optional. Defines the combination of templates to create wrappers for, when more than one is provided.\n  - `\"all\"`: Default. All combinations of supplied types (cartesian product)\n  - `\"fixed\"`: Linear combination of types. i.e. `template[0]type[0]` with `template[1]type[0]`, `template[0]type[1]` with `template[1]type[1]`.\n- `\"templates\"`: An array of templates that the C++ function accepts.\n\nThe `\"templates\"` field is divided into:\n- `\"name\"`: The placeholder type for the template (e.g. `T`) in the C++ function signature.\n- `\"types\"`: Array of types that are accepted by the template. These must be C friendly (i.e. no std::XYZ).\n\n## Test\n\nA complete test case can be found in the `test` folder, demonstrating the intended usage of the script in a project. It involves the following steps:\n\n- run this script to generate the wrappers.\n- compile the library.\n- compile the C user code.\n- compile the C++ user code.\n- run simple tests for the compiled executables.\n\nThe complete project can be built using cmake presets (for cmake \u003e= 3.28):\n\n```\ncd test\ncmake --workflow --preset default\ncmake --workflow --preset split\n```\n\nFor older versions of cmake, you will need to configure manually (you can use the presets.json as a guide). It requires the python package dependencies to be satisfied.\n\n## Dependencies\n\nDependencies are listed in `requirements.txt`, and can be installed with:\n\n```\npip install -r requirements.txt\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthecomputekid%2Fscrapper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthecomputekid%2Fscrapper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthecomputekid%2Fscrapper/lists"}