{"id":19838690,"url":"https://github.com/miikama/ncdl-gen","last_synced_at":"2026-01-27T16:32:33.023Z","repository":{"id":93495117,"uuid":"555274742","full_name":"miikama/ncdl-gen","owner":"miikama","description":"A Parser for NetCDF cdl language","archived":false,"fork":false,"pushed_at":"2025-10-02T19:18:16.000Z","size":350,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-02T21:08:56.470Z","etag":null,"topics":["cpp","netcdf","parser","vscode"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miikama.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-21T09:04:59.000Z","updated_at":"2025-10-02T19:15:39.000Z","dependencies_parsed_at":"2023-12-14T20:51:56.899Z","dependency_job_id":"772c7eb3-76a6-4669-afdc-3239de0b5292","html_url":"https://github.com/miikama/ncdl-gen","commit_stats":{"total_commits":138,"total_committers":2,"mean_commits":69.0,"dds":"0.49275362318840576","last_synced_commit":"28191b4c9a0d5226faf1b7a8412f741b9c46530e"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/miikama/ncdl-gen","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miikama%2Fncdl-gen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miikama%2Fncdl-gen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miikama%2Fncdl-gen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miikama%2Fncdl-gen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miikama","download_url":"https://codeload.github.com/miikama/ncdl-gen/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miikama%2Fncdl-gen/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28816563,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T12:25:15.069Z","status":"ssl_error","status_checked_at":"2026-01-27T12:25:05.297Z","response_time":168,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","netcdf","parser","vscode"],"created_at":"2024-11-12T12:18:42.453Z","updated_at":"2026-01-27T16:32:33.015Z","avatar_url":"https://github.com/miikama.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ncdlgen\n\n`ncdlgen` in a nutshell: A library that enables creating interfaces for reading and writing structured typed data in multiple formats.\n\nncdlgen is a C++ library that provides a [NetCDF](https://github.com/Unidata/netcdf-c) cdl-_parser_ and a _code generator_ for C++ code that reads/writes NetCDF files to reads/writes data with [ZeroMQ](https://zeromq.org/).\n\nNetCDF is a platform independent data file format for structured data. It is commonly used in Earth observation missions as data storage format. NetCDF cdl-files describe structured data and are used together with the NetCDF library tools. The Netcdf CDL grammar is available [here](https://manpages.ubuntu.com/manpages/focal/man1/ncgen.1.html).\n\nZeroMQ provides sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast.\n\nIn the future `ncdlgen` could be used as a code generation tool for other structured data formats as well.\n\n## Example\n\nSee concrete example for using `ncdlgen` as a library to enable code generation and writing to different pipes under `examples/`. Main current features include generating interface code based on `.cdl` file and reading the same data in multiple formats.\n\nInterface is described via .cdl file:\n\n```\nnetcdf Data {\n  variables:\n    int foo ;\n    float bar ;\n}\n```\n\nAnd passing this data\n\n```c++\n// Data in generated during compilation with the interface code generator\ngenerated::Data data{{1}, {2.0}};\n\n// Create ZeroMQPipe at local socket and send data\nncdlgen::ZeroMQPipe pipe();\ngenerated::write(pipe, data);\n\n// Receive the data (other process, node etc.)\ngenerated::read(pipe, data);\n\n// Write received data to file\nncdlgen::NetCDFPipe ncpipe(\"data.nc\");\ngenerated::read(ncpipe, data);\n```\n\n## Installation\n\nClone the repository\n\n```sh\ncd ncdl-gen\ngit clone git@github.com:miikama/ncdl-gen.git\n```\n\n### Dependencies\n\nGet all dependencies with conan. Alternatively, download dependencies and configure `cmake` by hand\n\n```sh\nconan install .\n```\n\n### Install with conan\n\nInstall the library for downstream consumers (with conan)\n\n```sh\nconan build .\nconan export .\n```\n\n### Install with cmake\n\nAfter installing libraries with conan `ncdlgen` can be install with\n\n```sh\ncd build\ncmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=~/ncdlgen -DCMAKE_PREFIX_PATH=$(pwd)/Release/generators ..\nmake -j6 \u0026\u0026 make install\n```\n\nAfter running `conan install .` all the required `packageConfig.cmake` files for `ncdlgen`are now under `build/Release/generators` for finding the dependencies. It is also possible to manually download all the primary and transitive dependencies without Conan. Currently primary dependencies are\n\n- fmt\n- netCDF\n- GTest\n- cppzmq\n\n\u003e NOTE: add permanent setting with `conan profile update settings.compiler.libcxx=libstdc++11 default`\n\nTo build and run the tests, enable them separately by setting `cmake -DBUILD_TESTING=ON .. \u0026\u0026 make \u0026\u0026 make test`.\n\n## Parser\n\nTake an example cdl-file\n\n```\nnetcdf simple {\n\n    group: foo {\n\n        dimensions:\n            dim = 5 ;\n\n        variables:\n            int bar ;\n            float baz ;\n            ushort bee(dim) ;\n            int foobar(dim, dim) ;\n    }\n}\n```\n\nResult of parsing the file contents:\n\n```sh\n${installation_directory}/parser data/simple.cdl\nGroup simple\n  Group foo\n      Dimensions\n          dim = 5\n      Variables\n          int bar\n          float baz\n          ushort bee (dim)\n          int foobar (dim, dim)\n```\n\n## Code generator\n\nTake the same example `data/simple.cdl` file but use it as an input for the code-generator:\n\n```sh\n${installation_directory}/generator data/simple.cdl --header --target_pipes NetCDFPipe --interface_class_name generated_simple\n```\n\nresults in the following generated code\n\n```c++\n#pragma once\n\n#include \"stdint.h\"\n\n#include \"netcdf_pipe.h\"\n\n#include \u003cvector\u003e\n\n#include \"vector_interface.h\"\n\nnamespace ncdlgen {\n\nstruct simple\n{\n  struct foo\n  {\n      int bar;\n      float baz;\n      std::vector\u003cushort\u003e bee;\n      std::vector\u003cstd::vector\u003cint\u003e\u003e foobar;\n  };\n\n  foo foo_g{};\n};\n\nvoid read(NetCDFPipe\u0026 pipe, simple\u0026);\n\nvoid read(NetCDFPipe\u0026 pipe, simple::foo\u0026);\n\nvoid write(NetCDFPipe\u0026 pipe, const simple\u0026);\n\nvoid write(NetCDFPipe\u0026 pipe, const simple::foo\u0026);\n\n};\n```\n\nThe corresponding source file can be generated with\n\n```sh\n${installation_directory}/generator data/simple.cdl --source --target_pipes NetCDFPipe --interface_class_name generated_simple\n```\n\n### Generation as part of CMake build\n\nThis can be integrated as part of a CMake build (as done for the test/CMakelFiles.txt)\n\n```cmake\n# Run generator to create test wrappers\nadd_custom_command(\n                   OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.h\n                   OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.cpp\n                   COMMAND generator ${CMAKE_SOURCE_DIR}/data/simple.cdl --header \u003e ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.h\n                   COMMAND generator ${CMAKE_SOURCE_DIR}/data/simple.cdl --source \u003e ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.cpp\n                   DEPENDS generator\n                   DEPENDS ${CMAKE_SOURCE_DIR}/data/simple.cdl\n                   VERBATIM\n                   )\n# Add dependency to generated code\nadd_custom_target(generated-test-code\n                  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.h\n                  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.cpp  )\nset_source_files_properties(${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.h\n                            ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.cpp\n                            PROPERTIES GENERATED TRUE)\nset(GENERATED_SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/generated_simple.cpp)\n```\n\n### Generator configurability and extra pipes\n\nIf you want, the available pipe read/write entries can either be disabled with\n\n```shell\n./generator data/simple.cdl --header --target_pipes  {} --interface_class_name generated_simple --interface_namespace_name generated\n```\n\nOr you can generate for all the supported pipes\n\n```shell\n./generator data/simple.cdl --header --target_pipes NetCDFPipe ZeroMQPipe --interface_class_name generated_simple --interface_namespace_name generated\n```\n\nWhich generates the following additional interfaces on\n\n```c++\n#pragma once\n\n#include \"stdint.h\"\n\n#include \"pipes/netcdf_pipe.h\"\n#include \"pipes/zeromq_pipe.h\"\n\n#include \u003cvector\u003e\n\n#include \"vector_interface.h\"\n\nnamespace generated {\n\nstruct simple\n{\n  struct foo\n  {\n      int bar;\n      float baz;\n      std::vector\u003cuint16_t\u003e bee;\n      std::vector\u003cstd::vector\u003cint\u003e\u003e foobar;\n  };\n\n  foo foo_g{};\n};\n\nvoid read(NetCDFPipe\u0026 pipe, simple\u0026);\n\nvoid read(ZeroMQPipe\u0026 pipe, simple\u0026);\n\nvoid read(NetCDFPipe\u0026 pipe, simple::foo\u0026);\n\nvoid read(ZeroMQPipe\u0026 pipe, simple::foo\u0026);\n\nvoid write(NetCDFPipe\u0026 pipe, const simple\u0026);\n\nvoid write(ZeroMQPipe\u0026 pipe, const simple\u0026);\n\nvoid write(NetCDFPipe\u0026 pipe, const simple::foo\u0026);\n\nvoid write(ZeroMQPipe\u0026 pipe, const simple::foo\u0026);\n\n};\n```\n\n### Using generated code\n\nThe generated code for reading and writing to pipes can be used to read/write the contents of the entire file or its subgroups\n\n```c++\n#include \"generated_simple.h\"\n\nncdlgen::simple root;\nncdlgen::NetCDFPipe pipe{\"generated.nc\"};\npipe.open();\n\n// Write contents of 'simple' struct to a netcdf file\nncdlgen::write(pipe, root);\n\n// Read the contents of a netcdf file into 'simple' struct\nread(pipe, root);\n\npipe.close();\n\n// Configure ZeroMQPipe\nncdlgen::ZeroMQPipe zeromq_pipe {};\n\n// Push the contents through ZeroMQPipe\nncdlgen::write(zeromq_pipe, root);\n\n// Read the contents through ZeroMQPipe\nncdlgen::read(zeromq_pipe, root);\n```\n\n## ncdlgen as dependency\n\nSee example for downstream usage under the [example](examples) directory.\n\nClone the `ncdlgen` repository. In repository root, run\n\n```sh\nconan install .\nconan build .\nconan export .\n```\n\nIf using conan yourself, you can make a small conanfile with the wanted dependencies\n\n```\n[requires]\n ncdlgen/0.3.0\n netcdf/4.8.1\n cppzmq/4.10.0\n```\n\nAnd run `conan install . --build=missing`. Which compiles and installs the `ncdlgen` library.\n\nThen, configure your project with `cmake` with\n\n```sh\ncd build\ncmake -DCMAKE_PREFIX_PATH=$(pwd)/Release/generators -DCMAKE_BUILD_TYPE=RELEASE ..\n```\n\nThen compile.\n\n\u003e The example uses the `generator` binary compiled with the ncdlgen build. This has to be in `PATH`\n\n```sh\nexport PATH=\"$PATH:\u003cncdlgen-install-dir-with-generator-binary\u003e\"\nmake\n```\n\nAfter installing ncdlgen you can use the library in your projects `CMakeLists.txt`\n\n```cmake\ncmake_minimum_required(VERSION 3.16)\nproject(\"ncdlgen-examples\")\n\nfind_package(ncdlgen REQUIRED)\n\nadd_executable(custom_parser custom_parser.cpp)\ntarget_link_libraries(custom_parser PRIVATE ncdlgen::ncdlgen)\n`\n```\n\nThe example `CMakeLists.txt` runs the interface generator to build `example_data.h` interface during compilation. This file is included in the example `custom_parser.cpp` file.\n\n## Build using Docker\n\nThere is a `Dockerfile` that setups a build environment for the current user. Build the docker file with\n\n```sh\ndocker build \\\n    --build-arg USER=$(whoami) \\\n    --build-arg USER_ID=$(id -u $(whoami)) \\\n    --build-arg GROUP_ID=$(id -g $(whoami)) \\\n    --tag=ncdgen \\\n    .\n```\n\nAfter building, start the container with\n\n```sh\ndocker run --rm -it -v $(pwd):/home/$(whoami) ncdgen bash\n```\n\nThis mounts the repository at the home directory of the container user.\n\n## Changelist\n\nMain features for each release\n\n0.1.0\n\n- Initial relase with NetCDF cdl parser\n- Code generator for generating code for interface reading/writing\n\n0.2.0\n\n- Support Conan 2\n- Support multidimensional containers in interfaces\n- Support multidimensional containers in code generation\n- Update gtest version\n- Improve code generation configurability\n- Support global attributes outside of variables: section\n- Resolve untyped attribute types by finding corresponding variable\n- Make NetCDF and optional dependency\n\n0.3.0\n\n- Rename NetCDFInterface as NetCDFPipe\n- Introduce ZeroMQPipe\n- Add cppzmq/4.10.0 optional dependency\n- Add cli11/2.4.2 dependency\n- Improve code generation configurability\n- Make project exportable with conan\n\n## Building VSCode extension\n\nOfficial guides\n\n[Getting started](https://code.visualstudio.com/api/get-started/your-first-extension) with extensions\n\n[VSCode language extensions](https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide)\n\nThe language extensions add a `language` [Contribution Point](https://code.visualstudio.com/api/references/contribution-points)\n\nText mate grammar guides\n\n[Writing grammar](https://macromates.com/manual/en/language_grammars)\n\n[Notes about textmate language grammar](https://www.apeth.com/nonblog/stories/textmatebundle.html)\n\nSome inspiration is derived from the first example I found, which is the Jakt language in SerenityOS\n\n[Jakt language syntax](https://github.com/SerenityOS/jakt/blob/main/editors/vscode/syntaxes/jakt.tmLanguage.json)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiikama%2Fncdl-gen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiikama%2Fncdl-gen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiikama%2Fncdl-gen/lists"}