{"id":26129520,"url":"https://github.com/mathworks-ref-arch/matlab-apache-kafka","last_synced_at":"2025-06-13T21:05:28.019Z","repository":{"id":78586812,"uuid":"224416315","full_name":"mathworks-ref-arch/matlab-apache-kafka","owner":"mathworks-ref-arch","description":"MATLAB Interface for Apache Kafka","archived":false,"fork":false,"pushed_at":"2022-09-27T20:38:27.000Z","size":2232,"stargazers_count":16,"open_issues_count":2,"forks_count":6,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-13T18:55:31.183Z","etag":null,"topics":["kafka"],"latest_commit_sha":null,"homepage":"","language":"MATLAB","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mathworks-ref-arch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-27T11:35:13.000Z","updated_at":"2024-06-24T19:12:38.000Z","dependencies_parsed_at":"2023-06-10T03:45:49.194Z","dependency_job_id":null,"html_url":"https://github.com/mathworks-ref-arch/matlab-apache-kafka","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/mathworks-ref-arch/matlab-apache-kafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-apache-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-apache-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-apache-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-apache-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mathworks-ref-arch","download_url":"https://codeload.github.com/mathworks-ref-arch/matlab-apache-kafka/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathworks-ref-arch%2Fmatlab-apache-kafka/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259719716,"owners_count":22901239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka"],"created_at":"2025-03-10T19:49:16.310Z","updated_at":"2025-06-13T21:05:27.984Z","avatar_url":"https://github.com/mathworks-ref-arch.png","language":"MATLAB","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MATLAB Interface *for Apache Kafka*\n\nMATLAB® interface for Apache Kafka®. This package provides Kafka clients for\nMATLAB, Simulink and Embedded Coder.\n\nKafka® is used for building real-time data pipelines and streaming apps.\nIt is horizontally scalable, fault-tolerant, fast and widely used.\n\n## Requirements\n\nRequires MATLAB release R2018a or later. Also requires Simulink when using the Kafka Client blocks.\nThe package is currently only supported for Windows and Linux. The Embedded Coder solution is only supported\non Linux. \n\n### MathWorks Products (https://www.mathworks.com)\n\n* MATLAB\n* [Optional] Simulink\n* [Optional] MATLAB Coder\n* [Optional] Simulink Coder\n* [Optional] Embedded Coder\n\n### 3rd Party Products\n\nFor building the mex functions and S-functions.\n* A C/C++ compiler for your platform, in accordance with\n[MATLAB supported compilers](https://www.mathworks.com/support/requirements/supported-compilers.html).\n* [CMake](https://cmake.org/), for building dependencies on Windows\n* [librdkafka](https://github.com/edenhill/librdkafka), version 1.0 or higher\n* Some other libraries, as declared in the installation of librdkafka.\n* [Optional] [jansson](http://www.digip.org/jansson/) JSON library, if you're using the Simulink JSON-Converter.\n\n## Introduction\n\n[Apache Kafka®](http://kafka.apache.org/)  is a community distributed event streaming platform capable of handling\ntrillions of events a day. Initially conceived as a messaging queue, Kafka is based on an abstraction of a\ndistributed commit log. Since being created and open sourced by LinkedIn in 2011, Kafka has quickly evolved\nfrom messaging queue to a full-fledged event streaming platform.\n\nThis project contains C/C++-based Kafka Clients, producers and consumers, for both MATLAB and Simulink. In Simulink,\ncode generation with Embedded Coder is also supported.\n\nThe usage in MATLAB is for prototyping, and if you want to use it in a production system, we recommend compiling\nyour code for *MATLAB Production Server*.\n\nFor usage in Simulink, this is also prototyping. Generate C code using Embedded Coder to use in a production environment.\n\n## Installation\n\nFirst install *librdkafka*, and optionally *jansson*, see [the documentation](Documentation/Installation.md).\n\nTo install the MATLAB part, just do\n```matlab\ncd Software/MATLAB\nstartup\n```\nThis will add the needed paths. If you want them to be saved for future sessions, run\n```matlab\nsavepath\n```\n\nYou will need to compile the Mex function and optionally the S-functions. This is done with the two commands:\n```matlab\nkafka_build_mex\nkafka_build_sfuns\n```\n\nIf you intend to generate dockerfiles too, you need to build the base images before you can build from these dockerfiles. To build the base images, run\n```matlab\nkafka_build_dockerfiles\n```\n\nFor help refer to [the documentation](Documentation/Installation.md).\n\n## Usage\n\n### MATLAB Kafka Producer\n\nUsing the Kafka producer is straightforward.\n\n```matlab\nP = kafka.Producer('\u003cmybroker\u003e', '\u003cmytopic\u003e');\nP.publish('mykey', 'my message');\n```\nThe same producer can, and should be used for sending several\nmessages to the same topic.\n\nThe key and message arguments should be in a form that can automatically be converted to ```int8```. If you have a structure,\nyou should first convert it to JSON.\n```matlab\n\u003e\u003e S = struct('item', '3422', 'value', [3,4,5])\nS =\n  struct with fields:\n\n     item: '3422'\n    value: [3 4 5]\n\u003e\u003e jsstr = jsonencode(S)\njsstr =\n    '{\"item\":\"3422\",\"value\":[3,4,5]}'\n```\n\nThe Kafka consumer is similar.\n```matlab\nC = kafka.Consumer(brokers, topic, group);\n[key, val, errMsg] = C.consume(timeoutMillis);\n```\n\nPlease see the [documentation](Documentation/README.md) for more information.\n\n### Simulink blocks\nThe current version contains 3 blocks for Kafka communication. A consumer block, a producer block,\nand a very simple block to convert flat JSON structures.\n\nThe Kafka functionality works both for simulation and code generation.\n\nRead more in the [corresponding documentation ](Documentation/BasicUsage.md#simulink-clients)\n\n### Embedded Coder\nThis repository also contains an Embedded Coder target for use with Kafka. It will generate code\nfrom the Simulink model and the Kafka blocks. This code can also be dockerized,\nwith the use of a Dockerfile that is generated with the code.\n\nRead more in the [corresponding documentation](Documentation/BasicUsage.md#embedded-coder-target)\n\n\n## Documentation\nSee [documentation](Documentation/README.md) for more information.\n\n\n## License\nThe license for the MATLAB Interface *for Apache Kafka* is available in the [LICENSE.md](LICENSE.md)\nfile in this GitHub repository.\nThis package uses certain third-party content which is licensed under separate license agreements.\nSee the 3rd party packages for the respective license details.\n\n## Enhancement Request\nProvide suggestions for additional features or capabilities using the following link:   \nhttps://www.mathworks.com/products/reference-architectures/request-new-reference-architectures.html\n\n## Support\nEmail: `mwlab@mathworks.com`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathworks-ref-arch%2Fmatlab-apache-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathworks-ref-arch%2Fmatlab-apache-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathworks-ref-arch%2Fmatlab-apache-kafka/lists"}