{"id":24736381,"url":"https://github.com/engineering-research-and-development/o2k-connector","last_synced_at":"2025-10-10T05:31:12.470Z","repository":{"id":44542980,"uuid":"504163504","full_name":"Engineering-Research-and-Development/o2k-connector","owner":"Engineering-Research-and-Development","description":"Lightweight connector that enables context data sharing from Orion Context Broker towards Apache Kafka","archived":false,"fork":false,"pushed_at":"2025-01-31T10:35:47.000Z","size":326,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T16:21:55.742Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Engineering-Research-and-Development.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-06-16T13:19:14.000Z","updated_at":"2025-01-31T10:35:51.000Z","dependencies_parsed_at":"2023-09-29T12:10:53.415Z","dependency_job_id":"c3ccb728-940c-4230-ac71-5d28610ef0a4","html_url":"https://github.com/Engineering-Research-and-Development/o2k-connector","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Engineering-Research-and-Development/o2k-connector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Fo2k-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Fo2k-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Fo2k-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Fo2k-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Engineering-Research-and-Development","download_url":"https://codeload.github.com/Engineering-Research-and-Development/o2k-connector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Engineering-Research-and-Development%2Fo2k-connector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002866,"owners_count":26083468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-27T21:25:00.740Z","updated_at":"2025-10-10T05:31:12.102Z","avatar_url":"https://github.com/Engineering-Research-and-Development.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# O2K-Connector: a FIWARE Orion to Kafka Connector\n\n[![](https://nexus.lab.fiware.org/static/badges/chapters/core.svg)](https://www.fiware.org/developers/catalogue/)\n[![License badge](https://img.shields.io/github/license/Engineering-Research-and-Development/o2k-connector.svg)](https://opensource.org/licenses/AGPL-3.0)\n[![](https://img.shields.io/badge/tag-fiware+o2k-orange.svg?logo=stackoverflow)](http://stackoverflow.com/questions/tagged/fiware+o2k)\u003cbr/\u003e\n[![Quay badge](https://img.shields.io/badge/quay.io-fiware%2Fo2k--connector-grey?logo=red%20hat\u0026labelColor=EE0000)](https://quay.io/repository/fiware/o2k-connector)\n[![Docker badge](https://img.shields.io/badge/docker-rdlabengpa%2Fo2k--connector-blue?logo=docker)](https://registry.hub.docker.com/r/rdlabengpa/o2k-connector)\n\u003cbr/\u003e\n![Status](https://nexus.lab.fiware.org/repository/raw/public/badges/statuses/incubating.svg)\n\u003cbr/\u003e\n\u003cimg align=\"right\" width=\"150\" src=\"/docs/images/o2k_logo.png\" /\u003e \n\nO2K-Connector is a lightweight connector, built in Python, that enables context data sharing from Orion Context Broker towards Apache Kafka. In more detail, O2K-Connector subscribes to Orion Context Broker and publishes the received context-data to a built-in kafka topic.\n\u003cbr/\u003e\n\n| \u003cimg style=\"height:1em\" src=\"https://quay.io/static/img/quay_favicon.png\"/\u003e [quay.io](https://quay.io/repository/fiware/o2k-connector) | :whale: [Docker Hub](https://hub.docker.com/r/rdlabengpa/o2k-connector) | \n|---|---|\n\n## O2K-Connector example placement in a FIWARE/Apache Architecture\n\n# ![O2K-Connector-placement](docs/images/schema.png)\n\n## Contents\n\n-   [Install](#install)\n  -   [Configuration](#configuration)\n  -   [Docker Install](#docker---recommended)\n-   [API](#api)\n-   [Testing](#testing)\n-   [License](#license)\n\n## Configuration\n\nO2K-Connector can be fully customized and configured by the means of environment variables listed on __*o2k.env*__ file.\n\n\u003ctable role=\"table\"\u003e\n    \u003cthead\u003e\n        \u003ctr align=\"center\"\u003e\n            \u003cth\u003eEnvironment variables\u003c/th\u003e\n            \u003cth\u003eDescription\u003c/th\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eLOG_LEVEL\u003c/td\u003e\n          \u003ctd\u003eLevel of O2K Debug. Possible values: INFO, DEBUG, ERROR\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eLOGS_PATH\u003c/td\u003e\n          \u003ctd\u003ePath of O2K-Connector logs\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eO2K_HOST\u003c/td\u003e\n          \u003ctd\u003eListener for O2K-Connector\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eO2K_PORT\u003c/td\u003e\n          \u003ctd\u003ePort of O2K-Connector\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSCHEMA_URL\u003c/td\u003e\n          \u003ctd\u003eKafka Schema url for avro messages\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eBOOTSTRAP_SERVERS\u003c/td\u003e\n          \u003ctd\u003eKafka Boostrap Servers\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eKAFKA_TOPIC\u003c/td\u003e\n          \u003ctd\u003eUser-specified Kafka Topic, if left blank, it will be auto-generated on the fly taking into account FIWARE_SERVICE and FIWARE_SERVICEPATH environments for NGSI V2 and entity id for NGSI-LD\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eORION_HOST\u003c/td\u003e\n          \u003ctd\u003eHostname of Orion Context Broker\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eORION_PORT\u003c/td\u003e\n          \u003ctd\u003ePort of Orion Context Broker\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eFIWARE_SERVICE\u003c/td\u003e\n          \u003ctd\u003eFiware Service for NGSI V2\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eFIWARE_SERVICEPATH\u003c/td\u003e\n          \u003ctd\u003eFiware ServicePath for NGSI V2s\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eMQTT_HOST\u003c/td\u003e\n          \u003ctd\u003eHostname of MQTT Server, if used\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eMQTT_PORT\u003c/td\u003e\n          \u003ctd\u003ePort of MQTT Server, if used\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eMQTT_TOPIC\u003c/td\u003e\n          \u003ctd\u003eTopic to which publish on MQTT Server, if used\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_JSON_PATH\u003c/td\u003e\n          \u003ctd\u003ePath of subscription configuration files\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_JSON_FILENAME\u003c/td\u003e\n          \u003ctd\u003eFilename of subscription configuration file for NGSI V2 subscription\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_JSON_FILENAME_LD\u003c/td\u003e\n          \u003ctd\u003eFilename of subscription configuration file for NGSI-LD subscription\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_JSON_FILENAME_MQTT\u003c/td\u003e\n          \u003ctd\u003eFilename of subscription configuration file for NGSI-V2 subscription and MQTT underlying subscription engine\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_SCHEMA_FILE_PATH\u003c/td\u003e\n          \u003ctd\u003eFilename of subscription schema file for NGSI V2 subscription\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_SCHEMA_FILE_PATH_LD\u003c/td\u003e\n          \u003ctd\u003eFilename of subscription schema file for NGSI-LD subscription\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSUBSCRIPTION_SCHEMA_FILE_PATH_MQTT\u003c/td\u003e\n          \u003ctd\u003eFilename of subscription schema file for NGSI-V2 subscription and MQTT underlying subscription engine\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eORION_VERSION\u003c/td\u003e\n          \u003ctd\u003eOrion Context Broker version. Possible values: V2, LD\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eORION_SUBSCRIPTION\u003c/td\u003e\n          \u003ctd\u003eOrion Context Broker subscription engine. Possible values: http, mqtt\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eMULTIPLE_SUBSCRIPTIONS\u003c/td\u003e\n          \u003ctd\u003eFor NGSI V2 only: set to \"false\" to handle a unique subscription with multiple attributes\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eKAFKA_ENABLE_SSL\u003c/td\u003e\n          \u003ctd\u003eSet to \"true\" if want to use SSL connection to Kafka\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eKAFKA_SSL_CA\u003c/td\u003e\n          \u003ctd\u003eCertificate Authority key (path file inside the container) to use for SSL connection to Kafka\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eKAFKA_SSL_KEY\u003c/td\u003e\n          \u003ctd\u003eClient key (path file inside the container) to use for SSL connection to Kafka\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eKAFKA_SSL_CERTIFICATE\u003c/td\u003e\n          \u003ctd\u003eCertificate key (path file inside the container) to use for SSL connection to Kafka\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n## Getting Started - Install\n\nTo instantiate O2K-Connector you can use docker-compose which will take care of the creation of the container running python. Alternatively you can locally run the connector by running the python script.\n\nN.B:\nThe connector will create a Kafka Topic structured as follows:\n\u003ccode\u003e{FIWARE_SERVICE}\\_{FIWARE_SERVICEPATH*}\\_{NGSI_ENTITY_ID}\\_{NGSI_ENTITY_TYPE}\u003c/code\u003e\n\n*FIWARE_SERVICEPATH is deprived of '/' character\n\n### Docker - Recommended\n\n1. Configure subscription payload configuration _'subscription.json_' inside \"conf\" folder.\n2. Configure your environment variables in the _'docker-compose.yml'_ file\n3. Run docker-compose command into project root folder:\n```sh\ndocker-compose up\n```\n\n## API\n\n\u003ctable role=\"table\"\u003e\n    \u003cthead\u003e\n        \u003ctr align=\"center\"\u003e\n            \u003cth\u003eHTTP Method\u003c/th\u003e\n            \u003cth\u003eService\u003c/th\u003e\n            \u003cth\u003eDescription\u003c/th\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003ePOST\u003c/td\u003e\n          \u003ctd\u003e/notify\u003c/td\u003e\n          \u003ctd\u003eV2 Context data ingestion route\u003c/td\u003e\n      \u003c/tr\u003e\n        \u003ctr\u003e\n          \u003ctd\u003ePOST\u003c/td\u003e\n          \u003ctd\u003e/notifyld?subscriptionId=urn:ngsi-ld:Type:Entity\u003c/td\u003e\n          \u003ctd\u003eLD Context data ingestion route\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\n## Testing\n\nTo test O2K-Connector, few services are required, in particular:\n - Orion Context Broker\n - A fully working Kafka Environment\n\nAt startup O2K-Connector subscribes to Orion Context Broker which is required to be running. Context-data will start being published to a given Kafka Topic (generate on-the-fly) as soon as possible.\n\nTo text context-data consumpion, run kafka-console-consumer command:\n\n\u003ccode\u003ekafka-console-consumer --bootstrap-server broker-1:29092,broker-2:29093,broker-3:29094 --topic {FIWARE_SERVICE}\\_{FIWARE_SERVICEPATH*}\\_{NGSI_ENTITY_ID}\\_{NGSI_ENTITY_TYPE}\u003c/code\u003e\n\n*FIWARE_SERVICEPATH is deprived of '/' character\n\n## License\nO2K-Connector is licensed under\n\n*GNU Affero General Public License v3.0*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineering-research-and-development%2Fo2k-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fengineering-research-and-development%2Fo2k-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineering-research-and-development%2Fo2k-connector/lists"}