{"id":22043088,"url":"https://github.com/coatyio/dda","last_synced_at":"2025-10-14T19:09:12.860Z","repository":{"id":209044321,"uuid":"694072270","full_name":"coatyio/dda","owner":"coatyio","description":"Data Distribution Agent - Data-centric Communication and Collaboration as a Service","archived":false,"fork":false,"pushed_at":"2025-02-13T12:50:19.000Z","size":488,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-14T19:07:40.289Z","etag":null,"topics":["collaborative","data-centric-communication","decentralized","distributed-system","grpc","grpc-web","multi-agent-system","raft-consensus-algorithm","sidecar-agent"],"latest_commit_sha":null,"homepage":"https://coatyio.github.io/dda/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coatyio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-09-20T09:21:07.000Z","updated_at":"2025-02-13T12:46:04.000Z","dependencies_parsed_at":"2023-11-24T18:25:28.610Z","dependency_job_id":"2794e365-d482-433f-b069-1548e8405669","html_url":"https://github.com/coatyio/dda","commit_stats":null,"previous_names":["coatyio/dda"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/coatyio/dda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatyio%2Fdda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatyio%2Fdda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatyio%2Fdda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatyio%2Fdda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coatyio","download_url":"https://codeload.github.com/coatyio/dda/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatyio%2Fdda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279020646,"owners_count":26086895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collaborative","data-centric-communication","decentralized","distributed-system","grpc","grpc-web","multi-agent-system","raft-consensus-algorithm","sidecar-agent"],"created_at":"2024-11-30T12:14:23.523Z","updated_at":"2025-10-14T19:09:12.845Z","avatar_url":"https://github.com/coatyio.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Distribution Agent\n\n[![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/coatyio/dda.svg)](https://github.com/coatyio/dda/blob/main/go.mod)\n[![Go Report Card](https://goreportcard.com/badge/github.com/coatyio/dda)](https://goreportcard.com/report/github.com/coatyio/dda)\n[![Coverage Report](https://coatyio.github.io/dda/coverage-badge.svg)](https://coatyio.github.io/dda/coverage.html)\n[![Go Reference](https://pkg.go.dev/badge/github.com/coatyio/dda.svg)](https://pkg.go.dev/github.com/coatyio/dda)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/coatyio/dda/blob/main/LICENSE)\n[![Latest Release](https://img.shields.io/github/v/release/coatyio/dda)](https://github.com/coatyio/dda/releases/latest)\n\n## Table of Contents\n\n* [Introduction](#introduction)\n* [Overview](#overview)\n* [Data-Centric Communication](#data-centric-communication)\n* [Quick Start](#quick-start)\n  * [Prebuilt Binaries](#prebuilt-binaries)\n  * [Docker Image](#docker-image)\n* [Examples](#examples)\n* [Contributing](#contributing)\n* [License](#license)\n\n## Introduction\n\nThis project realizes the Data Distribution Agent (DDA), a loosely coupled\nmulti-agent system that enables unified and universal data-centric communication\nand collaboration in distributed and decentralized applications. Its main goal\nis to remove the inherent complexity in developing communication technologies\nfor distributed systems by exposing _Data-centric Communication as a Service_ to\napplication developers in the form of a middleware without requiring in-depth\nknowledge of messaging protocols and communication networks. DDA is designed for\nuniversal data exchange, enabling human-to-human, machine-to-machine, or hybrid\ncommunication.\n\nDDA follows the _Sidecar_/_Sidekick_ decomposition design pattern, where each\nDDA sidecar is attached to a primary application component typically co-located\non the same host. The application component provides the domain-specific core\nfunctionality of a single logical node within the distributed system. The DDA\nsidecar provides unified peripheral services to its local application component\nincluding generic data-centric patterns for communication between components,\ndistributed state synchronization, and local persistent storage. DDA sidecar\nservices are exposed by modern open-standard gRPC APIs that can be consumed\nconveniently by application components written in different languages using\ndifferent frameworks, including low-code platforms. A DDA sidecar is independent\nfrom its primary application component in terms of runtime environment and\nprogramming language and can be easily used with containers.\n\nDDA sidecars are very lightweight and work across edge, cloud, on-premise, and\nhybrid environments, either as processes or containerized. Technically, a DDA\nsidecar is running as a standalone platform-specific binary prebuilt from pure\nGo. It is available on a variety of platforms, including Linux, macOS, and\nWindows.\n\nAs an alternative to the sidecar pattern, DDA functionality is also available as\na lightweight library (Golang reference implementation) which can be directly\nembedded into a primary application component written in Go. The DDA library\nenables a deeper level of integration and optimized service invocation with less\noverhead, notably latency in calls. This approach is particularly suitable for\nlatency sensitive small footprint components on constrained devices where the\nresource cost of deploying a sidecar for each instance is not worth the\nadvantage of isolation.\n\nWe believe DDA is a major step forward in practice that allows developers to\ncreate more powerful collaborative applications with less complexity and in less\ntime, running anywhere.\n\nDDA comes with complete reference documentation, a developer guide, and a\ncollection of ready-to-run best practice code examples (for details see\n[here](https://coatyio.github.io/dda)).\n\nThe software design and coding principles on which DDA is based follow the MAYA\n(Most Advanced. Yet Acceptable.) design strategy and the YAGNI (You Aren’t Gonna\nNeed It) principle.\n\n## Overview\n\nA DDA exposes a collection of application-level peripheral services that\n_abstract_ commonly used functionality in decentralized applications:\n\n* data-centric communication over interchangeable messaging protocols\n* distributed state synchronization based on consensus algorithms that\n  guarantee strong consistency\n* local persistent key-value storage\n\nApplication components that do not embed the DDA library directly can consume\nthese services through modern open-standard APIs based on:\n\n* [gRPC](https://grpc.io/) - A high performance, open source RPC framework\n  supporting a wide range of programming languages and platforms with a common\n  declarative semantic API protocol\n* [gRPC-Web](https://github.com/grpc/grpc-web) - A JavaScript implementation of\n  gRPC for browser clients\n\nIn addition, a DDA supports observability, i.e. the ability to measure and infer\nits internal state by analyzing [OpenTelemetry](https://opentelemetry.io)\ntraces, logs, and metrics that it generates and exports.\n\n## Data-Centric Communication\n\nUtilizing DDA, you can build a distributed application with decoupled\ndata-centric interaction between decentrally organized application components,\nwhich are loosely coupled and communicate with each other in (soft) real-time\nover interchangeable open-standard publish-subscribe messaging protocols, such\nas MQTT 5 (reference implementation), Zenoh, Kafka, or DDS (on the roadmap).\nProviding a generic abstraction layer on top of interchangeable messaging\nprotocols avoids potential vendor lock-in and allows you to choose a protocol\nthat best fits your existing networking infrastructure.\n\nA typical usage is in IoT prosumer scenarios where smart components act in an\nautonomous, collaborative, and ad-hoc fashion. You can dynamically and\nspontaneously add new components and new features without distracting the\nexisting system in order to adapt to your ever-changing scenarios. All types of\ncomponents such as sensors, mobile apps, edge and cloud services can be\nconsidered equivalent.\n\nAt its core, a DDA exposes a minimal, yet complete set of communication patterns\nfor application components to talk to each other by routed one-way/two-way and\none-to-many/many-to-one communication flows without the need to know about each\nother. Subject of communication is domain-specific data transmitted to\ninterested parties in the form of:\n\n* a one-way _Event_ to route _data in motion_ from an event producer (source) to\n  interested event consumers,\n* a two-way _Action_ to issue a remote operation with _data in use_ to be\n  executed by interested actors and to receive its results,\n* a two-way _Query_ to query remote and/or distributed _data at rest_ from\n  interested retrieving components.\n\nStructured Event, Action, and Query data is described in a common way adhering\nclosely to the [CloudEvents](https://cloudevents.io/) specification. Thereby, a\nDDA combines the characteristics of both classic request-response and\npublish-subscribe communication to make data in motion, data in use, and data at\nrest available throughout the application by expressing interest in them. The\ntwo-way patterns support multiple short- and long-lived responses over time\nand/or over parties enabling sophisticated features such as querying live data\nwhich is updated over time or receiving progressive results for a remote\noperation call. In contrast to classic client-server systems, all DDAs are equal\nin that they can act both as producers/requesters and consumers/responders.\n\nOne of the unique features of DDA communication is the fact that a single Action\nor Query request in principle can yield multiple responses over time, even from\nthe same responder. The use case specific logic implemented by the requester\ndetermines how to handle such responses. For example, the requester can decide\nto\n\n* just take the first response and ignore all others,\n* only take responses received within a given time interval,\n* only take responses until a specific condition is met,\n* handle any response over time, or\n* process responses as defined by any other application-specific logic.\n\nWe think decentralized collaborative applications often have a natural need for\nsuch powerful forms of interaction. Restricting interaction to strongly coupled\none-to-one communication (classic RPC, REST) quickly becomes complex,\ninflexible, and unmaintainable. Restricting interaction to loosely coupled\none-way communication (classic publish-subscribe) lacks the power of\ntransmitting information in direct responses. Which is why a DDA combines the\nbest of both worlds to allow developers to create more powerful collaborative\napplications with less complexity and in less time.\n\nUsing the minimal set of communication patterns, higher-level patterns of\ninteraction and collaboration between decentralized application components can\nbe composed, e.g.\n\n* coordinating dynamic workflows, workloads, and computations over a set of\n  distributed workers,\n* distributing and synchronizing shared state,\n* discovering and tracking remote application entities,\n* negotiating specialized side channels for streaming high-volume and\n  high-frequency data,\n* handling backpressure in producer-consumer scenarios by pull-based approaches,\n* all types of auctioning, such as a first-price sealed-bid auction (aka blind\n  auction) where each bidder submits a sealed bid and the best bidder wins.\n\nWhile some of these patterns will be integrated into DDA as generic peripheral\nservices in the near future, more domain-specific ones are left to dedicated\napplications.\n\n## Quick Start\n\nTo run a DDA sidecar, you may use\n\n* a prebuilt standalone platform-specific binary, or\n* a Docker image.\n\nIn both cases, the dda program is configured by a YAML configuration file\ndeployed along with it. A fully documented default configuration file `dda.yaml`\nis part of all DDA sidecar\n[deployments](https://github.com/coatyio/dda/releases). For details on how to\nprovide a configuration file run the binary or image with the `-h` command line\noption.\n\nTo use DDA as a library, refer to the [DDA Developer\nGuide](https://coatyio.github.io/dda/DEVGUIDE.html).\n\n### Prebuilt Binaries\n\nUse one of the prebuilt platform-specific binaries delivered as [GitHub\nrelease](https://github.com/coatyio/dda/releases) assets.\n\nNote that each GitHub release asset also includes a default DDA configuration\nfile and definition files of the public DDA APIs, i.e. Protobuf files and\nJavaScript stubs to consume the DDA sidecar services as a client using gRPC or\ngRPC-Web, respectively.\n\nAlternatively, in a Golang development environment you can build and run the DDA\nmodule from source using the Golang toolchain:\n\n```sh\ngo install github.com/coatyio/dda/cmd/dda@latest\n\ndda -h\n```\n\n### Docker Image\n\nUse one of the versioned Docker images deployed in the [GitHub Container\nRegistry](https://ghcr.io/coatyio/dda). The DDA configuration file `dda.yaml`\nand associated assets, such as certificates, must be bind-mounted on path `/dda`\nin the container. Configured DDA API service ports must be exposed by the\ncontainer:\n\n```sh\ndocker run --rm -p 8800:8800 -p 8900:8900 -v /path-to-config-folder:/dda ghcr.io/coatyio/dda:\u003crelease-version\u003e\n```\n\n\u003e __NOTE__: In a containerized DDA sidecar that makes use of the local\n\u003e persistent storage service use a volume or a bind mount and specify a\n\u003e destination path that corresponds with the storage location configured in the\n\u003e DDA configuration file. You may also simply reuse the DDA configuration mount\n\u003e point `/dda`, e.g. using `/dda/store` as storage location inside the\n\u003e container.\n\n## Examples\n\nThis project is accompanied by\n[dda-examples](https://github.com/coatyio/dda-examples), a repository that\nprovides a collection of ready-to-run best practice code examples demonstrating\nhow to use DDA as a sidecar or library in a decentralized desktop or web\napplication:\n\n* `compute` - a pure Go app with distributed components utilizing either the DDA\n  library directly or a co-located DDA sidecar over gRPC\n* `light-control` - a distributed web app in JavaScript/HTML connecting to its\n  co-located DDA sidecar over gRPC-Web\n\nFor detailed documentation take a look at the README of the individual example\nprojects and delve into the [DDA Developer\nGuide](https://coatyio.github.io/dda/DEVGUIDE.html).\n\n## Contributing\n\nContributions to DDA are welcome and appreciated. Please follow the recommended\npractice for idiomatic Go [programming](https://go.dev/doc/effective_go) and\n[documentation](https://tip.golang.org/doc/comment).\n\nThe DDA project is organized as a single Go module which consists of multiple\npackages that realize functionality of individual peripheral DDA services and\nits open APIs. As a contributor, you may want to add new DDA services, add\nfeatures to existing services, apply patches, and publish new releases.\n\nTo set up the project on your developer machine, install a compatible\n[Go](https://go.dev/) version as specified in the `go.mod` file.\n\nNext, install the [Task](https://taskfile.dev/) build tool, either by using one\nof the predefined binaries delivered as [GitHub\nrelease](https://github.com/go-task/task/releases) assets or by using the Golang\ntoolchain:\n\n```sh\ngo install github.com/go-task/task/v3/cmd/task@latest\n```\n\nRun `task install` to install Go tools required to build, test, and release DDA,\nand to install DDA module dependencies.\n\nAll build, test, and release related tasks can be performed using the Task tool.\nFor a list of available tasks, run `task`.\n\n\u003e __NOTE__: If you intend to make changes to Protobuf service definitions,\n\u003e install the latest release of the [Protobuf\n\u003e compiler](https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os)\n\u003e and include the path to the `protoc` program under `bin` directory in your\n\u003e environment’s PATH variable. Copy the contents of the `include` directory\n\u003e somewhere as well, for example into `/usr/local/include/`. Next, to generate\n\u003e JavaScript code for gRPC-Web download the latest releases from [this\n\u003e repo](https://github.com/protocolbuffers/protobuf-javascript/releases) and\n\u003e [this repo](https://github.com/grpc/grpc-web/releases) and add the protoc\n\u003e plugins `protoc-gen-js` under `bin` directory and the `protoc-gen-grpc-web`\n\u003e program to your environment's PATH variable. Finally, run `task protoc` to\n\u003e generate new Protobuf, gRPC, and gRPC-Web service stubs.\n\nTo release a new DDA version run `task release`. It triggers a GitHub Actions\nrelease workflow by pushing the current branch including a version tag with\nannotated release notes read from the console.\n\nFor testing purposes run `task release-dry` to create a local release for which\nrelease assets are deployed in the `dist` folder and Docker images are pushed\nlocally.\n\n## License\n\nCode and documentation copyright 2023 Siemens AG.\n\nCode is licensed under the [MIT License](https://opensource.org/licenses/MIT).\n\nDocumentation is licensed under a\n[Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoatyio%2Fdda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoatyio%2Fdda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoatyio%2Fdda/lists"}