{"id":29513004,"url":"https://github.com/snipsco/sda","last_synced_at":"2026-03-17T22:32:24.047Z","repository":{"id":66892011,"uuid":"81083419","full_name":"snipsco/sda","owner":"snipsco","description":"Secure distributed aggregation of high-dimensional vectors","archived":false,"fork":false,"pushed_at":"2017-03-31T13:51:43.000Z","size":1044,"stargazers_count":58,"open_issues_count":0,"forks_count":20,"subscribers_count":27,"default_branch":"dev","last_synced_at":"2025-07-17T13:45:05.458Z","etag":null,"topics":["cryptography","homomorphic-encryption","machine-learning","privacy","secret-sharing","secure-computation","statistics"],"latest_commit_sha":null,"homepage":"https://snipsco.github.io/sda/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snipsco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-02-06T12:08:15.000Z","updated_at":"2025-07-12T21:15:40.000Z","dependencies_parsed_at":"2023-05-19T05:45:40.143Z","dependency_job_id":null,"html_url":"https://github.com/snipsco/sda","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/snipsco/sda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snipsco%2Fsda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snipsco%2Fsda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snipsco%2Fsda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snipsco%2Fsda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snipsco","download_url":"https://codeload.github.com/snipsco/sda/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snipsco%2Fsda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30633335,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T17:32:55.572Z","status":"ssl_error","status_checked_at":"2026-03-17T17:32:38.732Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cryptography","homomorphic-encryption","machine-learning","privacy","secret-sharing","secure-computation","statistics"],"created_at":"2025-07-16T12:44:20.419Z","updated_at":"2026-03-17T22:32:24.027Z","avatar_url":"https://github.com/snipsco.png","language":"Rust","funding_links":[],"categories":["Software"],"sub_categories":["Retired software"],"readme":"# Overview\n\n## Purpose\n\nSDA is a framework used at Snips for Secure Distributed Aggregation. It\nimplements a simple and efficient [multi-party computation protocol](https://en.wikipedia.org/wiki/Secure_multi-party_computation) for\ncomputing aggregations (sums for now) of data from several participants while\nkeeping all inputs private.\n\nIn exchange for computing only a certain class of functions, the system has been\noptimised to run on relatively weak and sporadic devices such as mobile phones.\nIn particular, one aim of SDA is to combine locally trained machine learning\nmodels from mobile phones into a global model, and doing so privately.\n\n## Walkthrough\n\nA runnable version of this scenario [docs/simple-cli-example.sh](can be found\nhere).\n\nSDA relies on interaction between several _agents_ helped by a single _server_.\n\n### Server\n\nFirst we can build and start the server:\n\n```\ncd server-cli\ncargo build\ncargo run -- --jfs tmp/simple-data/server httpd\n```\n\nThis starts a SDA server that will listen for localhost on port 8888, and use\njson files in the directory to store its state. `--help` will show options\nto move data around or change the listening socket. For production setup\na MongoDB alternative storage is offered.\n\n### Agents\n\nNext we need a _recipient_. This is the person or organisation that is setting\nup the aggregation specification and who will receive the final aggregated\nresult.\n\nAs well as the recipient, we will create three possible _clerks_: these three\nagents, by providing the server with a public encryption key, become candidates\nto take part in distributed computations. Private data is never at risk of being\nexposed to any clerk, and the protocol has furthermore been designed to minimise\ntheir work load. \n\nFor this walkthrough we will use the [command line client](/cli) to show the\ninteractions of agents, but there is also a [client library](/client) for\nincorporating SDA in other applications. In the following, the -i allows us to\nspecify different identity storages to simulate several agents on the same\ncomputer.\n\n```\n(cd cli \u0026\u0026 cargo build)\nalias sda=./cli/target/debug/sda\n\nfor i in recipient clerk-1 clerk-2 clerk-3\ndo\n    sda -i tmp/simple-data/agent/$i agent create\n    sda -i tmp/simple-data/agent/$i agent keys create\ndone\n```\n\nWe will also create three _participants_ in the process: they will offer the\ndata to be aggregated, without making it public to the server or any other\nagent in the operation.\n\n```\nfor i in part-1 part-2 part-3\ndo\n    sda -i tmp/simple-data/agent/$i agent create\ndone\n```\n\n### Aggregation\n\nNow that we have enough clerks to share the secrets and spread the trust, the\nrecipient can create the aggregation (having created the participants do not\nmatter).\n\n```\nAGGID=ad3142d8-9a83-4f40-a64a-a8c90b701bde\nRECIPIENT_KEY_ID=$(grep -l '\"ek\"' tmp/simple-data/agent/recipient/keys/* | sed 's/.*\\///;s/\\.json//')\nsda -i tmp/simple-data/agent/recipient aggregations create --id $AGGID \"aggro\" 10 433 $RECIPIENT_KEY_ID 3\nsda -i tmp/simple-data/agent/recipient aggregations begin $AGGID\n```\n\nWe need a bit of shell plumbing to grab the recipient key for now, but the\ngist of these command is to create an aggregation (with a provided id), and a\nname. It will aggregate participations of 10 numbers, taken from a \n(semi-exclusive) 0..433 interval. We also specify the key we want the final \nresult to be encrypted under and the number of ways (3) we want to split the\nparticipants secrets.\n\nThe \"begin\" command will actually pick a committee of 3 clerks (among the\nclerks and recipient) and thus \"open\" the aggregation for participation.\n\n### Participation\n\nAt this point each participant can send its contribution to be aggregated.\n\n```\nsda -i tmp/simple-data/agent/part-1 participate $AGGID 0 1 2 3 4 5 6 7 8 9\nsda -i tmp/simple-data/agent/part-2 participate $AGGID 0 0 0 0 0 0 0 0 0 0\nsda -i tmp/simple-data/agent/part-3 participate $AGGID 0 1 0 1 0 1 0 1 0 1\n```\n\nThese secret inputs are split between the elected clerks, each part encrypted\nwith the corresponding key, and sent to the server. Splitting the participations\nare done using [secret sharing](https://en.wikipedia.org/wiki/Secret_sharing) so\nthat no group of agents below a specified _privacy threshold_ can recover the\ninputs from the shares. Likewise, any outsider such as the server cannot recover\nthe inputs since to do so one would need to decrypt the shares using secret keys\nknown only by the clerks.\n\n### Clerking\n\nWhen the recipient determines that enough participations have been made, the\naggregation can be closed to move it to the next stage:\n\n```\nsda -i tmp/simple-data/agent/recipient aggregations end $AGGID\n```\n\nThe server will organize the data in \"jobs\" that the three clerk from the\ncommittee will have to eventually work upon.\n\n```\nfor i in recipient clerk-1 clerk-2 clerk-3\ndo\n    sda -i tmp/simple-data/agent/$i clerk --once\ndone\n```\n\nWithout the `--once` parameter, the command would behave as a long running\nprocess that checks the server queue periodically and perform whatever task the\nserver has in store.\n\nHere it will just check once. Three out of the four potential clerks have actual\nclerking jobs, the remaining one none.\n\nEach clerk actually aggregates its part of the multi-party computation protocol,\nand then sends back its share of the result to the server, encrypted under the\nrecipient's key.\n\n### Final reveal\n\nThe recipient can reconstruct the final aggregated output from the results of\nthe clerks:\n\n```\nsda -i tmp/simple-data/agent/recipient aggregations reveal $AGGID\n```\n\nIn our case, it should read: `0 2 2 4 4 6 6 8 8 10`.\n\n## Doing more, with APIs\n\nThe command line client only expose a subset of the API, at least for now. It is\nnot meant to be the primary mode of interacting with an SDA service. The Rust\nAPI to the client, or the REST API to the server allow more flexibility:\n\n* tweaking the sharing scheme: the Packed Shamir Scheme provides resilience\n    over clerks failure, as well as reducing message size.\n* using a masking scheme to protect participants privacy against a collusion\n    between clerks\n* using the [Paillier cryptosystem](https://en.wikipedia.org/wiki/Paillier_cryptosystem) \n  to scale up the system to any number of participants\n* allowing candidate clerks to link their profile to some external\n    authenticating system to improve participants trust in the system\n* allow recipient to actually chose the clerks that should get in the committee\n    for its aggregation\n\n## Structure\n\n### Core\n- [protocol](/protocol) documents the SDA service interface, as implemented by the server and consumed by the client\n- [client](/client) contains the core client code, published as `sda-client`, with a minimum of options and dependencies\n- [server](/server) is the minimum server\n\n### Server and network\n\n- [server-http](/server-http) is the REST interface for the server\n- [client-http](/client-http) is the matching REST proxy\n- [server-store-mongodb](/server-store-server) is the MongoDB production storage for the server\n\nVarious combination for these crates can be tested with [integration-tests](/integration-tests).\n\n# Command line interface\n\n- [cli](/cli) is the agent command line interface. The executable name is `sda`.\n- [server-cli](/server-cli) binds all of the server pieces together in a command line interface called `sdad`.\n\n### Wrappers\n\nThese wrappers are meant to be used in application (typically mobile or\nembedded apps). They have not been released yet, they need a bit of cleanup\nbut will come soon.\n\n- /embeddable-client wraps [client](/client) and [client-http](/client-http) to exposes the client functionality in a C-friendly\n- /javaclient builds on top of it a semantic interface for Java application (including Android)\n- /swiftclient does the same for Swift, targetting macOS and iOS application integration.\n\n# License\n\nLicensed under either of\n\n * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall\nbe dual licensed as above, without any additional terms or conditions.\n \n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnipsco%2Fsda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnipsco%2Fsda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnipsco%2Fsda/lists"}