{"id":21399314,"url":"https://github.com/streamr-dev/streamr-spark-integrations","last_synced_at":"2025-07-07T18:33:22.408Z","repository":{"id":44841522,"uuid":"189041727","full_name":"streamr-dev/streamr-spark-integrations","owner":"streamr-dev","description":"Example integrations from Streamr to Apache Spark","archived":false,"fork":false,"pushed_at":"2022-12-06T18:32:14.000Z","size":209,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-06-30T06:08:46.965Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/streamr-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-28T14:13:08.000Z","updated_at":"2019-09-13T10:38:48.000Z","dependencies_parsed_at":"2023-01-23T12:00:23.978Z","dependency_job_id":null,"html_url":"https://github.com/streamr-dev/streamr-spark-integrations","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/streamr-dev/streamr-spark-integrations","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamr-dev%2Fstreamr-spark-integrations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamr-dev%2Fstreamr-spark-integrations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamr-dev%2Fstreamr-spark-integrations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamr-dev%2Fstreamr-spark-integrations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/streamr-dev","download_url":"https://codeload.github.com/streamr-dev/streamr-spark-integrations/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamr-dev%2Fstreamr-spark-integrations/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264130517,"owners_count":23562043,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T15:14:04.904Z","updated_at":"2025-07-07T18:33:22.363Z","avatar_url":"https://github.com/streamr-dev.png","language":"Scala","readme":"# Apache spark integrations from Streamr\n\nYou can find the maven library for the integration [here](https://github.com/streamr-dev/streamr-spark)\nYou can import it to your maven project easily with:\n\n```\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.streamr.labs\u003c/groupId\u003e\n  \u003cartifactId\u003estreamr_spark\u003c/artifactId\u003e\n  \u003cversion\u003e0.2\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nThis repository contains different ways to integrate Streamr to Apache Spark.\n\nDirect integrations from Streamr to Spark are done with Streamr's Java client for Java and Scala versions of Spark. Non-direct example integrations for Streamr-NiFi-Kafka-Spark-Kafka-NiFi-Streamr and a Streamr integration to Spark file streaming are also provided. The NiFi + Kafka integration might be useful for people who wish to either subscribe or publish data via NiFi and/or Kafka. The file streaming JS integration tool is useful for people who wish to do operations on historical data from Streamr and possibly store it for later use.\n\nThe direct integrations use the [Helsinki Tram Stream](https://www.streamr.com/marketplace/products/31e8df5243ce49cfa250002f60b73e475f39b44723ca4fbcbf695198d19c6b08) as their data input.\n\n[Java Direct Integration](./java-spark)\n\n[Scala Integrations](./scala-spark)\n\n[JS file integration tool](./streamr-file-spark-streaming)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamr-dev%2Fstreamr-spark-integrations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamr-dev%2Fstreamr-spark-integrations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamr-dev%2Fstreamr-spark-integrations/lists"}