https://github.com/jaceklaskowski/spark-activator

Spark Streaming with Scala and Akka Activator template
https://github.com/jaceklaskowski/spark-activator

Last synced: about 2 months ago
JSON representation

Spark Streaming with Scala and Akka Activator template

• Host: GitHub
• URL: https://github.com/jaceklaskowski/spark-activator
• Owner: jaceklaskowski
• License: apache-2.0
• Created: 2014-05-07T21:04:21.000Z (about 11 years ago)
• Default Branch: spark-2.0.0
• Last Pushed: 2016-01-31T11:55:11.000Z (over 9 years ago)
• Last Synced: 2025-03-29T05:34:46.256Z (2 months ago)
• Language: Scala
• Homepage: https://typesafe.com/activator/template/spark-streaming-scala-akka
• Size: 35.2 KB
• Stars: 44
• Watchers: 9
• Forks: 26
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
Spark Streaming with Scala and Akka activator template

======================================================

[![Build Status](https://travis-ci.org/jaceklaskowski/spark-activator.svg?branch=master)](https://travis-ci.org/jaceklaskowski/spark-activator) [![Stories in Ready](https://badge.waffle.io/jaceklaskowski/spark-activator.png?label=ready&title=Ready)](https://waffle.io/jaceklaskowski/spark-activator)

This is an [Typesafe Activator](http://typesafe.com/platform/getstarted) template to demonstrate [Apache Spark](http://spark.apache.org) for near-real-time data stream processing using [Scala](http://www.scala-lang.org/) and [Akka](http://akka.io/) using the [Spark Streaming](http://spark.apache.org/docs/latest/streaming-programming-guide.html) extension.

The motivation was [Go Reactive Activator Contest: Scala Days Edition](http://typesafe.com/blog/go-reactive-activator-contest-scala-days-edition)

# Run the activator project

Once you download the activator template, execute `./activator clean run`.

When the words `Hello from Spark Streaming with Scala and Akka` arrive to the actor and Spark Streaming gets notified about them (by the `store` method), the flow finishes and so does the demo.

Watch out for the following message in the logs (the `Time` line may show a different value):

    -------------------------------------------

    Time: 1422714451000 ms

    -------------------------------------------

    Hello from Spark Streaming with Scala and Akka

Please note that the `clean` task is important after the first run as some files get serialized and then deserialized that have often caused deserialization problems. As a workaround, running `clean` gets rid of the potential issue quickly and easily.