https://github.com/twitter/tormenta
Scala extensions for Storm
https://github.com/twitter/tormenta
Last synced: 6 months ago
JSON representation
Scala extensions for Storm
- Host: GitHub
- URL: https://github.com/twitter/tormenta
- Owner: twitter
- License: apache-2.0
- Created: 2012-09-14T18:43:20.000Z (over 13 years ago)
- Default Branch: develop
- Last Pushed: 2019-06-07T01:01:52.000Z (over 6 years ago)
- Last Synced: 2025-06-07T23:53:07.780Z (7 months ago)
- Language: Scala
- Homepage:
- Size: 459 KB
- Stars: 134
- Watchers: 133
- Forks: 37
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGES.md
- License: LICENSE
Awesome Lists containing this project
README
## Tormenta
[](http://travis-ci.org/twitter/tormenta)
[](https://codecov.io/github/twitter/tormenta)
[](https://index.scala-lang.org/twitter/tormenta/tormenta-core)
[](https://gitter.im/twitter/tormenta?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
Scala extensions for the [Storm](https://github.com/nathanmarz/storm) distributed computation system. Tormenta adds a type-safe wrapper over Storm's Kafka spout. This type safety allows the user to push mapping and filtering transformations down to the level of the spout itself:
```scala
import com.twitter.tormenta.scheme._
import com.twitter.tormenta.spout._
// produces strings:
val scheme: Scheme[String] = Scheme { bytes => Some(new String(bytes)) }
// produces integers w/ string length:
val mappedScheme: Scheme[Int] = scheme.map(_.length)
// filters out all tuples less than 5:
val filteredScheme: Scheme[Int] = mappedScheme.filter(_ > 5)
// produces lengths for input strings > length of 5
val spout: KafkaSpout[Int] = new KafkaSpout(filteredScheme, zkHost, brokerZkPath, topic, appID, zkRoot)
```
To use a `Spout[T]` in a Storm topology, call the `getSpout` method:
```scala
topologyBuilder.setSpout("spoutName", spout.getSpout, 10)
```
Now you're cooking with gas.
## Documentation
To learn more and find links to tutorials and information around the web, check out the [Tormenta Wiki](https://github.com/twitter/tormenta/wiki).
The latest ScalaDocs are hosted on Tormenta's [Github Project Page](http://twitter.github.io/tormenta).
## Contact
Discussion occurs primarily on the [Tormenta mailing list](https://groups.google.com/forum/#!forum/tormenta-user). Issues should be reported on the [GitHub issue tracker](https://github.com/twitter/tormenta/issues).
## Get Involved + Code of Conduct
Pull requests and bug reports are always welcome!
We use a lightweight form of project governence inspired by the one used by Apache projects.
Please see [Contributing and Committership](https://github.com/twitter/analytics-infra-governance#contributing-and-committership) for our code of conduct and our pull request review process.
The TL;DR is send us a pull request, iterate on the feedback + discussion, and get a +1 from a [Committer](COMMITTERS.md) in order to get your PR accepted.
The current list of active committers (who can +1 a pull request) can be found here: [Committers](COMMITTERS.md)
A list of contributors to the project can be found here: [Contributors](https://github.com/twitter/tormenta/graphs/contributors)
## Maven
Tormenta modules are available on Maven Central. The current groupid and version for all modules is, respectively, `"com.twitter"` and `0.12.0`.
Current published artifacts are
* `tormenta-core_2.12`
* `tormenta-core_2.11`
* `tormenta-core_2.10`
* `tormenta-kafka_2.12`
* `tormenta-kafka_2.11`
* `tormenta-kafka_2.10`
* `tormenta-twitter_2.12`
* `tormenta-twitter_2.11`
* `tormenta-twitter_2.10`
The suffix denotes the scala version.
## Authors
* Oscar Boykin
* Sam Ritchie
## License
Copyright 2017 Twitter, Inc.
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).