{"id":50670863,"url":"https://github.com/agapic/twitch-streamer","last_synced_at":"2026-06-08T11:01:20.317Z","repository":{"id":57716850,"uuid":"96376262","full_name":"agapic/Twitch-Streamer","owner":"agapic","description":"Spark Streaming library for reading chat messages from Twitch.tv","archived":false,"fork":false,"pushed_at":"2017-07-17T21:05:49.000Z","size":71,"stargazers_count":9,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-01T14:52:27.323Z","etag":null,"topics":["irc-chat","java","scala","spark","stream"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agapic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-06T01:40:46.000Z","updated_at":"2020-10-21T21:32:27.000Z","dependencies_parsed_at":"2022-09-26T21:40:21.605Z","dependency_job_id":null,"html_url":"https://github.com/agapic/Twitch-Streamer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/agapic/Twitch-Streamer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agapic%2FTwitch-Streamer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agapic%2FTwitch-Streamer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agapic%2FTwitch-Streamer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agapic%2FTwitch-Streamer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agapic","download_url":"https://codeload.github.com/agapic/Twitch-Streamer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agapic%2FTwitch-Streamer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34059157,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["irc-chat","java","scala","spark","stream"],"created_at":"2026-06-08T11:00:40.377Z","updated_at":"2026-06-08T11:01:20.305Z","avatar_url":"https://github.com/agapic.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Spark-Streaming-Twitch: live stream Twitch.tv data with Spark Streaming\n\n[![][license img]][license]\n\n`Twitch-Streamer` uses Twitch's [Chat and IRC API](https://dev.twitch.tv/docs/v5/guides/irc/) to stream in messages from specified\nTwitch.tv channels. It is a light-weight wrapper over Spark Streaming and Twitch's Chat IRC data feed. The goal of this project is to fully utilize the strengths of Spark Streaming to allow others to perform analyses of Twitch's live stream chatrooms.\n\n## Full Documentation\n### Scaladocs\nClients (yourself!) will interact directly using the scala classes -- these are important!\n\n[Find them here](http://andrewgapic.com/spark-streaming-twitch/scaladocs/#com.andrewgapic.package)\n### Javadocs\nThe nitty gritty internals of the project were written in Java.\n\n[Find them here](http://andrewgapic.com/spark-streaming-twitch/javadocs/)\n\n## Communication\n- Twitter: [@AndrewGapic](http://twitter.com/andrewgapic)\n- [GitHub Issues](https://github.com/agapic/twitch-streamer/issues)\n\n## Getting Started\n### Using Maven\n```\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.andrewgapic\u003c/groupId\u003e\n    \u003cartifactId\u003espark-streaming-twitch\u003c/artifactId\u003e\n    \u003cversion\u003e1.0.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n### Using SBT \n```\nlibraryDependencies += \"com.andrewgapic\" %% \"spark-streaming-twitch\" % \"1.0.0\"\n```\n\n## Developing\n### Built With\nThis project is built with Java 1.8, Scala 2.11.8, and Spark Streaming 2.11.8.\n### Build\n```\n$ git clone https://github.com/agapic/twitch-streamer.git\n$ cd twitch-streamer/\n$ mvn clean install\n```\n**Note**: If you're interested in helping with developing the project further, there are lots of features and optimizations that could be done. For example, concurrency isn't as optimal as it could be; the entire stream is being fed through one thread. Other asynchronous\ncalls could potentially also have their own thread. \n\n## Usage\n`Twitch-Streamer` can be used with either Scala or Java, and was built with this notion in mind. Generally, Scala is a better fit since Spark was built in Scala; but, as always, use your favourite language. `Twitch-Streamer` uses the Builder pattern to construct a new Receiver object; please refer to the scaladocs for a complete listing of mutator methods.\n\n### twitch_auth.txt\n1. You can obtain the `twitch_client_id` by registering for your application here: https://www.twitch.tv/settings/connections. Alternatively, you can follow the instructions here: https://blog.twitch.tv/client-id-required-for-kraken-api-calls-afbb8e95f843.\n2. Your `twitch_username` can be any string that hasn't joined the IRC chat already. Typically, I just use my actual username.\n3. Your `twitch_password` can be retrieved here: ttp://twitchapps.com/tmi/ after your application has been registered.\n\n```\ntwitch_client_id \u003cclientid\u003e\ntwitch_username \u003cusername\u003e\ntwitch_password \u003cgo here: http://twitchapps.com/tmi/\u003e It's an oauth: password, not your twitch account password.\n```\n\n### Messages\n`Twitch-Streamer` introduces an abstraction called a `Message`. It transforms a line of text from Twitch's IRC chat into a \n`Message`, which allows clients to get the author of the message, the channel name, and the actual message content.\n\n### Scala API\n```\nimport com.andrewgapic.spark.streaming.TwitchStreamBuilder\nimport com.andrewgapic.stream.Message\nval gamesSet: Set[String] = Set(\"League+of+Legends\")\nval stream: ReceiverInputDStream[Message] = new TwitchStreamBuilder().setGames(gamesSet).build(ssc)\n```\n\n### Java API\n```\nimport com.andrewgapic.spark.streaming.TwitchStreamBuilder;\nimport com.andrewgapic.stream.Message;\nSet\u003cString\u003e gamesSet = new HashSet\u003c\u003e();\ngamesSet.add(\"League+of+Legends\");\nJavaReceiverInputDStream\u003cMessage\u003e stream = new TwitchStreamBuilder().setGames(gamesSet).build(jssc);\n```\n\n### More advanced usage (Scala)\n**Note**: spaces in game names must be replaced by a `+` character. This will be done automatically in future versions.\n```\nimport com.andrewgapic.spark.streaming.TwitchStreamBuilder\nimport com.andrewgapic.stream.Message\nval sparkConf = new SparkConf().setAppName(\"TwitchTest\")\nval ssc = new StreamingContext(sparkConf, Seconds(2))\nval gamesSet: Set[String] = Set(\"League+of+Legends\")\nval channelsSet: Set[String] = Set(\"TSM_Dyrus\")\nval language: String = \"en\" //english\nval storageLevel: StorageLevel = StorageLevel.MEMORY_AND_DISK_SER_2\nval schedulingInterval: FiniteDuration = 600 seconds // refresh channels every 10 minutes\nval stream: ReceiverInputDStream[Message] = new TwitchStreamBuilder()\n                                            .setGames(gamesSet)\n                                            .setChannels(channelsSet)\n                                            .setLanguage(language)\n                                            .setStorageLevel(storageLevel)\n                                            .setSchedulingInterval(schedulingInterval)\n                                            .build(ssc)\n```\n\n## Examples\nThere are two examples in the `examples` folder; one in Scala (`ChannelAndWordsCount`), and one in Java (`JavaWordsCount`).\nThe Scala example does two things: displays the top 15 words by word frequency (ignores stopwords), and displays the top channels by message frequency. The Java example only displays word frequency.\n\n\n## Bugs and Feedback\n\nFor bugs, questions and discussions please use the [GitHub Issues](https://github.com/agapic/twitch-streamer/issues).\n\n## LICENSE\n\nCopyright 2017, Andrew Gapic.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n[license]:LICENSE\n[license img]:https://img.shields.io/badge/License-Apache%202-blue.svg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagapic%2Ftwitch-streamer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagapic%2Ftwitch-streamer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagapic%2Ftwitch-streamer/lists"}