https://github.com/cbuntain/replaytweetstream

Given a gzipped file of tweets, replay them as though you were streaming them from Twitter's data stream
https://github.com/cbuntain/replaytweetstream

Last synced: 27 days ago
JSON representation

Given a gzipped file of tweets, replay them as though you were streaming them from Twitter's data stream

• Host: GitHub
• URL: https://github.com/cbuntain/replaytweetstream
• Owner: cbuntain
• License: apache-2.0
• Created: 2015-01-12T20:19:35.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2015-06-09T20:49:14.000Z (over 9 years ago)
• Last Synced: 2023-05-03T00:43:14.596Z (over 1 year ago)
• Language: Java
• Size: 152 KB
• Stars: 3
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# replayTweetStream

Given a gzipped file of tweets, replay them as though you were streaming them from Twitter's data stream.

Currently, only supports a single GZipped file and replays at the per-second level with gaps between messages as determined by their created_at timestamps. 

THe code assumes tweet files are temporally ordered.

The code was recently updated to support configurable senders. Simply implement the Sender interface, and you should be good. I've included an implementation to send to the console and to write to a server socket, and both seem to work.

I've also tested this streaming capability with Spark Streaming over a socket and achieved good results.

## Todo Items

- Add support for multiple files

- Add support for streaming files from HDFS