{"id":13395125,"url":"https://github.com/tweetstream/tweetstream","last_synced_at":"2025-05-14T20:10:38.579Z","repository":{"id":671335,"uuid":"314686","full_name":"tweetstream/tweetstream","owner":"tweetstream","description":"A simple EventMachine-based library for consuming Twitter's Streaming API.","archived":false,"fork":false,"pushed_at":"2021-08-24T17:51:40.000Z","size":413,"stargazers_count":1107,"open_issues_count":28,"forks_count":154,"subscribers_count":32,"default_branch":"master","last_synced_at":"2025-04-28T11:36:16.974Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://rdoc.info/github/tweetstream/tweetstream","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tweetstream.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2009-09-22T20:20:04.000Z","updated_at":"2025-04-09T05:26:29.000Z","dependencies_parsed_at":"2022-08-16T10:40:08.781Z","dependency_job_id":null,"html_url":"https://github.com/tweetstream/tweetstream","commit_stats":null,"previous_names":["intridea/tweetstream"],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tweetstream%2Ftweetstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tweetstream%2Ftweetstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tweetstream%2Ftweetstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tweetstream%2Ftweetstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tweetstream","download_url":"https://codeload.github.com/tweetstream/tweetstream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252084812,"owners_count":21692163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T17:01:42.878Z","updated_at":"2025-05-14T20:10:38.555Z","avatar_url":"https://github.com/tweetstream.png","language":"Ruby","funding_links":[],"categories":["Third-party APIs","Ruby"],"sub_categories":[],"readme":"# TweetStream\n\n[![Gem Version](http://img.shields.io/gem/v/tweetstream.svg)][gem]\n[![Build Status](http://img.shields.io/travis/tweetstream/tweetstream.svg)][travis]\n[![Dependency Status](http://img.shields.io/gemnasium/tweetstream/tweetstream.svg)][gemnasium]\n[![Code Climate](http://img.shields.io/codeclimate/github/tweetstream/tweetstream.svg)][codeclimate]\n[![Coverage Status](http://img.shields.io/coveralls/tweetstream/tweetstream.svg)][coveralls]\n\n[gem]: https://rubygems.org/gems/tweetstream\n[travis]: http://travis-ci.org/tweetstream/tweetstream\n[gemnasium]: https://gemnasium.com/tweetstream/tweetstream\n[codeclimate]: https://codeclimate.com/github/tweetstream/tweetstream\n[coveralls]: https://coveralls.io/r/tweetstream/tweetstream\n\nTweetStream provides simple Ruby access to [Twitter's Streaming API](https://dev.twitter.com/docs/streaming-apis).\n\n## Installation\n\n    gem install tweetstream\n\n## Usage\n\nUsing TweetStream is quite simple:\n\n```ruby\nrequire 'tweetstream'\n\nTweetStream.configure do |config|\n  config.consumer_key       = 'abcdefghijklmnopqrstuvwxyz'\n  config.consumer_secret    = '0123456789'\n  config.oauth_token        = 'abcdefghijklmnopqrstuvwxyz'\n  config.oauth_token_secret = '0123456789'\n  config.auth_method        = :oauth\nend\n\n# This will pull a sample of all tweets based on\n# your Twitter account's Streaming API role.\nTweetStream::Client.new.sample do |status|\n  # The status object is a special Hash with\n  # method access to its keys.\n  puts \"#{status.text}\"\nend\n```\n\nYou can also use it to track keywords or follow a given set of\nuser ids:\n\n```ruby\n# Use 'track' to track a list of single-word keywords\nTweetStream::Client.new.track('term1', 'term2') do |status|\n  puts \"#{status.text}\"\nend\n\n# Use 'follow' to follow a group of user ids (integers, not screen names)\nTweetStream::Client.new.follow([14252, 53235]) do |status|\n  puts \"#{status.text}\"\nend\n```\n\nThe methods available to TweetStream::Client are kept in parity\nwith the methods available on the Streaming API wiki page.\n\n## Changes in 2.0\n\nTweetStream 2.0 introduces a number of requested features and bug fixes. For\nthe complete list refer to the [changelog](CHANGELOG.md#version-200). Notable\nadditions in 2.0 include:\n\n### OAuth\n\nOAuth is now the default authentication method.  Both userstreams and Site\nStreams exclusively work with OAuth.  TweetStream still supports Basic Auth,\nhowever it is no longer the default.  If you are still using Basic Auth, you\nshould plan to move to OAuth as soon as possible.\n\n### Site Stream Support\n\nSite Streams are now fully supported, including the connection management functionality.\n\n### Compatibility with the Twitter gem\n\nTweetStream now emits objects from the [Twitter gem](https://github.com/sferik/twitter) instead of custom hashes. These objects are already defined in the `twitter` gem and are superior to the custom objects in the following ways:\n\n1. Object equivalence (`#==` returns true if `#id`s are the same).\n2. The `#created_at` method returns a `Date` instead of a `String`.\n3. Allows boolean methods to be called with a question mark (e.g.\n   `User#protected?`)\n\nAdditionally, any new features that are added to objects in the\n`twitter` gem (e.g. identity map) will be automatically inherited by TweetStream.\n\n### em-twitter\n\nWe've replaced the underlying gem that connects to the streaming API. [twitter-stream](https://github.com/voloko/twitter-stream) has been replaced with [em-twitter](https://github.com/spagalloco/em-twitter).\nIt offers functionality parity with twitter-stream while also supporting several new features.\n\n### Removal of on_interval callback\n\nWe have removed the `on_interval` callback.  If you require interval-based timers, it is possible to run\nTweetStream inside an already running EventMachine reactor in which you can define `EM::Timer` or `EM::PeriodicTimer`\nfor time-based operations:\n\n```ruby\nEM.run do\n  client = TweetStream::Client.new\n\n  EM::PeriodicTimer.new(10) do\n    # do something on an interval\n  end\nend\n```\n\n### Additional Notes\n\nThe parser configuration method has been removed as MultiJson automatically detects existing parsers.\n\n## Using the Twitter Userstream\n\nUsing the Twitter userstream works similarly to regular streaming, except you use the `userstream` method.\n\n```ruby\n# Use 'userstream' to get message from your stream\nclient = TweetStream::Client.new\n\nclient.userstream do |status|\n  puts status.text\nend\n```\n\n## Using Twitter Site Streams\n\n```ruby\nclient = TweetStream::Client.new\n\nclient.sitestream(['115192457'], :followings =\u003e true) do |status|\n  puts status.inspect\nend\n```\n\nOnce connected, you can [control the Site Stream connection](https://dev.twitter.com/docs/streaming-apis/streams/site/control):\n\n```ruby\n# add users to the stream\nclient.control.add_user('2039761')\n\n# remove users from the stream\nclient.control.remove_user('115192457')\n\n# obtain a list of followings of users in the stream\nclient.control.friends_ids('115192457') do |friends|\n  # do something\nend\n\n# obtain the current state of the stream\nclient.control.info do |info|\n  # do something\nend\n```\n\nNote that per Twitter's documentation, connection management features are not\nimmediately available when connected\n\nYou also can use method hooks for both regular timeline statuses and direct messages.\n\n```ruby\nclient = TweetStream::Client.new\n\nclient.on_direct_message do |direct_message|\n  puts \"direct message\"\n  puts direct_message.text\nend\n\nclient.on_timeline_status do |status|\n  puts \"timeline status\"\n  puts status.text\nend\n\nclient.userstream\n```\n\n## Authentication\n\nTweetStream supports OAuth and Basic Auth.  `TweetStream::Client` now accepts\na hash:\n\n```ruby\nTweetStream::Client.new(:username =\u003e 'you', :password =\u003e 'pass')\n```\n\nAlternatively, you can configure TweetStream via the configure method:\n\n```ruby\nTweetStream.configure do |config|\n  config.consumer_key       = 'cVcIw5zoLFE2a4BdDsmmA'\n  config.consumer_secret    = 'yYgVgvTT9uCFAi2IuscbYTCqwJZ1sdQxzISvLhNWUA'\n  config.oauth_token        = '4618-H3gU7mjDQ7MtFkAwHhCqD91Cp4RqDTp1AKwGzpHGL3I'\n  config.oauth_token_secret = 'xmc9kFgOXpMdQ590Tho2gV7fE71v5OmBrX8qPGh7Y'\n  config.auth_method        = :oauth\nend\n```\n\nIf you are using Basic Auth:\n\n```ruby\nTweetStream.configure do |config|\n  config.username     = 'username'\n  config.password     = 'password'\n  config.auth_method  = :basic\nend\n```\n\nTweetStream assumes OAuth by default.  If you are using Basic Auth, it is recommended\nthat you update your code to use OAuth as Twitter is likely to phase out Basic Auth\nsupport.  Basic Auth is only available for public streams as User Stream and Site Stream\nfunctionality [only support OAuth](https://dev.twitter.com/docs/streaming-apis/connecting#Authentication).\n\n## Parsing JSON\n\nTweetStream supports swappable JSON backends via MultiJson. Simply require your preferred\nJSON parser and it will be used to parse responses.\n\n## Handling Deletes and Rate Limitations\n\nSometimes the Streaming API will send messages other than statuses.\nSpecifically, it does so when a status is deleted or rate limitations\nhave caused some tweets not to appear in the stream. To handle these,\nyou can use the on_delete, on_limit and on_enhance_your_calm methods. Example:\n\n```ruby\n@client = TweetStream::Client.new\n\n@client.on_delete do |status_id, user_id|\n  Tweet.delete(status_id)\nend\n\n@client.on_limit do |skip_count|\n  # do something\nend\n\n@client.on_enhance_your_calm do\n  # do something\nend\n\n@client.track('intridea')\n```\n\nThe on_delete and on_limit methods can also be chained:\n\n```ruby\nTweetStream::Client.new.on_delete{ |status_id, user_id|\n  Tweet.delete(status_id)\n}.on_limit { |skip_count|\n  # do something\n}.track('intridea') do |status|\n  # do something with the status like normal\nend\n```\n\nYou can also provide `:delete` and/or `:limit`\noptions when you make your method call:\n\n```ruby\nTweetStream::Client.new.track('intridea',\n  :delete =\u003e proc{ |status_id, user_id| # do something },\n  :limit  =\u003e proc{ |skip_count| # do something }\n) do |status|\n  # do something with the status like normal\nend\n```\n\nTwitter recommends honoring deletions as quickly as possible, and\nyou would likely be wise to integrate this functionality into your\napplication.\n\n## Errors and Reconnecting\n\nTweetStream uses EventMachine to connect to the Twitter Streaming\nAPI, and attempts to honor Twitter's guidelines in terms of automatic\nreconnection. When Twitter becomes unavailable, the block specified\nby you in `on_error` will be called. Note that this does not\nindicate something is actually wrong, just that Twitter is momentarily\ndown. It could be for routine maintenance, etc.\n\n```ruby\nTweetStream::Client.new.on_error do |message|\n  # Log your error message somewhere\nend.track('term') do |status|\n  # Do things when nothing's wrong\nend\n```\n\nHowever, if the maximum number of reconnect attempts has been reached,\nTweetStream will raise a `TweetStream::ReconnectError` with\ninformation about the timeout and number of retries attempted.\n\nOn reconnect, the block specified by you in `on_reconnect` will be called:\n\n```ruby\nTweetStream::Client.new.on_reconnect do |timeout, retries|\n  # Do something with the reconnect\nend.track('term') do |status|\n  # Do things when nothing's wrong\nend\n```\n\nYou can also know when there is an error with the authentication with\n`on_unauthorized`:\n\n```ruby\nTweetStream::Client.new.on_unauthorized do\n  # Alert about this\nend.track('term') do |status|\n  # Do things when nothing's wrong\nend\n```\n\n## Terminating a TweetStream\n\nIt is often the case that you will need to change the parameters of your\ntrack or follow tweet streams. In the case that you need to terminate\na stream, you may add a second argument to your block that will yield\nthe client itself:\n\n```ruby\n# Stop after collecting 10 statuses\n@statuses = []\nTweetStream::Client.new.sample do |status, client|\n  @statuses \u003c\u003c status\n  client.stop if @statuses.size \u003e= 10\nend\n```\n\nWhen `stop` is called, TweetStream will return from the block\nthe last successfully yielded status, allowing you to make note of\nit in your application as necessary.\n\n## Daemonizing\n\nIt is also possible to create a daemonized script quite easily\nusing the TweetStream library:\n\n```ruby\n# The third argument is an optional process name\nTweetStream::Daemon.new('tracker').track('term1', 'term2') do |status|\n  # do something in the background\nend\n```\n\nIf you put the above into a script and run the script with `ruby scriptname.rb`,\nyou will see a list of daemonization commands such as start, stop, and run.\n\nA frequent use case is to use TweetStream along with ActiveRecord to insert new\nstatuses to a database. The library TweetStream uses the `daemons` gem for\ndaemonization which forks a new process when the daemon is created. After forking,\nyou'll need to reconnect to the database:\n\n```ruby\nENV[\"RAILS_ENV\"] ||= \"production\"\n\nroot = File.expand_path(File.join(File.dirname(__FILE__), '..'))\nrequire File.join(root, \"config\", \"environment\")\n\ndaemon = TweetStream::Daemon.new('tracker', :log_output =\u003e true)\ndaemon.on_inited do\n  ActiveRecord::Base.connection.reconnect!\n  ActiveRecord::Base.logger = Logger.new(File.open('log/stream.log', 'w+'))\nend\ndaemon.track('term1') do |tweet|\n  Status.create_from_tweet(tweet)\nend\n```\n\n## Proxy Support\n\nTweetStream supports a configurable proxy:\n\n```ruby\nTweetStream.configure do |config|\n  config.proxy = { :uri =\u003e 'http://myproxy:8081' }\nend\n```\n\nYour proxy will now be used for all connections.\n\n## REST\n\nTo access the Twitter REST API, we recommend the [Twitter][] gem.\n\n[twitter]: https://github.com/sferik/twitter\n\n## Contributors\n\n* [Michael Bleigh](https://github.com/mbleigh) (initial gem)\n* [Steve Agalloco](https://github.com/spagalloco) (current maintainer)\n* [Erik Michaels-Ober](https://github.com/sferik) (current maintainer)\n* [Countless others](https://github.com/intridea/tweetstream/graphs/contributors)\n\n## Copyright\n\nCopyright (c) 2012-2013 Intridea, Inc. (http://www.intridea.com/). See\n[LICENSE](LICENSE.md) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftweetstream%2Ftweetstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftweetstream%2Ftweetstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftweetstream%2Ftweetstream/lists"}