{"id":23670802,"url":"https://github.com/jeffreymorganio/twitter-collection-js","last_synced_at":"2026-05-06T03:31:32.738Z","repository":{"id":145373011,"uuid":"50602636","full_name":"jeffreymorganio/twitter-collection-js","owner":"jeffreymorganio","description":"Collect tweets filtered from the Twitter streaming API with Node.js.","archived":false,"fork":false,"pushed_at":"2020-08-15T09:24:23.000Z","size":5,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-21T04:39:05.168Z","etag":null,"topics":["collection","nodejs","twitter-streaming-api"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jeffreymorganio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-01-28T18:07:57.000Z","updated_at":"2020-08-15T09:24:25.000Z","dependencies_parsed_at":"2023-06-03T16:30:30.547Z","dependency_job_id":null,"html_url":"https://github.com/jeffreymorganio/twitter-collection-js","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/jeffreymorganio/twitter-collection-js","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffreymorganio%2Ftwitter-collection-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffreymorganio%2Ftwitter-collection-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffreymorganio%2Ftwitter-collection-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffreymorganio%2Ftwitter-collection-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jeffreymorganio","download_url":"https://codeload.github.com/jeffreymorganio/twitter-collection-js/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffreymorganio%2Ftwitter-collection-js/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32677855,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-06T02:33:58.958Z","status":"ssl_error","status_checked_at":"2026-05-06T02:33:39.611Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collection","nodejs","twitter-streaming-api"],"created_at":"2024-12-29T09:39:54.105Z","updated_at":"2026-05-06T03:31:32.726Z","avatar_url":"https://github.com/jeffreymorganio.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# twitter-collection-js\n\nThe Twitter collection tool is a Node.js utility for collecting tweets filtered from the [Twitter streaming API](https://dev.twitter.com/streaming/overview).\n\n## Requirements\n\nAfter cloning the `twitter-collection-js` repository, run `npm install` to download the Node.js modules on which the Twitter collection tool depends.\n\n## Twitter Credentials\n\nTo allow the Twitter collection tool to make an authenticated connection to the Twitter Streaming API, you need to supply your own Twitter application credentials. Twitter application credentials are a pair of consumer key and secret and a pair of access token key and secret, which you can set up using your Twitter account on the [Twitter applications page](http://apps.twitter.com) using [these instructions](https://dev.twitter.com/oauth/overview/application-owner-access-tokens \"Learn how to generate application credentials at TWitter\").\n\nWhen you have created the credentials for your Twitter application, copy and paste the corresponding values into the `twitter-credentials.js` file:\n\n```javascript\n/*\n * You need to supply your own values for the following keys.\n * Get yours at http://apps.twitter.com\n */\nmodule.exports = {\n  consumer_key: 'YOUR CONSUMER KEY',\n  consumer_secret: 'YOUR CONSUMER SECRET',\n  access_token_key: 'YOUR ACCESS TOKEN KEY',\n  access_token_secret: 'YOUR ACCESS TOKEN SECRET'\n};\n```\n\n## Filtering Tweets\n\nThe tweets from the streaming API are filtered using the keywords and phrases in the `filter-terms.txt` file. Each keyword or phrase should be written on a new line, e.g.:\n\n```\nstorm\nweather\nraining cats and dogs\n```\n\nThese keywords and phrases are composed into a comma-separated string that is passed to the Twitter streaming API:\n\n```\nstorm,weather,raining cats and dogs\n```\n\nThis string has the following semantics:\n\n```\nstorm OR weather OR (raining AND cats AND dogs)\n```\n\nas described in the documentation for the [track streaming endpoint](https://dev.twitter.com/streaming/overview/request-parameters#track) of the Twitter API.\n\n## Collecting Tweets\n\nTo start collecting tweets, run `collect.js`:\n\n```\n./collect.js\n```\n\nwhich will output tweets on the command line. To store the tweets in a file, redirect the output of `collect.js` into a file:\n\n```\n./collect.js \u003e collection.json\n```\n\n## Monitoring Collections\n\nThe `monitor-collection.sh` script provides a simple collection monitoring tool:\n\n```\nwhile true; do\n  wc -l collection.json\n  sleep 10s\ndone\n```\n\nEvery 10 seconds the script outputs the current size of the collection. The size of the collection is the number of tweets, which is found by calculating the number of lines (`wc -l`) in the collection file:\n\n```\nwc -l collection.json\n```\n\nYou should change `collection.json` to the name of the file into which the tweets you are collecting are being stored.\n\nTo change the frequency with which the monitor outputs the size of the collection, change the `10s` to the appropriate value for the \u003ca href=\"https://en.wikipedia.org/wiki/Sleep_(Unix)\"\u003e`sleep`\u003c/a\u003e command.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffreymorganio%2Ftwitter-collection-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjeffreymorganio%2Ftwitter-collection-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffreymorganio%2Ftwitter-collection-js/lists"}