{"id":13892880,"url":"https://github.com/uclatommy/tweetfeels","last_synced_at":"2025-04-06T04:10:21.004Z","repository":{"id":62585667,"uuid":"82366919","full_name":"uclatommy/tweetfeels","owner":"uclatommy","description":"Real-time sentiment analysis in Python using twitter's streaming api","archived":false,"fork":false,"pushed_at":"2018-11-10T15:45:09.000Z","size":298,"stargazers_count":255,"open_issues_count":6,"forks_count":48,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-03-30T03:05:11.524Z","etag":null,"topics":["data-mining","data-science","python-3-6","sentiment-analysis","twitter"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uclatommy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-18T06:31:58.000Z","updated_at":"2025-02-12T20:51:48.000Z","dependencies_parsed_at":"2022-11-03T22:04:29.424Z","dependency_job_id":null,"html_url":"https://github.com/uclatommy/tweetfeels","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uclatommy%2Ftweetfeels","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uclatommy%2Ftweetfeels/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uclatommy%2Ftweetfeels/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uclatommy%2Ftweetfeels/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uclatommy","download_url":"https://codeload.github.com/uclatommy/tweetfeels/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247430870,"owners_count":20937874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-mining","data-science","python-3-6","sentiment-analysis","twitter"],"created_at":"2024-08-06T17:01:18.116Z","updated_at":"2025-04-06T04:10:20.973Z","avatar_url":"https://github.com/uclatommy.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cimage src=\"https://uclatommy.github.io/tweetfeels/images/tweetfeels.svg\" width=\"100%\" height=\"180\"\u003e\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://travis-ci.org/uclatommy/tweetfeels\"\u003e\n    \u003cimage src=\"https://travis-ci.org/uclatommy/tweetfeels.svg?branch=master\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://github.com/uclatommy/tweetfeels/issues\"\u003e\n     \u003cimg src=\"https://img.shields.io/github/issues/uclatommy/tweetfeels.svg\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://www.python.org/\"\u003e\n     \u003cimg src=\"https://img.shields.io/badge/python-3.6%2B-blue.svg\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://www.clahub.com/agreements/uclatommy/tweetfeels\"\u003e\n     \u003cimg src=\"https://img.shields.io/badge/CLA-open-brightgreen.svg\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n# Introduction\nTweetfeels relies on [VADER sentiment analysis](https://github.com/cjhutto/vaderSentiment) to provide sentiment scores to user-defined topics. It does this by utilizing Twitter's streaming API to listen to real-time tweets around a particular topic. Some possible applications for this include:\n* Calculating the social sentiment of particular political figures or issues and analyzing scores across geographic regions.\n* Calculating sentiment scores for brands.\n* Using sentiment scores as training features for a learning algorithm to determine stock buy and sell triggers.\n* And more!\n\n# Install Methods\n1. The easiest way is to install from PyPI:\n    ```\n    \u003e pip3 install tweetfeels\n    ```\n\n2. If you've installed from PyPI and want to upgrade:\n    ```\n    \u003e pip3 install --upgrade tweetfeels\n    ```\n\n3. You can also install by cloning this repo:\n    ```\n    \u003e git clone https://github.com/uclatommy/tweetfeels.git\n    \u003e cd tweetfeels\n    \u003e python3 setup.py install\n    ```\n\n## Additional Requirements\n1. You will need to obtain Twitter OAuth keys and supply them to tweetfeels in order to connect to Twitter's streaming API. Go [here](https://twittercommunity.com/t/how-to-get-my-api-key/7033) for instructions on how to obtain your keys.\n\n2. Minimum python version of 3.6\n\n3. If for some reason pip did not install the vader lexicon:\n    ```\n    \u003e python3 -m nltk.downloader vader_lexicon\n    ```\n\n# Examples\n*Note: Authorization keys in the examples are masked for privacy.*\n\nFor all examples, we use a few common boilerplate lines:\n```python\nfrom tweetfeels import TweetFeels\n\nconsumer_key = '*************************'\nconsumer_secret = '**************************************************'\naccess_token = '**************************************************'\naccess_token_secret = '*********************************************'\nlogin = [consumer_key, consumer_secret, access_token, access_token_secret]\n```\n\n#### Stream tweets related to keyword \"Trump\" for 10 seconds, then calculate a sentiment score for the last 10 seconds.\n```python\n\u003e\u003e\u003e trump_feels = TweetFeels(login, tracking=['trump'])\n\u003e\u003e\u003e trump_feels.start(10)\nTimer completed. Disconnecting now...\n\u003e\u003e\u003e trump_feels.sentiment.value\n-0.0073007430343252711\n```\n\n#### Stream tweets continuously and print current sentiment score every 10 seconds\n```python\n\u003e\u003e\u003e from threading import Thread\n\u003e\u003e\u003e import time\n\u003e\u003e\u003e\n\u003e\u003e\u003e def print_feels(seconds=10):\n...     while go_on:\n...         time.sleep(seconds)\n...         print(f'[{time.ctime()}] Sentiment Score: {trump_feels.sentiment.value}')\n...\n\u003e\u003e\u003e go_on = True\n\u003e\u003e\u003e t = Thread(target=print_feels)\n\u003e\u003e\u003e trump_feels.start()\n\u003e\u003e\u003e t.start()\n[Mon Feb 20 23:42:02 2017] Sentiment Score: -0.010528112416665309\n[Mon Feb 20 23:42:13 2017] Sentiment Score: -0.007496043169013409\n[Mon Feb 20 23:42:25 2017] Sentiment Score: -0.015294713038619036\n[Mon Feb 20 23:42:36 2017] Sentiment Score: -0.030362951884842962\n[Mon Feb 20 23:42:48 2017] Sentiment Score: -0.042087318872206333\n[Mon Feb 20 23:42:59 2017] Sentiment Score: -0.041308681936680865\n[Mon Feb 20 23:43:10 2017] Sentiment Score: -0.056203371039128994\n[Mon Feb 20 23:43:22 2017] Sentiment Score: -0.07374769163753854\n[Mon Feb 20 23:43:34 2017] Sentiment Score: -0.09549338153348486\n[Mon Feb 20 23:43:46 2017] Sentiment Score: -0.10943157911799692\n[Mon Feb 20 23:43:57 2017] Sentiment Score: -0.1406756546353098\n[Mon Feb 20 23:44:08 2017] Sentiment Score: -0.12366467180485821\n[Mon Feb 20 23:44:20 2017] Sentiment Score: -0.14460675229624026\n[Mon Feb 20 23:44:32 2017] Sentiment Score: -0.13149386547613803\n[Mon Feb 20 23:44:43 2017] Sentiment Score: -0.14568801433828418\n[Mon Feb 20 23:44:55 2017] Sentiment Score: -0.14505295656838593\n[Mon Feb 20 23:45:06 2017] Sentiment Score: -0.12853750933261338\n[Mon Feb 20 23:45:17 2017] Sentiment Score: -0.11649611157554504\n[Mon Feb 20 23:45:29 2017] Sentiment Score: -0.11382260762980569\n[Mon Feb 20 23:45:40 2017] Sentiment Score: -0.11121839471955856\n[Mon Feb 20 23:45:52 2017] Sentiment Score: -0.11083390577340985\n[Mon Feb 20 23:46:03 2017] Sentiment Score: -0.10879727669948112\n[Mon Feb 20 23:46:15 2017] Sentiment Score: -0.10137079133168492\n[Mon Feb 20 23:46:26 2017] Sentiment Score: -0.10075971619875508\n[Mon Feb 20 23:46:38 2017] Sentiment Score: -0.1194907722483259\n[Mon Feb 20 23:46:49 2017] Sentiment Score: -0.1328795394197093\n[Mon Feb 20 23:47:01 2017] Sentiment Score: -0.13734346200202507\n[Mon Feb 20 23:47:12 2017] Sentiment Score: -0.1157629833027525\n[Mon Feb 20 23:47:24 2017] Sentiment Score: -0.11030256885649424\n[Mon Feb 20 23:47:35 2017] Sentiment Score: -0.12185876174059834\n[Mon Feb 20 23:47:47 2017] Sentiment Score: -0.11323251979604802\n[Mon Feb 20 23:47:58 2017] Sentiment Score: -0.11307793897469191\n\u003e\u003e\u003e trump_feels.stop()\n```\n\n**Note:** Trump is an extremely high volume topic. We ran this for roughly 6 minutes and gathered nearly 15,000 tweets! For lower volume topics, you may want to poll the sentiment value less frequently than every 10 seconds.\n\n#### Stream tweets continuously for another topic and save to a different database.\n\n```python\n\u003e\u003e\u003e tesla_feels = TweetFeels(login, tracking=['tesla', 'tsla', 'gigafactory', 'elonmusk'], db='tesla.sqlite')\n\u003e\u003e\u003e t = Thread(target=print_feels, args=(tesla_feels, 120))\n\u003e\u003e\u003e tesla_feels.start()\n\u003e\u003e\u003e t.start()\n[Mon Feb 20 17:39:15 2017] Sentiment Score: 0.03347735418362685\n[Mon Feb 20 17:41:15 2017] Sentiment Score: 0.09408120307200825\n[Mon Feb 20 17:43:15 2017] Sentiment Score: 0.12554072120979093\n[Mon Feb 20 17:45:16 2017] Sentiment Score: 0.12381491277579157\n[Mon Feb 20 17:47:16 2017] Sentiment Score: 0.17121666657137832\n[Mon Feb 20 17:49:16 2017] Sentiment Score: 0.22588283902409384\n[Mon Feb 20 17:51:16 2017] Sentiment Score: 0.23587583668725887\n[Mon Feb 20 17:53:16 2017] Sentiment Score: 0.2485916177213093\n```\n\n#### Use the sentiments generator to replay captured data and plot\n```python\nimport pandas as pd\nfrom datetime import timedelta, datetime\nimport matplotlib.pyplot as plt\nimport matplotlib.dates as mdates\ndata1 = {s.end: s.value for s in tesla_feels.sentiments(delta_time=timedelta(minutes=15), nans=True)}\ndata2 = {s.end: s.volume for s in tesla_feels.sentiments(delta_time=timedelta(minutes=15), nans=True)}\ndf1 = pd.DataFrame.from_dict(data1, orient='index')\ndf2 = pd.DataFrame.from_dict(data2, orient='index')\nfig, axes = plt.subplots(nrows=2, ncols=1)\nfig.set_size_inches(15, 5)\nplt.subplot(211).axes.get_xaxis().set_visible(False)\ndf1[0].plot(kind='line', title='Tesla Sentiment')\nplt.subplot(212)\ndf2[0].plot(kind='area', title='Volume')\n```\n\u003cimage src=\"https://uclatommy.github.io/tweetfeels/images/volume.svg\" width=\"100%\" height=\"300\"\u003e\n\n# Methodology\nThere are a multitude of ways in which you could combine hundreds or thousands of tweets across time in order to calculate a single sentiment score. One naive method might be to bin tweets into discretized time-boxes. For example, perhaps you average the individual sentiment scores every 10 seconds so that the current sentiment is the average over the last 10 seconds. In this method, your choice of discretization length is arbitrary and will have an impact on the perceived variance of the score. It also disregards any past sentiment calculations.\n\nTo correct for these effects, we time-box every minute by default and do not discard the sentiment from prior calculations. Instead, we phase out older tweet sentiments geometrically as we add in new tweets:\n\n![f1]\n\nWhere ![f2] is the aggregate sentiment at time t, ![f3] is the sentiment score for the current time-box, and ![f5] is the fall-off factor between 0 and 1. We start the calculation with ![f4], which is why you will see the sentiment score move away from zero until it stabilizes around the natural value. Within each time-box we are using a weighted average of sentiment scores. For each tweet, we utilize the associated user's followers and friends count as the measure of influence.\n\nSome tweets will also have a neutral score (0.0). In these cases, we exclude it from aggregation.\n\nHere's an example of different model parameterizations of real-time Tesla sentiment:\n\u003cimage src=\"https://uclatommy.github.io/tweetfeels/images/tesla-sentiment.svg\" width=\"100%\" height=\"300\"\u003e\n\n[f1]: http://chart.apis.google.com/chart?cht=tx\u0026chl=S_{t}=%5calpha{S_{t-1}}%2B(1-%5calpha)s_t\n[f2]: http://chart.apis.google.com/chart?cht=tx\u0026chl=S_t\n[f3]: http://chart.apis.google.com/chart?cht=tx\u0026chl=s_t\n[f4]: http://chart.apis.google.com/chart?cht=tx\u0026chl=S_0=0\n[f5]: http://chart.apis.google.com/chart?cht=tx\u0026chl=%5calpha\n\n## Caveats\nThe trained dataset that comes with [vaderSentiment](https://github.com/cjhutto/vaderSentiment) is optimized for social media, so it can recognize the sentiment embedded in neologisms, internet shorthand, and even emoticons. However, it can only measure the aggregate sentiment value of a sentence or group of words. It does not measure whether or not a tweet agrees or disagrees with a particular ideology, political figure, or party. Although it is generally true that statements of disagreement will tend to have a negative sentiment. As an illustration, have a look at a few sentiment scores from the trump dataset:\n\n| | Sentiment | Tweet |\n| :---: | :--- | :--- |\n| 1 | -0.5106 | RT @TEN_GOP: BREAKING: Massive riots happening now in Sweden. Stockholm in flames. Trump was right again! |\n| 2 | -0.8744 | RT @kurteichenwald: Intel shows our ally, Sweden, has no rise in crime. Trump saw on Fox it does. So he ignores intel, attacks our ally. ht… |\n| 3 | 0.7003 | RT @NoBoomGaming: I'm a glass half full kind of guy. Now that Trump won, think of all the new memes we'll have over the next four years! |\n| 4 | 0.6249 | RT @SandraTXAS: Nikki Haley is kicking a$$ at the UN👊💥💥 Trump made a great choice for envoy to the UN!! #Israel #MAGA |\n\nThe first tweet is clearly voicing support for Donald Trump yet we get a negative score. The second tweet is clearly in opposition and it also produces a very negative sentiment. The fourth tweet is a case of sentiment aligning with approval. Clearly, sentiment scores should not be confused with ideological alignment or approval because it can go both ways! You can approve and make a negative comment and you can disapprove and make a positive sounding comment! Don't even get me started on sarcastic tweets (see third one).\n\nSentiment scores tend to be more meaningful to non-ideological topics such as products and services. For example, here are some tweets from the Tesla dataset:\n\n| | Sentiment | Tweet |\n| :---: | :--- | :--- |\n| 1 | -0.296 | Tesla is ‘illegally selling cars’ in Connecticut, says Dealership Association as they try to stop direct-sale bill |\n| 2 | -0.5859 | Supercharger Realtime Availability Map is offline until further notice. I am no longer receiving data as Tesla asked for it to be cut off. |\n| 3 | 0.5859 | Elon Musk Steps Forward To Help Tesla Driver Who Sacrificed Car To Save Stroke Victim via @aplusapp |\n| 4 | 0.4404 | RT @ElectrekCo: Tesla Model 3: aluminum part supplier announces investment to increase output ahead of Model 3 production…  |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuclatommy%2Ftweetfeels","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuclatommy%2Ftweetfeels","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuclatommy%2Ftweetfeels/lists"}