{"id":13666016,"url":"https://github.com/gvegayon/twitterreport","last_synced_at":"2025-03-20T01:31:36.695Z","repository":{"id":34759462,"uuid":"38740870","full_name":"gvegayon/twitterreport","owner":"gvegayon","description":"Out-of-the-box analysis and reporting tools for twitter","archived":false,"fork":false,"pushed_at":"2017-11-29T07:47:53.000Z","size":4886,"stargazers_count":39,"open_issues_count":4,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-28T21:59:12.149Z","etag":null,"topics":["d3js","jaccard","leaflet","sentiment-analysis","tweets","twitter","wordcloud"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gvegayon.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-07-08T08:09:41.000Z","updated_at":"2024-10-07T12:55:38.000Z","dependencies_parsed_at":"2022-09-14T00:50:22.987Z","dependency_job_id":null,"html_url":"https://github.com/gvegayon/twitterreport","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvegayon%2Ftwitterreport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvegayon%2Ftwitterreport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvegayon%2Ftwitterreport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvegayon%2Ftwitterreport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gvegayon","download_url":"https://codeload.github.com/gvegayon/twitterreport/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244033788,"owners_count":20386998,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["d3js","jaccard","leaflet","sentiment-analysis","tweets","twitter","wordcloud"],"created_at":"2024-08-02T06:00:55.620Z","updated_at":"2025-03-20T01:31:36.276Z","avatar_url":"https://github.com/gvegayon.png","language":"R","funding_links":[],"categories":["R"],"sub_categories":[],"readme":"twitterreport\n=============\n\n[![Build Status](https://travis-ci.org/gvegayon/twitterreport.svg?branch=master)](https://travis-ci.org/gvegayon/twitterreport) [![Build status](https://ci.appveyor.com/api/projects/status/a7ki7jlc5qht4dmn?svg=true)](https://ci.appveyor.com/project/gvegayon/twitterreport) [![DOI](https://zenodo.org/badge/19832/gvegayon/twitterreport.svg)](https://zenodo.org/badge/latestdoi/19832/gvegayon/twitterreport)\n\nOut-of-the-box analysis and reporting tools for twitter\n\nAbout\n-----\n\nWhile there are some (very neat) R packages focused on twitter (namely `twitteR` and `stramR`), `twitterreport` is centered on providing analysis and reporting tools for twitter data. The package's current version features:\n\n-   Access to twitter API\n-   Extracting mentions/hashtags/urls from text (tweets)\n-   Gender tagging by matching user names with gender datasets included in the package (**es** and **en**)\n-   Creating (mentions) networks and visualizing them using D3js\n-   Sentiment analysis (basic, but useful) using lexicons included in the package (again, **es** and **en**)\n-   Creating time series charts of hashtags/users/etc. and visualizing them using D3js\n-   Create wordclouds (after removing stop words and processing the text)\n-   Map visualization using the leaflet package\n-   Topics identification through the Jaccard coeff (words similarity)\n\nYou can take a look at a live example at \u003chttp://www.its.caltech.edu/~gvegayon/twitter/report_example.html\u003e, and at the source code of that example at \u003chttps://github.com/gvegayon/twitterreport/blob/master/vignettes/report_example.Rmd\u003e\n\nSome of the functions here were firstly developed in the project *nodoschile.cl* (no longer running). You can visit the project's testimonial website \u003chttp://nodos.modularity.cl\u003e and the website (part of nodoschile) that motivated `twitterreports` at \u003chttp://modularity.cl/presidenciales\u003e.\n\nInstallation\n------------\n\nWhile the package is still in development, you can always use `devtools` to install the most recent version.\n\n``` r\ndevtools::install_git('gvegayon/twitterreport')\n```\n\nExamples\n--------\n\n### Getting tweets from a set of users\n\n``` r\n# Firts, load the package!\nlibrary(twitterreport)\n\n# List of twitter accounts\nusers \u003c- c('MarsRovers', 'senatormenendez', 'sciencemagazine')\n\n# Getting the twitts (first gen the token)\nkey \u003c- tw_gen_token('myapp','key', 'secret')\ntweets \u003c- lapply(users, tw_api_get_statuses_user_timeline, twitter_token=key)\n \n# Processing the data (and taking a look)\ntweets \u003c- do.call(rbind, tweets)\nhead(tweets)\n```\n\n### Creating a (fancy) network of mentions\n\n``` r\n# Loading data\ndata(\"senators\")\ndata(\"senators_profile\")\ndata(\"senate_tweets\")\n\ntweets_components \u003c- tw_extract(senate_tweets$text)\ngroups \u003c- data.frame(\n  name      = senators_profile$tw_screen_name,\n  group     = factor(senators$party),\n  real_name = senators$Name,\n  stringsAsFactors = FALSE)\ngroups$name \u003c- tolower(groups$name)\n\nsenate_network \u003c- tw_network(\n  tolower(senate_tweets$screen_name),\n  lapply(tweets_components$mention,unique),only.from = TRUE,\n  group=groups, min.interact = 3)\n\nplot(senate_network, nodelabel='real_name')\n```\n\n![](README_files/figure-markdown_github/network.png)\n\nIn the following examples we will use data on US senators extracted from twitter using the REST API (you can find it in the package)\n\n### Creating a wordcloud\n\nThe function `tw_words` takes a character vector (of tweets for example) and extracts all the stopwords+symbols. And the `plot` method for its output creates a wordcloud\n\n``` r\ndata(senate_tweets)\ntab \u003c- tw_words(senate_tweets$text)\n\n# What did it do?\nsenate_tweets$text[1:2];tab[1:2]\n```\n\n    ## [1] \"“I am saddened by the news that four Marines lost their lives today in the service of our country.” #Chattanooga\"         \n    ## [2] \".@SenAlexander statement on today’s “tragic and senseless” murder of four Marines in #Chattanooga: http://t.co/H9zWdJPbiE\"\n\n    ## [[1]]\n    ##  [1] \"saddened\"    \"news\"        \"four\"        \"marines\"     \"lost\"       \n    ##  [6] \"lives\"       \"today\"       \"service\"     \"country\"     \"chattanooga\"\n    ## \n    ## [[2]]\n    ## [1] \"senalexander\" \"statement\"    \"todays\"       \"tragic\"      \n    ## [5] \"senseless\"    \"murder\"       \"four\"         \"marines\"     \n    ## [9] \"chattanooga\"\n\n``` r\n# Plot\nset.seed(123) # (so the wordcloud looks the same always)\nplot(tab, max.n.words = 40)\n```\n\n![](README_files/figure-markdown_github/wordcloud-1.png)\n\n### Identifying individuals gender\n\nUsing english and spanish names, the `tw_gender` function matches the character argument (which can be a vector) with either a male or female name (or unidentified).\n\n``` r\ndata(senators_profile)\n\n# Getting the names\nsen \u003c- tolower(senators_profile$tw_name)\nsen \u003c- gsub('\\\\bsen(ator|\\\\.)\\\\s+','',sen)\nsen \u003c- gsub('\\\\s+.+','',sen)\n\ntab \u003c- table(tw_gender(sen))\nbarplot(tab)\n```\n\n![](README_files/figure-markdown_github/gender-1.png)\n\nSentiment analysis\n------------------\n\nHere we have an example clasifying senate tweets on the \\#irandeal.\n\n``` r\nirandeal \u003c- subset(senate_tweets, grepl('irandeal',text, ignore.case = TRUE))\nirandeal$sentiment \u003c- tw_sentiment(irandeal$text, normalize = TRUE)\n\nhist(irandeal$sentiment, col = 'lightblue', \n     xlab ='Valence (strength of sentiment)')\n```\n\n![](README_files/figure-markdown_github/Sentiments-1.png)\n\nA map using leaflet\n-------------------\n\nThe function `tw_leaflet` provides a nice wrapper for the function `leaflet` of\nthe package of the same name. Using D3js, we can visualize the number of tweets grouped up geographically as the following example shows:\n\n``` r\ntw_leaflet(senate_tweets,~coordinates, nclusters=4,radii = ~sqrt(n)*3e5)\n```\n\n![](README_files/figure-markdown_github/leaflet_map.png)\n\nNote that in this case there are 14 tweets with the `coordinates` column non-empty, leading to 4 different senators that have such information. Using the `nclusters` option, the `tw_leaflet` groups the data using the `hclust` function of the stats package. So the user doesn't need to worry about aggregating data.\n\nWords closeness\n---------------\n\nAn interesting issue to review is how are words related to each other. Using the Jaccard coefficient we are able to estimate a measure of distance between two words. The `jaccard_coef` function implements such algorithm, and it allows us to get a better understanding of topics, as the following example\n\n``` r\n# Computing the jaccard coefficient\njaccard \u003c- jaccard_coef(senate_tweets$text,max.size = 1000)\n\n# See what words are related with abortion\nwords_closeness('veterans',jaccard,.025)\n```\n\n    ##        word         coef\n    ## 1  veterans 318.00000000\n    ## 2        va   0.08982036\n    ## 3      care   0.08510638\n    ## 4     honor   0.04389313\n    ## 5    access   0.04201681\n    ## 6   deserve   0.04176334\n    ## 7    health   0.04022989\n    ## 8  benefits   0.03827751\n    ## 9    mental   0.03733333\n    ## 10  honored   0.03505155\n    ## 11     home   0.03440860\n    ## 12  service   0.03266788\n    ## 13     july   0.03108808\n    ## 14   combat   0.02964960\n    ## 15 services   0.02857143\n    ## 16   choice   0.02549575\n    ## 17    thank   0.02529960\n\nWe can also do this using the output from `tw_extract`, this is, by passing a list of character vectors (this is much fasters)\n\n``` r\nhashtags \u003c- tw_extract(senate_tweets$text, obj = 'hashtag')$hashtag\n\n# Again, but using a list\njaccard \u003c- jaccard_coef(hashtags,max.size = 15000)\njaccard\n```\n\n    ## Jaccard index Matrix (Sparse) of 3283x3283 elements\n    ## Contains the following words (access via $freq):\n    ##          wrd   n\n    ## 1   irandeal 202\n    ## 2       iran 179\n    ## 3     scotus 141\n    ## 4        tpa 132\n    ## 5      netde 119\n    ## 6 mepolitics 117\n\n``` r\n# See what words are related with abortion\nwords_closeness('veterans',jaccard,.025)\n```\n\n    ##          word        coef\n    ## 1    veterans 78.00000000\n    ## 2 honorflight  0.06382979\n    ## 3          va  0.05154639\n    ## 4  miasalutes  0.05000000\n    ## 5     4profit  0.04166667\n    ## 6   choiceact  0.03658537\n    ## 7 40mileissue  0.02564103\n    ## 8        hepc  0.02531646\n\nAuthor\n------\n\nGeorge G. Vega Yon\n\ng vegayon at caltech\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgvegayon%2Ftwitterreport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgvegayon%2Ftwitterreport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgvegayon%2Ftwitterreport/lists"}