{"id":22211625,"url":"https://github.com/kwokhing/rfacebook-demo","last_synced_at":"2025-10-27T20:33:58.948Z","repository":{"id":189295789,"uuid":"96661202","full_name":"KwokHing/Rfacebook-Demo","owner":"KwokHing","description":"Demo on using the Rfacebook package to scrape/mine user generated contents (UGCs) from public FB pages. Simple sentiment, text, trend and posting frequency analysis are performed on these scraped UGCs","archived":false,"fork":false,"pushed_at":"2020-09-19T03:45:09.000Z","size":2461,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-30T05:26:57.292Z","etag":null,"topics":["d3-heatmap","exploratory-data-analysis","r","rfacebook","sentiment-analysis","sentiment-lexcions","social-media","text-analysis","trend-analysis","user-generated-content","wordcloud"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KwokHing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-07-09T04:34:27.000Z","updated_at":"2020-10-09T09:59:43.000Z","dependencies_parsed_at":null,"dependency_job_id":"d071ade6-94d2-45ea-afc0-aa4b60b52032","html_url":"https://github.com/KwokHing/Rfacebook-Demo","commit_stats":null,"previous_names":["kwokhing/rfacebook-demo"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FRfacebook-Demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FRfacebook-Demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FRfacebook-Demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FRfacebook-Demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KwokHing","download_url":"https://codeload.github.com/KwokHing/Rfacebook-Demo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245407763,"owners_count":20610232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["d3-heatmap","exploratory-data-analysis","r","rfacebook","sentiment-analysis","sentiment-lexcions","social-media","text-analysis","trend-analysis","user-generated-content","wordcloud"],"created_at":"2024-12-02T20:35:43.490Z","updated_at":"2025-10-27T20:33:58.845Z","avatar_url":"https://github.com/KwokHing.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"### Update: Permission is required for using Facebook Graph API to get information from public pages not owned by yourself ###\n\n## Facebook Analysis using Rfacebook Package ## \n\n*The Rfacebook package is authored and maintained by Pablo Barbera and Michael Piccirilli*  \n\nThis demo will provide a brief introduction in \n- setting up and using the Rfacebook package to scrape public pages posts and comments. \n- performing a simple lexicon-based sentiment analysis \n- performing text frequency analysis with the use of word cloud will be applied\n- plotting a simple trend analysis based on the page's \"likes\", \"shares\" and \"comments\" \n- creating a heatmap of the page posts by day and time \n- and the page's posting frequency. \n\n## Codes Walkthrough\n\n\n1) Installing the [__Rfacebook package__](https://cran.r-project.org/web/packages/Rfacebook/Rfacebook.pdf)  \n\n\n```R\ninstall.packages(\"Rfacebook\")\nlibrary(Rfacebook)\n```\n\n### Facebook authentication ###\n\n\n2) Getting Temporary Access Token  \n- Get temporary (2hrs) FB access token via: [https://developers.facebook.com/tools/explorer](https://developers.facebook.com/tools/explorer) \u0026gt; click on __Get Token__  \n- Replace __token \u0026lt;- \"\"__ with your token code \n- *You will need Graph API v2.6 and higher to get Reactions (love, haha, wow, sad, angry)* \n\n3) Creating an App on Facebook Platform\n- Create an app via: [https://developers.facebook.com/](https://developers.facebook.com/)  \n- Replace __app_id=\"xxx\"__ with your App ID code\n- Replace __app_secret=\"xxx\"__ with your App Secret code\n\n\n```R\ntoken \u003c- \"xxxxxxxxxx\"\nfb_oauth \u003c- fbOAuth(app_id=\"xxx\",app_secret=\"xxx\",extended_permissions = TRUE)\n\nsave(fb_oauth, file=\"fb_oauth\")\nload(\"fb_oauth\")\n```\n\n### Get Posts from Facebook Page ###\n\n4) Get the latest 5000 posts their respective __Reactions__ information from FB Page [__thatsmyfairprice__](https://www.facebook.com/thatsmyfairprice/)\n\n\n```R\nntucPosts \u003c- getPage(\"thatsmyfairprice\", token, n = 5000, reactions = T)\n```\n\n\n```R\nhead(ntucPosts)\n```\n\n\n\u003ctable\u003e\n\u003cthead\u003e\u003ctr\u003e\u003cth\u003eid\u003c/th\u003e\u003cth\u003elikes_count\u003c/th\u003e\u003cth\u003efrom_id\u003c/th\u003e\u003cth\u003efrom_name\u003c/th\u003e\u003cth\u003emessage\u003c/th\u003e\u003cth\u003ecreated_time\u003c/th\u003e\u003cth\u003etype\u003c/th\u003e\u003cth\u003elink\u003c/th\u003e\u003cth\u003estory\u003c/th\u003e\u003cth\u003ecomments_count\u003c/th\u003e\u003cth\u003eshares_count\u003c/th\u003e\u003cth\u003elove_count\u003c/th\u003e\u003cth\u003ehaha_count\u003c/th\u003e\u003cth\u003ewow_count\u003c/th\u003e\u003cth\u003esad_count\u003c/th\u003e\u003cth\u003eangry_count\u003c/th\u003e\u003cth\u003epost_datetime\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\n\u003ctbody\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408_10150633686196409                                                                            \u003c/td\u003e\u003ctd\u003e5                                                                                                         \u003c/td\u003e\u003ctd\u003e124299741408                                                                                              \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                                        \u003c/td\u003e\u003ctd\u003e1973-01-01T08:00:00+0000                                                                                  \u003c/td\u003e\u003ctd\u003elink                                                                                                      \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/124299741408/posts/10150633686196409/                                            \u003c/td\u003e\u003ctd\u003eNTUC FairPrice added a life event from 1973: Founded in 1973.                                             \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1973-01-01 08:00:00                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408_10150856484056409                                                                            \u003c/td\u003e\u003ctd\u003e8                                                                                                         \u003c/td\u003e\u003ctd\u003e124299741408                                                                                              \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                                        \u003c/td\u003e\u003ctd\u003e1983-01-01T08:00:00+0000                                                                                  \u003c/td\u003e\u003ctd\u003ephoto                                                                                                     \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856484011409/10150856484011409/?type=3            \u003c/td\u003e\u003ctd\u003eNTUC FairPrice added a life event from 1983: NTUC FairPrice Co-operative Ltd Formed.                      \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1983-01-01 08:00:00                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408_10150856485356409                                                                            \u003c/td\u003e\u003ctd\u003e3                                                                                                         \u003c/td\u003e\u003ctd\u003e124299741408                                                                                              \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                                        \u003c/td\u003e\u003ctd\u003e1983-06-01T07:00:00+0000                                                                                  \u003c/td\u003e\u003ctd\u003elink                                                                                                      \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/124299741408/posts/10150856485356409/                                            \u003c/td\u003e\u003ctd\u003eNTUC FairPrice added a life event from June 1983: Used Textbook Project Launched.                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1983-06-01 07:00:00                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408_10150856487136409                                                                            \u003c/td\u003e\u003ctd\u003e6                                                                                                         \u003c/td\u003e\u003ctd\u003e124299741408                                                                                              \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                                        \u003c/td\u003e\u003ctd\u003e1985-01-01T08:00:00+0000                                                                                  \u003c/td\u003e\u003ctd\u003ephoto                                                                                                     \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856487101409/10150856487101409/?type=3            \u003c/td\u003e\u003ctd\u003eNTUC FairPrice added a life event from 1985: FairPrice Housebrand Products Introduced.                    \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1985-01-01 08:00:00                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408_10150856489041409                                                                            \u003c/td\u003e\u003ctd\u003e1                                                                                                         \u003c/td\u003e\u003ctd\u003e124299741408                                                                                              \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                                        \u003c/td\u003e\u003ctd\u003e1991-01-01T08:00:00+0000                                                                                  \u003c/td\u003e\u003ctd\u003ephoto                                                                                                     \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856489011409/10150856489011409/?type=3            \u003c/td\u003e\u003ctd\u003eNTUC FairPrice added a life event from 1991: First Retailer to Implement Bar-Coding at Check-Out Counters.\u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1991-01-01 08:00:00                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408_10150856491601409                                                                            \u003c/td\u003e\u003ctd\u003e3                                                                                                         \u003c/td\u003e\u003ctd\u003e124299741408                                                                                              \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                                        \u003c/td\u003e\u003ctd\u003e1994-01-01T08:00:00+0000                                                                                  \u003c/td\u003e\u003ctd\u003ephoto                                                                                                     \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856491566409/10150856491566409/?type=3            \u003c/td\u003e\u003ctd\u003eNTUC FairPrice added a life event from 1994: Everyday Low Price (EDLP) Essential Items Introduced.        \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e0                                                                                                         \u003c/td\u003e\u003ctd\u003e1994-01-01 08:00:00                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\nCheck how many posts are scraped from the FB page\n\n\n```R\nnrow(ntucPosts)\n```\n\n\n2473\n\n\n5) Checking the distribution of post type of this FB page\n\n\n```R\ntable(ntucPosts$type)\n```\n\n\n    \n     event   link   note  photo status  video \n         7    375      4   1555    321    211 \n\n\n\n```R\nplot(table(ntucPosts$type),ylab=\"Count of Post Type\")\n```\n\n\n![png](images/output_13_0.png)\n\n\n6) Let's take a look at the number of __Likes__ \u0026amp; __Reactions__ in each Post Type\n\n\n```R\ninstall.packages(\"sqldf\")\nlibrary(sqldf)\n```\n\n\n```R\npostType \u003c- sqldf(\"select type, count(type) as total_type_count,\n                  sum(shares_count) as total_shares,\n                  sum(likes_count) as total_likes, \n                  sum(love_count) as total_love,\n                  sum(haha_count) as total_haha,\n                  sum(wow_count) as total_wow,\n                  sum(sad_count) as total_sad,\n                  sum(angry_count) as total_angry\n                  from ntucPosts group by type\")\n```\n\n\n```R\npostType\n```\n\n\n\u003ctable\u003e\n\u003cthead\u003e\u003ctr\u003e\u003cth\u003etype\u003c/th\u003e\u003cth\u003etotal_type_count\u003c/th\u003e\u003cth\u003etotal_shares\u003c/th\u003e\u003cth\u003etotal_likes\u003c/th\u003e\u003cth\u003etotal_love\u003c/th\u003e\u003cth\u003etotal_haha\u003c/th\u003e\u003cth\u003etotal_wow\u003c/th\u003e\u003cth\u003etotal_sad\u003c/th\u003e\u003cth\u003etotal_angry\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\n\u003ctbody\u003e\n\t\u003ctr\u003e\u003ctd\u003eevent \u003c/td\u003e\u003ctd\u003e   7  \u003c/td\u003e\u003ctd\u003e    2 \u003c/td\u003e\u003ctd\u003e  2786\u003c/td\u003e\u003ctd\u003e  10  \u003c/td\u003e\u003ctd\u003e  1   \u003c/td\u003e\u003ctd\u003e   0  \u003c/td\u003e\u003ctd\u003e  0   \u003c/td\u003e\u003ctd\u003e  0   \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003elink  \u003c/td\u003e\u003ctd\u003e 375  \u003c/td\u003e\u003ctd\u003e 9710 \u003c/td\u003e\u003ctd\u003e 83335\u003c/td\u003e\u003ctd\u003e 560  \u003c/td\u003e\u003ctd\u003e 93   \u003c/td\u003e\u003ctd\u003e 484  \u003c/td\u003e\u003ctd\u003e 12   \u003c/td\u003e\u003ctd\u003e 18   \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003enote  \u003c/td\u003e\u003ctd\u003e   4  \u003c/td\u003e\u003ctd\u003e    4 \u003c/td\u003e\u003ctd\u003e    26\u003c/td\u003e\u003ctd\u003e   0  \u003c/td\u003e\u003ctd\u003e  0   \u003c/td\u003e\u003ctd\u003e   0  \u003c/td\u003e\u003ctd\u003e  0   \u003c/td\u003e\u003ctd\u003e  0   \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003ephoto \u003c/td\u003e\u003ctd\u003e1555  \u003c/td\u003e\u003ctd\u003e34243 \u003c/td\u003e\u003ctd\u003e424959\u003c/td\u003e\u003ctd\u003e3014  \u003c/td\u003e\u003ctd\u003e209   \u003c/td\u003e\u003ctd\u003e1419  \u003c/td\u003e\u003ctd\u003e172   \u003c/td\u003e\u003ctd\u003e138   \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003estatus\u003c/td\u003e\u003ctd\u003e 321  \u003c/td\u003e\u003ctd\u003e 2016 \u003c/td\u003e\u003ctd\u003e 17255\u003c/td\u003e\u003ctd\u003e   6  \u003c/td\u003e\u003ctd\u003e 47   \u003c/td\u003e\u003ctd\u003e  20  \u003c/td\u003e\u003ctd\u003e  8   \u003c/td\u003e\u003ctd\u003e192   \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003evideo \u003c/td\u003e\u003ctd\u003e 211  \u003c/td\u003e\u003ctd\u003e38630 \u003c/td\u003e\u003ctd\u003e 82989\u003c/td\u003e\u003ctd\u003e1593  \u003c/td\u003e\u003ctd\u003e295   \u003c/td\u003e\u003ctd\u003e 392  \u003c/td\u003e\u003ctd\u003e 55   \u003c/td\u003e\u003ctd\u003e 65   \u003c/td\u003e\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\n### Get All Comments from the Scraped Facebook Posts ###\n\n7) Loop through all the posts and get their respective comments\n\n\n```R\nntucComment \u003c- list()\nfor (i in 1:length(ntucPosts$id)){\n  ntucComment[[i]] \u003c- getPost(ntucPosts$id[i], token, likes=F, comments=T)\n  ntucComment[[i]][['reactions']] \u003c- getReactions(post=ntucPosts$id[i], token)\n  if (nrow(ntucComment[[i]][[\"comments\"]]) == 0)\n    ntucComment[[i]][['comments']][1,] \u003c- c(NA,NA,NA,NA,NA,NA,NA)\n}\nntucComments \u003c- do.call(rbind, lapply(ntucComment, data.frame, stringsAsFactors=FALSE))\n```\n\n\n```R\nhead(ntucComments)\n```\n\n\n\u003ctable\u003e\n\u003cthead\u003e\u003ctr\u003e\u003cth\u003epost.from_id\u003c/th\u003e\u003cth\u003epost.from_name\u003c/th\u003e\u003cth\u003epost.message\u003c/th\u003e\u003cth\u003epost.created_time\u003c/th\u003e\u003cth\u003epost.type\u003c/th\u003e\u003cth\u003epost.link\u003c/th\u003e\u003cth\u003epost.id\u003c/th\u003e\u003cth\u003epost.likes_count\u003c/th\u003e\u003cth\u003epost.comments_count\u003c/th\u003e\u003cth\u003epost.shares_count\u003c/th\u003e\u003cth\u003e...\u003c/th\u003e\u003cth\u003ecomments.likes_count\u003c/th\u003e\u003cth\u003ecomments.comments_count\u003c/th\u003e\u003cth\u003ecomments.id\u003c/th\u003e\u003cth\u003ereactions.id\u003c/th\u003e\u003cth\u003ereactions.likes_count\u003c/th\u003e\u003cth\u003ereactions.love_count\u003c/th\u003e\u003cth\u003ereactions.haha_count\u003c/th\u003e\u003cth\u003ereactions.wow_count\u003c/th\u003e\u003cth\u003ereactions.sad_count\u003c/th\u003e\u003cth\u003ereactions.angry_count\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\n\u003ctbody\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e1973-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003elink                                                                                          \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/124299741408/posts/10150633686196409/                                \u003c/td\u003e\u003ctd\u003e124299741408_10150633686196409                                                                \u003c/td\u003e\u003ctd\u003e5                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e124299741408_10150633686196409                                                                \u003c/td\u003e\u003ctd\u003e5                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e1983-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003ephoto                                                                                         \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856484011409/10150856484011409/?type=3\u003c/td\u003e\u003ctd\u003e124299741408_10150856484056409                                                                \u003c/td\u003e\u003ctd\u003e8                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e124299741408_10150856484056409                                                                \u003c/td\u003e\u003ctd\u003e8                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e1983-06-01T07:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003elink                                                                                          \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/124299741408/posts/10150856485356409/                                \u003c/td\u003e\u003ctd\u003e124299741408_10150856485356409                                                                \u003c/td\u003e\u003ctd\u003e3                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e124299741408_10150856485356409                                                                \u003c/td\u003e\u003ctd\u003e3                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e1985-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003ephoto                                                                                         \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856487101409/10150856487101409/?type=3\u003c/td\u003e\u003ctd\u003e124299741408_10150856487136409                                                                \u003c/td\u003e\u003ctd\u003e6                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e124299741408_10150856487136409                                                                \u003c/td\u003e\u003ctd\u003e6                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e1991-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003ephoto                                                                                         \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856489011409/10150856489011409/?type=3\u003c/td\u003e\u003ctd\u003e124299741408_10150856489041409                                                                \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e124299741408_10150856489041409                                                                \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e1994-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003ephoto                                                                                         \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150856491566409/10150856491566409/?type=3\u003c/td\u003e\u003ctd\u003e124299741408_10150856491601409                                                                \u003c/td\u003e\u003ctd\u003e3                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e124299741408_10150856491601409                                                                \u003c/td\u003e\u003ctd\u003e3                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\n\n```R\nnrow(ntucComments)\n```\n\n\n19518\n\n\nThere are __19518__ rows of comments scraped but there are not actually __19518__ comments being made. This is because some of the posts do not have any comments made and their respective comments columns will show *NA*.\n\n### Dataframe Variable Names Standardization ###\n\nFirst, let's proceed to standardize the dataframe variable names\n\n\n```R\nnames(ntucComments)\n```\n\n\n\u003col class=\"list-inline\"\u003e\n\t\u003cli\u003e'post.from_id'\u003c/li\u003e\n\t\u003cli\u003e'post.from_name'\u003c/li\u003e\n\t\u003cli\u003e'post.message'\u003c/li\u003e\n\t\u003cli\u003e'post.created_time'\u003c/li\u003e\n\t\u003cli\u003e'post.type'\u003c/li\u003e\n\t\u003cli\u003e'post.link'\u003c/li\u003e\n\t\u003cli\u003e'post.id'\u003c/li\u003e\n\t\u003cli\u003e'post.likes_count'\u003c/li\u003e\n\t\u003cli\u003e'post.comments_count'\u003c/li\u003e\n\t\u003cli\u003e'post.shares_count'\u003c/li\u003e\n\t\u003cli\u003e'comments.from_id'\u003c/li\u003e\n\t\u003cli\u003e'comments.from_name'\u003c/li\u003e\n\t\u003cli\u003e'comments.message'\u003c/li\u003e\n\t\u003cli\u003e'comments.created_time'\u003c/li\u003e\n\t\u003cli\u003e'comments.likes_count'\u003c/li\u003e\n\t\u003cli\u003e'comments.comments_count'\u003c/li\u003e\n\t\u003cli\u003e'comments.id'\u003c/li\u003e\n\t\u003cli\u003e'reactions.id'\u003c/li\u003e\n\t\u003cli\u003e'reactions.likes_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions.love_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions.haha_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions.wow_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions.sad_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions.angry_count'\u003c/li\u003e\n\u003c/ol\u003e\n\n\n\n8) We will replace all __.__ with _ instead\n\n\n```R\nnames(ntucComments) \u003c- gsub(\"\\\\.\", \"_\", names(ntucComments))\nnames(ntucComments)\n```\n\n\n\u003col class=\"list-inline\"\u003e\n\t\u003cli\u003e'post_from_id'\u003c/li\u003e\n\t\u003cli\u003e'post_from_name'\u003c/li\u003e\n\t\u003cli\u003e'post_message'\u003c/li\u003e\n\t\u003cli\u003e'post_created_time'\u003c/li\u003e\n\t\u003cli\u003e'post_type'\u003c/li\u003e\n\t\u003cli\u003e'post_link'\u003c/li\u003e\n\t\u003cli\u003e'post_id'\u003c/li\u003e\n\t\u003cli\u003e'post_likes_count'\u003c/li\u003e\n\t\u003cli\u003e'post_comments_count'\u003c/li\u003e\n\t\u003cli\u003e'post_shares_count'\u003c/li\u003e\n\t\u003cli\u003e'comments_from_id'\u003c/li\u003e\n\t\u003cli\u003e'comments_from_name'\u003c/li\u003e\n\t\u003cli\u003e'comments_message'\u003c/li\u003e\n\t\u003cli\u003e'comments_created_time'\u003c/li\u003e\n\t\u003cli\u003e'comments_likes_count'\u003c/li\u003e\n\t\u003cli\u003e'comments_comments_count'\u003c/li\u003e\n\t\u003cli\u003e'comments_id'\u003c/li\u003e\n\t\u003cli\u003e'reactions_id'\u003c/li\u003e\n\t\u003cli\u003e'reactions_likes_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions_love_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions_haha_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions_wow_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions_sad_count'\u003c/li\u003e\n\t\u003cli\u003e'reactions_angry_count'\u003c/li\u003e\n\u003c/ol\u003e\n\n\n\n9) Create a function that coverts FB date format to GMT\n\n\n```R\nformat.facebook.date \u003c- function(datestring) {\n  date \u003c- as.POSIXct(datestring, format = \"%Y-%m-%dT%H:%M:%S+0000\", tz=\"GMT\")\n}\n```\n\n10) Adding two new variable __post_datetime__ \u0026amp; __comments_datetime__ to __ntucComments__ dataframe\n\nThis two variable will use the __format.facebook.date__ function to convert the original FB date format to GMT format\n\n\n```R\nntucComments$comments_datetime \u003c- format.facebook.date(ntucComments$comments_created_time)\nntucComments$post_datetime \u003c- format.facebook.date(ntucComments$post_created_time)\n```\n\n### Comments Variable Cleaning ###\n\n11) Convert the comments variable column to ASCII\n\n\n```R\nntucComments$comments_message \u003c- iconv(ntucComments$comments_message, \"ASCII\", \"UTF-8\", sub=\"\")\n```\n\n12) Remove comments made by the organisation itself\n\n\n```R\nntucCommentsClean \u003c- subset(ntucComments, comments_from_name != \"NTUC FairPrice\")\n```\n\n13) Substituting emoticons with text/descriptions\n\n\n```R\nntucCommentsClean$comments_message \u003c- gsub(\":-)\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\";-)\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":)\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\";)\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\"=p\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":p\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":P\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\"=P\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\"=)\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":-)\", \" happy \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\"\u003c3\", \" love \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":\\\\(\", \" sad \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":-\\\\(\", \" sad \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":x\", \" oops \", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\":X\", \" oops \", ntucCommentsClean$comments_message)\n```\n\n14) Substituting line breaks, tabs, digits, url and punctuations with empty space\n\n\n```R\nntucCommentsClean$comments_message \u003c- gsub(\"\\n\", \" \", ntucCommentsClean$comments_message) \nntucCommentsClean$comments_message \u003c- gsub(\"\\t\", \" \", ntucCommentsClean$comments_message) \nntucCommentsClean$comments_message \u003c- gsub(\"\\\\d\", \"\", ntucCommentsClean$comments_message) \nntucCommentsClean$comments_message \u003c- gsub(\"http[^[:blank:]]+\", \"\", ntucCommentsClean$comments_message)\nntucCommentsClean$comments_message \u003c- gsub(\"[[:punct:]]\", \"\", ntucCommentsClean$comments_message)\n```\n\n15) Removing rows with less than 3 characters (I deemed to be not useful at all for text analysis)\n\n\n```R\nntucCommentsClean \u003c- subset(ntucCommentsClean, nchar(comments_message) \u003e 2)\n```\n\n### Lexicon-based Sentiment Analysis ###\n\n16) load positive and negative lexicons\n\n\n```R\npositives = readLines(\"positive-words.txt\")\nnegatives = readLines(\"negative-words.txt\")\n```\n\n17) Adding custom abbreviation / internet slangs to lexicon\n\n\n```R\npositives = c(positives, 'thx', 'congrats', 'luv')\nnegatives = c(negatives, 'wtf', 'cancellation')\n```\n\n18) Installing required packages\n\n\n```R\ninstall.packages(\"stringr\")\ninstall.packages(\"dplyr\")\ninstall.packages(\"plyr\")\nlibrary(stringr)\nlibrary(dplyr)\nlibrary(plyr)\n```\n\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    \n    Attaching package: 'dplyr'\n    \n    The following objects are masked from 'package:stats':\n    \n        filter, lag\n    \n    The following objects are masked from 'package:base':\n    \n        intersect, setdiff, setequal, union\n    \n    ------------------------------------------------------------------------------\n    You have loaded plyr after dplyr - this is likely to cause problems.\n    If you need functions from both plyr and dplyr, please load plyr first, then dplyr:\n    library(plyr); library(dplyr)\n    ------------------------------------------------------------------------------\n    \n    Attaching package: 'plyr'\n    \n    The following objects are masked from 'package:dplyr':\n    \n        arrange, count, desc, failwith, id, mutate, rename, summarise,\n        summarize\n    \n\n\n19) Creating a simple scoring sentiment analysis function\n\n\n```R\nscore.sentiment = function(sentences, pos.words, neg.words, .progress='none')\n{\n  require(plyr)\n  require(stringr)\n  \n  scores = laply(sentences, function(sentence, pos.words, neg.words) {\n    \n    sentence = gsub('[[:punct:]]', '', sentence)\n    sentence = gsub('[[:cntrl:]]', '', sentence)\n    sentence = gsub('\\\\d+', '', sentence)\n    sentence = tolower(sentence)\n    \n    word.list = str_split(sentence, '\\\\s+')\n    words = unlist(word.list)\n    \n    pos.matches = match(words, pos.words)\n    neg.matches = match(words, neg.words)\n\n    pos.matches = !is.na(pos.matches)\n    neg.matches = !is.na(neg.matches)\n    \n    score = sum(pos.matches) - sum(neg.matches)\n    \n    return(score)\n  }, pos.words, neg.words, .progress=.progress )\n  \n  scores.df = data.frame(score=scores, text=sentences)\n  return(scores.df)\n}\n```\n\n\n```R\nntucSentiScores \u003c- score.sentiment(ntucCommentsClean$comments_message,positives,negatives,.progress = \"text\")\n```\n\n      |======================================================================| 100%\n\n\n\n```R\nhist(ntucSentiScores$score,xlab=\"Sentiment Score\")\n```\n\n\n![png](images/output_52_0.png)\n\n\nhe histogram shows that most of the comments have a score hovering around the 0 score mark. The histogram depicts a normal distribution indicating that most of the comments are neutral with some positive and negative comments\n\n20) Now, let's add a new variable __sentiment_polar__ to determine the sentiment of the comment\n\n\n```R\nntucCommentsClean$sentiment \u003c- ntucSentiScores$score\nntucCommentsClean$sentiment_polar \u003c- ifelse(ntucCommentsClean$sentiment == 0, \"Neutral\", ifelse(ntucCommentsClean$sentiment \u003e 0, \"Positive\", \"Negative\"))\n```\n\n\n```R\nhead(ntucCommentsClean, n=2)\n```\n\n\n\u003ctable\u003e\n\u003cthead\u003e\u003ctr\u003e\u003cth\u003e\u003c/th\u003e\u003cth\u003epost_from_id\u003c/th\u003e\u003cth\u003epost_from_name\u003c/th\u003e\u003cth\u003epost_message\u003c/th\u003e\u003cth\u003epost_created_time\u003c/th\u003e\u003cth\u003epost_type\u003c/th\u003e\u003cth\u003epost_link\u003c/th\u003e\u003cth\u003epost_id\u003c/th\u003e\u003cth\u003epost_likes_count\u003c/th\u003e\u003cth\u003epost_comments_count\u003c/th\u003e\u003cth\u003epost_shares_count\u003c/th\u003e\u003cth\u003e...\u003c/th\u003e\u003cth\u003ereactions_likes_count\u003c/th\u003e\u003cth\u003ereactions_love_count\u003c/th\u003e\u003cth\u003ereactions_haha_count\u003c/th\u003e\u003cth\u003ereactions_wow_count\u003c/th\u003e\u003cth\u003ereactions_sad_count\u003c/th\u003e\u003cth\u003ereactions_angry_count\u003c/th\u003e\u003cth\u003ecomments_datetime\u003c/th\u003e\u003cth\u003epost_datetime\u003c/th\u003e\u003cth\u003esentiment\u003c/th\u003e\u003cth\u003esentiment_polar\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\n\u003ctbody\u003e\n\t\u003ctr\u003e\u003cth\u003e13\u003c/th\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e2004-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003ephoto                                                                                         \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150858221106409/10150858221106409/?type=3\u003c/td\u003e\u003ctd\u003e124299741408_10150858221126409                                                                \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e3                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e2012-06-08 09:23:48                                                                           \u003c/td\u003e\u003ctd\u003e2004-01-01 08:00:00                                                                           \u003c/td\u003e\u003ctd\u003e 0                                                                                            \u003c/td\u003e\u003ctd\u003eNeutral                                                                                       \u003c/td\u003e\u003c/tr\u003e\n\t\u003ctr\u003e\u003cth\u003e15\u003c/th\u003e\u003ctd\u003e124299741408                                                                                  \u003c/td\u003e\u003ctd\u003eNTUC FairPrice                                                                                \u003c/td\u003e\u003ctd\u003eNA                                                                                            \u003c/td\u003e\u003ctd\u003e2004-01-01T08:00:00+0000                                                                      \u003c/td\u003e\u003ctd\u003ephoto                                                                                         \u003c/td\u003e\u003ctd\u003ehttps://www.facebook.com/thatsmyfairprice/photos/p.10150858221106409/10150858221106409/?type=3\u003c/td\u003e\u003ctd\u003e124299741408_10150858221126409                                                                \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e3                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e...                                                                                           \u003c/td\u003e\u003ctd\u003e1                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e0                                                                                             \u003c/td\u003e\u003ctd\u003e2015-03-04 19:44:11                                                                           \u003c/td\u003e\u003ctd\u003e2004-01-01 08:00:00                                                                           \u003c/td\u003e\u003ctd\u003e-1                                                                                            \u003c/td\u003e\u003ctd\u003eNegative                                                                                      \u003c/td\u003e\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\n\n```R\ntable(ntucCommentsClean$sentiment_polar) \n```\n\n\n    \n    Negative  Neutral Positive \n        1418     8884     6986 \n\n\n\n```R\nplot(table(ntucCommentsClean$sentiment_polar),ylab=\"Frequency\")\n```\n\n\n![png](images/output_57_0.png)\n\n\n\n```R\nmean(ntucCommentsClean$sentiment)\n```\n\n\n0.56461129106895\n\n\nmean sentiment score of 0.56 indicates a mostly neutral and positive comments\n\n*You can continue to check the comments sentiments based on the post type as well*  \n*For e.g. sentiment for __post_type = \"link\"__*\n\n\n```R\nntucLink \u003c- subset(ntucCommentsClean, post_type == \"link\") \nhist(ntucLink$sentiment, xlab = \"Sentiment Score\", main = \"Sentiment Histogram of NTUC Link Posts\")\ntable(ntucLink$sentiment_polar)\nmean(ntucLink$sentiment)\nplot(table(ntucLink$sentiment_polar),ylab=\"Frequency\")\n```\n\n\n    \n    Negative  Neutral Positive \n         159      908      522 \n\n\n\n0.351793580868471\n\n\n\n![png](images/output_60_2.png)\n\n\n\n![png](images/output_60_3.png)\n\n\n### Text Frequency Analysis ###\n\n21) Installing required packages for Text Analysis\n\n\n```R\nNeeded = c(\"tm\", \"SnowballC\", \"RColorBrewer\", \"wordcloud\")  \ninstall.packages(Needed, dependencies=TRUE)\n```\n\n    Installing packages into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    Warning message:\n    \"dependencies 'Rcampdf', 'Rgraphviz', 'tm.lexicon.GeneralInquirer' are not available\"also installing the dependency 'Rpoppler'\n    \n    Warning message in install.packages(Needed, dependencies = TRUE):\n    \"installation of package 'Rpoppler' had non-zero exit status\"\n\n22) Creating corpus, lower case transformation, removing of punctuation / numbers and stopwords\n\n\n```R\nlibrary(tm)\n# create corpus\ncorpus = Corpus(VectorSource(ntucCommentsClean$comments_message))\n# Conversion to lower case\ncorpus = tm_map(corpus, content_transformer(tolower)) \n# Removal of punctuation\ncorpus = tm_map(corpus, removePunctuation)\n# Removal of numbers\ncorpus = tm_map(corpus, removeNumbers)\n# Removal of stopwords\ncorpus = tm_map(corpus, removeWords, stopwords(\"english\"))\n```\n\n    Loading required package: NLP\n\n\n23) Generating wordcloud\n\n\n```R\nlibrary(wordcloud)\nwordcloud(corpus, random.order = F, min.freq=2, max.words=250,\n          colors = brewer.pal(8, \"Dark2\"))\n```\n\n\n![png](images/output_66_0.png)\n\n\n### Page Trend Analysis ###\n\n24) Creating function for aggregation matric\n\n\n```R\naggregate.matric \u003c- function(metric){\n  m \u003c- aggregate(ntucPosts[[paste0(metric, \"_count\")]],\n                 list(month = ntucPosts$month), \n                 mean)\n  m$month \u003c- as.Date(paste0(m$month, \"-15\"))\n  m$metric \u003c- metric\n  return(m)\n}\n```\n\n\n```R\nntucPosts$timestamp \u003c- format.facebook.date(ntucPosts$created_time)\nntucPosts$month \u003c- format(ntucPosts$timestamp, \"%Y-%m\")\n```\n\n\n```R\ndf.list \u003c- lapply(c(\"likes\", \"comments\", \"shares\"), aggregate.matric)\ndf \u003c- do.call(rbind, df.list)\n```\n\n25) Installing ggplot2 package\n\n\n```R\ninstall.packages(\"ggplot2\")\ninstall.packages(\"scales\")\nlibrary(ggplot2)\nlibrary(scales)\n```\n\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    \n    Attaching package: 'ggplot2'\n    \n    The following object is masked from 'package:NLP':\n    \n        annotate\n    \n\n\n26) Creating the trend plot\n\n\n```R\nggplot(df, aes(x = month, y = x, group = metric)) +\n  geom_line(aes(color = metric)) +\n  scale_x_date(date_breaks = \"years\", labels = date_format(\"%Y\")) +\n  scale_y_log10(\"Average count per post\", breaks = c(100, 500, 1000)) +\n  theme_bw() +\n  theme(axis.title.x = element_blank(), axis.text.x=element_text(angle = -90, hjust = 0)) +\n  ggtitle(\"NTUC Page CTR Performance\") \n```\n\n\n\n\n![png](images/output_74_1.png)\n\n\n### Organisation FB Post Day/Time Heatmap ###\n\n27) Installing lubridate package to work with Date and Time\n\n\n```R\ninstall.packages(\"lubridate\")\nlibrary(lubridate)\n```\n\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    \n    Attaching package: 'lubridate'\n    \n    The following object is masked from 'package:base':\n    \n        date\n    \n\n\n28) Creating the relevant date and time variables\n\n\n```R\nntucPosts$datetime \u003c- format.facebook.date(ntucPosts$created_time)\nntucPosts$dayweek \u003c- wday(ntucPosts$datetime, label=T)\nntucPosts$dayint \u003c- wday(ntucPosts$datetime)\nntucPosts$sghour \u003c- with_tz(ntucPosts$datetime, \"Asia/Singapore\")\nntucPosts$hour \u003c- hour(ntucPosts$sghour)\n```\n\n29) Installing heatmap package\n\n\n```R\ninstall.packages(\"d3heatmap\")\nlibrary(d3heatmap)\n```\n\n    Installing package into '/home/nbcommon/R'\n    (as 'lib' is unspecified)\n    also installing the dependency 'htmlwidgets'\n    \n\n\n30) Creating heatmap of posts by day and time\n\n\n```R\nheatmapFrame \u003c- matrix(0,nrow=24,ncol=7);\nrownames(heatmapFrame) \u003c- 0:23\ncolnames(heatmapFrame) \u003c- c(\"Sun\", \"Mon\", \"Tue\", \"Wed\", \"Thu\", \"Fri\", \"Sat\")\n\nfor (i in 1:24) {\n  for (j in 1:7) {\n    heatmapFrame[i,j] \u003c- nrow(subset(ntucPosts,dayint==j \u0026 hour==i-1))\n  }\n}\nd3heatmap(heatmapFrame, scale = \"column\",dendrogram = \"none\", color = scales::col_quantile(\"Blues\", NULL, 5))\n```\n\n![gif](images/HeatMap_ntucPosts.gif)\n\nYou can amend the codes to create and show the day and time heatmap for customers' comments using the above provided code \n\n### Organisation Posts Frequency ###\n\n31) Finally, let's look at the posts frequency by the organisation\n\n\n```R\ntime.interval \u003c- min(ntucPosts$datetime) %--% max(ntucPosts$datetime)\nsampledays \u003c- round(time.interval / ddays(1))\n\nntucPostFreq \u003c- sampledays / nrow(ntucPosts)\n```\n\n\n```R\nntucPostFreq\n```\n\n\n6.57096643752527\n\n\nIt seems that the organisation would create a new post every __6.5__ days\n\n\n### Future Improvements ###\n\nFurther analysis such as:\n- comparision of organisation's post timing and customers' active timing can be performed to determine if the organisation is posting contents at the right time\n- topic analysis (via the use of LDA) on customers' contents can be perform to determine what topics/subjects customers are talking about or are interested in\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwokhing%2Frfacebook-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwokhing%2Frfacebook-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwokhing%2Frfacebook-demo/lists"}