{"id":22211631,"url":"https://github.com/kwokhing/exploratory-data-analysis-on-smrt-tweets","last_synced_at":"2025-07-27T11:32:28.505Z","repository":{"id":189295765,"uuid":"135796018","full_name":"KwokHing/Exploratory-Data-Analysis-on-SMRT-Tweets","owner":"KwokHing","description":"Demo on performing exploratory data analysis (EDA) on train service disruptions based on scrapped (user generated contents) tweets from the train operator's (SMRT) twitter account","archived":false,"fork":false,"pushed_at":"2019-12-05T11:23:57.000Z","size":1313,"stargazers_count":3,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-08-19T08:42:07.281Z","etag":null,"topics":["data-analysis","data-cleaning","data-collection","data-preparation","exploratory-data-analysis","exploratory-data-visualizations","folium","geospatial-data","leaflet-map","python","python3","regex","scraping","selenium","selenium-python","social-media","text-processing","user-generated-content","web-scraping","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KwokHing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-06-02T07:09:50.000Z","updated_at":"2023-08-19T08:42:08.649Z","dependencies_parsed_at":null,"dependency_job_id":"2ec15cbd-9f49-4321-bc5f-66206c3886d1","html_url":"https://github.com/KwokHing/Exploratory-Data-Analysis-on-SMRT-Tweets","commit_stats":null,"previous_names":["kwokhing/exploratory-data-analysis-on-smrt-tweets"],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FExploratory-Data-Analysis-on-SMRT-Tweets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FExploratory-Data-Analysis-on-SMRT-Tweets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FExploratory-Data-Analysis-on-SMRT-Tweets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KwokHing%2FExploratory-Data-Analysis-on-SMRT-Tweets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KwokHing","download_url":"https://codeload.github.com/KwokHing/Exploratory-Data-Analysis-on-SMRT-Tweets/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227801443,"owners_count":17822018,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","data-cleaning","data-collection","data-preparation","exploratory-data-analysis","exploratory-data-visualizations","folium","geospatial-data","leaflet-map","python","python3","regex","scraping","selenium","selenium-python","social-media","text-processing","user-generated-content","web-scraping","webscraping"],"created_at":"2024-12-02T20:35:45.276Z","updated_at":"2024-12-02T20:35:45.690Z","avatar_url":"https://github.com/KwokHing.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Project Overview\n\nThis demo will provide a brief introduction in performing a rudimentary analysis on train service disruptions in Singapore. Data scrapped are from the SMRT's twitter account and wikipedia containing the relevant train stations information such as name and code \n\n- scraping of data from website (twitter) using Selenium \n- scraping of tabular data from website (wikipedia) using Xpath\n- exploratory data analysis (EDA) on the scrapped data\n- data cleaning, data prepration and processing \n- loading of .shp (shape) files into Python\n- geospatial analysis on frequency of service disruptions using Folium \u0026 Leaflet\n\nThere are two primary methods of extracting data from the SMRT tweets (twitter website). The first method was to use the provided twitter API for getting SMRT tweets, while the second method was to scrap information out from the HTML codes on the official SMRT twitter website (https://twitter.com/smrt_singapore). Due to limitation on the number of tweets the twitter's API could be pulled and an expected substantial number of SMRT tweets involved (approximately 4000 tweets), the latter method was employed to overcome twitter API's rate limitation.\n\nThis codes are submitted as a web scraping project for NTU's WKW H6752 - Data Extraction Techniques module.\n\n![png](images/output_28.png)\n\n## Getting started\nOpen `1_scrape_tweets.ipynb` and `2_geospatial_EDA_tweets.ipynb` on a jupyter notebook environment, or Google colab. The notebook consists of further technical details.\n\n- `1_scrape_tweets.ipynb` shows the steps taken to scrape tweets from twitter using Selenium\n- `2_geospatial_EDA_tweets.ipynb`shows the steps taken to generate a heat map on the frequency of train breakdowns \n\n## Improvements\nTo perform scraping and generate SBS train breakdowns heat map as well.\n\n\u003c!---\n# Exploratory Data Analysis on service disruption based on tweets\n\n\n## Extraction of SMRT Tweets using Selenium Web Driver\n\n```python\nimport time\nfrom selenium import webdriver\nfrom bs4 import BeautifulSoup as bs\n\n### Use either Firefox or Chrome to load the webpages\nbrowser = webdriver.Firefox()\n#browser = webdriver.Chrome()\n\n### URL to scrap\nurl = \"https://twitter.com/smrt_singapore?lang=en\"\n\nbrowser.get(url)\n```\n\n\n```python\n### Selenium script to auto scroll to the end of page, with interval of 3 seconds between scroll.\n### For purpose of testing, skip this part of code.\n\n#from selenium import webdriver\n#browser = webdriver.Firefox()\n#browser.get(\"https://twitter.com/smrt_singapore?lang=en\")\nlenOfPage = browser.execute_script(\"window.scrollTo(0, document.body.scrollHeight); \\\n                                var lenOfPage=document.body.scrollHeight;return lenOfPage;\")\nmatch=False\nwhile(match==False):\n    lastCount = lenOfPage\n    time.sleep(3)    # pause for 3 seconds before next scroll\n    lenOfPage = browser.execute_script(\"window.scrollTo(0, document.body.scrollHeight);\\\n                                var lenOfPage=document.body.scrollHeight;return lenOfPage;\")\n    if lastCount==lenOfPage:\n        match=True\n```\n\n\n```python\n### Scrap the HTML after fully load the web page and load into BeautifulSoup.\n\nsource_data = browser.page_source\nbs_data = bs(source_data, \"lxml\")\n```\n\n\n```python\n### Verify data extraction for one tweet is correct.\n\narticle_info = bs_data.find(\"p\", {\"class\": 'TweetTextSize TweetTextSize--normal js-tweet-text tweet-text'})\nprint(article_info.text)\n\narticle_info = bs_data.find('a', {'class': 'tweet-timestamp js-permalink js-nav js-tooltip' })['title']\nprint (article_info)\n```\n\n    All 35 EWL stns from Tuas Link to Pasir Ris \u0026 from Tanah Merah to Changi Airport will have shorter operational hrs every weekend from 6 to 29 April. Shuttle bus svcs will be available between the stns. Plan ahead \u0026 take other train lines and bus svcs. https://bit.ly/2Gvq1Br pic.twitter.com/u47bY0op8M\n    2:38 AM - 2 Apr 2018\n\n\n\n```python\n### Verify REGEX pattern extracts correctly\n\nimport re\n\n# Sample tweet\ns = '[NSL] CLEARED: Train svc from #YewTee to #JurongEast is running normally now.'\n\nrepatt = r'[\\bbetween\\b|\\bbtwn\\b|\\bfrom\\b] \\#(\\w*)[\\s\\w]*[and|to|\u0026]*[\\s]*\\#[\\s]*(\\w*)'\n#repatt = r\"\\bCLEARED\\b|\\bcleared\\b|\\bCleared\\b|\\bhave resume\\b|\\bhave resumed\\b|\\bhas resumed\\b|\\bservice resumed\\b|\\bnormal\\b|\\bnormally\\b|\\bceased\\b|\\bcease\\b\"    # indication that fault is cleared or services is resumed\n#repatt = r'\\bEWL\\b|\\bNSL\\b|\\bCCL\\b'\n#repatt  = r'[F|f][ree][\\w\\s\\d\\\u0026\\#]*end'\n\nextract = re.search(repatt, s)\nif extract:  \n    print (\"From: %s  To: %s\\n\" % (extract.group(1), extract.group(2)))\n    #print (extract.group(0))\nelse:\n    print(\"Extraction Error\")\n```\n\n    From: YewTee  To: JurongEast\n\n\n\n\n```python\n### Actual script to extract all the relevant data (tweets, date/time, Station Names, MRT Status)\n\nfrom datetime import datetime\nimport re\n\narticle_infoAll = bs_data.find_all(\"div\", {\"class\": 'content'})\n\nsmrt_tweet_data = []  # list container to hold all extracted SMRT tweet\n\n### Regex extraction pattern\n### ========================\nre_mrtline = r'^[\\[]*(\\bNSL\\b|\\bEWL\\b|\\bCCL\\b|\\bBPLRT\\b|\\bDTL\\b)'\nre_mrtstatus = r'\\bCLEARED\\b|\\bcleared\\b|\\bCleared\\b|\\bhave resume\\b|\\bhave resumed\\b|\\bhas resumed\\b|\\bservice resumed\\b|\\bnormal\\b|\\bnormally\\b|\\bceased\\b|\\bcease\\b'    # indication that fault is cleared or services is resumed\nre_patt = r'[F|f][ree][\\w\\s\\d\\\u0026\\#]*ended'\nre_stn = r'[\\bbetween\\b|\\bbtwn\\b|\\bfrom\\b] \\#(\\w*)[\\s\\w]*[and|to|\u0026]*[\\s]*\\#[\\s]*(\\w*)'\n\nfor item in article_infoAll:\n\n    tweet_row = [] # list container to hold extracted tweet info for each SMRT tweet\n\n    try:\n\n        ### Extract tweet body from html\n        ### ============================\n        tweet_body = item.find(\"p\", {\"class\": 'TweetTextSize TweetTextSize--normal js-tweet-text tweet-text'})\n        tweet_row.append(tweet_body.text)\n\n        ### Extract date and time from the html\n        ### ===================================\n        tweet_datetime = item.find('a', {'class': 'tweet-timestamp js-permalink js-nav js-tooltip' })['title']\n        tweet_row.append(tweet_datetime)\n\n        ### Convert extract tweet_datetime into datetime object so that it can be calculated\n        ### =======================================\n        date = datetime.strptime(tweet_datetime, '%I:%M %p - %d %b %Y')\n        print(\"Converted date/time: %s\" % date)\n        tweet_row.append(date)        \n\n        ### ================================================================================\n        ### If tweet starts with [xxx], it indicate the tweet had MRT status information\n        ### Extract tweet using regex to determine status type, i.e.'cleared', 'update', 'ok'\n        ### ================================================================================\n\n        ### Extract and check if tweet starts with [xxx]\n        ### ============================================\n        mrtline = re.search(re_mrtline, tweet_body.text)\n\n        if mrtline:  # if the tweet starts with [XXX], then it is a status tweet\n            ### Extract MRT Line inside []\n            ### ==========================\n            print(\"Extracted MRT Line: \" + mrtline.group(1))\n            tweet_row.append(mrtline.group(1))\n\n            ### Extract MRT Line status based on keywords\n            ### =========================================\n            mrtstatus = re.search(re_mrtstatus, tweet_body.text)\n            if mrtstatus:   # if is status is stated 'cleared' in tweeter\n                print(\"Status: Cleared\")\n                print(\"Extracted tweet: %s\\n\" % tweet_body.text)\n                tweet_row.append('cleared')\n            else:\n                patt=re.search(re_patt, tweet_body.text)\n                if patt:    # check if there are other indicators that inferred as 'cleared'. Such as 'Free ... ended'.\n                    print(\"Status: Cleared\")\n                    print(\"Extracted tweet: %s\\n\" % tweet_body.text)\n                    tweet_row.append('cleared')\n                else:       # if is a status tweet but no 'cleared' status, then the tweet will be a disruption status.\n                    print(\"Status: Update\")\n                    print(\"Extracted tweet: %s\" % tweet_body.text.replace('\\n','')) # some tweet has newline\n                    tweet_row.append('update')\n\n                    ### Extract station names\n                    ### =====================\n                    stn = re.search(re_stn, tweet_body.text)\n                    if extract:\n                        print (\"From: %s  To: %s\\n\" % (stn.group(1), stn.group(2)))  \n                        tweet_row.append(stn.group(1))   # from station\n                        tweet_row.append(stn.group(2))   # to station\n\n        else:    # if no mrt line mention in [xxx] format, then it is a normal information tweets.\n            tweet_row.append('None')\n            tweet_row.append('ok')\n            print('Status: OK')\n            print('Extracted tweet: %s\\n' % tweet_body.text)   # for checking only\n\n        ### Store the extracted tweet info into the list container\n        smrt_tweet_data.append(tweet_row)\n\n    except:\n        print(\"Extraction Error!\\n\")\n        continue\n```\n\n    Converted date/time: 2018-04-02 02:38:00\n    Status: OK\n    Extracted tweet: All 35 EWL stns from Tuas Link to Pasir Ris \u0026 from Tanah Merah to Changi Airport will have shorter operational hrs every weekend from 6 to 29 April. Shuttle bus svcs will be available between the stns. Plan ahead \u0026 take other train lines and bus svcs. https://bit.ly/2Gvq1Br pic.twitter.com/u47bY0op8M\n\n    Converted date/time: 2018-03-08 00:53:00\n    Status: OK\n    Extracted tweet: Gentle reminder that svc improvement has been made to shuttle bus svc 6 for better connectivity. Shuttle bus svc 6 will now be extended to ply from Raffles Place to Paya Lebar stns (both bounds). http://bit.ly/2oyza4d pic.twitter.com/22CPlP1JYR\n\n    Converted date/time: 2018-03-08 00:44:00\n    Status: OK\n    Extracted tweet: All EWL stns from Tuas Link to Pasir Ris \u0026 from Tanah Merah to Changi Airport will have shorter operational hrs every weekend \u0026 selected weekdays from 2 Mar to 1 Apr.Shuttle bus svcs will be avail btwn the stns.Plan ahead \u0026 take other train lines/bus svcs. http://bit.ly/2FyKFkd pic.twitter.com/qj7qMVo9LJ\n\n    Converted date/time: 2018-03-03 20:02:00\n    Status: OK\n    Extracted tweet: Here’s a sneak peek of our SMRT’s new uniforms, fresh from the oven! It’s been nine years since the last transformation and our staff are excited to serve you with this fresh look starting tomorrow! Look out for them at all our MRT stations and say hi!pic.twitter.com/FDi5WbZnLL\n\n\n\n\n\n```python\n### Tabulate all the extracted data into table using panda\nimport pandas as pd\n\ndf = pd.DataFrame(smrt_tweet_data, columns=['Tweet', 'Extracted Date/Time', 'Date/Time', 'MRT_Line', 'Status', 'From_Stn', 'To_Stn'])\n\ndf.head(10)\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eAll 35 EWL stns from Tuas Link to Pasir Ris \u0026amp; ...\u003c/td\u003e\n      \u003ctd\u003e2:38 AM - 2 Apr 2018\u003c/td\u003e\n      \u003ctd\u003e2018-04-02 02:38:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eGentle reminder that svc improvement has been ...\u003c/td\u003e\n      \u003ctd\u003e12:53 AM - 8 Mar 2018\u003c/td\u003e\n      \u003ctd\u003e2018-03-08 00:53:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eAll EWL stns from Tuas Link to Pasir Ris \u0026amp; fro...\u003c/td\u003e\n      \u003ctd\u003e12:44 AM - 8 Mar 2018\u003c/td\u003e\n      \u003ctd\u003e2018-03-08 00:44:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eHere’s a sneak peek of our SMRT’s new uniforms...\u003c/td\u003e\n      \u003ctd\u003e8:02 PM - 3 Mar 2018\u003c/td\u003e\n      \u003ctd\u003e2018-03-03 20:02:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eAll EWL stns from Tuas Link to Pasir Ris \u0026amp; fro...\u003c/td\u003e\n      \u003ctd\u003e3:19 AM - 1 Mar 2018\u003c/td\u003e\n      \u003ctd\u003e2018-03-01 03:19:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e5\u003c/th\u003e\n      \u003ctd\u003eTrack improvement works in the North-South Lin...\u003c/td\u003e\n      \u003ctd\u003e3:26 AM - 19 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-19 03:26:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e6\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train svcs from #AngMoKio to #R...\u003c/td\u003e\n      \u003ctd\u003e6:17 PM - 18 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-18 18:17:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e7\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: Due to on going track improvemen...\u003c/td\u003e\n      \u003ctd\u003e6:01 PM - 18 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-18 18:01:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRafflesPlace\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e8\u003c/th\u003e\n      \u003ctd\u003e[NSL] : Due to on going track improvement work...\u003c/td\u003e\n      \u003ctd\u003e4:29 PM - 18 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-18 16:29:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRafflesPlace\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e9\u003c/th\u003e\n      \u003ctd\u003eWishing everyone a Happy and Prosperous Lunar ...\u003c/td\u003e\n      \u003ctd\u003e8:01 AM - 15 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-15 08:01:00\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eok\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Number of SMRT tweets extracted\ndf.shape\n```\n\n\n\n\n    (710, 7)\n\n\n\n\n```python\n### Save all extracted tweets into csv\ndf.to_csv(\"smrt_tweet_extract.csv\")\n```\n\n\n```python\n### Clean up the data\n### Remove 'ok' status to focus on tweet with status\n\ndf = df[df.Status != 'ok']\n\ndf.head(10)   \n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e6\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train svcs from #AngMoKio to #R...\u003c/td\u003e\n      \u003ctd\u003e6:17 PM - 18 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-18 18:17:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e7\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: Due to on going track improvemen...\u003c/td\u003e\n      \u003ctd\u003e6:01 PM - 18 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-18 18:01:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRafflesPlace\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e8\u003c/th\u003e\n      \u003ctd\u003e[NSL] : Due to on going track improvement work...\u003c/td\u003e\n      \u003ctd\u003e4:29 PM - 18 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-18 16:29:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRafflesPlace\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e10\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train svcs from #AngMoKio to #R...\u003c/td\u003e\n      \u003ctd\u003e6:07 PM - 13 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-13 18:07:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e11\u003c/th\u003e\n      \u003ctd\u003e[NSL Update]: Due to maintenance work, trains ...\u003c/td\u003e\n      \u003ctd\u003e5:04 PM - 13 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-13 17:04:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRaffles\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e12\u003c/th\u003e\n      \u003ctd\u003e[NSL]: Due to maintenance work, south-bound tr...\u003c/td\u003e\n      \u003ctd\u003e1:45 PM - 13 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-13 13:45:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRafflesPlace\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e13\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: Fault cleared, train svcs from #...\u003c/td\u003e\n      \u003ctd\u003e6:38 PM - 6 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-06 18:38:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e14\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: Due to on going track improvemen...\u003c/td\u003e\n      \u003ctd\u003e6:04 PM - 6 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-06 18:04:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n      \u003ctd\u003eRafflesPlace\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e15\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: Fault cleared, train svcs are pr...\u003c/td\u003e\n      \u003ctd\u003e5:18 PM - 6 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-06 17:18:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e16\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: Fault cleared, train svcs are pr...\u003c/td\u003e\n      \u003ctd\u003e4:46 PM - 6 Feb 2018\u003c/td\u003e\n      \u003ctd\u003e2018-02-06 16:46:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Number of tweets with MRT Line status update\ndf.shape\n```\n\n\n\n\n    (583, 7)\n\n\n\n\n```python\n### Sort dataframe according to MRT line (ascending) and Date/time (descending)\n### This will help in the calculation of disruption duration\ndf.sort_values(['MRT_Line', 'Date/Time'], ascending=[True, False], inplace=True)\n\ndf.head(10)  \n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e20\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Fault cleared. Normal train services a...\u003c/td\u003e\n      \u003ctd\u003e12:54 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:54:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e21\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]: No train service between #ChoaChuKang...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e22\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular and bridging bus...\u003c/td\u003e\n      \u003ctd\u003e3:09 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 03:09:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e23\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services on the entire B...\u003c/td\u003e\n      \u003ctd\u003e2:30 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:30:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e24\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Service B on the BPLRT inner l...\u003c/td\u003e\n      \u003ctd\u003e2:04 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:04:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e165\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free bus and bridging bus ser...\u003c/td\u003e\n      \u003ctd\u003e1:45 AM - 9 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-09 01:45:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e166\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Normal service on the BPLRT h...\u003c/td\u003e\n      \u003ctd\u003e1:25 AM - 9 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-09 01:25:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e203\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nCLEARED: Train services on the Servic...\u003c/td\u003e\n      \u003ctd\u003e11:56 PM - 11 Aug 2017\u003c/td\u003e\n      \u003ctd\u003e2017-08-11 23:56:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e205\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nCLEARED: Train services on the Servic...\u003c/td\u003e\n      \u003ctd\u003e2:15 AM - 27 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-27 02:15:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e206\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nUPDATE: Train fault between #Senja an...\u003c/td\u003e\n      \u003ctd\u003e2:05 AM - 27 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-27 02:05:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eSenja\u003c/td\u003e\n      \u003ctd\u003eJelapang\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Re-index the dataframe to prepare to calculate disruption duration\ndf = df.reset_index(drop=True)\ndf.head(10)\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Fault cleared. Normal train services a...\u003c/td\u003e\n      \u003ctd\u003e12:54 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:54:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]: No train service between #ChoaChuKang...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular and bridging bus...\u003c/td\u003e\n      \u003ctd\u003e3:09 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 03:09:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services on the entire B...\u003c/td\u003e\n      \u003ctd\u003e2:30 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:30:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Service B on the BPLRT inner l...\u003c/td\u003e\n      \u003ctd\u003e2:04 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:04:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e5\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free bus and bridging bus ser...\u003c/td\u003e\n      \u003ctd\u003e1:45 AM - 9 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-09 01:45:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e6\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Normal service on the BPLRT h...\u003c/td\u003e\n      \u003ctd\u003e1:25 AM - 9 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-09 01:25:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e7\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nCLEARED: Train services on the Servic...\u003c/td\u003e\n      \u003ctd\u003e11:56 PM - 11 Aug 2017\u003c/td\u003e\n      \u003ctd\u003e2017-08-11 23:56:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e8\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nCLEARED: Train services on the Servic...\u003c/td\u003e\n      \u003ctd\u003e2:15 AM - 27 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-27 02:15:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e9\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nUPDATE: Train fault between #Senja an...\u003c/td\u003e\n      \u003ctd\u003e2:05 AM - 27 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-27 02:05:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eSenja\u003c/td\u003e\n      \u003ctd\u003eJelapang\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Calculate the disruption duration and convert into minutes.\n### Taking the [Clear time] - [1st Fault Reported]\n### Extract the affected train stations and placed it together with the disruption duration\n\ncleared_found=False\ncurr_status=''\nlast_datetime=''\ncurr_datetime=''\ncleared_datetime=''\ncurr_mrtlinr=''\n\ncurr_fromstn=''\ncurr_tostn=''\nlast_fromstn=''\nlast_tostn=''\n\ndur_list = []\nto_list=[]\nfr_list=[]\n\nfor index in range(len(df)):\n    curr_status = df.Status[index]\n    curr_datetime = df['Date/Time'][index]\n\n    curr_fromstn = df['From_Stn'][index]\n    curr_tostn = df['To_Stn'][index]\n\n    if index == 0:\n        curr_mrtline = df.MRT_Line[index]\n        last_mrtline = df.MRT_Line[index]\n    else:\n        curr_mrtline = df.MRT_Line[index]\n\n    dur_list.append('0')  # default duration = 0 minutes\n\n    to_list.append('None')\n    fr_list.append('None')\n\n    if curr_status=='cleared' and cleared_found==False:\n        cleared_datetime = curr_datetime\n        cleared_index = index              # remember the index so that the fault duration can be updated\n        cleared_found = True\n\n    elif cleared_found==True and (curr_status=='cleared' or curr_status=='ok'):\n        #datetime_diff = cleared_datetime - last_datetime    # calculate the time difference\n        datetime_diff = (cleared_datetime - last_datetime).total_seconds()/60    # calculate the time difference and convert into minutes\n        print(\"List index position          : %s\" % cleared_index)\n        print(\"Disruption Cleared Time      : %s\" % cleared_datetime)\n        print(\"1st Disruption Reported Time : %s\" % last_datetime)\n        print(\"Disruption Duration (minutes): %s\" % datetime_diff)\n        print(\"From Station                 : %s\" % last_fromstn)\n        print(\"To Station                   : %s\\n\" % last_tostn)\n\n        dur_list[cleared_index]=datetime_diff     # store the fault duration time in the list\n        fr_list[cleared_index]=last_fromstn       # store the from station in the list\n        to_list[cleared_index]=last_tostn         # store the to station in the list\n\n        cleared_found=False        \n\n        if curr_status=='cleared':\n            cleared_datetime = curr_datetime\n            cleared_index = index\n            cleared_found = True\n\n    last_datetime = curr_datetime\n    last_mrtline = curr_mrtline\n\n    last_fromstn = curr_fromstn\n    last_tostn = curr_tostn\n\n```\n\n    List index position          : 0\n    Disruption Cleared Time      : 2018-01-18 00:54:00\n    1st Disruption Reported Time : 2018-01-18 00:12:00\n    Disruption Duration (minutes): 42.0\n    From Station                 : ChoaChuKang\n    To Station                   : Phoenix\n\n    List index position          : 2\n    Disruption Cleared Time      : 2018-01-12 03:09:00\n    1st Disruption Reported Time : 2018-01-12 03:09:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 3\n    Disruption Cleared Time      : 2018-01-12 02:30:00\n    1st Disruption Reported Time : 2018-01-12 02:30:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 4\n    Disruption Cleared Time      : 2018-01-12 02:04:00\n    1st Disruption Reported Time : 2018-01-12 02:04:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 5\n    Disruption Cleared Time      : 2017-09-09 01:45:00\n    1st Disruption Reported Time : 2017-09-09 01:45:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 6\n    Disruption Cleared Time      : 2017-09-09 01:25:00\n    1st Disruption Reported Time : 2017-09-09 01:25:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 7\n    Disruption Cleared Time      : 2017-08-11 23:56:00\n    1st Disruption Reported Time : 2017-08-11 23:56:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 8\n    Disruption Cleared Time      : 2017-07-27 02:15:00\n    1st Disruption Reported Time : 2017-07-27 02:05:00\n    Disruption Duration (minutes): 10.0\n    From Station                 : Senja\n    To Station                   : Jelapang\n\n    List index position          : 10\n    Disruption Cleared Time      : 2017-07-19 23:03:00\n    1st Disruption Reported Time : 2017-07-19 23:03:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 11\n    Disruption Cleared Time      : 2017-07-19 22:58:00\n    1st Disruption Reported Time : 2017-07-19 22:45:00\n    Disruption Duration (minutes): 13.0\n    From Station                 : ChoaChuKang\n    To Station                   : BukitPanjang\n\n    List index position          : 13\n    Disruption Cleared Time      : 2017-03-28 07:02:00\n    1st Disruption Reported Time : 2017-03-28 07:02:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 14\n    Disruption Cleared Time      : 2017-03-28 06:28:00\n    1st Disruption Reported Time : 2017-03-28 06:28:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 15\n    Disruption Cleared Time      : 2017-02-10 00:12:00\n    1st Disruption Reported Time : 2017-02-10 00:12:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 16\n    Disruption Cleared Time      : 2017-02-09 23:43:00\n    1st Disruption Reported Time : 2017-02-09 23:43:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 17\n    Disruption Cleared Time      : 2016-10-26 04:37:00\n    1st Disruption Reported Time : 2016-10-26 04:37:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 18\n    Disruption Cleared Time      : 2016-10-26 04:32:00\n    1st Disruption Reported Time : 2016-10-26 04:32:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 19\n    Disruption Cleared Time      : 2016-10-26 04:12:00\n    1st Disruption Reported Time : 2016-10-26 02:51:00\n    Disruption Duration (minutes): 81.0\n    From Station                 : BukitPanjang\n    To Station                   : Phoenix\n\n    List index position          : 21\n    Disruption Cleared Time      : 2016-09-28 14:20:00\n    1st Disruption Reported Time : 2016-09-28 14:20:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 22\n    Disruption Cleared Time      : 2016-09-28 00:36:00\n    1st Disruption Reported Time : 2016-09-28 00:36:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 23\n    Disruption Cleared Time      : 2016-09-27 22:53:00\n    1st Disruption Reported Time : 2016-09-27 19:56:00\n    Disruption Duration (minutes): 177.0\n    From Station                 : ChoaChuKang\n    To Station                   : BukitPanjang\n\n    List index position          : 26\n    Disruption Cleared Time      : 2016-09-27 18:52:00\n    1st Disruption Reported Time : 2016-09-27 18:52:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 27\n    Disruption Cleared Time      : 2016-09-27 09:42:00\n    1st Disruption Reported Time : 2016-09-27 09:42:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 28\n    Disruption Cleared Time      : 2016-09-27 09:27:00\n    1st Disruption Reported Time : 2016-09-27 07:55:00\n    Disruption Duration (minutes): 92.0\n    From Station                 : ChoaChuKang\n    To Station                   : BukitPanjang\n\n    List index position          : 30\n    Disruption Cleared Time      : 2016-04-25 07:16:00\n    1st Disruption Reported Time : 2016-04-25 07:16:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 31\n    Disruption Cleared Time      : 2016-04-25 07:12:00\n    1st Disruption Reported Time : 2016-04-25 07:12:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 32\n    Disruption Cleared Time      : 2016-04-25 06:52:00\n    1st Disruption Reported Time : 2016-04-25 06:52:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 33\n    Disruption Cleared Time      : 2017-11-14 17:46:00\n    1st Disruption Reported Time : 2017-11-14 17:46:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 34\n    Disruption Cleared Time      : 2017-11-14 16:34:00\n    1st Disruption Reported Time : 2017-11-14 14:28:00\n    Disruption Duration (minutes): 126.00000000000001\n    From Station                 : BotanicGardens\n    To Station                   : HawParVilla\n\n    List index position          : 41\n    Disruption Cleared Time      : 2017-09-10 19:27:00\n    1st Disruption Reported Time : 2017-09-10 18:25:00\n    Disruption Duration (minutes): 62.00000000000001\n    From Station                 : PayaLebar\n    To Station                   : BuonaVista\n\n    List index position          : 46\n    Disruption Cleared Time      : 2017-07-21 07:28:00\n    1st Disruption Reported Time : 2017-07-21 07:08:00\n    Disruption Duration (minutes): 20.0\n    From Station                 : Caldecott\n    To Station                   : HarbourFront\n\n    List index position          : 49\n    Disruption Cleared Time      : 2017-02-23 15:43:00\n    1st Disruption Reported Time : 2017-02-23 15:32:00\n    Disruption Duration (minutes): 11.0\n    From Station                 : bishan\n    To Station                   : HarbourFront\n\n    List index position          : 51\n    Disruption Cleared Time      : 2016-11-01 20:30:00\n    1st Disruption Reported Time : 2016-11-01 20:30:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 52\n    Disruption Cleared Time      : 2016-11-01 20:24:00\n    1st Disruption Reported Time : 2016-11-01 20:24:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 53\n    Disruption Cleared Time      : 2016-11-01 20:04:00\n    1st Disruption Reported Time : 2016-11-01 20:04:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 54\n    Disruption Cleared Time      : 2016-11-01 19:46:00\n    1st Disruption Reported Time : 2016-11-01 17:09:00\n    Disruption Duration (minutes): 157.0\n    From Station                 : BotanicGardens\n    To Station                   : HarbourFront\n\n    List index position          : 63\n    Disruption Cleared Time      : 2016-11-01 16:39:00\n    1st Disruption Reported Time : 2016-11-01 16:37:00\n    Disruption Duration (minutes): 2.0000000000000004\n    From Station                 : PasirPanjang\n    To Station                   : one\n\n    List index position          : 65\n    Disruption Cleared Time      : 2016-09-19 19:15:00\n    1st Disruption Reported Time : 2016-09-19 19:15:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 66\n    Disruption Cleared Time      : 2016-09-19 19:03:00\n    1st Disruption Reported Time : 2016-09-19 16:13:00\n    Disruption Duration (minutes): 170.0\n    From Station                 : DhobyGhaut\n    To Station                   : MacPherson\n\n    List index position          : 74\n    Disruption Cleared Time      : 2016-09-19 16:06:00\n    1st Disruption Reported Time : 2016-09-19 15:57:00\n    Disruption Duration (minutes): 9.0\n    From Station                 : PayaLebar\n    To Station                   : MacPherson\n\n    List index position          : 76\n    Disruption Cleared Time      : 2016-09-05 06:13:00\n    1st Disruption Reported Time : 2016-09-05 05:22:00\n    Disruption Duration (minutes): 51.0\n    From Station                 : PayaLebar\n    To Station                   : MacPherson\n\n    List index position          : 80\n    Disruption Cleared Time      : 2016-08-31 23:59:00\n    1st Disruption Reported Time : 2016-08-31 23:59:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 81\n    Disruption Cleared Time      : 2016-08-30 21:44:00\n    1st Disruption Reported Time : 2016-08-30 21:44:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 82\n    Disruption Cleared Time      : 2016-04-25 06:31:00\n    1st Disruption Reported Time : 2016-04-25 06:31:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 83\n    Disruption Cleared Time      : 2016-04-25 06:23:00\n    1st Disruption Reported Time : 2016-04-25 06:23:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 84\n    Disruption Cleared Time      : 2018-01-10 14:43:00\n    1st Disruption Reported Time : 2018-01-10 14:43:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 85\n    Disruption Cleared Time      : 2018-01-10 14:33:00\n    1st Disruption Reported Time : 2018-01-10 14:19:00\n    Disruption Duration (minutes): 14.0\n    From Station                 : OutramPark\n    To Station                   : Bugis\n\n    List index position          : 87\n    Disruption Cleared Time      : 2018-01-01 17:02:00\n    1st Disruption Reported Time : 2018-01-01 17:02:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 88\n    Disruption Cleared Time      : 2018-01-01 16:59:00\n    1st Disruption Reported Time : 2018-01-01 15:53:00\n    Disruption Duration (minutes): 66.00000000000001\n    From Station                 : TanahMerah\n    To Station                   : ChangiAirport\n\n    List index position          : 91\n    Disruption Cleared Time      : 2018-01-01 15:17:00\n    1st Disruption Reported Time : 2018-01-01 13:49:00\n    Disruption Duration (minutes): 88.0\n    From Station                 : ChangiAirport\n    To Station                   : TanahMerah\n\n    List index position          : 96\n    Disruption Cleared Time      : 2017-12-06 14:24:00\n    1st Disruption Reported Time : 2017-12-06 14:24:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 97\n    Disruption Cleared Time      : 2017-12-06 14:05:00\n    1st Disruption Reported Time : 2017-11-15 04:29:00\n    Disruption Duration (minutes): 30816.0\n    From Station                 : JooKoon\n    To Station                   : TuasLink\n\n    List index position          : 106\n    Disruption Cleared Time      : 2017-11-15 01:34:00\n    1st Disruption Reported Time : 2017-11-15 01:34:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 107\n    Disruption Cleared Time      : 2017-11-15 01:15:00\n    1st Disruption Reported Time : 2017-11-10 07:15:00\n    Disruption Duration (minutes): 6840.0\n    From Station                 : TiongBahru\n    To Station                   : PasirRis\n\n    List index position          : 120\n    Disruption Cleared Time      : 2017-11-10 06:58:00\n    1st Disruption Reported Time : 2017-11-10 06:58:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 121\n    Disruption Cleared Time      : 2017-11-10 06:37:00\n    1st Disruption Reported Time : 2017-11-10 05:53:00\n    Disruption Duration (minutes): 44.0\n    From Station                 : Bugis\n    To Station                   : Queenstown\n\n    List index position          : 124\n    Disruption Cleared Time      : 2017-11-04 00:34:00\n    1st Disruption Reported Time : 2017-11-04 00:34:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 125\n    Disruption Cleared Time      : 2017-11-04 00:30:00\n    1st Disruption Reported Time : 2017-11-03 23:53:00\n    Disruption Duration (minutes): 37.0\n    From Station                 : Queenstown\n    To Station                   : JurongEast\n\n    List index position          : 129\n    Disruption Cleared Time      : 2017-09-27 17:37:00\n    1st Disruption Reported Time : 2017-09-27 16:15:00\n    Disruption Duration (minutes): 82.0\n    From Station                 : Tampines\n    To Station                   : PasirRis\n\n    List index position          : 132\n    Disruption Cleared Time      : 2017-09-27 15:47:00\n    1st Disruption Reported Time : 2017-09-27 15:47:00\n    Disruption Duration (minutes): 0.0\n    From Station                 : None\n    To Station                   : None\n\n    List index position          : 133\n    Disruption Cleared Time      : 2017-09-27 15:15:00\n    1st Disruption Reported Time : 2017-09-27 14:54:00\n    Disruption Duration (minutes): 21.0\n    From Station                 : TanahMerah\n    To Station                   : PasirRis\n\n\n\n\n\n\n```python\n### Combine the duration, from and to list into existing df panda dataframe\ndf['Duration'] = dur_list\ndf['New_Fr_Stn'] = fr_list\ndf['New_To_Stn'] = to_list\n\ndf.head(70)\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n      \u003cth\u003eDuration\u003c/th\u003e\n      \u003cth\u003eNew_Fr_Stn\u003c/th\u003e\n      \u003cth\u003eNew_To_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Fault cleared. Normal train services a...\u003c/td\u003e\n      \u003ctd\u003e12:54 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:54:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e42\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]: No train service between #ChoaChuKang...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular and bridging bus...\u003c/td\u003e\n      \u003ctd\u003e3:09 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 03:09:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services on the entire B...\u003c/td\u003e\n      \u003ctd\u003e2:30 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:30:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Service B on the BPLRT inner l...\u003c/td\u003e\n      \u003ctd\u003e2:04 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:04:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e5\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free bus and bridging bus ser...\u003c/td\u003e\n      \u003ctd\u003e1:45 AM - 9 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-09 01:45:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e6\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Normal service on the BPLRT h...\u003c/td\u003e\n      \u003ctd\u003e1:25 AM - 9 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-09 01:25:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e7\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nCLEARED: Train services on the Servic...\u003c/td\u003e\n      \u003ctd\u003e11:56 PM - 11 Aug 2017\u003c/td\u003e\n      \u003ctd\u003e2017-08-11 23:56:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e8\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nCLEARED: Train services on the Servic...\u003c/td\u003e\n      \u003ctd\u003e2:15 AM - 27 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-27 02:15:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e10\u003c/td\u003e\n      \u003ctd\u003eSenja\u003c/td\u003e\n      \u003ctd\u003eJelapang\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e9\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]\\nUPDATE: Train fault between #Senja an...\u003c/td\u003e\n      \u003ctd\u003e2:05 AM - 27 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-27 02:05:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eSenja\u003c/td\u003e\n      \u003ctd\u003eJelapang\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e10\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Normal service resumed. Free ...\u003c/td\u003e\n      \u003ctd\u003e11:03 PM - 19 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-19 23:03:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e11\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Train service between #ChoaCh...\u003c/td\u003e\n      \u003ctd\u003e10:58 PM - 19 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-19 22:58:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e13\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e12\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] No train service betwn #ChoaChuKang to...\u003c/td\u003e\n      \u003ctd\u003e10:45 PM - 19 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-19 22:45:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e13\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Update: Train services have resumed. B...\u003c/td\u003e\n      \u003ctd\u003e7:02 AM - 28 Mar 2017\u003c/td\u003e\n      \u003ctd\u003e2017-03-28 07:02:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e14\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Update: Train services have resumed. B...\u003c/td\u003e\n      \u003ctd\u003e6:28 AM - 28 Mar 2017\u003c/td\u003e\n      \u003ctd\u003e2017-03-28 06:28:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e15\u003c/th\u003e\n      \u003ctd\u003e[BPLRT CLEARED] Free Regular and Shuttle Bus S...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 10 Feb 2017\u003c/td\u003e\n      \u003ctd\u003e2017-02-10 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e16\u003c/th\u003e\n      \u003ctd\u003e[BPLRT Update] Normal Train Services have resu...\u003c/td\u003e\n      \u003ctd\u003e11:43 PM - 9 Feb 2017\u003c/td\u003e\n      \u003ctd\u003e2017-02-09 23:43:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e17\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services have resumed. F...\u003c/td\u003e\n      \u003ctd\u003e4:37 AM - 26 Oct 2016\u003c/td\u003e\n      \u003ctd\u003e2016-10-26 04:37:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e18\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services have resumed. F...\u003c/td\u003e\n      \u003ctd\u003e4:32 AM - 26 Oct 2016\u003c/td\u003e\n      \u003ctd\u003e2016-10-26 04:32:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e19\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services have resumed. F...\u003c/td\u003e\n      \u003ctd\u003e4:12 AM - 26 Oct 2016\u003c/td\u003e\n      \u003ctd\u003e2016-10-26 04:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e81\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e20\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Due to a train fault between #BukitPan...\u003c/td\u003e\n      \u003ctd\u003e2:51 AM - 26 Oct 2016\u003c/td\u003e\n      \u003ctd\u003e2016-10-26 02:51:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e21\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Service A and Service B are now runnin...\u003c/td\u003e\n      \u003ctd\u003e2:20 PM - 28 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-28 14:20:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e22\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]Svcs on the inner loop (anticlockwise d...\u003c/td\u003e\n      \u003ctd\u003e12:36 AM - 28 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-28 00:36:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e23\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Svcs on the inner loop (anticlockwise ...\u003c/td\u003e\n      \u003ctd\u003e10:53 PM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 22:53:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e177\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e24\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] No train service btwn #ChoaChuKang and...\u003c/td\u003e\n      \u003ctd\u003e8:25 PM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 20:25:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e25\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] No train service between #ChoaChuKang ...\u003c/td\u003e\n      \u003ctd\u003e7:56 PM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 19:56:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e26\u003c/th\u003e\n      \u003ctd\u003e[BPLRT Update] Free regular bus \u0026amp; free bridgin...\u003c/td\u003e\n      \u003ctd\u003e6:52 PM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 18:52:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e27\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular bus \u0026amp; free bridg...\u003c/td\u003e\n      \u003ctd\u003e9:42 AM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 09:42:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e28\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Train service between #ChoaCh...\u003c/td\u003e\n      \u003ctd\u003e9:27 AM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 09:27:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e92\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e29\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] No train service between #ChoaChuKang ...\u003c/td\u003e\n      \u003ctd\u003e7:55 AM - 27 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-27 07:55:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003eBukitPanjang\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e...\u003c/th\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e40\u003c/th\u003e\n      \u003ctd\u003e[CCL]: Due to a signal  fault trains are movin...\u003c/td\u003e\n      \u003ctd\u003e2:28 PM - 14 Nov 2017\u003c/td\u003e\n      \u003ctd\u003e2017-11-14 14:28:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eHawParVilla\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e41\u003c/th\u003e\n      \u003ctd\u003e[CCL] Update: Train services are running norma...\u003c/td\u003e\n      \u003ctd\u003e7:27 PM - 10 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-10 19:27:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e62\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003eBuonaVista\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e42\u003c/th\u003e\n      \u003ctd\u003e[CCL Update] Pls add 30mins additional travell...\u003c/td\u003e\n      \u003ctd\u003e6:54 PM - 10 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-10 18:54:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003eBuonaVista\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e43\u003c/th\u003e\n      \u003ctd\u003e[CCL Update] Pls add 30mins additional travell...\u003c/td\u003e\n      \u003ctd\u003e6:40 PM - 10 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-10 18:40:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003eBuonaVista\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e44\u003c/th\u003e\n      \u003ctd\u003e[CCL] UPDATE: Free regular bus services availa...\u003c/td\u003e\n      \u003ctd\u003e6:27 PM - 10 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-10 18:27:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003eBuonaVista\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e45\u003c/th\u003e\n      \u003ctd\u003e[CCL] Pls add 15mins additional travelling tim...\u003c/td\u003e\n      \u003ctd\u003e6:25 PM - 10 Sep 2017\u003c/td\u003e\n      \u003ctd\u003e2017-09-10 18:25:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003eBuonaVista\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e46\u003c/th\u003e\n      \u003ctd\u003e[CCL] UPDATE: Normal train service resumed tow...\u003c/td\u003e\n      \u003ctd\u003e7:28 AM - 21 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-21 07:28:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e20\u003c/td\u003e\n      \u003ctd\u003eCaldecott\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e47\u003c/th\u003e\n      \u003ctd\u003e[CCL] UPDATE: Free regular bus services are av...\u003c/td\u003e\n      \u003ctd\u003e7:09 AM - 21 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-21 07:09:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eCaldecott\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e48\u003c/th\u003e\n      \u003ctd\u003e[CCL]: Estimate 15mins additional travelling t...\u003c/td\u003e\n      \u003ctd\u003e7:08 AM - 21 Jul 2017\u003c/td\u003e\n      \u003ctd\u003e2017-07-21 07:08:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eCaldecott\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e49\u003c/th\u003e\n      \u003ctd\u003eCCL: CLEARED: Train Services between #Bishan a...\u003c/td\u003e\n      \u003ctd\u003e3:43 PM - 23 Feb 2017\u003c/td\u003e\n      \u003ctd\u003e2017-02-23 15:43:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e11\u003c/td\u003e\n      \u003ctd\u003ebishan\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e50\u003c/th\u003e\n      \u003ctd\u003eCCL: Estimate 5 mins additional travelling tim...\u003c/td\u003e\n      \u003ctd\u003e3:32 PM - 23 Feb 2017\u003c/td\u003e\n      \u003ctd\u003e2017-02-23 15:32:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ebishan\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e51\u003c/th\u003e\n      \u003ctd\u003e[CCL] Bus bridging and free regular bus svcs h...\u003c/td\u003e\n      \u003ctd\u003e8:30 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 20:30:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e52\u003c/th\u003e\n      \u003ctd\u003e[CCL] Train svcs have resumed. Bus bridging sv...\u003c/td\u003e\n      \u003ctd\u003e8:24 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 20:24:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e53\u003c/th\u003e\n      \u003ctd\u003e[CCL] Train svcs have resumed. Bus bridging sv...\u003c/td\u003e\n      \u003ctd\u003e8:04 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 20:04:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e54\u003c/th\u003e\n      \u003ctd\u003e[CCL] Train svcs have resumed. Bus bridging sv...\u003c/td\u003e\n      \u003ctd\u003e7:46 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 19:46:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e157\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e55\u003c/th\u003e\n      \u003ctd\u003e[CCL] Train svcs just resumed. Bus bridging sv...\u003c/td\u003e\n      \u003ctd\u003e7:27 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 19:27:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e56\u003c/th\u003e\n      \u003ctd\u003e[CCL]Bus bridging svcs avail between #Bishan a...\u003c/td\u003e\n      \u003ctd\u003e7:20 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 19:20:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e57\u003c/th\u003e\n      \u003ctd\u003e[CCL} Bus bridging svcs avail between #Bishan ...\u003c/td\u003e\n      \u003ctd\u003e7:01 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 19:01:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e58\u003c/th\u003e\n      \u003ctd\u003e[CCL]No train svc btwn #BotanicGardens and #Se...\u003c/td\u003e\n      \u003ctd\u003e6:50 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 18:50:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eSerangoon\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e59\u003c/th\u003e\n      \u003ctd\u003e[CCL] No train svc btwn #BotanicGardens and #S...\u003c/td\u003e\n      \u003ctd\u003e6:32 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 18:32:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eSerangoon\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e60\u003c/th\u003e\n      \u003ctd\u003e[CCL] No train svc between #BotanicGardens and...\u003c/td\u003e\n      \u003ctd\u003e6:26 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 18:26:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eMarymount\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e61\u003c/th\u003e\n      \u003ctd\u003e[CCL] No train service between #BotanicGardens...\u003c/td\u003e\n      \u003ctd\u003e6:06 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 18:06:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eMarymount\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e62\u003c/th\u003e\n      \u003ctd\u003e[CCL] UPDATE: Estimate 10 mins additional trav...\u003c/td\u003e\n      \u003ctd\u003e5:09 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 17:09:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eBotanicGardens\u003c/td\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e63\u003c/th\u003e\n      \u003ctd\u003e[CCL] CLEARED: Fault cleared but trains and st...\u003c/td\u003e\n      \u003ctd\u003e4:39 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 16:39:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n      \u003ctd\u003ePasirPanjang\u003c/td\u003e\n      \u003ctd\u003eone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e64\u003c/th\u003e\n      \u003ctd\u003e[CCL]\\n  UPDATE: Estimate 10 mins additional t...\u003c/td\u003e\n      \u003ctd\u003e4:37 PM - 1 Nov 2016\u003c/td\u003e\n      \u003ctd\u003e2016-11-01 16:37:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ePasirPanjang\u003c/td\u003e\n      \u003ctd\u003eone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e65\u003c/th\u003e\n      \u003ctd\u003e[CCL] CLEARED: Train service is running normal...\u003c/td\u003e\n      \u003ctd\u003e7:15 PM - 19 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-19 19:15:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e66\u003c/th\u003e\n      \u003ctd\u003e[CCL] Update: Free regular bus service has cea...\u003c/td\u003e\n      \u003ctd\u003e7:03 PM - 19 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-19 19:03:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e170\u003c/td\u003e\n      \u003ctd\u003eDhobyGhaut\u003c/td\u003e\n      \u003ctd\u003eMacPherson\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e67\u003c/th\u003e\n      \u003ctd\u003e[CCL] Update: Estimate 10mins additional trave...\u003c/td\u003e\n      \u003ctd\u003e6:53 PM - 19 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-19 18:53:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eDhobyGhaut\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e68\u003c/th\u003e\n      \u003ctd\u003e[CCL] Update: Estimate 15 mins additional trav...\u003c/td\u003e\n      \u003ctd\u003e6:41 PM - 19 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-19 18:41:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eDhobyGhaut\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e69\u003c/th\u003e\n      \u003ctd\u003e[CCL] Update: Free regular bus services are av...\u003c/td\u003e\n      \u003ctd\u003e5:49 PM - 19 Sep 2016\u003c/td\u003e\n      \u003ctd\u003e2016-09-19 17:49:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003ePayaLebar\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e70 rows × 10 columns\u003c/p\u003e\n\u003c/div\u003e\n\n\n\n\n## Checking for outliers\n\n\n```python\n### Check for outliers, i.e see any adnormal spikes that may indicate data error\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn\n\ndf = df.sort_values('Date/Time', ascending=True)\nplt.plot(df['Date/Time'], df['Duration'])\nplt.xticks(rotation='vertical')\nplt.show()\n```\n\n\n![png](images/output_17_0.png)\n\n\n\n```python\n### Another view to check for outliers details that has duration \u003e 1 day (1440 durations),\n### which may indicate data error\n\ncount=0\noutlier_list=[]\nfor index in range(len(df)):\n    #if df.Duration[index].days \u003e=  1:\n    if int(df.Duration[index]) \u003e=  1440:  # 1440 = 24 hours\n        print ('Disruption \u003e1 day (1440 minutes) at index %s: %s, %s, %s' % (index, df.Duration[index], df['Date/Time'][index], df.MRT_Line[index]))\n        count = count + 1\n        outlier_list.append(index)  # store the outlier index\n\nprint(\"Count of disruption \u003e1 day (1440 minutes): %s\" % count)\nprint(outlier_list)\n\n### Sort the outlier index\noutlier_list.sort()\noutlier_list=outlier_list[::-1]\nprint(outlier_list)\n```\n\n    Disruption \u003e1 day (1440 minutes) at index 97: 30816.0, 2017-12-06 14:05:00, EWL\n    Disruption \u003e1 day (1440 minutes) at index 107: 6840.0, 2017-11-15 01:15:00, EWL\n    Disruption \u003e1 day (1440 minutes) at index 403: 11615.0, 2017-08-31 03:55:00, NSL\n    Count of disruption \u003e1 day (1440 minutes): 3\n    [97, 107, 403]\n    [403, 107, 97]\n\n\n\n```python\n### Remove outliers from the dataframe after investigating the data.\n\nfor i in outlier_list:\n    print(\"Removing data at index %s.\" % i)\n    df = df[df.index!= i]\n```\n\n    Removing data at index 403.\n    Removing data at index 107.\n    Removing data at index 97.\n\n\n\n```python\n### Check if outlier rows has been deleted\ndf.shape\n```\n\n\n\n\n    (580, 10)\n\n\n\n\n```python\n### Re-Sort the MRT Line and Date/time after removing outliers\ndf.sort_values(['MRT_Line', 'Date/Time'], ascending=[True, False], inplace=True)\ndf.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n      \u003cth\u003eDuration\u003c/th\u003e\n      \u003cth\u003eNew_Fr_Stn\u003c/th\u003e\n      \u003cth\u003eNew_To_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Fault cleared. Normal train services a...\u003c/td\u003e\n      \u003ctd\u003e12:54 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:54:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e42\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]: No train service between #ChoaChuKang...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular and bridging bus...\u003c/td\u003e\n      \u003ctd\u003e3:09 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 03:09:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services on the entire B...\u003c/td\u003e\n      \u003ctd\u003e2:30 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:30:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Service B on the BPLRT inner l...\u003c/td\u003e\n      \u003ctd\u003e2:04 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:04:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n## EDA on average service disruption (mins)\n\n```python\n### Compute the average disruption duration\ncount=0\ntotal=0\nfor index in range(len(df)):\n    if index in outlier_list: pass\n    elif int(df.Duration[index]) \u003e 0:\n        total = total + int(df.Duration[index])\n        count = count + 1\n        datetime = df['Date/Time'][index]\n        #print(index)\n\naverage = total/count\n\nprint('Number of disruption since %s                 : %s' % (datetime, count))\nprint(\"Average SMRT Line disruption duration since %s: %.2f minutes\" % (datetime, average))\n```\n\n    Number of disruption since 2016-05-11 01:41:00                 : 114\n    Average SMRT Line disruption duration since 2016-05-11 01:41:00: 75.33 minutes\n\n\n\n```python\n### Plot disruption duration (in minutes) against date/time with mean\n\ndf = df.sort_values('Date/Time', ascending=True)\n\ny_mean = [average for i in df['Date/Time']]\n\nplt.plot(df['Date/Time'], df['Duration'])\nplt.plot(df['Date/Time'], y_mean, label='Mean', linestyle='--')\n\nlegend = plt.legend(loc='best')  # insert a legend\n\nplt.show()\n```\n\n\n![png](images/output_23_0.png)\n\n\n## Analysing relationship between number of tweets and disruption occurances \n\n```python\n### Prepare dataset for analysing relationship between number of tweets and disruption\n### ==================================================================================\n\n### Remove all non-'cleared' status\n\ndf2 = df[df.Status != 'update']\ndf2.head(10)   \n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n      \u003cth\u003eDuration\u003c/th\u003e\n      \u003cth\u003eNew_Fr_Stn\u003c/th\u003e\n      \u003cth\u003eNew_To_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e83\u003c/th\u003e\n      \u003ctd\u003e[CCL] Normal services have resumed. Free shutt...\u003c/td\u003e\n      \u003ctd\u003e6:23 AM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 06:23:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e82\u003c/th\u003e\n      \u003ctd\u003e[CCL] Normal services have resumed. Free shutt...\u003c/td\u003e\n      \u003ctd\u003e6:31 AM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 06:31:00\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e32\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Train services have resumed on the BPL...\u003c/td\u003e\n      \u003ctd\u003e6:52 AM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 06:52:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e31\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Train services have resumed on BPLRT. ...\u003c/td\u003e\n      \u003ctd\u003e7:12 AM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 07:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e30\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Free shuttle bus services have ceased.\u003c/td\u003e\n      \u003ctd\u003e7:16 AM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 07:16:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e580\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train service from #Kranji to #...\u003c/td\u003e\n      \u003ctd\u003e3:49 PM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 15:49:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e579\u003c/th\u003e\n      \u003ctd\u003e[NSL]CLEARED: Free public \u0026amp; shuttle buses are ...\u003c/td\u003e\n      \u003ctd\u003e3:51 PM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 15:51:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e578\u003c/th\u003e\n      \u003ctd\u003e[NSL]Fault cleared but some trains \u0026amp; stations ...\u003c/td\u003e\n      \u003ctd\u003e4:20 PM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 16:20:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e577\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train services between Kranji a...\u003c/td\u003e\n      \u003ctd\u003e4:29 PM - 25 Apr 2016\u003c/td\u003e\n      \u003ctd\u003e2016-04-25 16:29:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e333\u003c/th\u003e\n      \u003ctd\u003e[EWL] CLEARED: Train service has resumed betwe...\u003c/td\u003e\n      \u003ctd\u003e1:33 AM - 2 May 2016\u003c/td\u003e\n      \u003ctd\u003e2016-05-02 01:33:00\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e49\u003c/td\u003e\n      \u003ctd\u003eJooKoon\u003c/td\u003e\n      \u003ctd\u003eBoonLay\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Check data type\n\ndf['Duration'] = df['Duration'].astype(int)\ndf.dtypes\n```\n\n\n\n\n    Tweet                          object\n    Extracted Date/Time            object\n    Date/Time              datetime64[ns]\n    MRT_Line                       object\n    Status                         object\n    From_Stn                       object\n    To_Stn                         object\n    Duration                        int64\n    New_Fr_Stn                     object\n    New_To_Stn                     object\n    dtype: object\n\n\n\n\n```python\n### Remove duration=0 and put the clean up data into a new dataframe\ndf2 = df[df.Duration != 0]\ndf2.head(10)\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n      \u003cth\u003eDuration\u003c/th\u003e\n      \u003cth\u003eNew_Fr_Stn\u003c/th\u003e\n      \u003cth\u003eNew_To_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e333\u003c/th\u003e\n      \u003ctd\u003e[EWL] CLEARED: Train service has resumed betwe...\u003c/td\u003e\n      \u003ctd\u003e1:33 AM - 2 May 2016\u003c/td\u003e\n      \u003ctd\u003e2016-05-02 01:33:00\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e49\u003c/td\u003e\n      \u003ctd\u003eJooKoon\u003c/td\u003e\n      \u003ctd\u003eBoonLay\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e331\u003c/th\u003e\n      \u003ctd\u003e[EWL] CLEARED: Free regular bus service betwee...\u003c/td\u003e\n      \u003ctd\u003e1:53 AM - 2 May 2016\u003c/td\u003e\n      \u003ctd\u003e2016-05-02 01:53:00\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e16\u003c/td\u003e\n      \u003ctd\u003eJooKoon\u003c/td\u003e\n      \u003ctd\u003eBoonLay\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e572\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train services from #Yishun tow...\u003c/td\u003e\n      \u003ctd\u003e1:41 AM - 11 May 2016\u003c/td\u003e\n      \u003ctd\u003e2016-05-11 01:41:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e40\u003c/td\u003e\n      \u003ctd\u003eYishun\u003c/td\u003e\n      \u003ctd\u003eYio\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e569\u003c/th\u003e\n      \u003ctd\u003e[NSL]UPDATE:Train service from Woodlands to Se...\u003c/td\u003e\n      \u003ctd\u003e1:53 AM - 23 Jun 2016\u003c/td\u003e\n      \u003ctd\u003e2016-06-23 01:53:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e8\u003c/td\u003e\n      \u003ctd\u003eWoodlands\u003c/td\u003e\n      \u003ctd\u003eSembawang\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e563\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Normal train service from #YioC...\u003c/td\u003e\n      \u003ctd\u003e7:49 AM - 29 Jun 2016\u003c/td\u003e\n      \u003ctd\u003e2016-06-29 07:49:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e90\u003c/td\u003e\n      \u003ctd\u003eWoodlands\u003c/td\u003e\n      \u003ctd\u003eAngMoKio\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e561\u003c/th\u003e\n      \u003ctd\u003e[NSL] UPDATE: South Bound normal service resum...\u003c/td\u003e\n      \u003ctd\u003e8:22 PM - 12 Jul 2016\u003c/td\u003e\n      \u003ctd\u003e2016-07-12 20:22:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e10\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003eKhatib\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e559\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Normal train service from #Bish...\u003c/td\u003e\n      \u003ctd\u003e9:07 PM - 12 Jul 2016\u003c/td\u003e\n      \u003ctd\u003e2016-07-12 21:07:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e30\u003c/td\u003e\n      \u003ctd\u003eBishan\u003c/td\u003e\n      \u003ctd\u003eKhatib\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e555\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train service between #MarinaBa...\u003c/td\u003e\n      \u003ctd\u003e5:02 PM - 20 Jul 2016\u003c/td\u003e\n      \u003ctd\u003e2016-07-20 17:02:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e18\u003c/td\u003e\n      \u003ctd\u003eMarinaBay\u003c/td\u003e\n      \u003ctd\u003eMarinaSouthPier\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e327\u003c/th\u003e\n      \u003ctd\u003e[EWL] Update: Fault has been cleared. However,...\u003c/td\u003e\n      \u003ctd\u003e2:48 AM - 31 Jul 2016\u003c/td\u003e\n      \u003ctd\u003e2016-07-31 02:48:00\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e54\u003c/td\u003e\n      \u003ctd\u003eBugis\u003c/td\u003e\n      \u003ctd\u003ePasirRis\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e550\u003c/th\u003e\n      \u003ctd\u003e[NSL] CLEARED: Train service on the NSL has re...\u003c/td\u003e\n      \u003ctd\u003e6:19 PM - 1 Aug 2016\u003c/td\u003e\n      \u003ctd\u003e2016-08-01 18:19:00\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e21\u003c/td\u003e\n      \u003ctd\u003eYishun\u003c/td\u003e\n      \u003ctd\u003eWoodlands\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Plot the relationship between tweets and disruption data\nfig = plt.figure()\nax = plt.subplot(111)\n\ndf.groupby('MRT_Line').Status.count().plot(kind='bar', ax=ax, label='Tweets', color='g')\ndf2.groupby('MRT_Line').Status.count().plot(kind='bar', ax=ax, label='Disruption', color='b')\n\nlegend = plt.legend(loc='best')\n```\n\n\n![png](images/output_27_0.png)\n\n\n## EDA on disruption occurances \n\n```python\n### Prepare a new dataframe for disruption analysis\ndf3 = pd.DataFrame()\ndf3['MRT_Line'] = df2['MRT_Line']\ndf3['hours'] = df2['Date/Time'].dt.hour\ndf3['duration'] = df2['Duration']\ndf3['Date/Time'] = df2['Date/Time']\ndf3.head(10)\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003ehours\u003c/th\u003e\n      \u003cth\u003eduration\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e333\u003c/th\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e49\u003c/td\u003e\n      \u003ctd\u003e2016-05-02 01:33:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e331\u003c/th\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e16\u003c/td\u003e\n      \u003ctd\u003e2016-05-02 01:53:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e572\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e40\u003c/td\u003e\n      \u003ctd\u003e2016-05-11 01:41:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e569\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e8\u003c/td\u003e\n      \u003ctd\u003e2016-06-23 01:53:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e563\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e7\u003c/td\u003e\n      \u003ctd\u003e90\u003c/td\u003e\n      \u003ctd\u003e2016-06-29 07:49:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e561\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e20\u003c/td\u003e\n      \u003ctd\u003e10\u003c/td\u003e\n      \u003ctd\u003e2016-07-12 20:22:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e559\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e21\u003c/td\u003e\n      \u003ctd\u003e30\u003c/td\u003e\n      \u003ctd\u003e2016-07-12 21:07:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e555\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e17\u003c/td\u003e\n      \u003ctd\u003e18\u003c/td\u003e\n      \u003ctd\u003e2016-07-20 17:02:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e327\u003c/th\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n      \u003ctd\u003e54\u003c/td\u003e\n      \u003ctd\u003e2016-07-31 02:48:00\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e550\u003c/th\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n      \u003ctd\u003e18\u003c/td\u003e\n      \u003ctd\u003e21\u003c/td\u003e\n      \u003ctd\u003e2016-08-01 18:19:00\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n### Analyse the MRT service breakdown ocurrence\n\nfrom matplotlib.ticker import FormatStrFormatter\nimport seaborn as sns\n\nxFormatter = FormatStrFormatter('%02d:00')\n\nsnsplot = sns.lmplot('hours', 'duration', data=df3, hue='MRT_Line', fit_reg=False, size=7) #remove col='MRT_Line' to get a combine view,\nsnsplot.set(xlabel = \"Time (24hrs)\", ylabel = \"Disruption Duration (mins)\")#, title = \"Disruption Plot\")\n\naxes = snsplot.ax\naxes.xaxis.set_major_formatter(xFormatter)\n\nplt.show()\n```\n\n\n![png](images/output_29_0.png)\n\n\n\n```python\n### Plot disruption occurence chart for each MRT Line separately\n\nxFormatter = FormatStrFormatter('%02d:00')\n\nsnsplot = sns.lmplot('hours', 'duration', data=df3, hue='MRT_Line', col='MRT_Line', col_wrap=2, fit_reg=False, size=5) #remove col='MRT_Line' to get a combine view\n#sharex=True, sharey=True,\nsnsplot.set(xlabel = \"Time (24hrs)\", ylabel = \"Disruption Duration (mins)\")#, title = \"Disruption Plot\")\n\nplt.show()\n\n```\n\n\n![png](images/output_30_0.png)\n\n\n\n```python\n### Frequency of service disruption across time (hourly) of day\nfig, ax = plt.subplots()\ndf3.groupby('hours').MRT_Line.count().plot.bar()\nax.set_xlabel(\"Time of Day\")\n```\n\n\n\n\n    \u003cmatplotlib.text.Text at 0x1286df908\u003e\n\n\n\n\n![png](images/output_31_1.png)\n\n\n\n```python\n### Yearly frequency of service disruption\nfig, ax = plt.subplots()\ndf3.groupby(df3['Date/Time'].dt.year).MRT_Line.count().plot.bar()\nax.set_xlabel(\"Year\")\n```\n\n\n\n\n    \u003cmatplotlib.text.Text at 0x1289af7f0\u003e\n\n\n\n\n![png](images/output_32_1.png)\n\n\n\n```python\n### Monthly frequency of service disruption\n\nmths=['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']\n\nfig, ax = plt.subplots()\ndf3.groupby(df3['Date/Time'].dt.month).MRT_Line.count().plot.bar()\nax.set_xlabel(\"Month\")\nax.set_xticklabels([mths[i] for i in range(12)])\n```\n\n\n\n![png](images/output_33_1.png)\n\n\n\n```python\n### Day of week daily frequency of service disruption\n\ndays=['Sun','Mon','Tue','Wed','Thu','Fri','Sat']\n\nfig, ax = plt.subplots()\ndf3.groupby(df3['Date/Time'].dt.dayofweek).MRT_Line.count().plot.bar()\n\nax.set_xlabel(\"Day\")\nax.set_xticklabels([days[i] for i in range(7)])\n```\n\n\n\n\n\n![png](images/output_34_1.png)\n\n\n\n```python\nbrowser.quit()\n```\n\n## Scraping of stations details from Wikipedia\n\n```python\nimport matplotlib.pyplot as plt\n%matplotlib inline\n#matplotlib.rcParams['figure.figsize'] = [14, 8]\nimport requests\nfrom lxml import html\nimport pandas as pd \nimport re\n\n#################################################\n# Scraping of related MRT/LRT stations details #\n################################################\n\nremoveWords = set(['\\n', 'Reserved station', 'N/A', 'Reserved Station', '[a]'])\n```\n\n\n```python\n## Scraping North South Line stations from wiki page using xPath\nallStn=requests.get('https://en.wikipedia.org/wiki/List_of_Singapore_MRT_stations')\nallStnTree = html.fromstring(allStn.content)\nallStnName = allStnTree.xpath('//table[@class=\"wikitable\"]//td[2]//text()')\nallStnName.pop(allStnName.index('[a]'))\nallStnName.pop(allStnName.index('Canberra'))\n\nstartingXpath = '//table[@class=\"wikitable\"]//tr[3]//text()'\nstartingXpath = int(re.search(r'[0-9]+', startingXpath)[0])\n\nendingXpath = '//table[@class=\"wikitable\"]//tr[32]//text()'\nendingXpath = int(re.search(r'[0-9]+', endingXpath)[0])\n\nredEndPos = endingXpath - startingXpath - 1\n\nredStnName = allStnName[:redEndPos]\nredStnName = list(filter(lambda x: x not in removeWords, redStnName))\n\nnsl_df = pd.DataFrame(redStnName, columns=['Name'])\nnsl_df['Line'] = 'NSL'\nnsl_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eName\u003c/th\u003e\n      \u003cth\u003eLine\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eJurong East\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eBukit Batok\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eBukit Gombak\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eChoa Chu Kang\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eYew Tee\u003c/td\u003e\n      \u003ctd\u003eNSL\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n## Scraping East West Line stations from wiki page using xPath\nstartingXpath = '//table[@class=\"wikitable\"]//tr[32]//text()'\nstartingXpath = int(re.search(r'[0-9]+', startingXpath)[0])\n\nendingXpath = '//table[@class=\"wikitable\"]//tr[66]//text()'\nendingXpath = int(re.search(r'[0-9]+', endingXpath)[0])\n\ngreenPos = endingXpath - startingXpath - 1\ngreenEndPos = redEndPos + greenPos\n\ngreenStnName = allStnName[redEndPos:greenEndPos]\ngreenStnName = list(filter(lambda x: x not in removeWords, greenStnName))\n\newl_df = pd.DataFrame(greenStnName, columns=['Name'])\newl_df['Line'] = 'EWL'\newl_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eName\u003c/th\u003e\n      \u003cth\u003eLine\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eTampines\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eSimei\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eTanah Merah\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eBedok\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eKembangan\u003c/td\u003e\n      \u003ctd\u003eEWL\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n## Scraping Changi Airport Line stations from wiki page using xPath\nstartingXpath = '//table[@class=\"wikitable\"]//tr[66]//text()'\nstartingXpath = int(re.search(r'[0-9]+', startingXpath)[0])\n\nendingXpath = '//table[@class=\"wikitable\"]//tr[69]//text()' \nendingXpath = int(re.search(r'[0-9]+', endingXpath)[0])\n\nairportPos = endingXpath - startingXpath - 1\n\nairportStnName = ['Tanah Merah']\nairportStnName += allStnName[greenEndPos:greenEndPos+airportPos]\nairportStnName = list(filter(lambda x: x not in removeWords, airportStnName))\n\ncgl_df = pd.DataFrame(airportStnName, columns=['Name'])\ncgl_df['Line'] = 'CGL'\ncgl_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eName\u003c/th\u003e\n      \u003cth\u003eLine\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eTanah Merah\u003c/td\u003e\n      \u003ctd\u003eCGL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eChangi Airport\u003c/td\u003e\n      \u003ctd\u003eCGL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eHarbourFront\u003c/td\u003e\n      \u003ctd\u003eCGL\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n## Adding Changi Airport Line to East West Line dataframe\newl_df1 = ewl_df.iloc[4:]\n\ncgl_df1 = cgl_df.reindex(index=cgl_df.index[::-1])\n\newl_df2 = pd.concat([cgl_df1, ewl_df1])\newl_df2.reset_index(drop=True,inplace=True)\n\nairportStnName = ewl_df2['Name'].tolist()\n```\n\n\n```python\n## Scraping Circle Line stations from wiki page using xPath\nstartingXpath = '//table[@class=\"wikitable\"]//tr[89]//text()'\nstartingXpath = int(re.search(r'[0-9]+', startingXpath)[0])\n\nendingXpath = '//table[@class=\"wikitable\"]//tr[119]//text()' \nendingXpath = int(re.search(r'[0-9]+', endingXpath)[0])\n\ncirclePos = endingXpath - startingXpath - 1\n\ncclPos = allStnName.index(\"Punggol Coast\") + 1\n\ncircleStnName = allStnName[cclPos:cclPos+circlePos]\ncircleStnName = list(filter(lambda x: x not in removeWords, circleStnName))\ncircleStnName = [c.replace('one-north', 'One-North') for c in circleStnName]\n\nccl_df = pd.DataFrame(circleStnName, columns=['Name'])\nccl_df['Line'] = 'CCL'\nccl_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eName\u003c/th\u003e\n      \u003cth\u003eLine\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eDhoby Ghaut\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eBras Basah\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eEsplanade\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003ePromenade\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eNicoll Highway\u003c/td\u003e\n      \u003ctd\u003eCCL\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\n## Scraping Bukit Panjang LRT stations from wiki page using xPath \nbpLine=requests.get('https://en.wikipedia.org/wiki/Bukit_Panjang_LRT_line')\nbpTree = html.fromstring(bpLine.content)\nbpStnName = bpTree.xpath('//table[@class=\"wikitable\"]//td[2]//text()')\nbpStnName = list(filter(lambda x: x not in removeWords, bpStnName))\nbpStnName = bpStnName[5:]\n\nbplrt_df = pd.DataFrame(bpStnName, columns=['Name'])\nbplrt_df['Line'] = 'BPLRT'\nbplrt_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eName\u003c/th\u003e\n      \u003cth\u003eLine\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eChoa Chu Kang\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eSouth View\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eKeat Hong\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eTeck Whye\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n## Loading shapefile into Python\n\n```python\n#############################################################\n# Converting LTA's MRT \u0026 LRT shapefile to Pandas dataframe #\n###########################################################\n```\n\n\n```python\n!pip install pyshp\n```\n\n    Requirement already satisfied: pyshp in /anaconda3/lib/python3.6/site-packages\n\n\n\n```python\n# import pyshp\nimport shapefile \n\n# reading Singapore's MRT/LRT stations from LTA's shapefile\nmyshp = open('data/MRTLRTStnPtt.shp', \"rb\")\nmydbf = open('data/MRTLRTStnPtt.dbf', \"rb\")\nsf = shapefile.Reader(shp=myshp, dbf=mydbf)\nrecords = sf.shapeRecords()\n\n# checking how many records in the shapefile\nprint('There are', len(records), 'shape objects in this file')\n\n# view the data in the shapefile\nfor record in records[:5]:\n    print (record.record[0], record.shape.points[0])\n\n```\n\n    There are 183 shape objects in this file\n    1 [35782.955299999565, 33560.077600000426]\n    2 [16790.746600000188, 36056.30189999938]\n    3 [27962.310800000094, 44352.56799999997]\n    4 [20081.697399999946, 45214.54790000059]\n    5 [26163.47800000012, 30218.819599999115]\n\n\n\n```python\n# converting shape file from WSG84 to SVY21 (Singapore's Lat Lon standard)\nfrom utils.svy21 import SVY21\nsvy = SVY21()\n\n## saving all MRT \u0026 LRT stations name, \u0026 Lat Lon from shapefile to dataframe\nlatlon = []\n\nfor record in records:\n    rec = []\n    rec.append(record.record[1])\n    svylatlon = svy.computeLatLon(record.shape.points[0][1], record.shape.points[0][0])\n    rec.append(svylatlon[0])\n    rec.append(svylatlon[1])\n    latlon.append(rec)\n\nstnlatlon_df = pd.DataFrame(latlon, columns=['StnName', 'Lat', 'Lon'])\nstnlatlon_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eStnName\u003c/th\u003e\n      \u003cth\u003eLat\u003c/th\u003e\n      \u003cth\u003eLon\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eEUNOS MRT STATION\u003c/td\u003e\n      \u003ctd\u003e1.319778\u003c/td\u003e\n      \u003ctd\u003e103.903252\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eCHINESE GARDEN MRT STATION\u003c/td\u003e\n      \u003ctd\u003e1.342352\u003c/td\u003e\n      \u003ctd\u003e103.732596\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eKHATIB MRT STATION\u003c/td\u003e\n      \u003ctd\u003e1.417383\u003c/td\u003e\n      \u003ctd\u003e103.832980\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eKRANJI MRT STATION\u003c/td\u003e\n      \u003ctd\u003e1.425177\u003c/td\u003e\n      \u003ctd\u003e103.762165\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eREDHILL MRT STATION\u003c/td\u003e\n      \u003ctd\u003e1.289562\u003c/td\u003e\n      \u003ctd\u003e103.816816\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\ndef convertCase(stn):\n    newName = stn.title()\n    return newName\n\nstnlatlon_df['StnName'] = stnlatlon_df.apply(lambda row: convertCase(row['StnName']), axis=1)\nstnlatlon_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\u003cstyle scoped\u003e\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n\u003c/style\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eStnName\u003c/th\u003e\n      \u003cth\u003eLat\u003c/th\u003e\n      \u003cth\u003eLon\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eEunos Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.319778\u003c/td\u003e\n      \u003ctd\u003e103.903252\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eChinese Garden Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.342352\u003c/td\u003e\n      \u003ctd\u003e103.732596\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eKhatib Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.417383\u003c/td\u003e\n      \u003ctd\u003e103.832980\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eKranji Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.425177\u003c/td\u003e\n      \u003ctd\u003e103.762165\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eRedhill Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.289562\u003c/td\u003e\n      \u003ctd\u003e103.816816\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\ndef convertLRT(stn):\n    newName = stn.replace('Lrt', 'Mrt')\n    return newName\n\nstnlatlon_df['StnName'] = stnlatlon_df.apply(lambda row: convertLRT(row['StnName']), axis=1)\nstnlatlon_df.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eStnName\u003c/th\u003e\n      \u003cth\u003eLat\u003c/th\u003e\n      \u003cth\u003eLon\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eEunos Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.319778\u003c/td\u003e\n      \u003ctd\u003e103.903252\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eChinese Garden Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.342352\u003c/td\u003e\n      \u003ctd\u003e103.732596\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eKhatib Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.417383\u003c/td\u003e\n      \u003ctd\u003e103.832980\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eKranji Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.425177\u003c/td\u003e\n      \u003ctd\u003e103.762165\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003eRedhill Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.289562\u003c/td\u003e\n      \u003ctd\u003e103.816816\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\nstnlatlon_df = stnlatlon_df.drop_duplicates(['StnName']) \nstnlatlon_df.tail()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eStnName\u003c/th\u003e\n      \u003cth\u003eLat\u003c/th\u003e\n      \u003cth\u003eLon\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e174\u003c/th\u003e\n      \u003ctd\u003eBuangkok Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.382877\u003c/td\u003e\n      \u003ctd\u003e103.893121\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e177\u003c/th\u003e\n      \u003ctd\u003eRaffles Place Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.284125\u003c/td\u003e\n      \u003ctd\u003e103.851461\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e178\u003c/th\u003e\n      \u003ctd\u003eHolland Village Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.311834\u003c/td\u003e\n      \u003ctd\u003e103.796191\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e180\u003c/th\u003e\n      \u003ctd\u003eTelok Blangah Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.270753\u003c/td\u003e\n      \u003ctd\u003e103.809748\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e181\u003c/th\u003e\n      \u003ctd\u003eTelok Ayer Mrt Station\u003c/td\u003e\n      \u003ctd\u003e1.282289\u003c/td\u003e\n      \u003ctd\u003e103.848302\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n## Train stations data cleaning \u0026 processing\n\n```python\n##############################################################################################\n# Mapping and merging all different dataframe together                                      #\n# (scrape tweets dataframe, scraped wiki stations dataframe \u0026 shapefile stations dataframe) #\n############################################################################################\n```\n\n\n```python\nmrtTweets = pd.read_csv('smrt_tweet_status_extract.csv')\nmrtTweets = mrtTweets.drop(mrtTweets.columns[0], axis=1)\n\nmrtTweets.fillna(value='None',inplace=True)\n## Manual data standardization due to very irregular pattern/outlier\nmrtTweets['New_Fr_Stn'].replace('Marinabay', 'Marina Bay', inplace = True)\nmrtTweets['New_To_Stn'].replace('Marinabay', 'Marina Bay', inplace = True)\nmrtTweets['New_Fr_Stn'].replace(['JUR'], 'Jurong East', inplace = True)\nmrtTweets['New_To_Stn'].replace(['JUR'], 'Jurong East', inplace = True)\nmrtTweets['New_Fr_Stn'].replace(['MSP'], 'Marina South Pier', inplace = True)\nmrtTweets['New_To_Stn'].replace(['MSP'], 'Marina South Pier', inplace = True)\nmrtTweets['New_Fr_Stn'].replace(['TLK'], 'Tuas Link', inplace = True)\nmrtTweets['New_To_Stn'].replace(['TLK'], 'Tuas Link', inplace = True)\nmrtTweets['New_Fr_Stn'].replace(['JKN'], 'Joo Koon', inplace = True)\nmrtTweets['New_To_Stn'].replace(['JKN'], 'Joo Koon', inplace = True)\nmrtTweets['New_Fr_Stn'].replace(['Woodlandwill'], 'Woodlands', inplace = True)\nmrtTweets['New_To_Stn'].replace(['Woodlandwill'], 'Woodlands', inplace = True)\n\nmrtTweets.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n      \u003cth\u003eDuration\u003c/th\u003e\n      \u003cth\u003eNew_Fr_Stn\u003c/th\u003e\n      \u003cth\u003eNew_To_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Fault cleared. Normal train services a...\u003c/td\u003e\n      \u003ctd\u003e12:54 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:54:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e42.0\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]: No train service between #ChoaChuKang...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular and bridging bus...\u003c/td\u003e\n      \u003ctd\u003e3:09 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 03:09:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services on the entire B...\u003c/td\u003e\n      \u003ctd\u003e2:30 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:30:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Service B on the BPLRT inner l...\u003c/td\u003e\n      \u003ctd\u003e2:04 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:04:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\ndef parseStn(fromStn):\n    newStn = re.sub(r\"(\\w)([A-Z])\", r\"\\1 \\2\", str(fromStn))\n    newStn += ' dummy'\n    return newStn\n\nmrtTweets['new_From_Stn'] = mrtTweets.apply(lambda row: parseStn(str(row['New_Fr_Stn'])), axis=1)\nmrtTweets['new_To_Stn'] = mrtTweets.apply(lambda row: parseStn(row['New_To_Stn']), axis=1)\n\nmrtTweets.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eTweet\u003c/th\u003e\n      \u003cth\u003eExtracted Date/Time\u003c/th\u003e\n      \u003cth\u003eDate/Time\u003c/th\u003e\n      \u003cth\u003eMRT_Line\u003c/th\u003e\n      \u003cth\u003eStatus\u003c/th\u003e\n      \u003cth\u003eFrom_Stn\u003c/th\u003e\n      \u003cth\u003eTo_Stn\u003c/th\u003e\n      \u003cth\u003eDuration\u003c/th\u003e\n      \u003cth\u003eNew_Fr_Stn\u003c/th\u003e\n      \u003cth\u003eNew_To_Stn\u003c/th\u003e\n      \u003cth\u003enew_From_Stn\u003c/th\u003e\n      \u003cth\u003enew_To_Stn\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] Fault cleared. Normal train services a...\u003c/td\u003e\n      \u003ctd\u003e12:54 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:54:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e42.0\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003eChoa Chu Kang dummy\u003c/td\u003e\n      \u003ctd\u003ePhoenix dummy\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e[BPLRT]: No train service between #ChoaChuKang...\u003c/td\u003e\n      \u003ctd\u003e12:12 AM - 18 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-18 00:12:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003eupdate\u003c/td\u003e\n      \u003ctd\u003eChoaChuKang\u003c/td\u003e\n      \u003ctd\u003ePhoenix\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] CLEARED: Free regular and bridging bus...\u003c/td\u003e\n      \u003ctd\u003e3:09 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 03:09:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Train services on the entire B...\u003c/td\u003e\n      \u003ctd\u003e2:30 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:30:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e[BPLRT] UPDATE: Service B on the BPLRT inner l...\u003c/td\u003e\n      \u003ctd\u003e2:04 AM - 12 Jan 2018\u003c/td\u003e\n      \u003ctd\u003e2018-01-12 02:04:00\u003c/td\u003e\n      \u003ctd\u003eBPLRT\u003c/td\u003e\n      \u003ctd\u003ecleared\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n      \u003ctd\u003eNone dummy\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\ndef parseStn(line, fromStn, toStn):\n    startStnIndex = ''\n    endStnIndex = ''\n    affectedStn = []\n    if fromStn != 'None dummy':\n        fromPat = (re.search(r'[^\\s]+', str(fromStn))[0]).title()\n        toPat = (re.search(r'[^\\s]+', str(toStn))[0]).title()\n        if line == 'NSL':\n            startStnIndex = nsl_df[nsl_df['Name'].str.match(fromPat)].index[0]\n            endStnIndex = nsl_df[nsl_df['Name'].str.match(toPat)].index[0]\n            if startStnIndex \u003e endStnIndex:\n                affectedStn = redStnName[endStnIndex:startStnIndex+1]\n            else:\n                affectedStn = redStnName[startStnIndex:endStnIndex+1]\n        elif line == 'CCL':\n            startStnIndex = ccl_df[ccl_df['Name'].str.match(fromPat)].index[0]\n            endStnIndex = ccl_df[ccl_df['Name'].str.match(toPat)].index[0]\n            if startStnIndex \u003e endStnIndex:\n                affectedStn = circleStnName[endStnIndex:startStnIn","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwokhing%2Fexploratory-data-analysis-on-smrt-tweets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwokhing%2Fexploratory-data-analysis-on-smrt-tweets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwokhing%2Fexploratory-data-analysis-on-smrt-tweets/lists"}