{"id":15393640,"url":"https://github.com/alhankeser/citibike-analysis","last_synced_at":"2026-01-25T01:03:04.658Z","repository":{"id":126620175,"uuid":"185126553","full_name":"alhankeser/citibike-analysis","owner":"alhankeser","description":"Extracting and Transforming Citi Bike Data for Analysis","archived":false,"fork":false,"pushed_at":"2024-01-10T13:49:29.000Z","size":69113,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-15T23:56:00.371Z","etag":null,"topics":["citibike","data-analysis","data-science","data-visualization","etl","sql"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alhankeser.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-06T05:02:35.000Z","updated_at":"2024-02-01T00:39:28.000Z","dependencies_parsed_at":"2024-10-19T01:08:06.077Z","dependency_job_id":null,"html_url":"https://github.com/alhankeser/citibike-analysis","commit_stats":{"total_commits":46,"total_committers":4,"mean_commits":11.5,"dds":0.08695652173913049,"last_synced_commit":"05cc898162c7b49be83bbc02969bcd73a8c88fd9"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/alhankeser/citibike-analysis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alhankeser%2Fcitibike-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alhankeser%2Fcitibike-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alhankeser%2Fcitibike-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alhankeser%2Fcitibike-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alhankeser","download_url":"https://codeload.github.com/alhankeser/citibike-analysis/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alhankeser%2Fcitibike-analysis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28740400,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T22:12:27.248Z","status":"ssl_error","status_checked_at":"2026-01-24T22:12:10.529Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["citibike","data-analysis","data-science","data-visualization","etl","sql"],"created_at":"2024-10-01T15:19:49.000Z","updated_at":"2026-01-25T01:03:04.642Z","avatar_url":"https://github.com/alhankeser.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"_This project was completed a while ago and does not reflect my current level of knowledge in this domain._\n\n\n# Extracting and Transforming Citi Bike Data for Analysis \n## How is Citi Bike availability affected by various factors like time of day and weather?\n\n### Who:\nI am [Alhan Keser](https://blog.alhan.co/), a [10+ year specialist in Web Experimentation](https://www.linkedin.com/in/alhankeser/) (aka A/B Testing, Conversion Optimization).\n\n### What:\nThis is an original analysis of Citi Bike station data from May-June 2019 to find out what affect the day of week, time of day, and weather (temperature, precipitation, etc...) have on the availability of bikes at station-,  neighborhood-, and borough-levels. \n\n### Why:\n- I wanted to push myself to extract and transform my own data. Skipping the entire ETL process and going straight into analysis is a luxury: it does not reflect reality. \n- Doing a time-series analysis is something that I wanted practice with. \n- I commute by bike every day (despite weather) so I have first-hand evidence that Citi Bike riders tend to shy away from biking in inclement weather. It will be interesting to visualize the differences here.   \n\n### How:\n- **Combined original data sources:**\n    - [Citi Bike Live Station Status](https://feeds.citibikenyc.com/stations/stations.json)\n    - [Dark Sky Weather API](https://darksky.net/dev/docs)\n    - [Google Geocoding API](https://developers.google.com/maps/documentation/geocoding/intro)\n- **Created cron jobs** to collect Citi Bike station statuses for all ~858 stations, every 3 minutes, for ~2 months.\n    - Total rows in final table: 5,800,274\n    - \"Why stop after 2 months,\" you ask? Because my server ran out of space while I was on vacation. Here's what that looks like: \n\n![My server crashed July 14](https://blog.alhan.co/storage/images/posts/2/web-server-crashed_2_1568434613_sm.jpg)\n- **Created a mini-ETL process** to transform data into the final output used below. \n    - Along the way, there were many errors, some of which I will resolve here.\n\n### Table of Contents\n- [Packages](#Packages)\n- [Extracting](#Extracting)\n    - [Stations-Raw](#Stations-Raw)\n    - [Stations-Flat](#Stations-Flat)\n    - [Geocoding](#Geocoding)\n    - [Weather](#Weather)\n    - [Cron Jobs](#Cron-Jobs)\n- [Transforming](#Transforming)\n    - [Availability by Station](#Availability-by-Station)\n- [Cleaning](#Cleaning)\n    - [Reducing Complexity](#Reducing-Complexity)\n    - [Data Quality Issues](#Data-Quality-Issues)\n    - [Fixing Zip Data Type](#Fixing-the-zip-data_type-issue)\n    - [Fixing Missing Weather](#Fixing-the-missing-weather-issue)\n- [Exploratory Data Analysis](#Exploratory-Data-Analysis)\n    - [Hypotheses](#Hypotheses)\n\n### Packages\nImporting a few packages that will help with describing, cleaning and visualizing things. \n\n\n```python\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom darksky import forecast\nimport random\nimport warnings\nimport time\nfrom datetime import datetime as dt\nfrom dateutil.parser import parse\nfrom dotenv import load_dotenv\nimport os\nload_dotenv()\nwarnings.filterwarnings('ignore')\npd.set_option('display.max_columns', None)\n```\n\n### Extracting\nI started by find an interesting data source. In this case, I found the [Citi Bike Station Feed](https://feeds.citibikenyc.com/stations/stations.json) via the [NYC Open Data site](https://opendata.cityofnewyork.us/).\n\nThe feed shows the latest statuses of ~858 Citi Bike stations. Below is a list of values per station and sample data for each. Any keys left blank are often blank in the data source as well, which I'll address in later steps. \n\n| key | sample value |\n|------------:|:---------|\n| `id`        | 285|\n| `stationName` |\"Broadway \u0026 E 14St\"|\n| `availableDocks` |20|\n| `totalDocks` |53|\n| `latitude`|40.73454567|\n| `longitude`   |-73.99074142|\n| `statusValue` |\"In Service\"|\n| `statusKey`   |1|\n| `availableBikes` |31|\n| `stAddress1`  |\"Broadway \u0026 E 14 St\"|\n| `stAddress2`  |\"\"|\n| `city`        |\"\"|\n| `postalCode`  |\"\"|\n| `location`    |\"\"|\n| `altitude`    |\"\"|\n| `testStation` |false|\n| `lastCommunicationTime` |\"2019-09-12 08:38:21 PM\"|\n| `landMark`    |\"\"|\n\n#### Stations-Raw\n\nTo have a back-up in case any of the subsequent steps went awry, I wanted to store the source data in the simplest way possible: a table `stations_raw` that stored the following: \n\n|column_name|data_type|sample value|\n|-----------|-----------|----------|\n|id         |int4|31419|\n|status     |json|{\"executionTime\": \"2019-06-22 01:53:41 PM\", \"s...|\n\nOnce the table created, I needed a way to collect data. A quick solution -- for me -- was to create [a Laravel application](https://github.com/alhankeser/citibike-tracker/)  that [makes it easy create console commands](https://laravel.com/docs/5.8/artisan#writing-commands). In combination with [Laravel Forge](https://forge.laravel.com), it's easy to set up a cron job that triggers [the necessary command](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L49) at set intervals.\n\nOnce the commands created, I set up a [cron job](#Cron-Jobs) that ran once every 3 minutes. This resulted in the collection of 41,325 rows.\n\n#### Stations-Flat\nAs part of the same command that creates the [stations_raw](#Stations-Raw) table, I [flattened out the JSON](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L80) and created a table with a single row per 3-minute interval, per station. We'll call this table `stations_flat` (probably could have used a better naming convention throughout this project). \n\nHere is the structure of `stations_flat` and some sample data:\n\n|column_name|data_type|sample value|description|\n|-----------|---------|------------|-----------|\n|id         |int4     |10511778    |row id|\n|station_id |int4     |72          |unique id for each station|\n|available_bikes|int4 |4           |number of available bikes at the station|\n|available_docks|int4 |49          |number of available docks (places to park a bike) at the station|\n|station_status|text  |In Service  |whether the station is in or out of service|\n|last_communication_time|timestamp|2019-05-15 01:14:15|the last time the station sent back data|\n\nAfter just over 2 months of this, I ended up with **34,301,048 rows** in this table. Luckily, I took some steps to make the volume of data more manageable when analyzing outside of a high CPU/RAM environment. \n\n#### Stations-Static\nAs the name suggests, `stations_static` contains information about each station that doesn't change minute-to-minute. Since there was a likelihood that stations be added, removed, renamed, I inserted or updated on duplicate each time `stations_flat` was updated. \n\n|column_name|data_type|sample value|description|\n|-----------|---------|------------|-----------|\n|id| int4|3119|unique `station_id` found throughout db|\n|name| text|Vernon Blvd \u0026 50 Ave||\n|latitude |float8|40.74232744||\n|longitude |float8|-73.95411749||\n|status_key| int4|1||\n|postal_code| text|NULL||\n|st_address_1| text|Vernon Blvd \u0026 50 Ave||\n|st_address_2| text|NULL||\n|total_docks| int4|45||\n|status| text|In Service||\n|altitude| text|NULL||\n|location| text|NULL||\n|land_mark| text|NULL||\n|city| text|NULL||\n|is_test_station| int4|0||\n\n#### Geocoding\nAs can be seen from the `stations_static` table above, many of the location-related values are null. This was the case for all stations. I wanted to be able to group stations by neighborhood and zip. Also, I wanted to use zip to associate weather data to each station, without having to make separate requests for each station (to stay within the free tier of the [Dark Sky Weather API](https://darksky.net/dev/docs)). \n\nTo geocode from lat/long for each station into human-readeable location info, I used the [Google Geocoding API](https://developers.google.com/maps/documentation/geocoding/intro). [See the command I used to create the below table](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L92)\n\n|column_name|data_type|sample value|description|\n|-----------|---------|------------|-----------|\n|id|int4|1||\n|station_id|int4|3119|unique station id|\n|zip|text|11101|zip code of station|\n|hood_1|text|LIC|neighborhood or the closest thing provided by Google|\n|hood_2|text|Hunters Point|another level of neighborhood|\n|borough|text|Queens|one of 5 NYC boroughs or New Jersey|\n\n#### Weather\nGrouping stations by zip, I then called the [Dark Sky Weather API](https://darksky.net/dev/docs) once every hour to build the `weather` table. Reducing the scope to zip made it possible to stay within the free plan limits of Dark Sky. \n\n|column_name|data_type|sample value|description|\n|-----------|---------|------------|-----------|\n|id|int4|1||\n|time_interval|timestamptz|2019-05-02 01:00:00-04|the 15-minute interval of time to associate the weather data to|\n|summary|text|Foggy|a categorical label for weather conditions|\n|precip_intensity|float8|0|percent percipitation intensity|\n|temperature|float8|61.45|temperature in Fahrenheit|\n|apparent_temperature|float8|61.89|\"feels-like\" temperature|\n|dew_point|float8|60.82||\n|humidity|float8|0.98|percent humidity|\n|wind_speed|float8|3.11|speed in MPH|\n|wind_gust|float8|5.38|gusts in MPH|\n|cloud_cover|float8|1|percent cloud cover|\n|uv_index|float8|0||\n|visibility|float8|3.18||\n|ozone|float8|316.23||\n|status|text|observed|one of two values (\"predicted\"/\"observed\") depending on if the weather values are from the past or the future|\n\n#### Cron Jobs\nI'm not going to spend a lot of time on discussing cron jobs, but here are the patterns I was using to run everything. There is probably a more optimal approach that I am not aware of. \n\n|Cron       |Command|\n|-----------|-------|\n|\\*/3 \\* \\* \\* \\* | get:docks \u0026\u0026 update:availability 0 \u0026\u0026 update:weather|\n|0 \\*/2 \\* \\* \\*  | get:weather 0|\n\nView the code behind each command:\n- `get:docks` [view](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L49)\n- `update:availability 0` [view](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L179)\n- `update:weather` [view](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L359)\n- `get:weather 0` [view](https://github.com/alhankeser/citibike-tracker/blob/d61f82adde88c90430205785297abf9f3de07c4d/app/Console/Kernel.php#L276)\n\n### Transforming\nOnce I had the 1 raw table and 4 tables above updating as expected, I created a single, flat table that had the final data I intended to use for analysis. I kept most columns except some of the minutiae of the `weather` table. This table was updated regularly such that I could run analyses on an on-going basis and avoid having to run a massive, single query after all data had been collected.   \n\n#### Availability by Station\nBelow is the flat table `availability` that combined the above tables, purpose-built for analysis. Note that available_bikes is the minimum number of bikes available during any of the 3-minute intervals during which samples were collected over the course of each 15-minute interval. \n\n|column_name|data_type|\n|-----------|---------|\n|time_interval|timestamptz|\n|station_id|int4|\n|station_name|text|\n|station_status|text|\n|latitude|float8|\n|longitude|float8|\n|zip|text|\n|borough|text|\n|hood|text|\n|available_bikes|int4|\n|available_docks|int4|\n|weather_summary|text|\n|precip_intensity|float8|\n|temperature|float8|\n|humidity|float8|\n|wind_speed|float8|\n|wind_gust|float8|\n|cloud_cover|float8|\n|weather_status|text|\n|created_at|timestamptz|\n|updated_at|timestamptz|\n\n### Cleaning\n\n#### Reducing Complexity\nFirst things first, I wanted to reduce the number of stations I was analyzing. The `availability` table resulted in nearly 6 million rows after 2 months, so I decided to export a subset of \"interesting\" stations to begin analyzing. Below is the query I used to find the interesting stations, based on whether there is a high variability in number of bikes, that the bikes regularly get refilled, and that the station has a decent number of bikes. I also limited the number of stations per neighborhood to 1.  \nhttps://gist.github.com/alhankeser/9fbaf67a8ce052de72f22ab1630cd91c\n\n```sql\nwith variability as (\n    select\n\t\tborough,\n\t\thood,\n\t\tstation_name,\n\t\tstation_id,\n\t\tmax(available_bikes) as max_bikes,\n\t\tsum(case when available_bikes = 0 then 1 else 0 end) as times_no_bikes,\n\t\tsum(case when available_docks = 0 then 1 else 0 end) as times_replenished\n\tfrom \n\t\tavailability\n\twhere\n\t\tstation_status = 'In Service'\n\tgroup by\n\t\tstation_id, station_name, hood, borough\n),\npercentiles as ( \n\tselect\n\t\t*,\n        ntile(100) over (order by max_bikes asc) max_bikes_percentile,\n\t\tntile(100) over (order by times_no_bikes asc) no_bikes_percentile,\n\t \tntile(100) over (order by times_replenished asc) times_replenished_percentile\n\tfrom\n\t\tvariability\n\torder by times_no_bikes\n),\nranks as (\n\tselect\n\t\t*,\n\t\t(max_bikes_percentile + no_bikes_percentile + times_replenished_percentile) as score,\n\t\trank() over (partition by hood order by (max_bikes_percentile + no_bikes_percentile + times_replenished_percentile) desc) as rank\n\tfrom \n\t\tpercentiles\n\twhere\n\t\tmax_bikes_percentile \u003e 40 \n\t\tand no_bikes_percentile \u003e 50\n\t\tand times_replenished_percentile \u003e 50\n),\nranked_by_hood as (\n\tselect\n\t\t*\n\tfrom \n\t\tranks\n\twhere\n\t\trank = 1 \n\torder by\n\t\tscore desc\n)\nselect\n\ta.*\nfrom\n\tavailability as a\njoin \n\tranked_by_hood as rbh \n\ton a.station_id = rbh.station_id;\n```\n\nThe query above reduced the nearly 6 million rows down to 186,000. The csv export used for the analysis below can be [found here](https://github.com/alhankeser/citibike-analysis/blob/master/input/availability_interesting_original.csv). \n\n#### Data Quality Issues\nAs can be seen below, without much digging, it's easy to spot some data quality/consistency issues: \n1. `zip` is being converted to an integer and thus dropping the 0, which is may or may not be an issue. If wish to solve problem #1 then this chould be an issue. \n1. `weather_status` should be 'observed' for all locations rather than 'predicted' since the dates are in the past. \n\n\n```python\ndate_cols = ['time_interval', 'updated_at', 'created_at']\ndf = pd.read_csv('../input/availability_interesting_original.csv', parse_dates=date_cols)\n```\n\n\n```python\ndf.info()\n```\n\n    \u003cclass 'pandas.core.frame.DataFrame'\u003e\n    RangeIndex: 186030 entries, 0 to 186029\n    Data columns (total 21 columns):\n    station_id          186030 non-null int64\n    station_name        186030 non-null object\n    station_status      186030 non-null object\n    latitude            186030 non-null float64\n    longitude           186030 non-null float64\n    zip                 186030 non-null int64\n    borough             186030 non-null object\n    hood                186030 non-null object\n    available_bikes     186030 non-null int64\n    available_docks     186030 non-null int64\n    time_interval       186030 non-null datetime64[ns]\n    created_at          186030 non-null datetime64[ns]\n    weather_summary     88053 non-null object\n    precip_intensity    88053 non-null float64\n    temperature         88053 non-null float64\n    humidity            88053 non-null float64\n    wind_speed          88053 non-null float64\n    wind_gust           88053 non-null float64\n    cloud_cover         88053 non-null float64\n    weather_status      88053 non-null object\n    updated_at          186030 non-null datetime64[ns]\n    dtypes: datetime64[ns](3), float64(8), int64(4), object(6)\n    memory usage: 29.8+ MB\n\n\n\n```python\nprint(df.head(3)) #printing to improve how this looks in the README.md markdown file\n```\n\n       station_id station_name station_status   latitude  longitude   zip  \\\n    0        3195      Sip Ave     In Service  40.730897 -74.063913  7306   \n    1        3195      Sip Ave     In Service  40.730897 -74.063913  7306   \n    2        3195      Sip Ave     In Service  40.730897 -74.063913  7306   \n    \n          borough            hood  available_bikes  available_docks  \\\n    0  New Jersey  Journal Square                1               33   \n    1  New Jersey  Journal Square                0               34   \n    2  New Jersey  Journal Square                0               34   \n    \n            time_interval          created_at weather_summary  precip_intensity  \\\n    0 2019-05-12 22:45:00 2019-05-13 02:45:04        Overcast               0.0   \n    1 2019-05-12 22:30:00 2019-05-13 02:30:04        Overcast               0.0   \n    2 2019-05-12 22:15:00 2019-05-13 02:15:05        Overcast               0.0   \n    \n       temperature  humidity  wind_speed  wind_gust  cloud_cover weather_status  \\\n    0        44.86      0.91        6.85       9.65          1.0      predicted   \n    1        44.86      0.91        6.85       9.65          1.0      predicted   \n    2        44.86      0.91        6.85       9.65          1.0      predicted   \n    \n               updated_at  \n    0 2019-05-13 02:45:04  \n    1 2019-05-13 02:45:04  \n    2 2019-05-13 02:45:04  \n\n\n#### Fixing the zip data_type issue\nThe issue related to zip codes is related to New Jersey's that start with a zero. There are two options to fix this:  \n1. Mutate the existing column to a string and insert a 0 to the beginning of the incorrect zip.  \nOR  \n2. Read the csv with dtype specified as `str` for the `zip` column. \n\nGoing to go with option #2 and re-import the csv correctly, then check that `zip` is in fact treated as a string:\n\n\n```python\ndate_cols = ['time_interval', 'updated_at', 'created_at']\ndata_types = {'zip': str}\ndf = pd.read_csv('../input/availability_interesting_original.csv', parse_dates=date_cols, dtype=data_types)\n```\n\n\n```python\ndf['zip'].dtype\n```\n\n\n\n\n    dtype('O')\n\n\n\n#### Fixing the missing weather issue\nAs for why weather_status is not set to 'observed', which would mean that the weather data my be inaccurate (since only the predicted weather was captured), I will need to first measure the extent of the problem, then remedy by fetching the correct weather data.\n\nFirst, let's get an understanding of the rows affected: \n\n\n```python\nprint(df['weather_status'].unique())\n```\n\n    ['predicted' 'observed' nan]\n\n\n\n```python\nprint('Weather Status: ')\nprint(df[df['weather_status'] == 'predicted'].count()[0], str(round(df[df['weather_status'] == 'predicted'].count()[0]/df.count()[0]*100)) + '%' , ' Predicted')\nprint(df[df['weather_status'] == 'observed'].count()[0], str(round(df[df['weather_status'] == 'observed'].count()[0]/df.count()[0]*100)) + '%', ' Observed')\nprint(df[df['weather_status'].isna()].count()[0], str(round(df[df['weather_status'].isna()].count()[0]/df.count()[0]*100)) + '%'' NAs')\n```\n\n    Weather Status: \n    6448 3.0%  Predicted\n    81605 44.0%  Observed\n    97977 53.0% NAs\n\n\nIt is apparent that there is a larger issue here: we have `nan` values in the `weather_status` column! Let's assume that there is no option to go back and re-run the original commands that fetched the weather data in the first place and that I will have to do this all here...\n\nLet's see which stations are affected by the missing weather_status data: \n\n\n```python\nprint(df[df['weather_status'].isna()]['hood'].unique())\n```\n\n    ['UES' 'Williamsburg' 'LIC' 'New York County' 'Yorkville' 'UWS'\n     'Journal Square' 'Park Slope' 'Downtown Brooklyn' 'Chelsea'\n     'Prospect Heights' 'Crown Heights' 'Lincoln Square' 'Alphabet City'\n     'Canal Street' 'Financial District' 'Little Italy' 'Tribeca'\n     'Ukrainian Village' 'Battery Park City' 'West Village' 'Clinton Hill'\n     'Lower East Side' \"Hell's Kitchen\" 'Midtown East' 'Peter Cooper Village'\n     'Stuyvesant Town-Peter Cooper Village']\n\n\n\n```python\nprint(df[df['weather_status'].isna()]['zip'].unique())\n```\n\n    ['10075' '11249' '11101' '10022' '10028' '10024' '07306' '11215' '11201'\n     '10011' '11238' '10023' '10009' '10013' '10004' '10007' '10003' '10282'\n     '10014' '11205' '10002' '10036' '10010']\n\n\n\n```python\nprint(len(df[df['weather_status'].isna()]['time_interval'].unique()))\n```\n\n    4928\n\n\n\n```python\nprint('Start: ', df[df['weather_status'].isna()][['time_interval']].values[0])\nprint('Finish: ', df[df['weather_status'].isna()][['time_interval']].values[-1])\n```\n\n    Start:  ['2019-05-21T07:45:00.000000000']\n    Finish:  ['2019-07-11T23:45:00.000000000']\n\n\nI want to get weather data at the hour-level to keep things reasonable when fetching weather data, so I am going to round down the `time_interval` column:\n\n\n```python\ndf['time_hour'] = df['time_interval'].apply(lambda x: x.replace(microsecond=0, second=0, minute=0))\n```\n\nLooks like it worked as expected:\n\n\n```python\nprint(df[['zip', 'time_interval','time_hour']].head())\n```\n\n         zip       time_interval           time_hour\n    0  07306 2019-05-12 22:45:00 2019-05-12 22:00:00\n    1  07306 2019-05-12 22:30:00 2019-05-12 22:00:00\n    2  07306 2019-05-12 22:15:00 2019-05-12 22:00:00\n    3  07306 2019-05-12 22:00:00 2019-05-12 22:00:00\n    4  07306 2019-05-12 23:30:00 2019-05-12 23:00:00\n\n\nLooking at the `nan`'s and `predicted`'s in `weather_status`, which are the unique `zip` and `time_hour` combinations that we'll need to re-fetch data for?\n\n\n```python\ndf_weather_na = df[(df['weather_status'].isna()) | (df['weather_status'] == 'predicted')][['zip','time_hour']].sort_values('time_hour').drop_duplicates()\nprint(df_weather_na.head())\nprint('Rows:', len(df_weather_na))\n```\n\n             zip           time_hour\n    0      07306 2019-05-12 22:00:00\n    12809  11101 2019-05-12 22:00:00\n    13452  11201 2019-05-12 22:00:00\n    887    10003 2019-05-12 22:00:00\n    5502   10013 2019-05-12 22:00:00\n    Rows: 22566\n\n\nThe way that the [Dark Sky Weather API](https://darksky.net/dev/docs) works is that you can fetch a whole day's worth of data for each location and it's considered one request. So rather than making an individual request for each zip and hour combination, I will be making a request for every zip and _day_ combination. Let's further reduce the granularity of the table...\n\n\n```python\ndf_weather_na['time_day'] = df_weather_na['time_hour'].apply(lambda x: x.replace(hour=0))\nprint(df_weather_na.head())\nprint('Rows:', len(df_weather_na))\n```\n\n             zip           time_hour   time_day\n    0      07306 2019-05-12 22:00:00 2019-05-12\n    12809  11101 2019-05-12 22:00:00 2019-05-12\n    13452  11201 2019-05-12 22:00:00 2019-05-12\n    887    10003 2019-05-12 22:00:00 2019-05-12\n    5502   10013 2019-05-12 22:00:00 2019-05-12\n    Rows: 22566\n\n\nWe don't need `time_hour`: \n\n\n```python\ndf_weather_na.drop('time_hour', axis=1, inplace=True)\ndf_weather_na = df_weather_na.drop_duplicates()\n```\n\n\n```python\nprint(df_weather_na.head())\nprint('Rows:', len(df_weather_na))\n```\n\n             zip   time_day\n    0      07306 2019-05-12\n    12809  11101 2019-05-12\n    13452  11201 2019-05-12\n    887    10003 2019-05-12\n    5502   10013 2019-05-12\n    Rows: 1416\n\n\nTo call the Dark Sky Weather API, we'll need to have sample lat/long coordinates to send. To do so, I will grab the coordinates of one station within each zip to represent that zip. \n\n\n```python\ndf_weather_na['latitude'] = df_weather_na['zip'].apply(lambda x: df[df['zip'] == x]['latitude'].unique()[0])\ndf_weather_na['longitude'] = df_weather_na['zip'].apply(lambda x: df[df['zip'] == x]['longitude'].unique()[0])\nprint(df_weather_na.head())\n```\n\n             zip   time_day   latitude  longitude\n    0      07306 2019-05-13  40.730897 -74.063913\n    12809  11101 2019-05-13  40.742327 -73.954117\n    13452  11201 2019-05-13  40.692418 -73.989495\n    887    10003 2019-05-13  40.729538 -73.984267\n    5502   10013 2019-05-13  40.719105 -73.999733\n\n\nLooking at what I did above, there's a less resource-intensive way to do that. First, by creating a table of zip to coordinate combinations: \n\n\n```python\ndf_zip_coord = df[['zip','latitude', 'longitude']].drop_duplicates()\ndf_zip_coord['order'] = df_zip_coord.groupby('zip').latitude.rank(method='min')\nprint(df_zip_coord.sort_values('zip')[9:15])\n```\n\n            zip   latitude  longitude  order\n    5495  10013  40.719105 -73.999733    1.0\n    5499  10013  40.719392 -74.002472    2.0\n    7077  10014  40.736529 -74.006180    1.0\n    7640  10022  40.757148 -73.972078    1.0\n    7636  10022  40.763505 -73.971092    2.0\n    8960  10023  40.775160 -73.989187    1.0\n\n\n\n```python\ndf_zip_coord = df_zip_coord[df_zip_coord['order'] == 1]\ndf_zip_coord.drop('order', inplace=True, axis=1)\nprint(df_zip_coord.head())\n```\n\n            zip   latitude  longitude\n    0     07306  40.730897 -74.063913\n    224   10002  40.720664 -73.985180\n    886   10003  40.729538 -73.984267\n    1548  10004  40.703652 -74.011678\n    2210  10007  40.714979 -74.013012\n\n\n1. Original method timing:\n\n\n```python\nstart_time_1 = time.process_time()\ndf_weather_na['latitude'] = df_weather_na['zip'].apply(lambda x: df[df['zip'] == x]['latitude'].unique()[0])\ndf_weather_na['longitude'] = df_weather_na['zip'].apply(lambda x: df[df['zip'] == x]['longitude'].unique()[0])\nend_time_1 = time.process_time()\nelapsed_time_1 = round(end_time_1 - start_time_1, 4)\nprint('Process Time Elapsed: ', elapsed_time_1)\n```\n\n    Process Time Elapsed:  42.9605\n\n\n2. Faster method timing (using lookup table):\n\n\n```python\nstart_time_2 = time.process_time()\ndf_weather_na['latitude'] = df_weather_na['zip'].apply(lambda x: df_zip_coord[df_zip_coord['zip'] == x]['latitude'].get_values()[0])\ndf_weather_na['longitude'] = df_weather_na['zip'].apply(lambda x: df_zip_coord[df_zip_coord['zip'] == x]['longitude'].get_values()[0])\nend_time_2 = time.process_time()\nelapsed_time_2 = round(end_time_2 - start_time_2, 4)\nprint('Process Time Elapsed: ', elapsed_time_2)\n```\n\n    Process Time Elapsed:  2.028\n\n\n3. And an even faster method, now that we have the lookup table, is to simply merge: \n\n\n```python\nstart_time_3 = time.process_time()\ndf_weather_na = df_weather_na.merge(df_zip_coord, how='inner', on='zip')\nend_time_3 = time.process_time()\nelapsed_time_3 = round(end_time_3 - start_time_3, 4)\nprint('Process Time Elapsed: ', elapsed_time_3)\n```\n\n    Process Time Elapsed:  0.0052\n\n\n#### Re-fetching weather data\nNow that we've reduced the number of individual requests we'll need to make to the [Dark Sky Weather API](https://darksky.net/dev/docs), we can start to setup the re-fetching process. \n\nHere is our dataset of missing weather: \n\n\n```python\nprint(df_weather_na.head())\nprint(df_weather_na.shape)\n```\n\n         zip   time_day   latitude  longitude\n    0  07306 2019-05-12  40.730897 -74.063913\n    1  07306 2019-05-13  40.730897 -74.063913\n    2  07306 2019-05-14  40.730897 -74.063913\n    3  07306 2019-05-15  40.730897 -74.063913\n    4  07306 2019-05-16  40.730897 -74.063913\n    (1416, 4)\n\n\nAs a safety measure to avoid having to through these steps all over again, let's save this dataset to a csv:\n\n\n```python\ndf_weather_na.to_csv('../input/df_weather_na.csv', index=False)\n```\n\nNow on to creating a new DataFrame of correct weather data for the locations and days in `df_weather_na`, by hour. \n\n1. Get the api key:\n\n\n```python\ntry:\n    ds_key = os.getenv(\"DARK_SKY_API_KEY\")\nexcept:\n    pass\n```\n\n2. Create a function to build a DataFrame over time. Since I need to regulate how often I call the DS API within a 24h period, I created a set of functions to allow me to pick up where I leave off with no wasted API calls. After some trial and error, I've got it working below. Comments on each function provide more details. \n\n\n```python\nfilename = '../input/df_weather_fix.csv'\n\ndef get_weather_by_day(api_key, row):\n    \"\"\"Call the DarkSky API to request single day of weather and return a Forecast object\"\"\"\n    day = dt.strftime(row.time_day, '%Y-%m-%dT%H:%M:%S')\n    lat = row.latitude\n    long = row.longitude\n    weather = forecast(api_key, lat, long, time=day)\n    return weather\n\ndef create_weather_df(zip_code, weather):\n    \"\"\"Create a DataFrame with the same columns and naming convention as primary df\"\"\"\n    df_weather = pd.DataFrame(weather['hourly']['data'])\n    df_weather.rename(columns={ 'precipIntensity': 'precip_intensity',\n                                'windSpeed': 'wind_speed',\n                                'windGust': 'wind_gust',\n                                'cloudCover': 'cloud_cover',\n                                'summary': 'weather_summary',\n                                'time': 'time_hour'\n                                }, inplace=True)\n    df_weather = df_weather[['time_hour','precip_intensity','temperature',\\\n                             'humidity', 'wind_speed', 'wind_gust', \\\n                             'weather_summary', 'cloud_cover']]\n    df_weather['time_hour'] = df_weather['time_hour'].apply(lambda x: dt.utcfromtimestamp(x-14400).strftime('%Y-%m-%d %H:%M:%S'))\n    df_weather['time_hour'] = df_weather['time_hour'].apply(lambda x: parse(x))\n    df_weather['zip'] = str(zip_code)\n    df_weather['weather_status'] = 'observed'\n    return df_weather\n\ndef get_start_index_df():\n    \"\"\"Get the latest version of the fixed weather df and the index on which to start\"\"\"\n    try:\n        df_weather_fix = pd.read_csv(filename, dtype={'zip': str})\n        df_weather_filtered = df_weather_na[(df_weather_na['zip'] == df_weather_fix.iloc[-24]['zip'])\\\n                                        \u0026 (df_weather_na['time_day'] == df_weather_fix.iloc[-24]['time_hour'])]\n        return df_weather_fix, df_weather_filtered.index[0]+1\n    except:\n        df_weather_fix = pd.DataFrame(columns=['time_hour', 'precip_intensity',\\\n                                       'temperature', 'humidity',\\\n                                       'wind_speed', 'wind_gust',\\\n                                       'weather_summary', 'cloud_cover',\\\n                                       'zip', 'weather_status'])\n        return df_weather_fix, 0\n    \ndef get_weather_fix(ds_key, api_limit, df_weather_na):\n    \"\"\"Create an on-going df which contains missing weather data in increments as defined in api_limit\"\"\"\n    df_weather_fix, start_index = get_start_index_df()\n    api_limit = api_limit\n    for index, row in df_weather_na[start_index:].iterrows():\n        if index \u003c (api_limit + start_index):\n            weather = get_weather_by_day(ds_key, row)\n            df_weather_day = create_weather_df(row['zip'], weather)\n            df_weather_fix = df_weather_fix.append(df_weather_day)\n        else:\n            df_weather_fix.to_csv(filename, index=False)\n            print('Saved weather_fix to csv after', 'getting weather for', row['zip'], 'on', str(row['time_day']).split(' ')[0])\n            break\n        if index % 10 == 0:\n            df_weather_fix.to_csv(filename, index=False)\n            print('Saved weather_fix to csv after', 'getting weather for', row['zip'], 'on', str(row['time_day']).split(' ')[0])\n        time.sleep(3)\n    print('DONE!')\n    return df_weather_fix\n```\n\n\n```python\ndf_weather_fix = get_weather_fix(ds_key, 999, df_weather_na)\n```\n\n    Saved weather_fix to csv after getting weather for 07306 on 2019-05-22\n    Saved weather_fix to csv after getting weather for 07306 on 2019-06-01\n    Saved weather_fix to csv after getting weather for 07306 on 2019-06-11\n    Saved weather_fix to csv after getting weather for 07306 on 2019-06-21\n    Saved weather_fix to csv after getting weather for 07306 on 2019-07-01\n    Saved weather_fix to csv after getting weather for 07306 on 2019-07-11\n    Saved weather_fix to csv after getting weather for 11101 on 2019-05-20\n    Saved weather_fix to csv after getting weather for 11101 on 2019-05-30\n    Saved weather_fix to csv after getting weather for 11101 on 2019-06-09\n    Saved weather_fix to csv after getting weather for 11101 on 2019-06-19\n    Saved weather_fix to csv after getting weather for 11101 on 2019-06-29\n    Saved weather_fix to csv after getting weather for 11101 on 2019-07-09\n    Saved weather_fix to csv after getting weather for 11201 on 2019-05-18\n    Saved weather_fix to csv after getting weather for 11201 on 2019-05-28\n    Saved weather_fix to csv after getting weather for 11201 on 2019-06-07\n    Saved weather_fix to csv after getting weather for 11201 on 2019-06-17\n    Saved weather_fix to csv after getting weather for 11201 on 2019-06-27\n    Saved weather_fix to csv after getting weather for 11201 on 2019-07-07\n    Saved weather_fix to csv after getting weather for 10003 on 2019-05-16\n    Saved weather_fix to csv after getting weather for 10003 on 2019-05-26\n    Saved weather_fix to csv after getting weather for 10003 on 2019-06-05\n    Saved weather_fix to csv after getting weather for 10003 on 2019-06-15\n    Saved weather_fix to csv after getting weather for 10003 on 2019-06-25\n    Saved weather_fix to csv after getting weather for 10003 on 2019-07-05\n    Saved weather_fix to csv after getting weather for 10013 on 2019-05-14\n    Saved weather_fix to csv after getting weather for 10013 on 2019-05-24\n    Saved weather_fix to csv after getting weather for 10013 on 2019-06-03\n    Saved weather_fix to csv after getting weather for 10013 on 2019-06-13\n    Saved weather_fix to csv after getting weather for 10013 on 2019-06-23\n    Saved weather_fix to csv after getting weather for 10013 on 2019-07-03\n    Saved weather_fix to csv after getting weather for 11205 on 2019-05-16\n    Saved weather_fix to csv after getting weather for 11205 on 2019-05-26\n    Saved weather_fix to csv after getting weather for 11205 on 2019-06-05\n    Saved weather_fix to csv after getting weather for 11205 on 2019-06-15\n    Saved weather_fix to csv after getting weather for 11205 on 2019-06-25\n    Saved weather_fix to csv after getting weather for 11205 on 2019-07-08\n    Saved weather_fix to csv after getting weather for 10282 on 2019-05-18\n    Saved weather_fix to csv after getting weather for 10282 on 2019-05-28\n    Saved weather_fix to csv after getting weather for 10282 on 2019-06-07\n    Saved weather_fix to csv after getting weather for 10282 on 2019-06-17\n    Saved weather_fix to csv after getting weather for 10282 on 2019-06-27\n    Saved weather_fix to csv after getting weather for 10282 on 2019-07-07\n    Saved weather_fix to csv after getting weather for 11238 on 2019-05-16\n    Saved weather_fix to csv after getting weather for 11238 on 2019-05-26\n    Saved weather_fix to csv after getting weather for 11238 on 2019-06-05\n    Saved weather_fix to csv after getting weather for 11238 on 2019-06-15\n    Saved weather_fix to csv after getting weather for 11238 on 2019-06-25\n    Saved weather_fix to csv after getting weather for 11238 on 2019-07-05\n    Saved weather_fix to csv after getting weather for 10075 on 2019-05-14\n    Saved weather_fix to csv after getting weather for 10075 on 2019-05-24\n    Saved weather_fix to csv after getting weather for 10075 on 2019-06-03\n    Saved weather_fix to csv after getting weather for 10075 on 2019-06-13\n    Saved weather_fix to csv after getting weather for 10075 on 2019-06-23\n    Saved weather_fix to csv after getting weather for 10075 on 2019-07-03\n    Saved weather_fix to csv after getting weather for 10002 on 2019-05-12\n    Saved weather_fix to csv after getting weather for 10002 on 2019-05-22\n    Saved weather_fix to csv after getting weather for 10002 on 2019-06-01\n    Saved weather_fix to csv after getting weather for 10002 on 2019-06-11\n    Saved weather_fix to csv after getting weather for 10002 on 2019-06-21\n    Saved weather_fix to csv after getting weather for 10002 on 2019-07-01\n    Saved weather_fix to csv after getting weather for 10002 on 2019-07-11\n    Saved weather_fix to csv after getting weather for 10014 on 2019-05-20\n    Saved weather_fix to csv after getting weather for 10014 on 2019-05-30\n    Saved weather_fix to csv after getting weather for 10014 on 2019-06-09\n    Saved weather_fix to csv after getting weather for 10014 on 2019-06-19\n    Saved weather_fix to csv after getting weather for 10014 on 2019-06-29\n    Saved weather_fix to csv after getting weather for 10014 on 2019-07-11\n    Saved weather_fix to csv after getting weather for 11215 on 2019-05-20\n    Saved weather_fix to csv after getting weather for 11215 on 2019-05-30\n    Saved weather_fix to csv after getting weather for 11215 on 2019-06-09\n    Saved weather_fix to csv after getting weather for 11215 on 2019-06-19\n    Saved weather_fix to csv after getting weather for 11215 on 2019-06-29\n    Saved weather_fix to csv after getting weather for 11215 on 2019-07-09\n    Saved weather_fix to csv after getting weather for 10009 on 2019-05-18\n    Saved weather_fix to csv after getting weather for 10009 on 2019-05-28\n    Saved weather_fix to csv after getting weather for 10009 on 2019-06-07\n    Saved weather_fix to csv after getting weather for 10009 on 2019-06-17\n    Saved weather_fix to csv after getting weather for 10009 on 2019-06-27\n    Saved weather_fix to csv after getting weather for 10009 on 2019-07-07\n    Saved weather_fix to csv after getting weather for 10007 on 2019-05-16\n    Saved weather_fix to csv after getting weather for 10007 on 2019-05-26\n    Saved weather_fix to csv after getting weather for 10007 on 2019-06-05\n    Saved weather_fix to csv after getting weather for 10007 on 2019-06-15\n    Saved weather_fix to csv after getting weather for 10007 on 2019-06-25\n    Saved weather_fix to csv after getting weather for 10007 on 2019-07-05\n    Saved weather_fix to csv after getting weather for 10004 on 2019-05-14\n    Saved weather_fix to csv after getting weather for 10004 on 2019-05-24\n    Saved weather_fix to csv after getting weather for 10004 on 2019-06-03\n    Saved weather_fix to csv after getting weather for 10004 on 2019-06-13\n    Saved weather_fix to csv after getting weather for 10004 on 2019-06-23\n    Saved weather_fix to csv after getting weather for 10004 on 2019-07-03\n    Saved weather_fix to csv after getting weather for 10010 on 2019-05-12\n    Saved weather_fix to csv after getting weather for 10010 on 2019-05-22\n    Saved weather_fix to csv after getting weather for 10010 on 2019-06-01\n    Saved weather_fix to csv after getting weather for 10010 on 2019-06-11\n    Saved weather_fix to csv after getting weather for 10010 on 2019-06-21\n    Saved weather_fix to csv after getting weather for 10010 on 2019-07-01\n    Saved weather_fix to csv after getting weather for 10010 on 2019-07-11\n    Saved weather_fix to csv after getting weather for 10011 on 2019-05-20\n    Saved weather_fix to csv after getting weather for 10011 on 2019-05-29\n    DONE!\n\n\n\n```python\ndf_weather_fix.to_csv('../input/df_weather_fix.csv', index=False)\n```\n\nSuccessfully kept the total API requests to just below 1000 on Day 1:\n![Dark Sky API Calls today](https://blog.alhan.co/storage/images/posts/2/dark-sky-api-calls_2_1569210787_md.jpg)\n\n(And finished the remaining locations on Day 2)\n\n#### Joining the fixed weather data\nTo avoid creating unnecessary columns by joining the fixed weather data to the entire df, I separate `df` into two:\n- Observed\n- NAs and Predicted (the dataset that needed joining with the fixed weather)\n\n\n```python\ndate_cols = ['time_interval', 'updated_at', 'created_at']\ndata_types = {'zip': str}\ndf = pd.read_csv('../input/availability_interesting_original.csv', parse_dates=date_cols, dtype=data_types)\n\ndf['time_hour'] = df['time_interval'].apply(lambda x: x.replace(microsecond=0, second=0, minute=0))\n\n# Rows that are already clean and don't need changing:\ndf_weather_observed = df[df['weather_status'] == 'observed']\n\n# Rows that need weather data corrected:\ndf_weather_na_predicted = df[(df['weather_status'].isna()) | (df['weather_status'] == 'predicted')]\n\n# Removing the bad columns:\ndf_weather_na_predicted.drop(['precip_intensity', 'temperature', 'humidity',\\\n                              'wind_speed', 'wind_gust', 'weather_summary', \\\n                              'cloud_cover', 'weather_status'], axis=1, inplace=True)\n\n# Getting the newly created fixed weather data:\ndf_weather_fix = pd.read_csv('../input/df_weather_fix.csv', dtype={'zip': str}, parse_dates=['time_hour'])\n\n# Joining with the corrected weather data:\ndf_weather_fixed = df_weather_na_predicted.merge(df_weather_fix, how='left', on=['time_hour', 'zip'])\n\n# Concatenating the already fixed weather rows with the newly fixed rows:\ndf_fixed = pd.concat([df_weather_observed, df_weather_fixed])\ndf_fixed = df_fixed.drop_duplicates()\n\n# Quickly checking that it worked and that we didn't break anything:\nprint(df_weather_observed.info())\nprint(df_weather_fixed.info())\nprint(df_fixed.info())\nprint('NAs or Predicted:', len(df_fixed[(df_fixed['weather_status'].isna()) | (df_fixed['weather_status'] == 'predicted')].index))\n\n```\n\n    \u003cclass 'pandas.core.frame.DataFrame'\u003e\n    Int64Index: 81605 entries, 72 to 186029\n    Data columns (total 22 columns):\n    station_id          81605 non-null int64\n    station_name        81605 non-null object\n    station_status      81605 non-null object\n    latitude            81605 non-null float64\n    longitude           81605 non-null float64\n    zip                 81605 non-null object\n    borough             81605 non-null object\n    hood                81605 non-null object\n    available_bikes     81605 non-null int64\n    available_docks     81605 non-null int64\n    time_interval       81605 non-null datetime64[ns]\n    created_at          81605 non-null datetime64[ns]\n    weather_summary     81605 non-null object\n    precip_intensity    81605 non-null float64\n    temperature         81605 non-null float64\n    humidity            81605 non-null float64\n    wind_speed          81605 non-null float64\n    wind_gust           81605 non-null float64\n    cloud_cover         81605 non-null float64\n    weather_status      81605 non-null object\n    updated_at          81605 non-null datetime64[ns]\n    time_hour           81605 non-null datetime64[ns]\n    dtypes: datetime64[ns](4), float64(8), int64(3), object(7)\n    memory usage: 14.3+ MB\n    None\n    \u003cclass 'pandas.core.frame.DataFrame'\u003e\n    Int64Index: 104425 entries, 0 to 104424\n    Data columns (total 22 columns):\n    station_id          104425 non-null int64\n    station_name        104425 non-null object\n    station_status      104425 non-null object\n    latitude            104425 non-null float64\n    longitude           104425 non-null float64\n    zip                 104425 non-null object\n    borough             104425 non-null object\n    hood                104425 non-null object\n    available_bikes     104425 non-null int64\n    available_docks     104425 non-null int64\n    time_interval       104425 non-null datetime64[ns]\n    created_at          104425 non-null datetime64[ns]\n    updated_at          104425 non-null datetime64[ns]\n    time_hour           104425 non-null datetime64[ns]\n    precip_intensity    104425 non-null float64\n    temperature         104425 non-null float64\n    humidity            104425 non-null float64\n    wind_speed          104425 non-null float64\n    wind_gust           104425 non-null float64\n    weather_summary     104425 non-null object\n    cloud_cover         104425 non-null float64\n    weather_status      104425 non-null object\n    dtypes: datetime64[ns](4), float64(8), int64(3), object(7)\n    memory usage: 18.3+ MB\n    None\n    \u003cclass 'pandas.core.frame.DataFrame'\u003e\n    Int64Index: 186030 entries, 72 to 104424\n    Data columns (total 22 columns):\n    available_bikes     186030 non-null int64\n    available_docks     186030 non-null int64\n    borough             186030 non-null object\n    cloud_cover         186030 non-null float64\n    created_at          186030 non-null datetime64[ns]\n    hood                186030 non-null object\n    humidity            186030 non-null float64\n    latitude            186030 non-null float64\n    longitude           186030 non-null float64\n    precip_intensity    186030 non-null float64\n    station_id          186030 non-null int64\n    station_name        186030 non-null object\n    station_status      186030 non-null object\n    temperature         186030 non-null float64\n    time_hour           186030 non-null datetime64[ns]\n    time_interval       186030 non-null datetime64[ns]\n    updated_at          186030 non-null datetime64[ns]\n    weather_status      186030 non-null object\n    weather_summary     186030 non-null object\n    wind_gust           186030 non-null float64\n    wind_speed          186030 non-null float64\n    zip                 186030 non-null object\n    dtypes: datetime64[ns](4), float64(8), int64(3), object(7)\n    memory usage: 32.6+ MB\n    None\n    NAs or Predicted: 0\n\n\nRe-creating `df`, this time with fixed weather data:\n\n\n```python\ndf_fixed.to_csv('../input/availability_interesting_weather_fix.csv', index=False)\n```\n\n### Exploratory Data Analysis\nOn to the fun part. Now that I've got all of my station-by-station availability by 15-minute interval, it's time to explore. \n\n\n```python\ndate_cols = ['time_interval', 'updated_at', 'created_at', 'time_hour']\ndata_types = {'zip': str}\ndf = pd.read_csv('../input/availability_interesting_weather_fix.csv', dtype={'zip': str}, parse_dates=date_cols)\n```\n\n#### Hypotheses\n\nBelow is a list of hypotheses, in no particular order, that may be interesting to validate in the following analysis:  \n- There will be various categories of stations where usage patterns are similar.\n- On days where there is rain observed, overall usage will be lower.  \n- On business days before and after a holiday, there may be a decrease in overall Citi Bike usage. Might be worth excluding these days entirely from the analysis as they do no represent business days or weekends. Luckily, we only have the 4th of July to deal with as part of this dataset. \n\n\n```python\ndf.info()\n```\n\n    \u003cclass 'pandas.core.frame.DataFrame'\u003e\n    RangeIndex: 186030 entries, 0 to 186029\n    Data columns (total 22 columns):\n    available_bikes     186030 non-null int64\n    available_docks     186030 non-null int64\n    borough             186030 non-null object\n    cloud_cover         186030 non-null float64\n    created_at          186030 non-null datetime64[ns]\n    hood                186030 non-null object\n    humidity            186030 non-null float64\n    latitude            186030 non-null float64\n    longitude           186030 non-null float64\n    precip_intensity    186030 non-null float64\n    station_id          186030 non-null int64\n    station_name        186030 non-null object\n    station_status      186030 non-null object\n    temperature         186030 non-null float64\n    time_hour           186030 non-null datetime64[ns]\n    time_interval       186030 non-null datetime64[ns]\n    updated_at          186030 non-null datetime64[ns]\n    weather_status      186030 non-null object\n    weather_summary     186030 non-null object\n    wind_gust           186030 non-null float64\n    wind_speed          186030 non-null float64\n    zip                 186030 non-null object\n    dtypes: datetime64[ns](4), float64(8), int64(3), object(7)\n    memory usage: 31.2+ MB\n\n\n#### How does time of day effect Citi Bike availability?\n\nCreate time of day column:\n\n\n```python\ndf['hour'] = df['time_hour'].apply(lambda x: x.hour)\n```\n\nVisualize average availability by hour:\n\n\n```python\nsns.lineplot(x=df['hour'], y=df['available_bikes'])\n```\n\n\n\n\n    \u003cmatplotlib.axes._subplots.AxesSubplot at 0x120d68c50\u003e\n\n\n\n\n![png](README_files/README_76_1.png)\n\n\n\n```python\nsns.distplot(df['hour'])\n```\n\n\n\n\n    \u003cmatplotlib.axes._subplots.AxesSubplot at 0x120d53ba8\u003e\n\n\n\n\n![png](README_files/README_77_1.png)\n\n\n\n```python\ndf['time'] = df['time_interval'].apply(lambda x: x.time())\n```\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x=df['time'], y=df['available_bikes'])\n```\n\n\n\n\n    \u003cmatplotlib.axes._subplots.AxesSubplot at 0x125612eb8\u003e\n\n\n\n\n![png](README_files/README_79_1.png)\n\n\n\n```python\ndf['day'] = df['time_interval'].apply(lambda x: x.strftime('%A'))\n```\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df, hue='day')\nplt.show()\n```\n\n\n![png](README_files/README_81_0.png)\n\n\n\n```python\ndef get_day_type(x):\n    if x.weekday() \u003e 4:\n        return 'weekend'\n    return 'weekday'\n\ndf['day_type'] = df['time_interval'].apply(lambda x: get_day_type(x))\n```\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df, hue='day_type')\nplt.show()\n```\n\n\n![png](README_files/README_83_0.png)\n\n\n\n```python\nfor station in df['station_name'].unique():\n    fig, ax = plt.subplots(figsize=(16,10))\n    sns.lineplot(x='time', y='available_bikes', data=df[df['station_name'] == station], hue='day').set_title(station)\n    plt.xticks(df['time'].unique(), rotation=90)\n    plt.show()\n```\n\n\n![png](README_files/README_84_0.png)\n\n\n\n![png](README_files/README_84_1.png)\n\n\n\n![png](README_files/README_84_2.png)\n\n\n\n![png](README_files/README_84_3.png)\n\n\n\n![png](README_files/README_84_4.png)\n\n\n\n![png](README_files/README_84_5.png)\n\n\n\n![png](README_files/README_84_6.png)\n\n\n\n![png](README_files/README_84_7.png)\n\n\n\n![png](README_files/README_84_8.png)\n\n\n\n![png](README_files/README_84_9.png)\n\n\n\n![png](README_files/README_84_10.png)\n\n\n\n![png](README_files/README_84_11.png)\n\n\n\n![png](README_files/README_84_12.png)\n\n\n\n![png](README_files/README_84_13.png)\n\n\n\n![png](README_files/README_84_14.png)\n\n\n\n![png](README_files/README_84_15.png)\n\n\n\n![png](README_files/README_84_16.png)\n\n\n\n![png](README_files/README_84_17.png)\n\n\n\n![png](README_files/README_84_18.png)\n\n\n\n![png](README_files/README_84_19.png)\n\n\n\n![png](README_files/README_84_20.png)\n\n\n\n![png](README_files/README_84_21.png)\n\n\n\n![png](README_files/README_84_22.png)\n\n\n\n![png](README_files/README_84_23.png)\n\n\n\n![png](README_files/README_84_24.png)\n\n\n\n![png](README_files/README_84_25.png)\n\n\n\n![png](README_files/README_84_26.png)\n\n\n\n```python\nfor station in df['station_name'].unique():\n    fig, ax = plt.subplots(figsize=(16,10))\n    sns.lineplot(x='time', y='available_bikes', data=df[df['station_name'] == station], hue='day_type').set_title(station)\n    plt.xticks(df['time'].unique(), rotation=90)\n    plt.show()\n```\n\n\n![png](README_files/README_85_0.png)\n\n\n\n![png](README_files/README_85_1.png)\n\n\n\n![png](README_files/README_85_2.png)\n\n\n\n![png](README_files/README_85_3.png)\n\n\n\n![png](README_files/README_85_4.png)\n\n\n\n![png](README_files/README_85_5.png)\n\n\n\n![png](README_files/README_85_6.png)\n\n\n\n![png](README_files/README_85_7.png)\n\n\n\n![png](README_files/README_85_8.png)\n\n\n\n![png](README_files/README_85_9.png)\n\n\n\n![png](README_files/README_85_10.png)\n\n\n\n![png](README_files/README_85_11.png)\n\n\n\n![png](README_files/README_85_12.png)\n\n\n\n![png](README_files/README_85_13.png)\n\n\n\n![png](README_files/README_85_14.png)\n\n\n\n![png](README_files/README_85_15.png)\n\n\n\n![png](README_files/README_85_16.png)\n\n\n\n![png](README_files/README_85_17.png)\n\n\n\n![png](README_files/README_85_18.png)\n\n\n\n![png](README_files/README_85_19.png)\n\n\n\n![png](README_files/README_85_20.png)\n\n\n\n![png](README_files/README_85_21.png)\n\n\n\n![png](README_files/README_85_22.png)\n\n\n\n![png](README_files/README_85_23.png)\n\n\n\n![png](README_files/README_85_24.png)\n\n\n\n![png](README_files/README_85_25.png)\n\n\n\n![png](README_files/README_85_26.png)\n\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df[(df['station_name'] == 'Sip Ave') \u0026 (df['day_type'] == 'weekday')], hue='weather_summary')\nplt.xticks(df['time'].unique(), rotation=90)\nplt.show()\n```\n\n\n![png](README_files/README_86_0.png)\n\n\n\n```python\ndf['pi_rounded'] = df['precip_intensity'].apply(lambda x: round(x,1))\ndf.pi_rounded.unique()\n```\n\n\n\n\n    array([0. , 0.1, 0.2, 0.5, 0.3, 0.4, 0.6])\n\n\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df[(df['station_name'] == 'Sip Ave') \u0026 (df['day_type'] == 'weekday')], hue='pi_rounded')\nplt.xticks(df['time'].unique(), rotation=90)\nplt.show()\n```\n\n\n![png](README_files/README_88_0.png)\n\n\n\n```python\ndf['is_raining'] = df['precip_intensity'].apply(lambda x: x \u003e 0)\ndf.is_raining.unique()\n```\n\n\n\n\n    array([False,  True])\n\n\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df[(df['station_name'] == 'Sip Ave') \u0026 (df['day_type'] == 'weekday')], hue='is_raining')\nplt.xticks(df['time'].unique(), rotation=90)\nplt.show()\n```\n\n\n![png](README_files/README_90_0.png)\n\n\n\n```python\nfor station in df['station_name'].unique():\n    fig, ax = plt.subplots(figsize=(16,10))\n    sns.lineplot(x='time', y='available_bikes', data=df[(df['station_name'] == station) \u0026 (df['day_type'] == 'weekday')], hue='is_raining').set_title(station)\n    plt.xticks(df['time'].unique(), rotation=90)\n    plt.show()\n```\n\n\n![png](README_files/README_91_0.png)\n\n\n\n![png](README_files/README_91_1.png)\n\n\n\n![png](README_files/README_91_2.png)\n\n\n\n![png](README_files/README_91_3.png)\n\n\n\n![png](README_files/README_91_4.png)\n\n\n\n![png](README_files/README_91_5.png)\n\n\n\n![png](README_files/README_91_6.png)\n\n\n\n![png](README_files/README_91_7.png)\n\n\n\n![png](README_files/README_91_8.png)\n\n\n\n![png](README_files/README_91_9.png)\n\n\n\n![png](README_files/README_91_10.png)\n\n\n\n![png](README_files/README_91_11.png)\n\n\n\n![png](README_files/README_91_12.png)\n\n\n\n![png](README_files/README_91_13.png)\n\n\n\n![png](README_files/README_91_14.png)\n\n\n\n![png](README_files/README_91_15.png)\n\n\n\n![png](README_files/README_91_16.png)\n\n\n\n![png](README_files/README_91_17.png)\n\n\n\n![png](README_files/README_91_18.png)\n\n\n\n![png](README_files/README_91_19.png)\n\n\n\n![png](README_files/README_91_20.png)\n\n\n\n![png](README_files/README_91_21.png)\n\n\n\n![png](README_files/README_91_22.png)\n\n\n\n![png](README_files/README_91_23.png)\n\n\n\n![png](README_files/README_91_24.png)\n\n\n\n![png](README_files/README_91_25.png)\n\n\n\n![png](README_files/README_91_26.png)\n\n\n\n```python\nfor station in df['station_name'].unique():\n    fig, ax = plt.subplots(figsize=(16,10))\n    sns.lineplot(x='time', y='available_bikes', data=df[(df['station_name'] == station) \u0026 (df['day_type'] == 'weekend')], hue='is_raining').set_title(station)\n    plt.xticks(df['time'].unique(), rotation=90)\n    plt.show()\n```\n\n\n![png](README_files/README_92_0.png)\n\n\n\n![png](README_files/README_92_1.png)\n\n\n\n![png](README_files/README_92_2.png)\n\n\n\n![png](README_files/README_92_3.png)\n\n\n\n![png](README_files/README_92_4.png)\n\n\n\n![png](README_files/README_92_5.png)\n\n\n\n![png](README_files/README_92_6.png)\n\n\n\n![png](README_files/README_92_7.png)\n\n\n\n![png](README_files/README_92_8.png)\n\n\n\n![png](README_files/README_92_9.png)\n\n\n\n![png](README_files/README_92_10.png)\n\n\n\n![png](README_files/README_92_11.png)\n\n\n\n![png](README_files/README_92_12.png)\n\n\n\n![png](README_files/README_92_13.png)\n\n\n\n![png](README_files/README_92_14.png)\n\n\n\n![png](README_files/README_92_15.png)\n\n\n\n![png](README_files/README_92_16.png)\n\n\n\n![png](README_files/README_92_17.png)\n\n\n\n![png](README_files/README_92_18.png)\n\n\n\n![png](README_files/README_92_19.png)\n\n\n\n![png](README_files/README_92_20.png)\n\n\n\n![png](README_files/README_92_21.png)\n\n\n\n![png](README_files/README_92_22.png)\n\n\n\n![png](README_files/README_92_23.png)\n\n\n\n![png](README_files/README_92_24.png)\n\n\n\n![png](README_files/README_92_25.png)\n\n\n\n![png](README_files/README_92_26.png)\n\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df[(df['day_type'] == 'weekday')], hue='is_raining')\nplt.xticks(df['time'].unique(), rotation=90)\nplt.show()\n```\n\n\n![png](README_files/README_93_0.png)\n\n\n\n```python\ndf['date'] = df['time_hour'].apply(lambda x: x.date())\ndf['is_raining'] = df['is_raining'].apply(lambda x: int(x))\ndf_rainy_days = df.groupby(['station_id', 'date'])['is_raining'].max().reset_index()\n```\n\n\n```python\ndf_rainy_days.rename(columns={'is_raining': 'rainy_day'}, inplace=True)\n```\n\n\n```python\ndf_rainy_days.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\u003cstyle scoped\u003e\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n\u003c/style\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003estation_id\u003c/th\u003e\n      \u003cth\u003edate\u003c/th\u003e\n      \u003cth\u003erainy_day\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003e150\u003c/td\u003e\n      \u003ctd\u003e2019-05-02\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e150\u003c/td\u003e\n      \u003ctd\u003e2019-05-03\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n      \u003ctd\u003e150\u003c/td\u003e\n      \u003ctd\u003e2019-05-04\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e3\u003c/td\u003e\n      \u003ctd\u003e150\u003c/td\u003e\n      \u003ctd\u003e2019-05-05\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e4\u003c/td\u003e\n      \u003ctd\u003e150\u003c/td\u003e\n      \u003ctd\u003e2019-05-06\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\ndf = df.merge(df_rainy_days, on=['station_id', 'date'])\n```\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df[(df['day_type'] == 'weekday')], hue='rainy_day')\nplt.xticks(df['time'].unique(), rotation=90)\nplt.show()\n```\n\n\n![png](README_files/README_98_0.png)\n\n\n\n```python\nfig, ax = plt.subplots(figsize=(16,10))\nsns.lineplot(x='time', y='available_bikes', data=df[(df['day_type'] == 'weekend')], hue='rainy_day')\nplt.xticks(df['time'].unique(), rotation=90)\nplt.show()\n```\n\n\n![png](README_files/README_99_0.png)\n\n\nAuto-Generate README.md:\n\n\n```python\n!jupyter nbconvert --output-dir='..' --to markdown analysis.ipynb --output README.md\n```\n\n    [NbConvertApp] Converting notebook analysis.ipynb to markdown\n    [NbConvertApp] Support files will be in README_files/\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Making directory ../README_files\n    [NbConvertApp] Writing 59166 bytes to ../README.md\n\n\n[Powered by Dark Sky](https://darksky.net/poweredby/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falhankeser%2Fcitibike-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falhankeser%2Fcitibike-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falhankeser%2Fcitibike-analysis/lists"}