https://github.com/ophiase/big-data-project-ifeby310
Analysis website of the New York Shared Bike systems (Citibikes 🚲️) dataset. Extract Load Transform using pyspark in parquet format.
https://github.com/ophiase/big-data-project-ifeby310
bigdata spark
Last synced: 3 months ago
JSON representation
Analysis website of the New York Shared Bike systems (Citibikes 🚲️) dataset. Extract Load Transform using pyspark in parquet format.
- Host: GitHub
- URL: https://github.com/ophiase/big-data-project-ifeby310
- Owner: Ophiase
- License: apache-2.0
- Created: 2024-05-17T11:37:26.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-06-04T21:08:34.000Z (12 months ago)
- Last Synced: 2025-01-19T08:32:36.371Z (4 months ago)
- Topics: bigdata, spark
- Language: Jupyter Notebook
- Homepage: https://ophiase.github.io/Big-Data-Project-IFEBY310/
- Size: 17.9 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# [Analysis of Citibikes - Big Data](https://ophiase.github.io/Big-Data-Project-IFEBY310/)
Analysis of the *NYC Shared Bike systems* (**Citibikes**) dataset for a University Course.
A precompiled report is availible online at : [web report](https://ophiase.github.io/Big-Data-Project-IFEBY310/)
You can also run the precompiled report locally :
```bash
# precompiled version
cd docs
python3 -m http.server 8000 # create the server
xdg-open http://localhost:8000/ # open the local server
make preview_web # with quarto
```
## Installation
Python notebook dependencies :
- Graphics : `seaborn, altair, plotly`
- Geographic data : `geojson, geopandas, geopy, ipyleaflet`
- Others : `jupyter-lab, numpy, pandas, pyspark`
## Execution
Execute the jupyter notebook `report.ipynb`
## Sources
- Dataset :
- [https://citibikenyc.com/system-data](https://citibikenyc.com/system-data)
- [https://api.citybik.es/citi-bike-nyc.json](https://api.citybik.es/citi-bike-nyc.json)
- [https://s3.amazonaws.com/tripdata/index.html](https://s3.amazonaws.com/tripdata/index.html)
- Other informations :
- [https://bikeshare-research.org](https://bikeshare-research.org)
- [https://citibikenyc.com](https://citibikenyc.com)
- [Real time feed](https://citibikenyc.com/)
- [NYC Bike feeds data](https://gbfs.citibikenyc.com/gbfs/2.3/gbfs.json)
- [Station status feed](https://www.nyc.gov/html/dot/html/about/datafeeds.shtml#Bikes)
- [Bike routes](https://data.cityofnewyork.us/Transportation/New-York-City-Bike-Routes/7vsa-caz7)
- OpenStreetMap map of New York City