https://github.com/lucy0906/nyc-open-data-business-etl

This project analyze licenses issued by DCWP to businesses and individuals so that they may legally operate in New York City.
https://github.com/lucy0906/nyc-open-data-business-etl

data-visualization etl

Last synced: about 2 months ago
JSON representation

This project analyze licenses issued by DCWP to businesses and individuals so that they may legally operate in New York City.

• Host: GitHub
• URL: https://github.com/lucy0906/nyc-open-data-business-etl
• Owner: Lucy0906
• Created: 2024-04-28T20:38:15.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-09-16T19:39:07.000Z (4 months ago)
• Last Synced: 2024-09-17T00:55:45.496Z (4 months ago)
• Topics: data-visualization, etl
• Language: Jupyter Notebook
• Homepage: https://lucy0906.github.io/NYC-Open-Data-Business-ETL/
• Size: 20.3 MB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## CIS9440 HW1&2

## HW1 Data Sourcing 

## DataSource

Here is my data source: https://data.cityofnewyork.us/Business/Legally-Operating-Businesses/w7w3-xahh/about_data

This link directs to the NYC Open Data portal where the dataset can be accessed directly. Data Provided By Department of Consumer and Worker Protection (DCWP)

## Metadata

This dataset features licenses issued by DCWP to businesses and individuals so that they may legally operate in New York City.

## Datasize

This data has 281K rows 27 Columns and each row is aDCA-Issued License

## Description

This dataset reflects data as of 7/21/2023. The Department of Consumer and Worker Protection (DCWP) is working on an updated version of this dataset. This dataset features licenses issued by DCWP to businesses and individuals so that they may legally operate in New York City. This dataset is maintained by the City of New York and contains comprehensive information about businesses that are legally licensed to operate within the city limits. It includes details such as business names, addresses, industry types, license numbers, and status.

## Data dictionary 

Here is the data dictionary link: https://data.cityofnewyork.us/Business/Legally-Operating-Businesses/w7w3-xahh/about_data

## HW1 Storage

I use Azure Blob Storage to store data.

## HW1 Data Modeling

I use supabase to create the following diagram. Dimensional modeling for DCWP data involves creating a structure that facilitates analysis and reporting. This includes defining dimensions such as business type and date.

![image](https://github.com/Lucy0906/HW/assets/159973096/f5c927b8-5ae3-4389-8040-07320ca52c7c)

## HW2 Transformation

I use ETL tools to do the transformation and creat the data mapping.

## HW2 Data Modeling

I use supabase to create the following diagram. Dimensional modeling for DCWP data involves creating a structure that facilitates analysis and reporting. This includes defining dimensions such as business type and date.

![image](https://github.com/Lucy0906/HW/assets/159973096/cbe67581-0458-4bf3-b1ac-f6c40819de6d)

## HW2 Serving Data 

I use the tableau to do data visualization.

Visualizations:[https://public.tableau.com/app/profile/lu.chen2788/viz/HW1_17156589017020/Dashboard1?publish=yes](https://public.tableau.com/app/profile/lu.chen2788/viz/HW2viz/Sheet4)