https://github.com/shivabajelan/crowdfunding_etl
The goal of this project is to build an ETL pipeline using Python, Pandas, Python dictionary methods to extract and transform the data. Four CSV files will be created and they will be used to create an ERD and a table schema. Finally, the CSV file data will be uploaded into a Postgres database.
https://github.com/shivabajelan/crowdfunding_etl
erd etl-pipeline pandas postgresql python
Last synced: about 2 months ago
JSON representation
The goal of this project is to build an ETL pipeline using Python, Pandas, Python dictionary methods to extract and transform the data. Four CSV files will be created and they will be used to create an ERD and a table schema. Finally, the CSV file data will be uploaded into a Postgres database.
- Host: GitHub
- URL: https://github.com/shivabajelan/crowdfunding_etl
- Owner: Shivabajelan
- License: mit
- Created: 2024-02-13T14:47:28.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-08-08T14:25:07.000Z (9 months ago)
- Last Synced: 2025-01-15T16:04:47.795Z (3 months ago)
- Topics: erd, etl-pipeline, pandas, postgresql, python
- Language: Jupyter Notebook
- Homepage:
- Size: 1.77 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Crowdfunding_ETL
A project to build an ETL pipeline using Python, Pandas, and PostgreSQL for extracting, transforming, and loading crowdfunding data from Excel files into a relational database.
## Background
In this project, I practiced building an ETL pipeline to process crowdfunding data from Excel files. I extracted and transformed the data, created four CSV files, and then used the CSV files to create an Entity Relationship Diagram (ERD) and a table schema. Finally, I loaded the CSV files into a PostgreSQL database.
## Features
* Extract and transform crowdfunding and contact data from Excel files
* Create and export Category, Subcategory, Campaign, and Contact DataFrames as CSV files
* Design an ERD and table schema for the database
* Create and populate PostgreSQL database tables
## Usage
1. Run the Jupyter Notebook to extract and transform data, and create CSV files.
2. Use the provided crowdfunding_db_schema.sql file to create tables in the PostgreSQL database.
3. Load the CSV files into the corresponding PostgreSQL tables.
4. Query the database to verify the data has been loaded correctly.
## Built With
* Python
* Pandas
* PostgreSQL
* Jupyter Notebook
* Excel
## License
This project is licensed under the MIT License.