Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/praveendinesha/youtube-data-harvesting-and-warehousing
Comprehensive YouTube Data Harvesting and Analysis Project: Utilized Python's API for data extraction, enriched by SQL queries, and stored in MongoDB. Employed Streamlit for streamlined visualization, enabling efficient data collection, storage, and insightful analysis.
https://github.com/praveendinesha/youtube-data-harvesting-and-warehousing
cleaning integration mongodb nosql nosql-database postgreq python python-sql-integration python3 query rdbms scripts sql streamlit warehouse warehousing windows youtube youtube-api youtube-channel
Last synced: about 1 month ago
JSON representation
Comprehensive YouTube Data Harvesting and Analysis Project: Utilized Python's API for data extraction, enriched by SQL queries, and stored in MongoDB. Employed Streamlit for streamlined visualization, enabling efficient data collection, storage, and insightful analysis.
- Host: GitHub
- URL: https://github.com/praveendinesha/youtube-data-harvesting-and-warehousing
- Owner: Praveendinesha
- Created: 2023-10-03T08:11:10.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-28T22:30:47.000Z (12 months ago)
- Last Synced: 2024-11-15T01:43:58.557Z (about 1 month ago)
- Topics: cleaning, integration, mongodb, nosql, nosql-database, postgreq, python, python-sql-integration, python3, query, rdbms, scripts, sql, streamlit, warehouse, warehousing, windows, youtube, youtube-api, youtube-channel
- Language: Python
- Homepage:
- Size: 69.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# **YouTube Data Harvesting and Warhousing**
* This project aims to harvest data from YouTube using Python scripting and store it in a NoSQL (MongoDB) database as a data lake.
* The harvested data can then be fetched from the NoSQL database and migrated to a SQL (PostgreSQL) database for further analysis. Additionally, SQL queries can be executed on the postgreSQL database to answer specific questions related to the uploaded channel information.
**Prerequisites Before running the scripts, make sure you have the following dependencies installed:**
***Python 3.9 or later***
***MongoDB***
***PostgreSQL***