https://github.com/praveendinesha/youtube-data-harvesting-and-warehousing

Comprehensive YouTube Data Harvesting and Analysis Project: Utilized Python's API for data extraction, enriched by SQL queries, and stored in MongoDB. Employed Streamlit for streamlined visualization, enabling efficient data collection, storage, and insightful analysis.
https://github.com/praveendinesha/youtube-data-harvesting-and-warehousing

cleaning integration mongodb nosql nosql-database postgreq python python-sql-integration python3 query rdbms scripts sql streamlit warehouse warehousing windows youtube youtube-api youtube-channel

Last synced: 25 days ago
JSON representation

Comprehensive YouTube Data Harvesting and Analysis Project: Utilized Python's API for data extraction, enriched by SQL queries, and stored in MongoDB. Employed Streamlit for streamlined visualization, enabling efficient data collection, storage, and insightful analysis.

• Host: GitHub
• URL: https://github.com/praveendinesha/youtube-data-harvesting-and-warehousing
• Owner: Praveendinesha
• Created: 2023-10-03T08:11:10.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-12-28T22:30:47.000Z (10 months ago)
• Last Synced: 2024-10-11T20:01:43.901Z (25 days ago)
• Topics: cleaning, integration, mongodb, nosql, nosql-database, postgreq, python, python-sql-integration, python3, query, rdbms, scripts, sql, streamlit, warehouse, warehousing, windows, youtube, youtube-api, youtube-channel
• Language: Python
• Homepage:
• Size: 69.3 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# **YouTube Data Harvesting and Warhousing**

* This project aims to harvest data from YouTube using Python scripting and store it in a NoSQL (MongoDB) database as a data lake. 

* The harvested data can then be fetched from the NoSQL database and migrated to a SQL (PostgreSQL) database for further analysis. Additionally, SQL queries can be executed on the postgreSQL database to answer specific questions related to the uploaded channel information.

**Prerequisites Before running the scripts, make sure you have the following dependencies installed:**

***Python 3.9 or later***

***MongoDB***

***PostgreSQL***