https://github.com/edserranoc/scripting-with-python-and-sql-for-data-engineering
Scripting with Python and SQL for Data Engineering Course Course - Duke University
https://github.com/edserranoc/scripting-with-python-and-sql-for-data-engineering
databases-course json mysql python-programming sql web-scraping
Last synced: 6 months ago
JSON representation
Scripting with Python and SQL for Data Engineering Course Course - Duke University
- Host: GitHub
- URL: https://github.com/edserranoc/scripting-with-python-and-sql-for-data-engineering
- Owner: edserranoc
- License: gpl-3.0
- Created: 2025-03-31T05:17:12.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-04-24T23:40:21.000Z (7 months ago)
- Last Synced: 2025-04-25T00:31:52.805Z (7 months ago)
- Topics: databases-course, json, mysql, python-programming, sql, web-scraping
- Language: Jupyter Notebook
- Homepage: https://www.coursera.org/learn/scripting-with-python-sql-for-data-engineering-duke?specialization=python-bash-sql-data-engineering-duke
- Size: 3.23 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Scripting with Python and SQL for Data Engineering
## Course Overview
This repository contains notes, code snippets, and exercises from the Coursera course **"Scripting with Python and SQL for Data Engineering"** offered by Duke University. The course focuses on using Python and SQL for automating data workflows, managing databases, and performing data transformations.
## Topics Covered
### Week 1: Working with Data in Python
- Using built-in Python modules
- File handling (reading/writing files)
- Working with popular formats like JSON
- Serialize from Python to a JSON file
- Build a useful Python Decorator
### Week 2: Python Scripting and SQL
- Writing and running Python scripts
- Create database, store data adnd query with SQL from Python.
### Week 3: Web Scraping using Python
- Introduction to REST APIs
- Making HTTP requests with `requests` library
- Parsing JSON data
- Web scraping with `BeautifulSoup`
### Week 4: Working with MySQL
- Basics of SQL and relational databases
- Writing SQL queries
- Using `sqlite3` in Python
- Creating and managing tables in a database
- Performing CRUD operations (Create, Read, Update, Delete)
## Notes and Resources
- Course link: [Scripting with Python and SQL for Data Engineering](https://www.coursera.org/learn/scripting-with-python-sql-for-data-engineering-duke)
- [Python Official Documentation](https://docs.python.org/3/)
- [SQLite Documentation](https://www.sqlite.org/docs.html)
- [pandas Documentation](https://pandas.pydata.org/docs/)
- [Microsoft Learn](https://learn.microsoft.com/en-us/training/browse/?WT.mc_id=academic-0000-alfredodeza&subjects=artificial-intelligence)
- [Alfredo Deza: python-scripting Repository](https://github.com/alfredodeza/python-scripting)
- [Alfredo Deza: Parsing With HTMLParser](https://github.com/alfredodeza/scrapy-xpath)
## Progress Tracking
| Week | Topic | Status |
|------|-------------------------------|--------|
| 1 |Working with Data in Python | ✅ |
| 2 | Python Scripting and SQL | ✅ |
| 3 |Web Scraping using Python | ⏳ |
| 4 | Working with MySQL| ❌ |
✅ - Completed | ⏳ - In Progress | ❌ - Not Started
## License
This project is licensed under the GPL-3.0 license. See the LICENSE file for details.