{"id":30338815,"url":"https://github.com/victorcezeh/understanding-psycopg2","last_synced_at":"2026-05-03T20:39:54.469Z","repository":{"id":309780024,"uuid":"1037485829","full_name":"victorcezeh/understanding-psycopg2","owner":"victorcezeh","description":"A Python + PostgreSQL learning repo featuring two projects: a Class Roster system that evolves from basic scripts to OOP with a CLI, and a Real Estate ETL pipeline that cleans CSV data, loads it into PostgreSQL, and runs analytics.","archived":false,"fork":false,"pushed_at":"2025-09-27T04:08:30.000Z","size":74,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-27T06:08:19.393Z","etag":null,"topics":["dbeaver","git","postgresql","psycopg2","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/victorcezeh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-13T16:38:06.000Z","updated_at":"2025-09-27T04:08:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"9878f3a2-6000-46dd-8bd9-f850ba5d2377","html_url":"https://github.com/victorcezeh/understanding-psycopg2","commit_stats":null,"previous_names":["victorcezeh/understanding-psycopg2"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/victorcezeh/understanding-psycopg2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorcezeh%2Funderstanding-psycopg2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorcezeh%2Funderstanding-psycopg2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorcezeh%2Funderstanding-psycopg2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorcezeh%2Funderstanding-psycopg2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/victorcezeh","download_url":"https://codeload.github.com/victorcezeh/understanding-psycopg2/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorcezeh%2Funderstanding-psycopg2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32584646,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dbeaver","git","postgresql","psycopg2","python"],"created_at":"2025-08-18T06:40:13.040Z","updated_at":"2026-05-03T20:39:54.463Z","avatar_url":"https://github.com/victorcezeh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## 📑 Table of Contents\n\n1. [Project Overview](#postgresql--python-mini-data-engineering-projects)\n2. [Project Highlights](#-project-highlights)\n   1. [Class Roster Database](#1-class-roster-database)\n   2. [Real Estate Data Analysis](#2-real-estate-data-analysis)\n3. [Tech Stack](#-tech-stack)\n4. [Repository Structure](#-repository-structure)\n5. [Getting Started](#-getting-started)\n   1. [Install Dependencies](#1-install-dependencies)\n   2. [Configure Environment Variables](#2-configure-environment-variables)\n   3. [Run a Project](#3-run-a-project)\n6. [Learning Path Demonstrated](#-learning-path-demonstrated)\n7. [Next Steps \u0026 Extensions](#-next-steps--extensions)\n8. [Contributing](#-contributing)\n9. [Shout-out](#-shout-out)\n\n\n# PostgreSQL \u0026 Python Mini Data Engineering Projects\n\nA showcase of **end-to-end data workflows** using Python (`psycopg2`) and PostgreSQL. From foundational database operations to advanced CSV ingestion, transformation, and business intelligence queries.\n\nThis repository contains **two learning-to-production style projects**:\n\n\u003e **Note:** The CSV dataset used in this project is stored in the `data/` folder.\n\n1. **Class Roster Database** – [link-here](https://github.com/victorcezeh/understanding-psycopg2/tree/main/class-roster-database) A beginner-friendly system for storing and retrieving student data, demonstrating a clear progression from procedural scripts to object-oriented design and an interactive CLI.\n2. **Real Estate Data Analysis** – [link-here](https://github.com/victorcezeh/understanding-psycopg2/tree/main/real-estate-data-analysis) An advanced ETL pipeline for importing real estate data from CSV, handling complex data types, and running market analysis queries.\n\n---\n\n## 📌 Project Highlights\n\n### 1. Class Roster Database\n\n* **Progressive Design**: Procedural → OOP → User Interface\n* **Secure Queries**: Parameterized SQL to prevent injection\n* **Environment Config**: Credentials via `.env` file\n* **Error Handling**: psycopg2 exception management \u0026 cleanup\n* **User Interaction**: Command-line interface for live queries\n\n**Core Skills:** Basic schema design, CRUD operations, connection management, and interactive database applications.\n\n---\n\n### 2. Real Estate Data Analysis\n\n* **CSV → Database Pipeline**: Automated ETL process\n* **Data Type Mastery**: Integers, decimals, timestamps with timezones\n* **Transformation Layer**: Clean \u0026 map raw CSV fields before loading\n* **Business Analytics**: Aggregate queries for market insights\n* **Performance Aware**: Row-by-row processing with single-transaction commit\n\n**Core Skills:** ETL design, data type conversion, bulk inserts, aggregate analytics, and schema planning for production-like scenarios.\n\n---\n\n## 🤖 Tech Stack\n\n* **Language**: Python 3.x\n* **Database**: PostgreSQL\n* **Libraries**:\n\n  * `psycopg2-binary` – PostgreSQL connector\n  * `python-dotenv` – Environment variable management\n  * `python-dateutil` – Advanced date/time parsing (Real Estate project)\n\n---\n\n## 📂 Repository Structure\n\n```\nunderstanding-psycopg2/\n├── data/\n│   └── data.csv\n├── my_env/                   # Virtual environment (ignored in .gitignore)\n├── psycopg2-csv-demo/\n│   ├── property_analysis.py\n│   ├── README.md\n│   └── real_estate_import.py\n├── psycopg2-demo/\n│   ├── __pycache__/           # Python run cache (ignored)\n│   ├── class_roster_basic.py\n│   ├── db.py\n│   ├── README.md\n│   └── terminal.py\n├── .env                      # Local environment variables (ignored)\n├── .env.example              # Template for env configuration\n├── .gitignore\n├── README.md                 # This combined project README\n└── requirements.txt\n\n```\n\n---\n\n## 🚀 Getting Started\n\n### 1. Install Dependencies\n\n```bash\npip install psycopg2-binary python-dotenv python-dateutil\n```\n\n### 2. Configure Environment Variables\n\nCreate a `.env` file in the root directory based on the template below.\n\n**.env.example**\n\n```env\n# ==== Class Roster Database Credentials ====\nCLASS_ROASTER_DBNAME=class_roaster\nCLASS_ROASTER_HOST=localhost\nCLASS_ROASTER_USER=your_username\nCLASS_ROASTER_PASSWORD=your_password\nCLASS_ROASTER_PORT=5432\n\n# ==== Real Estate Database Credentials ====\nREAL_ESTATE_DBNAME=real_estate\nREAL_ESTATE_HOST=localhost\nREAL_ESTATE_USER=your_username\nREAL_ESTATE_PASSWORD=your_password\nREAL_ESTATE_PORT=5432\n\n# ==== CSV File Path for Real Estate Project ====\nPATH_TO_CSV=/path/to/your/data.csv\n```\n\n### 3. Run a Project\n\n* **Class Roster (Procedural)**\n\n  ```bash\n  python class_roster_basic.py\n  ```\n* **Class Roster (Interactive CLI)**\n\n  ```bash\n  python terminal.py\n  ```\n* **Real Estate (ETL Pipeline)**\n\n  ```bash\n  python real_estate_import.py\n  ```\n* **Real Estate (Analytics)**\n\n  ```bash\n  python property_analysis.py\n  ```\n\n---\n\n## 📖 Learning Path Demonstrated\n\n1. **Database Fundamentals** – Procedural scripting, manual resource control\n2. **OOP Abstraction** – Encapsulating DB logic in reusable classes\n3. **ETL Pipelines** – Extract, transform, and load from CSV to PostgreSQL\n4. **Analytics Layer** – Aggregations, grouping, and insights for decision-making\n\n---\n\n## 🔮 Next Steps \u0026 Extensions\n\n* Add UPDATE/DELETE operations in Class Roster\n* Integrate PostGIS for geospatial queries in Real Estate\n* Create REST API endpoints for both datasets\n* Visualize insights with matplotlib or Plotly\n* Implement connection pooling for performance scaling\n\n---\n\n## 🤝 Contributing\n\nThis repository is intended as both a **learning resource** and a **launchpad** for production-level data workflows.\nFeel free to fork, extend, or adapt these patterns for your own projects.\n\n---\n\n---\n\n## 📣 Shout-out \n\nBig shout-out to [Chad Mowbray](https://github.com/Chad-Mowbray) and [Code Platoon](https://www.codeplatoon.org/). \n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvictorcezeh%2Funderstanding-psycopg2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvictorcezeh%2Funderstanding-psycopg2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvictorcezeh%2Funderstanding-psycopg2/lists"}