{"id":29616616,"url":"https://github.com/pizofreude/data-engineer-notes","last_synced_at":"2026-04-19T10:32:23.554Z","repository":{"id":304098086,"uuid":"1017741883","full_name":"pizofreude/data-engineer-notes","owner":"pizofreude","description":"Personal notes and lab solutions for the Data Engineer Handbook Bootcamp","archived":false,"fork":false,"pushed_at":"2025-07-11T05:22:13.000Z","size":23640,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-11T08:24:34.486Z","etag":null,"topics":["apache-flink","apache-spark","communication","data-engineering","data-quality","database","dimensional-data-modeling","fact-data-modeling","impact","kafka"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pizofreude.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-11T03:10:59.000Z","updated_at":"2025-07-11T05:22:16.000Z","dependencies_parsed_at":"2025-07-11T08:35:04.168Z","dependency_job_id":null,"html_url":"https://github.com/pizofreude/data-engineer-notes","commit_stats":null,"previous_names":["pizofreude/data-engineer-notes"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/pizofreude/data-engineer-notes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pizofreude%2Fdata-engineer-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pizofreude%2Fdata-engineer-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pizofreude%2Fdata-engineer-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pizofreude%2Fdata-engineer-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pizofreude","download_url":"https://codeload.github.com/pizofreude/data-engineer-notes/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pizofreude%2Fdata-engineer-notes/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266222295,"owners_count":23894992,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-flink","apache-spark","communication","data-engineering","data-quality","database","dimensional-data-modeling","fact-data-modeling","impact","kafka"],"created_at":"2025-07-21T01:01:39.846Z","updated_at":"2026-04-19T10:32:23.542Z","avatar_url":"https://github.com/pizofreude.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Engineer Bootcamp Notes\n\nWelcome to my personal notes for the **Data Engineer Handbook Bootcamp**.\n\nThis repository is my learning journal, containing summaries, key concepts, and lab solutions for the 6-week bootcamp. It complements my forked [Data Engineer Handbook repo](https://github.com/pizofreude/data-engineer-handbook).\n\n---\n\n## Repository Structure\n\n```bash\ndata-engineer-notes/\n├── README.md\n├── resources.md\n├── assets/\n├── images/\n├── week00/\n│   ├── summary.md\n│   ├── key-concepts.md\n│   ├── lab-notes.md\n│   └── lab00/\n│       ├── solution.ipynb\n│       └── ...     # Artifacts from bootcamp materials\n├── week01/\n│   └── (similar structure)\n├── ...\n└── week06/\n    └── (similar structure)\n```\n\n### Week Notes\nEach week contains:\n- **Summary:** Key takeaways from the week\n- **Key Concepts:** Detailed explanations and examples of core ideas\n- **Lab Notes:** Observations, detailed notes, and troubleshooting during labs\n- **Labs:** Solutions for each lab\n\n---\n\n## Links\n\n- **Original Handbook:** [DataExpert-io/data-engineer-handbook](https://github.com/DataExpert-io/data-engineer-handbook)\n- **My Fork:** [pizofreude/data-engineer-handbook](https://github.com/pizofreude/data-engineer-handbook)\n\n---\n\n## Learning Progress\n\n- [X] Module 1: Bootcamp Orientation - Database setup and Boot Camp Kickoff [Week 0]\n    - [ ] Bootcamp Kickoff | 20 min\n    - [ ] Boot Camp Database Setup | 20 min\n- [ ] Module 2: Dimensional Data Modeling [Week 1]\n    - [ ] Dimensional Data Modeling Complex Data Type and Cumulation Day 1 Lecture | 43 min\n    - [ ] Dimensional Data Modeling Complex Data Type and Cumulation Day 1 Lab | 41 min\n    - [ ] Dimensional Data Modeling: Building Slowly Changing Dimensions Day 2 Lecture | 40 min\n    - [ ] Dimensional Data Modeling: Building Slowly Changing Dimensions Day 2 Lab | 45 min\n    - [ ] Dimensional Data Modeling: Graph Data Modeling Day 3 Lecture | 34 min\n    - [ ] Dimensional Data Modeling: Graph Data Modeling Day 3 Lab | 46 min\n    - [ ] Dimensional Data Modeling - Week 1 Assignment\n- [ ] Module 3: Fact Data Modeling [Week 2]\n    - [ ] Fact Data Modeling: Core Concepts, Deduplication Day 1 Lecture | 52 min\n    - [ ] Fact Data Modeling: Practical Insights into Data Modeling Day 1 Lab | 40 min\n    - [ ] Fact Data Modeling: Core Elements in Data Modeling Day 2 Lecture | 31 min\n    - [ ] Fact Data Modeling: Compact Tables for Efficient Data Representation Day 2 Lab | 45 min\n    - [ ] Fact Data Modeling: Minimizing Shuffle and Reducing Facts Day 3 Lecture | 32 min\n    - [ ] Fact Data Modeling: Practical Guide to Formatting and Aggregating Data Day 3 Lab | 30 min\n    - [ ] Fact Data Modeling - Week 2 Assignment\n- [ ] Module 4: Apache Spark Fundamentals [Week 3]\n    - [ ] Apache Spark: Architecture, Optimization, and Best Practices Day 1 Lecture | 48 min\n    - [ ] Apache Spark: Hands-On for Broadcast and Hash Joins Day 1 Lab | 26 min\n    - [ ] Apache Spark: Managing Spark Jobs and Notebooks Day 2 Lecture | 34 min\n    - [ ] Apache Spark: User-Defined Functions and Broadcast Join Day 2 Lab | 36 min\n    - [ ] Unit Testing Spark Jobs: Importance, Challenges, and Leadership Perspectives Lecture | 41 min\n    - [ ] Unit Testing Spark Jobs: Mastering Spark and PySpark Testing Lab | 27 min\n    - [ ] Spark Fundamentals - Week 3 Assignment\n- [ ] Module 5: Applying Analytical Patterns [Week 4]\n    - [ ] Applying Analytical Patterns: Exploring SQL, Scaling Projects and Aggregation Analysis Day 1 Lecture | 52 min\n    - [ ] Applying Analytical Patterns: Mastering Growth Accounting and Retention Analysis Day 1 Lab | 34 min\n    - [ ] Applying Analytical Patterns: Recursive CTEs and Window Functions Day 2 Lecture | 44 min\n    - [ ] Applying Analytical Patterns: Aggregations and Cardinality Reduction Day 2 Lab | 33 min\n    - [ ] Applying Analytical Patterns - Week 4 Assignment\n- [ ] Module 6: Real-time pipelines with Flink and Kafka [Week 5]\n    - [ ] Flink Lab Setup | 7 min\n    - [ ] Streaming Pipelines: Mastering Streaming and Real-time Pipelines Day 1 Lecture | 50 min\n    - [ ] Streaming Pipelines: Setting up Streaming Pipelines Day 1 Lab | 40 min\n    - [ ] Streaming Pipelines: Exploring Data Collection and Processing Day 2 Lecture | 31 min\n    - [ ] Streaming Pipelines: Kafka, Postgres, Spark Integrations and Parallelism Day 2 Lab | 39 min\n    - [ ] Flink - Week 5 Assignment\n- [ ] Module 7: Data Visualization and Impact [Week 6 Part 1]\n    - [ ] Data Visualization and Impact: Mastering Data Engineering Day 1 Lecture | 39 min\n    - [ ] Data Visualization and Impact: Hands-On with the CSV files Day 1 Lab | 8 min\n    - [ ] Data Visualization and Impact: Insights and Best Practices Day 2 Lecture | 23 min\n    - [ ] Data Visualization and Impact: Exploring Data Visualization and Aggregation Techniques Day 2 Lab | 37 min\n    - [ ] Data Visualization - Week 6 1st Assignment\n- [ ] Module 8: Data Pipeline Maintenance [Week 6 Part 2]\n    - [ ] Data Pipeline Maintenance: Navigating the Complexities of Data Engineering Day 1 Lecture | 67 min\n    - [ ] Data Pipeline Maintenance: Strategies for Maintenance and Dock Building Day 2 Lecture | 77 min\n    - [ ] Data Pipeline Maintenance - Week 6 2nd Assignment\n- [ ] Module 9: KPIs and Experimentation [Week 6 Part 3]\n    - [ ] KPIs and Experimentation: Decoding Business Success: Metrics, Growth Strategies and Collaborative Approaches Day 1 Lecture | 55 min\n    - [ ] KPIs and Experimentation: Setting up and Analysing Experiments Day 1 Lab | 36 min\n    - [ ] KPIs and Experimentation: Leading and Lagging Metrics Day 2 Lecture | 65 min\n    - [ ] KPIs and Experimentation - Week 6 3rd Assignment\n- [ ] Module 10: Data Quality Patterns [Week 7]\n    - [ ] Data Quality Patterns: MIDAS Process from Airbnb Day 1 Lecture | 45 min\n    - [ ] Data Quality Patterns: Spec-Building Document Day 1 Lab | 33 min\n    - [ ] Data Quality Patterns: WAP Patterns Day 2 Lecture | 27 min\n\n---\n\n## 💻 Daily Practice System\n\nThis repository now includes a comprehensive practice tracking system to organize daily coding practice across multiple platforms:\n\n- **[practice/](practice/)** - Platform-organized coding problems (LeetCode, StrataScratch, HackerRank, NeetCode, Codewars, etc.)\n- **[concepts/](concepts/)** - Reference notes on data structures, algorithms, SQL patterns, and system design\n- **[interview-prep/](interview-prep/)** - Interview-specific preparation materials (behavioral, technical, system design)\n- **[logs/](logs/)** - Daily practice logs and progress tracking with statistics dashboard\n\n### Quick Start\n\n```bash\n# Create today's log entry\n./scripts/new-day.sh\n\n# Start a new problem\n# ./scripts/create-problem.sh \u003cplatform\u003e \u003cdifficulty\u003e \"problem-name\"\n./scripts/create-problem.sh leetcode medium \"problem-name\"\n\n# Create a concept note\n# ./scripts/link-concept.sh \"concept-name\" \u003ccategory\u003e\n./scripts/link-concept.sh \"Window Functions\" sql-patterns\n\n# Generate weekly stats\npython scripts/generate-stats.py\n```\n\n### ✅ Manual Update Checklist:  For Every Problem You Solve\n\nHere's your **streamlined checklist** for logging each problem.\n\n\n#### 📋 **The 5-Step Workflow**\n\n##### **Step 1: Start Your Day** ⏱️ *30 seconds*\n\n**Run once per day (first thing in the morning):**\n\n```bash\n./scripts/new-day.sh\n```\n\n**✅ Done!** No manual edits needed for this step.\n\n---\n\n##### **Step 2: Scaffold the Problem** ⏱️ *30 seconds*\n\n**For each new problem you're about to solve:**\n\n```bash\n./scripts/create-problem.sh \u003cplatform\u003e \u003cdifficulty\u003e \"\u003cproblem-slug\u003e\"\n```\n\n**Examples:**\n```bash\n./scripts/create-problem.sh codewars easy \"absolute-value-log-base\"\n./scripts/create-problem.sh leetcode medium \"rank-scores\"\n./scripts/create-problem.sh stratascratch hard \"revenue-analysis\"\n```\n\n**✅ Done!** Folder created, template copied, ready to code.\n\n---\n\n##### **Step 3: Write Your Solution** ⏱️ *10-30 minutes (solving time)*\n\nNavigate to the problem folder: \n\n```bash\ncd practice/\u003cplatform\u003e/\u003cdifficulty\u003e/\u003cproblem-slug\u003e\n```\n\n**Open and write your solution:**\n\n```bash\ncode solution. sql    # For SQL problems\n# OR\ncode solution.py     # For Python/algorithm problems\n```\n\n**What to do:**\n- ✏️ **Paste your working solution code**\n- ✏️ **Add comments explaining key logic** (optional but recommended)\n- 💾 **Save the file**\n\n**Example:**\n```sql\n-- Calculate absolute value and logarithm base 64\nSELECT\n  ABS(number1) AS abs,\n  LOG(64, number2) AS log\nFROM decimals;\n```\n\n---\n\n##### **Step 4: Document Your Solution** ⏱️ *10-15 minutes*\n\n**Open the notes file:**\n\n```bash\ncode notes.md\n```\n\n**You need to manually update these sections:**\n\n##### **A. Metadata (Top of file)**\n```markdown\n# [Problem Name]  ← CHANGE THIS\n\n## 📋 Metadata\n- **Platform:** [Platform name]  ← CHANGE THIS\n- **Difficulty:** [Easy/Medium/Hard] (Optional:  add platform rating like \"7 kyu\")  ← CHANGE THIS\n- **Date Solved:** 2026-01-03  ← ✅ ALREADY FILLED BY SCRIPT\n- **Time Spent:** XX minutes  ← CHANGE THIS\n- **Status:** [✅ Solved | 🔄 Revisit | ❌ Stuck]  ← CHANGE THIS\n```\n\n**Example:**\n```markdown\n# Absolute Value and Log to Base\n\n## 📋 Metadata\n- **Platform:** Codewars\n- **Difficulty:** Easy (7 kyu)\n- **Date Solved:** 2026-01-03  ← Script filled this\n- **Time Spent:** 15 minutes\n- **Status:** ✅ Solved\n```\n\n---\n\n##### **B. Links**\n```markdown\n## 🔗 Links\n- [Problem URL]  ← PASTE THE ACTUAL URL HERE\n```\n\n**Example:**\n```markdown\n## 🔗 Links\n- https://www.codewars.com/kata/594a8f2f7ca3c692a4000041/train/sql\n```\n\n---\n\n##### **C. Topics \u0026 Tags (Check the boxes)**\n```markdown\n## 📚 Topics \u0026 Tags\n- [ ] SQL\n- [ ] Window Functions\n- [ ] Joins\n- [ ] CTEs\n- [ ] Python\n- [ ] Dynamic Programming\n```\n\n**Check the relevant ones:**\n```markdown\n##📚 Topics \u0026 Tags\n- [x] SQL  ← Put 'x' inside\n- [x] Mathematical Functions\n- [ ] Window Functions\n- [ ] Joins\n```\n\n---\n\n##### **D. Problem Statement**\n```markdown\n## 📝 Problem Statement\n[Paste the problem description here]\n\n### Example Input/Output\n```markdown\nInput: \nOutput: \n```\n\n\n**What to do:**\n- ✏️ Copy-paste the problem description from the platform\n- ✏️ Add example input/output (if provided)\n\n---\n\n##### **E. Approach**\n```markdown\n## 💡 Approach\n\n### Initial Thoughts\n[What was your first idea?  What patterns did you recognize?]\n\n### Solution Strategy\n1. Step 1\n2. Step 2\n3. Step 3\n```\n\n**What to do:**\n- ✏️ Write your thought process (2-3 sentences)\n- ✏️ List the steps you took (bullet points)\n\n**Example:**\n```markdown\n## 💡 Approach\n\n### Initial Thoughts\nStraightforward application of SQL math functions:  ABS for absolute value, LOG for logarithm with custom base.\n\n### Solution Strategy\n1. Use `ABS(number1)` to get absolute values\n2. Use `LOG(64, number2)` for logarithm base 64\n3. Alias columns as required (`abs`, `log`)\n```\n\n---\n\n##### **F. Solution**\n```markdown\n## 🖥️ Solution\n\n### Attempt 1 (Initial)\n```sql\n-- Your first solution here\n```\n\n**Result:** [Passed/Failed/Timeout]\n```\n\n**What to do:**\n- ✏️ Paste your solution code (can be same as `solution.sql`)\n- ✏️ Note if it passed or failed\n\n**If you optimized it, add:**\n```markdown\n### Attempt 2 (Optimized) ⭐\n```sql\n-- Improved solution\n```\n\n**Result:** ✅ Passed with better performance\n```\n\n---\n\n#### **G. Complexity Analysis**\n```markdown\n## ⚡ Complexity Analysis\n- **Time Complexity:** O(?)\n- **Space Complexity:** O(?)\n```\n\n**What to do:**\n- ✏️ Fill in the Big O notation\n- ✏️ If you don't know, write:  \"Time:  O(n) - single pass through table\"\n\n**Example:**\n```markdown\n## ⚡ Complexity Analysis\n- **Time Complexity:** O(n) - single pass through table\n- **Space Complexity:** O(n) - result set same size as input\n```\n\n---\n\n#### **H. Key Learnings**\n```markdown\n## 🎓 Key Learnings\n1. \n2. \n3. \n```\n\n**What to do:**\n- ✏️ Write 2-4 things you learned (this is THE MOST IMPORTANT SECTION!)\n\n**Example:**\n```markdown\n## 🎓 Key Learnings\n1. **ABS()** - Returns absolute value (distance from zero)\n2. **LOG(base, value)** - PostgreSQL syntax for custom base logarithm\n3. PostgreSQL uses `LOG(base, value)` while MySQL uses `LOG(value) / LOG(base)`\n4. Base-64 logarithm:  `LOG(64, 4096) = 2` because 64² = 4096\n```\n\n---\n\n#### **I. Related Concepts (Optional)**\n```markdown\n## 🏷️ Related Concepts\nSee:  `concepts/sql-patterns/[concept-file]. md`\n```\n\n**What to do:**\n- ✏️ If you created a concept note, link it here\n- ⏭️ Skip if you haven't created a concept yet\n\n**Example:**\n```markdown\n## 🏷️ Related Concepts\nSee:  `concepts/sql-patterns/sql-mathematical-functions.md`\n```\n\n---\n\n### **Step 5: Update Daily Log** ⏱️ *5 minutes*\n\n**Open today's log:**\n\n```bash\ncd ../../../../    # Return to repo root\ncode logs/2026/01-january. md\n```\n\n**Find today's date section** and fill in:\n\n#### **A. Time Spent**\n```markdown\n### Friday, January 03, 2026\n⏱️ Time:   X hours  ← CHANGE THIS\n```\n\n**Example:**\n```markdown\n⏱️ Time:  1.5 hours\n```\n\n---\n\n#### **B. Problems Completed**\n```markdown\n#### ✅ Completed\n1. \n```\n\n**Add each problem with:**\n- Problem name and difficulty\n- Key topics\n- Link to your solution\n- One-line key learning\n\n**Example:**\n```markdown\n#### ✅ Completed\n1. **Codewars - Absolute Value and Log to Base** (Easy/7kyu)\n   - Topics: ABS(), LOG(), Mathematical functions\n   - [Solution](../../practice/codewars/easy/absolute-value-log-base/)\n   - Key learning: PostgreSQL LOG(base, value) syntax differs from MySQL\n\n2. **LeetCode 178 - Rank Scores** (Medium)\n   - Topics: Window functions, DENSE_RANK\n   - [Solution](../../practice/leetcode/medium/178-rank-scores/)\n   - Key learning:  DENSE_RANK vs RANK vs ROW_NUMBER differences\n```\n\n---\n\n#### **C. Learnings**\n```markdown\n#### 💡 Learnings\n- \n```\n\n**Write 2-4 broader learnings from today:**\n\n**Example:**\n```markdown\n#### 💡 Learnings\n- Mathematical functions in SQL are database-specific (PostgreSQL vs MySQL syntax)\n- Always check for NULL values when using LOG() with user input\n- ABS() is useful for calculating distances and differences\n- Created concept note:  `concepts/sql-patterns/sql-mathematical-functions.md`\n```\n\n---\n\n#### **D. Tomorrow's Plan**\n```markdown\n#### 🎯 Tomorrow\n- [ ] \n```\n\n**Plan 2-3 things for tomorrow:**\n\n**Example:**\n```markdown\n#### 🎯 Tomorrow\n- [ ] LeetCode 180 - Consecutive Numbers (Window functions practice)\n- [ ] StrataScratch - Revenue analysis problem\n- [ ] Review:  Self-joins pattern\n```\n\n---\n\n### **Step 6: Commit \u0026 Push** ⏱️ *1 minute*\n\n```bash\ngit status\n\n# Add your changes\ngit add practice/\u003cplatform\u003e/\u003cdifficulty\u003e/\u003cproblem-slug\u003e/\ngit add logs/2026/01-january. md\n\n# If you created a concept note, add it too\ngit add concepts/\n\n# Commit with descriptive message\ngit commit -m \"✅ [Platform]:  [Problem Name] - [Key Topic]\"\n\n# Push to GitHub\ngit push\n```\n\n**Example commit messages:**\n```bash\ngit commit -m \"✅ Codewars:  Absolute Value and Log to Base - SQL math functions\"\ngit commit -m \"✅ LeetCode 178: Rank Scores - Window functions\"\ngit commit -m \"✅ StrataScratch: Revenue Analysis - CTEs and aggregations\"\n```\n\n---\n\n## 📊 **Weekly:  Update Stats Dashboard** ⏱️ *5 minutes*\n\n**Run every Sunday (or end of week):**\n\n```bash\npython scripts/generate-stats.py\n```\n\n**Copy the output:**\n```\n## 📊 All-Time Stats\n\n| Platform      | Easy | Medium | Hard | Total |\n|---------------|------|--------|------|-------|\n| Codewars      | 5    | 2      | 0    | 7     |\n| Leetcode      | 12   | 8      | 1    | 21    |\n| **Total**     | **17** | **10** | **1** | **28** |\n\n📅 Last Updated: 2026-01-05 20:30\n```\n\n**Paste it into:**\n```bash\ncode logs/README.md\n```\n\n**Replace the old stats section** with the new output.\n\n**Also update:**\n```markdown\n## 🔥 Current Streaks\n- **Daily Practice:** X days  ← UPDATE THIS MANUALLY\n```\n\n**Commit:**\n```bash\ngit add logs/README.md\ngit commit -m \"📊 Update weekly practice stats\"\ngit push\n```\n\n---\n\n## ✅ **Quick Reference Checklist**\n\nPrint this and keep it next to you:\n\n```\n□ Step 1: ./scripts/new-day.sh (once per day)\n\nFor each problem: \n□ Step 2: ./scripts/create-problem.sh \u003cplatform\u003e \u003cdifficulty\u003e \"\u003cslug\u003e\"\n□ Step 3: Write solution in solution.sql or solution.py\n□ Step 4: Fill in notes.md:\n    □ Change title\n    □ Update metadata (platform, difficulty, time, status)\n    □ Paste problem URL\n    □ Check topic tags\n    □ Paste problem statement\n    □ Write approach \u0026 strategy\n    □ Paste solution code\n    □ Add complexity analysis\n    □ Write key learnings (MOST IMPORTANT!)\n    □ Link concept note (if created)\n\n□ Step 5: Update logs/2026/01-january.md:\n    □ Time spent today\n    □ Add problem to \"Completed\" list\n    □ Write today's learnings\n    □ Plan tomorrow's focus\n\n□ Step 6: git add → commit → push\n\nWeekly: \n□ Sunday: Run generate-stats.py\n□ Update logs/README.md \u0026 the monthly log + practice/README.md with new stats\n```\n\n---\n\n## 💡 **Time-Saving Tips**\n\n### **Minimal Version (10 min per problem)**\nIf you're short on time, focus on:\n1. ✅ Solution code (`solution.sql`)\n2. ✅ Key learnings in `notes.md`\n3. ✅ Daily log entry\n\nSkip the rest for now, come back later to fill in. \n\n---\n\n### **Batch Update**\nIf you solve multiple problems:\n1. Scaffold all problems first\n2. Solve all problems\n3. Update all `notes.md` files\n4. Update daily log once (list all problems)\n5. Single commit at the end\n\n---\n\n### **Use Snippets/Shortcuts**\nCreate editor snippets for repetitive sections like complexity analysis, common tags, etc.\n\n---\n\n## 🎯 **Summary:  What You MUST Do Manually**\n\n| File | What to Update |\n|------|----------------|\n| **solution.sql** | Your code |\n| **notes.md** | Title, metadata, URL, approach, learnings |\n| **logs/YYYY/MM-month.md** | Time, problems list, learnings, tomorrow's plan |\n| **logs/README.md** | Weekly stats (copy from script output) |\n\n**Everything else is automated!** 🎉\n\n\n### Features\n\n- **Automation Scripts:** Quickly scaffold new problems and logs with templates\n- **Platform-Agnostic:** Automatically discovers and tracks any coding platform\n- **Comprehensive Templates:** Detailed templates for problems, concepts, and daily logs\n- **Progress Tracking:** Statistics generation and progress dashboards\n- **Knowledge Base:** Structured concept notes linked to practice problems\n\nSee [practice/README.md](practice/README.md) for detailed usage instructions and workflow.\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpizofreude%2Fdata-engineer-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpizofreude%2Fdata-engineer-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpizofreude%2Fdata-engineer-notes/lists"}