{"id":27872973,"url":"https://github.com/an4pdm/data_cleaning_for_cafe","last_synced_at":"2025-05-05T01:03:02.539Z","repository":{"id":287278285,"uuid":"964206251","full_name":"An4PDM/Data_cleaning_for_cafe","owner":"An4PDM","description":"This project aims to transform and clean data from a cafe by eliminating redundancies, improving data quality and persistency. It leverages data manipulation techniques using pandas, with a focus on enhancing data integrity and optimizing storage for future analysis.","archived":false,"fork":false,"pushed_at":"2025-04-24T23:04:48.000Z","size":340,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-05T01:02:19.799Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/An4PDM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-10T21:14:17.000Z","updated_at":"2025-04-24T23:04:51.000Z","dependencies_parsed_at":"2025-04-10T22:37:33.945Z","dependency_job_id":null,"html_url":"https://github.com/An4PDM/Data_cleaning_for_cafe","commit_stats":null,"previous_names":["an4pdm/data_cleaning_for_cafe"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FData_cleaning_for_cafe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FData_cleaning_for_cafe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FData_cleaning_for_cafe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/An4PDM%2FData_cleaning_for_cafe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/An4PDM","download_url":"https://codeload.github.com/An4PDM/Data_cleaning_for_cafe/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252420973,"owners_count":21745153,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-05T01:02:15.045Z","updated_at":"2025-05-05T01:03:02.526Z","avatar_url":"https://github.com/An4PDM.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ☕ *Data Cleaning for Cafe*\n\nThis project aims to transform and clean data from a cafe by eliminating redundancies, improving data quality and persistency. It leverages data manipulation techniques using **Pandas**, with a focus on enhancing data integrity and optimizing storage for future analysis.\n\n## 📌 Objectives\n\n- Detect and remove duplicate or inconsistent records\n- Convert invalid data types to numeric where applicable\n- Replace or handle missing values\n- Improve overall data structure and clarity\n\n## 🛠️ Tools \u0026 Libraries\n\n- Python\n- Pandas\n- Jupyter Notebook (for development and visualization)\n\n## 🧼 Data Cleaning Steps\n\nThe dataset was cleaned and transformed incrementally, with each step saved as a `.pkl` file for reproducibility and version control.\n\n### ✅ Checkpoints\n\n- **data_step1.pkl**\n\n  - Set all `Price Per Unit` values for each item correctly\n  - Converted `Quantity`, `Price Per Unit`, and `Total Spent` to numeric types\n  - Replaced non-numeric values with `NaN` (using `pd.to_numeric` with `errors='coerce'`)\n  - Imputed missing values in `Quantity` by dividing `Total Spent` by `Price Per Unit`\n  - Updated missing values in `Total Spent` by multiplying `Quantity` and `Price Per Unit`\n\n- **data_step2.pkl** (in progress)\n  - Removed NaN and replaced values in `Item` and `Payment Method` \n  - Removed redundant or duplicate rows\n  - Standardized column names (e.g., lowercase, underscores)\n  - Trimmed whitespace in string values\n\n### 🔄 File Naming Convention\n\nEach step is saved as `data_stepN.pkl`, where `N` indicates the transformation phase.\n\n\n## 💡 Key Learnings\n\n- Data validation and type conversion using `pd.to_numeric()`\n- Filtering rows with conditions (`isna()`, `notna()`)\n- Creating new DataFrames from cleaned Series\n- Good practices in data preprocessing for analysis\n\n## 📁 Output\n\nThe final cleaned DataFrame is ready for further use in dashboards, analysis, or machine learning tasks.\n\n---\n\nFeel free to fork or use it as a reference in your own data projects!\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fan4pdm%2Fdata_cleaning_for_cafe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fan4pdm%2Fdata_cleaning_for_cafe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fan4pdm%2Fdata_cleaning_for_cafe/lists"}