{"id":48695492,"url":"https://github.com/omnipotence-eth/manufacturing-quality-analytics","last_synced_at":"2026-04-11T07:13:35.627Z","repository":{"id":350213244,"uuid":"1204338707","full_name":"omnipotence-eth/manufacturing-quality-analytics","owner":"omnipotence-eth","description":"SQL + Python pipeline for semiconductor NCR analysis — supplier performance, defect Pareto, yield trends","archived":false,"fork":false,"pushed_at":"2026-04-09T11:08:12.000Z","size":1115,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-09T12:16:01.492Z","etag":null,"topics":["analytics","data-analysis","etl","manufacturing","matplotlib","pandas","postgresql","python","quality","sql"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/omnipotence-eth.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-07T23:20:03.000Z","updated_at":"2026-04-09T11:08:18.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/omnipotence-eth/manufacturing-quality-analytics","commit_stats":null,"previous_names":["omnipotence-eth/manufacturing-quality-analytics"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/omnipotence-eth/manufacturing-quality-analytics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnipotence-eth%2Fmanufacturing-quality-analytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnipotence-eth%2Fmanufacturing-quality-analytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnipotence-eth%2Fmanufacturing-quality-analytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnipotence-eth%2Fmanufacturing-quality-analytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/omnipotence-eth","download_url":"https://codeload.github.com/omnipotence-eth/manufacturing-quality-analytics/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnipotence-eth%2Fmanufacturing-quality-analytics/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31671998,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T17:19:37.612Z","status":"online","status_checked_at":"2026-04-11T02:00:05.776Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","data-analysis","etl","manufacturing","matplotlib","pandas","postgresql","python","quality","sql"],"created_at":"2026-04-11T07:13:35.532Z","updated_at":"2026-04-11T07:13:35.621Z","avatar_url":"https://github.com/omnipotence-eth.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cbr/\u003e\n\n# Manufacturing Quality Analytics\n\n**SQL + Python pipeline for semiconductor NCR analysis — supplier performance, defect Pareto, yield trends**\n\n\u003cbr/\u003e\n\n[![CI](https://github.com/omnipotence-eth/manufacturing-quality-analytics/actions/workflows/ci.yml/badge.svg)](https://github.com/omnipotence-eth/manufacturing-quality-analytics/actions/workflows/ci.yml)\n[![Python](https://img.shields.io/badge/python-3.11+-3776AB?style=flat-square\u0026logo=python\u0026logoColor=white)](https://python.org)\n[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-18-336791?style=flat-square\u0026logo=postgresql\u0026logoColor=white)](https://www.postgresql.org)\n[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-D7FF64?style=flat-square\u0026logo=ruff\u0026logoColor=black)](https://github.com/astral-sh/ruff)\n[![License: MIT](https://img.shields.io/badge/license-MIT-22C55E?style=flat-square)](LICENSE)\n\n\u003cbr/\u003e\n\n[Architecture](#architecture) \u0026nbsp;·\u0026nbsp; [Key Findings](#key-findings) \u0026nbsp;·\u0026nbsp; [SQL Highlights](#sql-highlights) \u0026nbsp;·\u0026nbsp; [Visualizations](#visualizations) \u0026nbsp;·\u0026nbsp; [Quick Start](#quick-start) \u0026nbsp;·\u0026nbsp; [Contributing](CONTRIBUTING.md)\n\n\u003cbr/\u003e\n\n\u003c/div\u003e\n\n---\n\n## What is This?\n\nA production-grade data analytics pipeline that ingests 2,500 simulated Non-Conformance Reports (NCRs) from a semiconductor manufacturing operation, loads them into PostgreSQL, and answers 10 business questions with SQL — then renders 7 professional visualizations in a Jupyter notebook.\n\nThe dataset is synthetically generated from real manufacturing quality domain knowledge (GM Quality + Shield AI supplier quality experience): realistic supplier defect curves, lot sizes drawn from log-normal distributions, a corrective action improvement trajectory baked into the worst-performing supplier, and shift-level quality variation matching industry norms.\n\n\u003e **What this demonstrates**: SQL fluency (CTEs, window functions, Pareto, self-joins), a clean Python data pipeline, manufacturing domain expertise, and production engineering habits — the exact combination DA/DE interviews test.\n\n---\n\n## Why\n\nManufacturing quality data lives in spreadsheets and disconnected databases. This project demonstrates how to build a proper analytics pipeline — PostgreSQL for structured storage, Python for ETL and statistical analysis, and SQL queries that answer real questions about defect rates, supplier performance, and process capability. Built from experience in automotive and aerospace quality engineering, it models the kind of NCR (non-conformance report) analysis that quality teams actually need but rarely have automated.\n\n---\n\n## Architecture\n\n```mermaid\ngraph LR\n    A[\"generate_synthetic.py\\n2,500 NCRs\"] --\u003e B[\"data/raw/\\nquality_records.csv\"]\n    B --\u003e C[\"load_data.py\\nSQLAlchemy · psycopg2\"]\n    C --\u003e D[(\"PostgreSQL 18\\nmanufacturing_qa\")]\n    D --\u003e E[\"sql/queries.sql\\n10 business queries\"]\n    D --\u003e F[\"analysis.ipynb\\nSQLAlchemy connection\"]\n    E -.-\u003e|reference| F\n    F --\u003e G[\"7 Visualizations\\nMatplotlib · Seaborn\"]\n    G --\u003e H[\"visuals/*.png\"]\n```\n\n### Data model\n\n| Column | Type | Description |\n|--------|------|-------------|\n| `ncr_number` | `varchar` | Unique NCR ID — `NCR-202301-0001` format |\n| `supplier_name` / `supplier_tier` | `varchar` / `int` | Supplier identity and qualification tier (1–3) |\n| `production_line` / `shift` | `varchar` | Where and when the defect was found |\n| `defect_code` / `defect_type` | `varchar` | 15-category defect taxonomy (D001–C015) |\n| `quantity_received` / `quantity_rejected` | `int` | Lot size and rejection volume |\n| `defect_rate` | `float` | Rejection rate for this NCR event |\n| `opened_date` / `closed_date` | `date` | NCR lifecycle timestamps |\n| `days_to_close` | `int` | Disposition cycle time |\n| `disposition` | `varchar` | Use As Is / Rework / Return to Supplier / Scrap |\n| `first_pass` | `bool` | Whether the lot passed first inspection |\n\n---\n\n## Key Findings\n\n1. **FastTrack Supply** leads defect rate at **6.9%** — 8.1× higher than best-in-class AeroParts Manufacturing (0.85%). Corrective action plan implemented July 2023 drove a measurable improvement through H2 2023.\n2. **Dimensional and Surface Finish defects** account for **40% of all rejected units** — Pareto-validated. These two categories are the only ones that warrant dedicated inspection protocols.\n3. **Night shift** runs **1.4× higher defect rate** than Day shift across all production lines. The gap is largest in Electronics Integration (LINE\\_B), flagging a staffing or training gap on nights.\n4. **4 of 7 suppliers** triggered the rolling 30-day repeat-offender threshold (3+ NCRs in 30 days), indicating systemic lot-level problems rather than random variation — mandatory CAP criteria met.\n\n---\n\n## SQL Highlights\n\n### Rolling 30-day repeat offender detection\n```sql\n-- Suppliers with 3+ NCRs in any rolling 30-day window\nWITH windowed AS (\n    SELECT\n        supplier_name,\n        opened_date,\n        COUNT(*) OVER (\n            PARTITION BY supplier_name\n            ORDER BY opened_date\n            RANGE BETWEEN INTERVAL '29 days' PRECEDING AND CURRENT ROW\n        ) AS ncrs_in_30d_window\n    FROM quality_records\n)\nSELECT DISTINCT\n    supplier_name,\n    MAX(ncrs_in_30d_window) OVER (PARTITION BY supplier_name) AS max_ncrs_in_any_30d\nFROM windowed\nWHERE ncrs_in_30d_window \u003e= 3\nORDER BY max_ncrs_in_any_30d DESC;\n```\n\n### Composite supplier scorecard (PERCENT_RANK + CTE)\n```sql\n-- Weighted composite: defect 50%, response time 30%, FPY 20%\nscored AS (\n    SELECT *,\n        ROUND((100.0 * (1 - PERCENT_RANK() OVER (ORDER BY defect_rate_pct DESC)))::numeric, 1) AS defect_score,\n        ROUND((100.0 * (1 - PERCENT_RANK() OVER (ORDER BY avg_close_days DESC)))::numeric, 1)  AS response_score,\n        ROUND((PERCENT_RANK() OVER (ORDER BY fpy_pct) * 100)::numeric, 1)                      AS fpy_score\n    FROM supplier_stats\n)\nSELECT *,\n    ROUND(0.50 * defect_score + 0.30 * response_score + 0.20 * fpy_score, 1) AS composite_score,\n    RANK() OVER (ORDER BY (...) DESC) AS overall_rank\nFROM scored;\n```\n\n### Pareto with running cumulative total\n```sql\nROUND(\n    100.0 * SUM(total_rejected) OVER (\n        ORDER BY total_rejected DESC\n        ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW\n    ) / SUM(total_rejected) OVER (),\n2) AS cumulative_pct\n```\n\n**All 10 queries:** defect rate by supplier, Pareto of defect types, monthly trend with rolling average, yield by production line, average time to disposition, repeat offender window function, defect rate by shift, co-occurring defect self-join, first pass yield by month, composite supplier scorecard.\n\n---\n\n## Visualizations\n\n### Defect Rate by Supplier\n![Defect Rate by Supplier](visuals/01_defect_rate_by_supplier.png)\n\n### Pareto of Defect Types\n![Pareto of Defect Types](visuals/02_pareto_defect_types.png)\n\n### Monthly NCR Trend with Rolling Average\n![Monthly Trend](visuals/03_monthly_trend.png)\n\n### Defect Rate Heatmap — Production Line × Shift\n![Heatmap](visuals/04_heatmap_line_shift.png)\n\n### Supplier Quality Scorecard\n![Supplier Scorecard](visuals/05_supplier_scorecard.png)\n\n### First Pass Yield by Month\n![First Pass Yield](visuals/06_first_pass_yield.png)\n\n### Shift Quality Comparison\n![Shift Comparison](visuals/07_shift_comparison.png)\n\n---\n\n## Quick Start\n\n### Prerequisites\n\n- Python 3.11+\n- PostgreSQL 18 (local or Docker)\n- conda or pip\n\n### Install\n\n```bash\ngit clone https://github.com/omnipotence-eth/manufacturing-quality-analytics.git\ncd manufacturing-quality-analytics\n\npip install -r requirements.txt\n```\n\n### Configure\n\n```bash\ncp .env.example .env\n# Edit .env:\n# DATABASE_URL=postgresql://user:password@localhost:5432/manufacturing_qa\n```\n\n### Run\n\n```bash\n# 1. Generate synthetic dataset (2,500 NCRs → data/raw/quality_records.csv)\npython src/generate_synthetic.py\n\n# 2. Create database and load data\n#    createdb manufacturing_qa  (if not already created)\npython src/load_data.py\n\n# 3. Open the analysis notebook\njupyter lab notebooks/analysis.ipynb\n# Run all cells — charts export automatically to visuals/\n\n# 4. Run standalone SQL queries\npsql $DATABASE_URL -f sql/queries.sql\n```\n\n### Tests\n\n```bash\npytest -q\n```\n\n---\n\n## Tech Stack\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eView full stack\u003c/strong\u003e\u003c/summary\u003e\n\n\u003cbr/\u003e\n\n| Layer | Technology | Notes |\n|-------|-----------|-------|\n| **Data generation** | Python, NumPy | Log-normal lot sizes, realistic supplier defect curves, FastTrack CAP improvement trajectory |\n| **Data pipeline** | Pandas, SQLAlchemy 2.x | Type enforcement, null guards, batch insert, schema confirmation |\n| **Database** | PostgreSQL 18, psycopg2 | `quality_records` table — 20 columns, 2,500 rows |\n| **SQL** | PostgreSQL SQL | CTEs, window functions (`RANGE BETWEEN`, `PERCENT_RANK`, `RANK`), self-joins, `PERCENTILE_CONT` |\n| **Analysis** | Jupyter Lab, Pandas | All queries run via SQLAlchemy connection — no CSV re-reads |\n| **Visualization** | Matplotlib, Seaborn | `seaborn-v0_8-whitegrid` style, consistent palette, exported PNG at 150 DPI |\n| **Config** | python-dotenv | `DATABASE_URL` from `.env` — never hardcoded |\n| **Code quality** | Ruff, mypy | Line length 100, `from __future__ import annotations`, typed public signatures |\n| **Testing** | pytest | 20 unit tests for synthetic data generator — schema, ranges, domain invariants |\n| **CI** | GitHub Actions | lint (ruff check + format) → test (pytest) on every push and PR |\n\n\u003c/details\u003e\n\n---\n\n## Documentation\n\n| Document | Contents |\n|----------|---------|\n| [CONTRIBUTING.md](CONTRIBUTING.md) | Branch strategy, ship workflow, audit checklist, commit standards, PR checklist |\n| [CHANGELOG.md](CHANGELOG.md) | Version history |\n| [SECURITY.md](SECURITY.md) | Security model and vulnerability reporting |\n| [sql/queries.sql](sql/queries.sql) | All 10 annotated standalone SQL queries |\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomnipotence-eth%2Fmanufacturing-quality-analytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fomnipotence-eth%2Fmanufacturing-quality-analytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomnipotence-eth%2Fmanufacturing-quality-analytics/lists"}