{"id":51627919,"url":"https://github.com/datacoolie/dekit","last_synced_at":"2026-07-13T03:01:16.434Z","repository":{"id":369078063,"uuid":"1265080972","full_name":"datacoolie/dekit","owner":"datacoolie","description":null,"archived":false,"fork":false,"pushed_at":"2026-07-03T12:46:30.000Z","size":77,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-03T14:27:07.999Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datacoolie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-06-10T12:54:46.000Z","updated_at":"2026-07-03T12:46:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/datacoolie/dekit","commit_stats":null,"previous_names":["datacoolie/dekit"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/datacoolie/dekit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacoolie%2Fdekit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacoolie%2Fdekit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacoolie%2Fdekit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacoolie%2Fdekit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datacoolie","download_url":"https://codeload.github.com/datacoolie/dekit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datacoolie%2Fdekit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35408466,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-13T02:00:06.543Z","response_time":119,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-13T03:01:15.624Z","updated_at":"2026-07-13T03:01:16.429Z","avatar_url":"https://github.com/datacoolie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dekit\n\nPortable instruction and skill toolkit for data engineering teams. Helps AI runners build, test, deploy, review, document, and maintain data pipelines without depending on platform-specific wrappers.\n\n## What's Inside\n\n### Runtime Contract\n\n- `AGENTS.md` is the repository entrypoint.\n- `.agents/instructions/` contains portable constraints and verification rules.\n- `.agents/skills/` contains task-specific behavior.\n- Platform adapters are optional and must not become a second source of truth.\n\n### Data Engineering Foundation\n\nAll skills inherit universal data engineering constraints defined in `.agents/instructions/data-engineering-constraints.md`:\nidempotency, schema evolution, data contracts, partitioning strategy, quality gates, modeling rules, performance, and security. Verification rules live in `.agents/instructions/verification.md`.\n\n### Bundled Skills by Workflow Mode\n\nThese are the baseline skills shipped with dekit. They are not an allowlist.\n\nAdditional skills can be added by users or installers. They can still work and auto-load when the active AI runner supports skill discovery for their location. List a skill here only when dekit owns and maintains it as part of the baseline toolkit.\n\n**Build \u0026 Scaffold** — creating new things:\n\n| Skill | Purpose |\n|---|---|\n| `plan` | Implementation planning, architecture decisions, phased roadmaps |\n| `spark-development` | PySpark patterns, joins, optimization, UDFs, debugging |\n| `sql-authoring` | Window functions, CTEs, pivots, dialect differences |\n| `data-modeling` | Kimball dimensional modeling, star schema, SCD, Data Vault |\n| `data-ingestion` | Ingestion patterns, transfer methods, change detection, landing zones, platform mapping |\n| `notebook-development` | Cell organization, parameterization, Fabric/Databricks/Jupyter patterns |\n\n**Debug \u0026 Operate** — diagnosing and fixing:\n\n| Skill | Purpose |\n|---|---|\n| `debug` | General debugging + Spark OOM/shuffle/skew, SQL explain plans, schema drift |\n| `dataops` | CI/CD, infrastructure provisioning, monitoring, rollback, cost controls |\n\n**Govern \u0026 Quality** — enforcing standards:\n\n| Skill | Purpose |\n|---|---|\n| `security` | STRIDE + OWASP security audit |\n| `data-quality` | Quality dimensions, assertions, contracts, quarantine patterns |\n| `code-review` | General review + SQL/Spark anti-patterns, notebook hygiene, metadata validation |\n| `test` | General testing + row count validation, schema assertions, reconciliation, SCD correctness |\n\n**Analyze \u0026 Research** — understanding and exploring:\n\n| Skill | Purpose |\n|---|---|\n| `brainstorm` | Open-ended ideation, option framing, assumption checks before research or planning |\n| `research` | Source-backed technology evaluation, best practices, and recommendations |\n| `scout` | Fast codebase exploration |\n| `docs-seeker` | External library/framework docs lookup |\n| `wiki` | Internal LLM wiki and project memory |\n\n**Utility** — supporting workflows:\n\n| Skill | Purpose |\n|---|---|\n| `git` | Git operations, conventional commits |\n| `docs` | End-user and public documentation |\n\n### ETL Skills (from datacoolie)\n\nThe `datacoolie-*` skills cover the full ETL lifecycle: `discover -\u003e architect -\u003e init -\u003e metadata -\u003e provision -\u003e deploy`.\n\nThey are not installed by reading this README. Install them when they are not already available:\n\n```bash\nnpx skills add datacoolie/datacoolie\n```\n\nFor a new datacoolie project, create the workspace `AGENTS.md` before doing any project work:\n\n```bash\nproject_name=\"sales_analytics\"\nworkspace_name=\"${project_name}_dcws\"\ncurl --create-dirs -o \"${workspace_name}/AGENTS.md\" https://raw.githubusercontent.com/datacoolie/datacoolie/main/ai/AGENTS.md\n```\n\nBy convention, `{workspace_name}` is `{project_name}_dcws`. Preserve an existing workspace `AGENTS.md` unless the user explicitly asks to replace it.\n\n## Plans \u0026 Templates\n\nReusable plan templates for data engineering work live in `plans/templates/`:\n\n| Template | Use when |\n|---|---|\n| `feature-implementation-template.md` | New ingestion pipelines, silver transforms, gold aggregates, data models |\n| `bug-fix-template.md` | Row count mismatches, schema drift, quality gate failures, pipeline errors |\n| `refactor-template.md` | Partition redesign, notebook modularization, query optimization, layer consolidation |\n| `template-usage-guide.md` | Selecting the right template and quality checklist |\n\nConvention: copy the template to `plans/YYMMDD-feature-name/plan.md`.\n\n## Relationship to datacoolie\n\n**datacoolie** is a pip-installable Python library that runs ETL pipelines.\n\n**dekit** wraps datacoolie with everything else a data engineering team needs: Spark development patterns, SQL authoring, data modeling, data quality, notebook best practices, internal wiki maintenance, end-user docs, plus general engineering workflows (git, debugging, testing, planning, code review).\n\n## Getting Started\n\n1. Clone this repo into your workspace\n2. Read `AGENTS.md`\n3. Configure your AI runner to load the relevant adapters\n4. For new Standard or Complex work, copy a template from `plans/templates/` to `plans/YYMMDD-feature-name/plan.md`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacoolie%2Fdekit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatacoolie%2Fdekit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacoolie%2Fdekit/lists"}