{"id":51265852,"url":"https://github.com/tomblancdev/data-science-devcontainer","last_synced_at":"2026-06-29T15:01:10.749Z","repository":{"id":364011719,"uuid":"1265076424","full_name":"tomblancdev/data-science-devcontainer","owner":"tomblancdev","description":"Portable, batteries-included data-science dev container: uv · ruff · polars · duckdb · Jupyter · marimo (Compose-based, offline-ready)","archived":false,"fork":false,"pushed_at":"2026-06-11T08:21:44.000Z","size":20,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-11T10:13:55.594Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tomblancdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-10T12:49:55.000Z","updated_at":"2026-06-11T08:21:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tomblancdev/data-science-devcontainer","commit_stats":null,"previous_names":["tomblancdev/data-science-devcontainer"],"tags_count":null,"template":true,"template_full_name":null,"purl":"pkg:github/tomblancdev/data-science-devcontainer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomblancdev%2Fdata-science-devcontainer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomblancdev%2Fdata-science-devcontainer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomblancdev%2Fdata-science-devcontainer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomblancdev%2Fdata-science-devcontainer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tomblancdev","download_url":"https://codeload.github.com/tomblancdev/data-science-devcontainer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomblancdev%2Fdata-science-devcontainer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34931592,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-29T02:00:05.398Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-29T15:01:07.995Z","updated_at":"2026-06-29T15:01:10.739Z","avatar_url":"https://github.com/tomblancdev.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Science Dev Container\n\nA portable, batteries-included **VS Code / containers.dev** environment for data\nanalysis. Open the folder, \"Reopen in Container\", and you get a fully reproducible\nstack with no host setup beyond Docker.\n\n\u003e Self-contained and codebase-agnostic — copy the whole folder (including the\n\u003e dotfiles) into any project to bootstrap it.\n\n## Quick start\n\n1. Install **Docker Desktop** (Windows/macOS) or Docker/Podman (Linux) + the VS\n   Code **Dev Containers** extension (`ms-vscode-remote.remote-containers`).\n   The default config is **Dockerfile-based — no `docker compose` required**.\n2. Open this folder in VS Code → **\"Reopen in Container\"**.\n   (If prompted, pick **\"Data Science (uv · ruff · polars · duckdb)\"** — the\n   plain Dockerfile config. A Compose variant is also offered; see below.)\n3. Wait for the one-time build, then:\n\n   ```bash\n   just            # list all tasks\n   just lab        # JupyterLab    → http://localhost:8888\n   just marimo     # reactive nb   → http://localhost:2718\n   just test       # pytest + coverage\n   ```\n\n4. Open `notebooks/00-welcome.ipynb` and pick the **`Python (uv · data-science)`** kernel.\n\n## Why these tools (the \"new tech\")\n\n| Concern            | Choice                     | Why it's best-in-class                                            |\n| ------------------ | -------------------------- | ---------------------------------------------------------------- |\n| Packages + Python  | **uv** (Astral)            | 10–100× faster than pip/poetry; installs \u0026 pins CPython; `uv.lock` for reproducibility |\n| Lint + format      | **ruff** (Astral)          | One Rust tool replacing black + flake8 + isort + pylint; fix-on-save |\n| Type check         | **mypy** + **`uvx ty`**    | Stable mypy, plus Astral's new Rust checker `ty` on demand (no install) |\n| Dataframes         | **Polars** + **narwhals**  | Multithreaded, lazy, Arrow-native; narwhals lets code target Polars *or* pandas |\n| SQL / analytics    | **DuckDB**                 | In-process OLAP over CSV/Parquet/Arrow; queries a Polars df zero-copy |\n| Notebooks          | **Jupyter** + **marimo**   | Classic `.ipynb` in VS Code, plus reactive git-friendly `.py` notebooks |\n| Explore visually   | **Data Wrangler**          | Point-and-click profiling/cleaning that emits Polars/pandas code |\n| Task runner        | **just**                   | Discoverable, dependency-free command shortcuts (`justfile`)     |\n| Clean diffs        | **nbstripout** + jupytext  | Strip outputs / pair notebooks to `.py` so reviews stay readable |\n\n## Layout\n\n```\n.\n├── .devcontainer/\n│   ├── devcontainer.json     # DEFAULT: Dockerfile-based, no Compose (Windows-friendly)\n│   ├── Dockerfile            # base + uv + native libs + CPython + venv baked at build\n│   ├── postCreate.sh         # kernel registration, ./.venv symlink, git hooks\n│   ├── compose.yaml          # optional Compose variant: build, mounts, env, secrets\n│   ├── compose.podman.yaml   # Compose: rootless-Podman override (userns keep-id)\n│   └── compose/\n│       └── devcontainer.json # optional \"… (Compose)\" config (shown in the picker)\n├── notebooks/00-welcome.ipynb\n├── src/ds_workspace/         # importable, testable helpers\n├── tests/                    # pytest smoke tests\n├── data/                     # git-ignored local data (placeholder ships)\n├── pyproject.toml            # deps, extras, ruff/mypy/pytest config\n├── .gitattributes            # force LF endings — Windows-safe shell scripts\n├── .pre-commit-config.yaml\n└── justfile                  # task shortcuts\n```\n\n## Container engine \u0026 variants\n\nTwo ways to run it — VS Code shows a picker on \"Reopen in Container\":\n\n1. **Dockerfile (default, recommended).** Just `devcontainer.json`, **no Docker\n   Compose required**. Works out of the box on Windows / macOS / Linux with\n   Docker Desktop, Podman, or Rancher Desktop. Mounts, ports and env are set\n   directly in `devcontainer.json`.\n2. **Compose** (`.devcontainer/compose/devcontainer.json`). Use this if you want\n   compose-managed secrets or plan to add more services; runtime lives in\n   `compose.yaml`.\n\n- **Environment** — Dockerfile variant: `containerEnv` in `devcontainer.json`.\n  Compose variant: `environment:` + an optional `../.env` (copy `.env.example`).\n- **Secrets** — Compose variant has a `secrets:` block (mounted at\n  `/run/secrets/\u003cname\u003e`, never baked into an image layer). For the Dockerfile\n  variant, mount a file via `mounts`.\n- **Ports** — both only **forward** (never hard-publish) via `forwardPorts`, so\n  VS Code auto-picks a free local port and rebuilds never hit \"port already\n  allocated\".\n\n### Windows notes\n\n- Use **Docker Desktop** (WSL2 backend) — the default config needs no\n  `docker compose`. Podman Desktop / Rancher Desktop work too.\n- `.gitattributes` forces **LF** line endings so `postCreate.sh` runs inside the\n  Linux container (CRLF would break it with `$'\\r': command not found`). If you\n  cloned *before* `.gitattributes` existed, normalize once:\n  `git add --renormalize . \u0026\u0026 git checkout .` (or just re-clone).\n\n### Networking: build vs. runtime\n\nThe whole Python environment (managed CPython **and** all packages) is installed\n**at image-build time** by the `Dockerfile`, into `/home/vscode/.venv`. Opening\nthe container then needs **no internet** — `postCreate` only registers the\nJupyter kernel and git hooks.\n\n- **Restricted runtime network?** You're fine — nothing is downloaded on open.\n  `UV_PYTHON_DOWNLOADS=never` makes uv fail fast (clear message) instead of\n  hanging if anything ever probes for a download.\n- **Restricted build network (proxy)?** Export `HTTP_PROXY` / `HTTPS_PROXY` /\n  `NO_PROXY` before \"Rebuild Container\" (or set them in `.env`); they flow into\n  the build for apt + uv.\n- **Changed dependencies?** Edit `pyproject.toml`, then `just sync` (needs\n  network once) or rebuild the container.\n\n\u003e Why this matters: a common failure is uv trying to download CPython/packages\n\u003e *on first open*, which dies on locked-down networks. Baking everything at\n\u003e build time — where the network is usually open — avoids that entirely.\n\n### Running on Podman (rootless)\n\nSo bind-mounted files stay writable, map the container user to your host UID:\n\n- **Dockerfile variant** — uncomment in `devcontainer.json`:\n  ```jsonc\n  \"runArgs\": [\"--userns=keep-id\"]\n  ```\n- **Compose variant** — list both files in the Compose config's\n  `dockerComposeFile` (Compose ignores `runArgs`, so the override carries it):\n  ```jsonc\n  \"dockerComposeFile\": [\"../compose.yaml\", \"../compose.podman.yaml\"]\n  ```\n  `compose.podman.yaml` adds `userns_mode: keep-id` + `security_opt:\n  [label=disable]` (SELinux hosts).\n\nOn Docker / Docker Desktop, leave both off.\n\n## Common tweaks\n\n- **Python version** — edit `.python-version` and the `PYTHON_VERSION` build arg\n  in `devcontainer.json`.\n- **Add a library** — `just add \u003cpkg\u003e` (runtime) or `just add-dev \u003cpkg\u003e` (tooling);\n  the lockfile updates automatically.\n- **Optional extras** — `ml`, `db`, `geo` are defined in `pyproject.toml`\n  (installed by default via `uv sync --all-extras`; trim as needed).\n- **AI assistant** — none is bundled. Add one to `extensions` in\n  `devcontainer.json` if you want it, e.g. `Continue.continue` (model-agnostic:\n  Claude, GPT, local Ollama) or `GitHub.copilot`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomblancdev%2Fdata-science-devcontainer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftomblancdev%2Fdata-science-devcontainer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomblancdev%2Fdata-science-devcontainer/lists"}