{"id":50771787,"url":"https://github.com/basnijholt/ai-lab","last_synced_at":"2026-06-11T19:31:44.760Z","repository":{"id":324992307,"uuid":"1099380551","full_name":"basnijholt/ai-lab","owner":"basnijholt","description":"My local AI stack on NixOS with dual RTX 3090 GPUs","archived":false,"fork":false,"pushed_at":"2026-01-01T15:17:41.000Z","size":39,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-06T19:22:07.473Z","etag":null,"topics":["agent-cli","ikllamacpp","kokoro","llamacpp","ollama"],"latest_commit_sha":null,"homepage":"","language":"Just","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/basnijholt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-18T23:23:34.000Z","updated_at":"2026-01-01T15:17:45.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/basnijholt/ai-lab","commit_stats":null,"previous_names":["basnijholt/ai"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/basnijholt/ai-lab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basnijholt%2Fai-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basnijholt%2Fai-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basnijholt%2Fai-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basnijholt%2Fai-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/basnijholt","download_url":"https://codeload.github.com/basnijholt/ai-lab/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basnijholt%2Fai-lab/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34215253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-cli","ikllamacpp","kokoro","llamacpp","ollama"],"created_at":"2026-06-11T19:31:44.614Z","updated_at":"2026-06-11T19:31:44.725Z","avatar_url":"https://github.com/basnijholt.png","language":"Just","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Lab Workspace\n\nThis repository serves as a **meta-workspace** for managing, building, and running various local AI tools and inference engines. It centralizes dependencies using [Nix](https://nixos.org/) and orchestrates tasks using [Just](https://github.com/casey/just).\n\nThe goal is to provide a reproducible, one-click setup for compiling high-performance inference backends (like `llama.cpp`) and running services (like TTS and ASR) without managing individual environments for each submodule.\n\n## 📂 Included Projects\n\nAll external projects are managed as git submodules in the `external/` directory:\n\n*   **[llama.cpp](https://github.com/ggerganov/llama.cpp):** Inference of LLaMA model in pure C/C++.\n*   **[ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp):** A fork of llama.cpp with optimizations.\n*   **[Ollama](https://github.com/ollama/ollama):** Get up and running with large language models.\n*   **[Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI):** A Dockerized/FastAPI wrapper for the Kokoro TTS model.\n*   **[agent-cli](https://github.com/basnijholt/agent-cli):** CLI agent tool (used here for its `faster-whisper` server script).\n\n## 🛠️ Prerequisites\n\n*   **[Nix](https://nixos.org/download.html):** Required for the environment.\n*   **[Direnv](https://direnv.net/)** (Recommended): Automatically loads the Nix environment when you enter the directory.\n*   **Git:** To manage the repository and submodules.\n\n## 🚀 Getting Started\n\n1.  **Clone the repository:**\n    ```bash\n    git clone --recursive git@github.com:basnijholt/ai.git\n    cd ai\n    ```\n\n2.  **Enter the environment:**\n    If you have `direnv` installed:\n    ```bash\n    direnv allow\n    ```\n    Otherwise, drop into the Nix shell manually:\n    ```bash\n    nix-shell\n    ```\n    *This provides `cmake`, `gcc`, `go`, `cuda`, `python`, `uv`, and `just` configured specifically for these projects.*\n\n3.  **Build everything:**\n    ```bash\n    just build\n    ```\n\n## 🤖 Commands\n\nThe `justfile` defines all available commands.\n\n### Global Operations\n| Command | Alias | Description |\n| :--- | :--- | :--- |\n| `just build` | `just b` | Compiles `llama.cpp`, `ik_llama.cpp`, and `ollama` from scratch. |\n| `just rebuild` | `just r` | Incrementally recompiles all projects. |\n| `just sync` | `just s` | Pulls the latest changes for **all** submodules from their upstream remotes. |\n| `just commit-submodules` | `just cs` | Commits submodule updates with an auto-generated message (only updated modules). |\n| `just clean` | `just c` | Removes build artifacts for all projects. |\n\n### Running Services\n| Command | Description |\n| :--- | :--- |\n| `just start-kokoro` | Starts the **Kokoro TTS** server (GPU accelerated). \u003cbr\u003e *Automatically handles python venv and model downloads.* |\n| `just start-faster-whisper` | Starts the **Faster Whisper** ASR server on port 8811 (CUDA, float16). |\n\n### Individual Project Commands\nYou can also target specific projects:\n\n*   **llama.cpp:** `build-llama`, `rebuild-llama`, `clean-llama`, `sync-llama`\n*   **ik_llama.cpp:** `build-ik`, `rebuild-ik`, `clean-ik`, `sync-ik`\n*   **Ollama:** `build-ollama`, `rebuild-ollama`, `clean-ollama`, `sync-ollama`\n*   **Kokoro:** `sync-kokoro`\n*   **Agent CLI:** `sync-agent-cli`\n\n## ⚙️ Configuration\n\n\u003e [!NOTE]\n\u003e This setup is specifically tailored for a machine with **NVIDIA CUDA-compatible hardware**.\n\n*   **Build Flags:** Configured in `justfile`. These include flags for CUDA support and hardware-specific architectures (e.g., targeting NVIDIA GPUs).\n*   **Environment:** Defined in `shell.nix`. It ensures `LD_LIBRARY_PATH` includes necessary CUDA and C++ libraries for Python extensions.\n\n## 🖥️ System Configuration\n\nMy complete NixOS configuration, which powers this setup, can be found in my [dotfiles](https://github.com/basnijholt/dotfiles).\n\n## 📝 License\n\nThis meta-repository is for personal organization. Each submodule retains its own license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbasnijholt%2Fai-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbasnijholt%2Fai-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbasnijholt%2Fai-lab/lists"}