{"id":50311462,"url":"https://github.com/azayzel/my-lm","last_synced_at":"2026-05-28T21:02:03.996Z","repository":{"id":354538480,"uuid":"1222865969","full_name":"Azayzel/my-lm","owner":"Azayzel","description":"Local, all-in-one Electron + Python playground for open-weight LLMs, SDXL image generation, RAG, and QLoRA fine-tuning — tuned to fit on a single 6 GB consumer GPU.","archived":false,"fork":false,"pushed_at":"2026-05-06T18:22:38.000Z","size":14964,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-06T18:40:49.417Z","etag":null,"topics":["ai","chatbot","desktop-app","electron","fine-tuning","generative-ai","huggingface","image-generation","llm","local-llm","lora","mongodb-atlas","python","pytorch","qlora","rag","sdxl","stable-diffusion","typescript","vector-search"],"latest_commit_sha":null,"homepage":"https://github.com/Azayzel/my-lm","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Azayzel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-27T19:34:32.000Z","updated_at":"2026-04-29T00:22:49.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Azayzel/my-lm","commit_stats":null,"previous_names":["azayzel/my-lm"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Azayzel/my-lm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azayzel%2Fmy-lm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azayzel%2Fmy-lm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azayzel%2Fmy-lm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azayzel%2Fmy-lm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Azayzel","download_url":"https://codeload.github.com/Azayzel/my-lm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azayzel%2Fmy-lm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33626142,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-28T02:00:06.440Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatbot","desktop-app","electron","fine-tuning","generative-ai","huggingface","image-generation","llm","local-llm","lora","mongodb-atlas","python","pytorch","qlora","rag","sdxl","stable-diffusion","typescript","vector-search"],"created_at":"2026-05-28T21:02:00.271Z","updated_at":"2026-05-28T21:02:03.990Z","avatar_url":"https://github.com/Azayzel.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# My-LM\n\n**A local, all-in-one playground for running, fine-tuning, and generating with open-weight models — on a single GPU.**\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![Node 18+](https://img.shields.io/badge/node-18+-green.svg)](https://nodejs.org/)\n[![Electron](https://img.shields.io/badge/Electron-31-47848F?logo=electron\u0026logoColor=white)](https://www.electronjs.org/)\n[![PyTorch 2.5 + CUDA 12.1](https://img.shields.io/badge/PyTorch-2.5%20%7C%20CUDA%2012.1-EE4C2C?logo=pytorch\u0026logoColor=white)](https://pytorch.org/)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)\n\n[**Install**](#-quick-start) · [**Docs**](docs/) · [**Features**](#-what-can-it-do) · [**Architecture**](docs/architecture.md) · [**Contributing**](CONTRIBUTING.md)\n\n\u003cimg src=\"docs/assets/my-lm-install-model-prompt.png\" alt=\"My-LM first-run experience\" width=\"900\" /\u003e\n\n\u003c/div\u003e\n\n---\n\n## What is My-LM?\n\nMy-LM is a **dark-mode Electron app + Python backend** that bundles four things you usually have to run separately:\n\n- 💬 a **chat LLM** (Qwen 2.5 / Llama 3.2 / Phi-3.5) with streaming and a system prompt\n- 🎨 an **SDXL image generator** with live latent previews, a face detailer, and 4× upscaling\n- 📚 a **RAG book recommender** (BookMind) over MongoDB Atlas Vector Search\n- 🧠 a **QLoRA fine-tuning loop** with live loss/epoch metrics and one-click adapter merge\n\nEverything runs **locally**, on your hardware. No tokens, no rate limits, no data leaving your machine.\n\nDesigned and tested on a **6 GB RTX 2060** — every feature has been tuned to fit that VRAM budget — but it scales up gracefully. Works on Windows, Linux, and (CPU/MPS) macOS.\n\n---\n\n## ✨ What can it do?\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd width=\"50%\" valign=\"top\"\u003e\n\n### 🎨 Generate images with live previews\nSDXL with `compel` long-prompt support, structured prompt builder, ETA, saved prompt presets, and **streaming latent previews** every 2 steps via TAESDXL — watch the image materialize in real time.\n\n\u003cimg src=\"docs/assets/realtime_streaming_generation.png\" alt=\"Live latent preview during SDXL generation\" /\u003e\n\n\u003c/td\u003e\n\u003ctd width=\"50%\" valign=\"top\"\u003e\n\n### 💬 First-class onboarding\nWelcome modal detects which essential models you're missing (~13 GB total) and offers one-click install from Hugging Face. No CLI gymnastics, no `git lfs` errors.\n\n\u003cimg src=\"docs/assets/gen_image_1.png\" alt=\"Prompt builder on the Generate screen\" /\u003e\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"50%\" valign=\"top\"\u003e\n\n### 📚 BookMind — RAG book recommender\nSemantic search over a MongoDB Atlas library using `$vectorSearch` on 384-dim embeddings, optionally blended with a user's stored taste vector, with **grounded LLM explanations** that can only mention books from the retrieved set.\n\n\u003cimg src=\"docs/assets/book_recommender.png\" alt=\"BookMind RAG recommender screen\" /\u003e\n\n\u003c/td\u003e\n\u003ctd width=\"50%\" valign=\"top\"\u003e\n\n### 🧠 QLoRA fine-tuning, no notebook required\n4-bit quantized base model + LoRA adapters, trainable on 6 GB VRAM. Live loss/epoch/LR metrics streamed from a `TrainerCallback`. **Merge Adapter** button fuses the LoRA back into the base model in one click.\n\n\u003cimg src=\"docs/assets/train.png\" alt=\"QLoRA fine-tuning screen\" /\u003e\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"50%\" valign=\"top\"\u003e\n\n### 📦 GPU-aware model catalog\nCurated picks across LLMs, SDXL/SD1.5, upscalers, face detectors, and preview VAEs. Detects your VRAM and **hides anything that won't fit** by default. Manual HF repo download with progress for everything else.\n\n\u003cimg src=\"docs/assets/manage_models.png\" alt=\"Model catalog\" /\u003e\n\n\u003c/td\u003e\n\u003ctd width=\"50%\" valign=\"top\"\u003e\n\n### 🖥 GPU dashboard + diagnostics\nLive `torch.cuda` info plus `nvidia-smi` output (memory, temperature, utilization). Sidebar status indicator shows LLM / Image / Python health at a glance.\n\n\u003cimg src=\"docs/assets/gpu_info.png\" alt=\"GPU information screen\" /\u003e\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003e **Also includes:** ADetailer-style face fix (YOLOv8 + SDXL img2img on each face, alpha-composited back), 4× ESRGAN upscaling tiled to fit 6 GB VRAM, image gallery with lightbox, persistent chat/prompt history, four standalone CLI tools.\n\nSee [docs/features.md](docs/features.md) for the full per-feature breakdown.\n\n---\n\n## 🚀 Quick start\n\n**Prerequisites:** Python 3.10+, Node.js 18+, NVIDIA GPU + CUDA drivers, `nvidia-smi` on PATH.\n\n```bash\ngit clone https://github.com/Azayzel/my-lm.git\ncd my-lm\n\n# Linux / macOS\n./setup.sh\n\n# Windows\nsetup.bat\n\n# Launch\ncd ui \u0026\u0026 npm start\n```\n\nThe setup script creates `.venv`, installs CUDA-matched PyTorch wheels, installs the `mylm` package in editable mode, and builds the Electron UI. On first launch you'll see the **Install Models** modal pictured above.\n\nFor BookMind, copy `.env.example` to `.env` and fill in your MongoDB Atlas connection. Full setup details + troubleshooting are in [docs/installation.md](docs/installation.md).\n\n---\n\n## 📚 Documentation\n\n| Doc                                          | What's in it                                       |\n| -------------------------------------------- | -------------------------------------------------- |\n| [docs/features.md](docs/features.md)         | Per-screen feature tour                            |\n| [docs/installation.md](docs/installation.md) | Setup, model downloads, troubleshooting            |\n| [docs/architecture.md](docs/architecture.md) | Process model, IPC, bridge protocol, memory budget |\n| [docs/cli.md](docs/cli.md)                   | Standalone CLI tools (`scripts/`)                  |\n| [docs/bookmind.md](docs/bookmind.md)         | BookMind RAG configuration                         |\n| [docs/packaging.md](docs/packaging.md)       | Building installers                                |\n| [docs/benchmarking.md](#-benchmarking)       | Agent benchmark harness                            |\n| [CONTRIBUTING.md](CONTRIBUTING.md)           | How to contribute                                  |\n| [SECURITY.md](SECURITY.md)                   | Reporting vulnerabilities                          |\n\n---\n\n## 🧱 Architecture at a glance\n\n```text\n┌─ Electron main (Node) ─────────────────────────────────┐\n│   ├─ llmBridge   ──spawns──▶  python scripts/llm_bridge.py\n│   ├─ imageBridge ──spawns──▶  python scripts/image_bridge.py\n│   ├─ trainBridge ──spawns──▶  python scripts/train_bridge.py\n│   ├─ bookBridge  ──spawns──▶  python scripts/book_bridge.py\n│   └─ preload exposes a typed API as window.My\n│                                                         │\n│ Renderer (sandboxed) — vanilla TS + webpack             │\n└─────────────────────────────────────────────────────────┘\n```\n\nLong-lived Python subprocesses exchange newline-delimited JSON over stdin/stdout. Screens can switch freely without interrupting any running operation.\n\n```text\nmy-lm/\n├── src/mylm/        Reusable Python library (pip-installable)\n├── scripts/         Bridge + CLI entry points (spawned by Electron)\n├── ui/              Electron app (TypeScript main + renderer)\n├── tests/           pytest suite\n├── datasets/        Example training data\n└── docs/            Long-form documentation\n```\n\n`models/`, `outputs/`, and `benchmark_results/` are runtime-only and gitignored.\n\n---\n\n## 📏 Memory budget on 6 GB\n\nThe app is designed so no single step blows out VRAM:\n\n- SDXL base pass uses `enable_model_cpu_offload()` to stream components on/off GPU\n- Text encoders move to GPU only for `compel` prompt encoding, then back to CPU\n- Face detailer **reuses the base pipe's weights** — no second SDXL load\n- 4× upscale runs **tiled at 384 px** with 32 px overlap\n- Training uses **4-bit NF4 quantization** + gradient checkpointing\n\n\u003e **Don't run chat + image + training simultaneously** — they each want the whole GPU.\n\n---\n\n## 📊 Benchmarking\n\nMy-LM includes a **multi-turn agent benchmark** (`scripts/agent_bench.py`) that measures LLM inference performance in agentic workloads — where context accumulates across turns and fixed KV windows can cause model reloads.\n\n### What it measures\n\n| Metric | Why it matters |\n| ------ | -------------- |\n| TTFT (time to first token) | Raw responsiveness per turn |\n| TTFT slope (ms/turn) | How latency grows as context accumulates |\n| Model reloads | Cold restarts caused by `num_ctx` changes |\n| Swap events | macOS/Linux memory pressure detection |\n| Peak RAM | KV cache memory footprint |\n| Tool-call errors | Format reliability under different temperatures |\n| Context tokens (final) | Context drift / trimming effectiveness |\n| Task success rate | End-to-end completion |\n\n### Tasks\n\n- **`code_debugger`** — read buggy Python files, run tests, identify failures, write fixes. Context grows with each file read and test output.\n- **`research_synth`** — read research documents, cross-reference findings, produce a structured synthesis. Context grows steadily with each doc.\n\n### Quick start\n\n```bash\n# Dry-run — validate task definitions, no Ollama required:\npython scripts/agent_bench.py --dry-run\n\n# Quick run (2 tasks, 2 trials):\npython scripts/agent_bench.py --models llama3.2:3b \\\n    --tasks code_debugger,research_synth --trials 2\n\n# Full run (all tasks, 5 trials):\npython scripts/agent_bench.py\n```\n\nResults are written to `benchmark_results/` as both JSON (raw data) and Markdown (formatted report). See the `--help` flag for all options.\n\n---\n\n## 🤝 Contributing\n\nPRs welcome! [CONTRIBUTING.md](CONTRIBUTING.md) covers the dev loop, coding conventions, and how to run linters/tests locally. Be excellent to each other — see the [Code of Conduct](CODE_OF_CONDUCT.md).\n\nFound a bug or have an idea? [Open an issue](https://github.com/Azayzel/my-lm/issues/new/choose) · Have a question? [Start a discussion](https://github.com/Azayzel/my-lm/discussions).\n\n---\n\n## 📝 License\n\n[MIT](LICENSE) © 2026 Josh Lavely\n\n\u003cdiv align=\"center\"\u003e\n\u003csub\u003eBuilt for tinkerers with modest GPUs and big curiosity.\u003c/sub\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazayzel%2Fmy-lm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazayzel%2Fmy-lm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazayzel%2Fmy-lm/lists"}