{"id":51272881,"url":"https://github.com/suzuke/code2lora-mlx","last_synced_at":"2026-06-29T19:30:27.303Z","repository":{"id":366922124,"uuid":"1275176465","full_name":"suzuke/code2lora-mlx","owner":"suzuke","description":"MLX (Apple Silicon) implementation of Code2LoRA: a hypernetwork that turns a repo into a repo-specific LoRA adapter for Qwen2.5-Coder — zero inference-time token overhead.","archived":false,"fork":false,"pushed_at":"2026-06-23T21:19:26.000Z","size":147,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-23T23:12:57.253Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/suzuke.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-20T10:56:24.000Z","updated_at":"2026-06-23T21:19:31.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/suzuke/code2lora-mlx","commit_stats":null,"previous_names":["suzuke/code2lora-mlx"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/suzuke/code2lora-mlx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suzuke%2Fcode2lora-mlx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suzuke%2Fcode2lora-mlx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suzuke%2Fcode2lora-mlx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suzuke%2Fcode2lora-mlx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/suzuke","download_url":"https://codeload.github.com/suzuke/code2lora-mlx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/suzuke%2Fcode2lora-mlx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34941024,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-29T02:00:05.398Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-29T19:30:26.564Z","updated_at":"2026-06-29T19:30:27.287Z","avatar_url":"https://github.com/suzuke.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# code2lora-mlx\n\nAn **MLX (Apple Silicon)** implementation of **Code2LoRA** — a hypernetwork that\nreads a whole code repository and, in a single forward pass, generates a\n**repository-specific LoRA adapter** for a frozen `Qwen2.5-Coder`. The adapter is\ninjected into the base model so completions become repo-aware **with zero\ninference-time token overhead** (the repo knowledge lives in the weights, not the\nprompt).\n\n\u003e Independent reimplementation for Apple Silicon. The method, the released\n\u003e hypernetwork checkpoints, and the RepoPeftBench datasets are from the paper\n\u003e **\"Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under\n\u003e Software Evolution\"** (Hotsko, Li, Deng, Nie — [arXiv:2606.06492](https://arxiv.org/abs/2606.06492))\n\u003e and the [`code2lora`](https://huggingface.co/code2lora) HuggingFace org. This\n\u003e repo is not affiliated with the authors.\n\n## What works here\n\n- **Faithful MLX inference of the official checkpoint.** Loads the released\n  `code2lora/code2lora-direct` hypernetwork (Qwen2.5-Coder-1.5B, rank 16) and\n  reproduces its behavior: on held-out repos the generated adapter **helps 8/8**\n  (mean cross-entropy 1.44 → 0.42). On a fresh cross-repo (`jd/tenacity`,\n  not in training) the test-assertion completion CE drops **0.84 → 0.31** (24/29\n  improved) — e.g. `assert find_ordinal(1) == ?` → base guesses wrong, adapted\n  emits the repo's actual `\"st\"`.\n- **Faithful training math.** The MLX hypernetwork's forward **and backward** match\n  the official PyTorch head to fp32 precision on a single step (gradient cosine\n  `1.0000000`, loss rel-diff `3.6e-7`) — so given equal data/compute it converges\n  to the same hypernetwork.\n- **Official repo encoder, reproduced.** `c2l/qwen3_repo_encoder.py` mirrors the\n  paper's `repo_state_embedding` recipe (Qwen3-Embedding-0.6B, `.py` files, chunk\n  2048/overlap 256, masked mean-pool, `L2norm(concat(mean_files, max_files))` →\n  2048-d). Validated against the official stored embeddings.\n\n## Install\n\nRequires [`uv`](https://docs.astral.sh/uv/) and Apple Silicon (Metal).\n\n```bash\ngit clone \u003cthis repo\u003e \u0026\u0026 cd code2lora-mlx\nuv sync          # mlx, mlx-lm, transformers, torch, sentence-transformers, ...\n```\n\n## Use it on your own repo\n\nThe official checkpoint is downloaded automatically from HuggingFace on first run.\n\n```bash\n# repo-aware completion: base vs adapted, side by side\nPYTHONPATH=. uv run python cli.py complete /path/to/python/repo \\\n  $'def test_thing():\\n    assert my_api(1) == '\n\n# quantitative: base-vs-adapted cross-entropy on held-out lines of the repo\nPYTHONPATH=. uv run python cli.py selftest /path/to/python/repo 15\n\n# just compute (and cache) the 2048-d repo embedding\nPYTHONPATH=. uv run python cli.py encode /path/to/python/repo\n```\n\n## How it works\n\n```\nrepo (.py files)\n   │  Qwen3-Embedding-0.6B, mean-pool, L2norm(concat(mean,max))\n   ▼\nrepo embedding ∈ R^2048\n   │  hypernetwork: trunk(2048-\u003e1024-\u003e1024) -\u003e per-module heads -\u003e tanh·exp(log_scale)\n   ▼\nLoRA adapter  (rank 16, for q/k/v/o/gate/up/down; shared across all 28 layers)\n   │  y = base(x) + (alpha/r)·(x·A^T)·B^T      alpha/r = 32/16 = 2\n   ▼\nfrozen Qwen2.5-Coder-1.5B  -\u003e  repo-aware completion (zero extra prompt tokens)\n```\n\n## Scope \u0026 honest notes\n\n- **Python only** as shipped: the encoder reads `.py`, and the released\n  hypernetwork was trained on Python repos for **test-assertion completion**\n  (`assert expr == value`). It shines on that task/domain; on arbitrary code\n  lines or out-of-domain repos the benefit is small. The *method* is\n  language-agnostic, but another language needs a retrained hypernetwork.\n- **Training from scratch** is supported and numerically faithful, but reproducing\n  the official-scale run (604 repos x 3 epochs, seq 4096) on an M3 Max is a\n  multi-day job (~9.8 s/step measured -\u003e ~7.5 days). The official H100 run is ~6 h.\n  For a working adapter you don't need to train — just use the released checkpoint.\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuzuke%2Fcode2lora-mlx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuzuke%2Fcode2lora-mlx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuzuke%2Fcode2lora-mlx/lists"}