{"id":18579341,"url":"https://github.com/chenhunghan/mlx-training-rs","last_synced_at":"2025-07-30T06:08:36.680Z","repository":{"id":216455264,"uuid":"741332890","full_name":"chenhunghan/mlx-training-rs","owner":"chenhunghan","description":"A CLI in Rust to generate synthetic data for MLX friendly training","archived":false,"fork":false,"pushed_at":"2024-01-13T07:44:26.000Z","size":1956,"stargazers_count":23,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-30T13:33:30.014Z","etag":null,"topics":["applesilicon","fine-tuning","llm","mlx","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chenhunghan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-10T07:13:53.000Z","updated_at":"2025-04-22T04:36:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"9f3dbb78-2541-44c2-b1b0-a955211beb48","html_url":"https://github.com/chenhunghan/mlx-training-rs","commit_stats":null,"previous_names":["chenhunghan/mlx-training-rs"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/chenhunghan/mlx-training-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhunghan%2Fmlx-training-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhunghan%2Fmlx-training-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhunghan%2Fmlx-training-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhunghan%2Fmlx-training-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chenhunghan","download_url":"https://codeload.github.com/chenhunghan/mlx-training-rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenhunghan%2Fmlx-training-rs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267820862,"owners_count":24149292,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["applesilicon","fine-tuning","llm","mlx","rust"],"created_at":"2024-11-06T23:40:16.214Z","updated_at":"2025-07-30T06:08:36.658Z","avatar_url":"https://github.com/chenhunghan.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mlx-training-rs\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/logo.jpeg\" width=\"320\" height=\"320\" alt=\"mlxt logo\" /\u003e\n\u003c/p\u003e\n\nA CLI to generate __synthetic__ data for MLX fine-tuning. The CLI is largely translated from the php version [here](https://apeatling.com/articles/simple-guide-to-local-llm-fine-tuning-on-a-mac-with-mlx/?utm_source=pocket_reader).\n\n## Demo\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/demo.gif\" width=\"900\" alt=\"Demo of mlxt\" /\u003e\n\u003c/p\u003e\n\n## QLoRa fine-tuning for dummies on Apple Silicon\n\nBased on [this](https://apeatling.com/articles/simple-guide-to-local-llm-fine-tuning-on-a-mac-with-mlx/?utm_source=pocket_reader), [this](https://www.reddit.com/r/LocalLLaMA/comments/191s7x3/a_simple_guide_to_local_llm_finetuning_on_a_mac/?share_id=hH4Vu8gxZgwYRvl_fIyOu\u0026utm_content=1\u0026utm_medium=ios_app\u0026utm_name=ioscss\u0026utm_source=share\u0026utm_term=1), [this](https://www.reddit.com/r/LocalLLaMA/comments/18ujt0n/using_gpus_on_a_mac_m2_max_via_mlx_update_on/) and [this](https://www.reddit.com/r/LocalLLaMA/comments/18wabkc/lessons_learned_so_far_lora_fine_tuning_on/).\n\n### Preparing\n\nInstall HomeBrew, it's a package manager that help use to install all other dependencies.\n\n```shell\n/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"\n```\n\nSetting up Python3 (if you haven't)\n```shell\nbrew install python@3.11\n```\n\nClone MLX and download the model for fine-tuning.\n```sh\ngit clone https://github.com/ml-explore/mlx-examples.git\n```\n\nDownload and convert [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2).\n\n```sh\ncd mlx-examples/llm/hf-lllm\npip install -r requirements.txt # or pip3\npython convert.py --hf-path mistralai/Mistral-7B-Instruct-v0.2 -q --mlx-path ./Mistral-7B-Instruct-v0.2-mlx-4bit\n```\nWe are adding `-q` for coverting into a 4-bit quantized MLX model to `./Mistral-7B-Instruct-v0.2-mlx-4bit`\n\nIt will tale some time...\n\nThe converted MLX version has something we don't need when fine-tuning the model, edit `./Mistral-7B-Instruct-v0.2-mlx-4bit/config.json`, replace all with:\n\n```json\n{\n    \"vocab_size\": 32000,\n    \"max_position_embeddings\": 32768,\n    \"hidden_size\": 4096,\n    \"intermediate_size\": 14336,\n    \"num_hidden_layers\": 32,\n    \"num_attention_heads\": 32,\n    \"sliding_window\": null,\n    \"num_key_value_heads\": 8,\n    \"hidden_act\": \"silu\",\n    \"initializer_range\": 0.02,\n    \"rms_norm_eps\": 1e-05,\n    \"use_cache\": true,\n    \"rope_theta\": 1000000.0,\n    \"attention_dropout\": 0.0,\n    \"return_dict\": true,\n    \"output_hidden_states\": false,\n    \"output_attentions\": false,\n    \"torchscript\": false,\n    \"torch_dtype\": \"bfloat16\",\n    \"use_bfloat16\": false,\n    \"tf_legacy_loss\": false,\n    \"pruned_heads\": {},\n    \"tie_word_embeddings\": false,\n    \"is_encoder_decoder\": false,\n    \"is_decoder\": false,\n    \"cross_attention_hidden_size\": null,\n    \"add_cross_attention\": false,\n    \"tie_encoder_decoder\": false,\n    \"max_length\": 20,\n    \"min_length\": 0,\n    \"do_sample\": false,\n    \"early_stopping\": false,\n    \"num_beams\": 1,\n    \"num_beam_groups\": 1,\n    \"diversity_penalty\": 0.0,\n    \"temperature\": 1.0,\n    \"top_k\": 50,\n    \"top_p\": 1.0,\n    \"typical_p\": 1.0,\n    \"repetition_penalty\": 1.0,\n    \"length_penalty\": 1.0,\n    \"no_repeat_ngram_size\": 0,\n    \"encoder_no_repeat_ngram_size\": 0,\n    \"bad_words_ids\": null,\n    \"num_return_sequences\": 1,\n    \"chunk_size_feed_forward\": 0,\n    \"output_scores\": false,\n    \"return_dict_in_generate\": false,\n    \"forced_bos_token_id\": null,\n    \"forced_eos_token_id\": null,\n    \"remove_invalid_values\": false,\n    \"exponential_decay_length_penalty\": null,\n    \"suppress_tokens\": null,\n    \"begin_suppress_tokens\": null,\n    \"architectures\": [\n        \"MistralForCausalLM\"\n    ],\n    \"finetuning_task\": null,\n    \"id2label\": {\n        \"0\": \"LABEL_0\",\n        \"1\": \"LABEL_1\"\n    },\n    \"label2id\": {\n        \"LABEL_0\": 0,\n        \"LABEL_1\": 1\n    },\n    \"tokenizer_class\": null,\n    \"prefix\": null,\n    \"bos_token_id\": 1,\n    \"pad_token_id\": null,\n    \"eos_token_id\": 2,\n    \"sep_token_id\": null,\n    \"decoder_start_token_id\": null,\n    \"task_specific_params\": null,\n    \"problem_type\": null,\n    \"model_type\": \"mistral\",\n    \"quantization\": {\n        \"group_size\": 64,\n        \"bits\": 4\n    }\n}\n```\n\n### Generating Training Data\n\nDelete example data in `mlx-examples/lora/data`, you can delete everything inside.\n\nInstall `mlxt`, the tool in this repo.\n```sh\nbrew install chenhunghan/homebrew-formulae/mlx-training-rs\n```\n\nGenerate a training on a topic you are interested in.\n```sh\nexport OPENAI_API_KEY=[don't tell me your key]\nmlxt --topic=\"[the topic you are interested, e.g. Large Language Model]\"\n```\n\n### Fine-tuning!\n\n```sh\ncd mlx-examples/lora\npip install -r requirements.txt # or pip3\npython lora.py --train --model ../llms/hf_llm/Mistral-7B-Instruct-v0.2-mlx-4bit --data ./data --batch-size 1 --lora-layers 4\n```\n\nTo chat with your fine-tuned model, see [here](https://github.com/ml-explore/mlx-examples/tree/main/lora#generate)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchenhunghan%2Fmlx-training-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchenhunghan%2Fmlx-training-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchenhunghan%2Fmlx-training-rs/lists"}