{"id":29817034,"url":"https://github.com/reinterpretcat/qwen3-rs","last_synced_at":"2025-07-28T20:11:52.259Z","repository":{"id":304055834,"uuid":"1017640744","full_name":"reinterpretcat/qwen3-rs","owner":"reinterpretcat","description":"An educational Rust project for exploring and running Qwen3 language family models","archived":false,"fork":false,"pushed_at":"2025-07-10T21:41:28.000Z","size":68,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-11T03:41:19.860Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reinterpretcat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-10T21:25:18.000Z","updated_at":"2025-07-10T21:41:32.000Z","dependencies_parsed_at":"2025-07-11T03:44:05.240Z","dependency_job_id":"2e0f8e07-6161-450b-ad23-d30d11012f8c","html_url":"https://github.com/reinterpretcat/qwen3-rs","commit_stats":null,"previous_names":["reinterpretcat/qwen3-rs"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/reinterpretcat/qwen3-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reinterpretcat%2Fqwen3-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reinterpretcat%2Fqwen3-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reinterpretcat%2Fqwen3-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reinterpretcat%2Fqwen3-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reinterpretcat","download_url":"https://codeload.github.com/reinterpretcat/qwen3-rs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reinterpretcat%2Fqwen3-rs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267578003,"owners_count":24110351,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-28T20:11:51.573Z","updated_at":"2025-07-28T20:11:52.252Z","avatar_url":"https://github.com/reinterpretcat.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Description\n\n**qwen3-rs** is an educational Rust project for exploring and running Qwen3 language family models. It is designed to be clear, modular, and approachable for learners, with minimal dependencies and many core algorithms reimplemented from scratch for transparency.\n\n\u003e **Note:** Parts of this codebase, including documentation and core algorithms, were generated or assisted by large language models (LLMs) to accelerate development and improve educational clarity. As a starting reference, the project [qwen3.c](https://github.com/adriancable/qwen3.c) was used for understanding model internals and file formats.\n\n\n## Project Goals\n\n- **Educational:** Learn how transformer architectures, quantization, and efficient inference work in Rust.\n- **Minimal Dependencies:** Most algorithms (tokenization, quantization, sampling, etc.) are implemented from scratch—no heavy ML or Python bindings.\n- **Modular:** Core library logic is separated from CLI tools for clarity and maintainability.\n- **Efficiency:** Uses memory mapping and zero-copy techniques for handling large model files.\n\n## Workspace Structure\n\n```\nqwen3-rs/\n├── docs                # LLM generated docs for key components\n├── Cargo.toml          # Workspace configuration\n├── qwen3-cli/          # Command-line interface crate\n├── qwen3-export/       # Model export crate\n├── qwen3-inference/    # LLM inference crate\n```\n\n## How to Use\n\n### 1. Get a HuggingFace Qwen3 model\n\n```bash\ngit clone https://huggingface.co/Qwen/Qwen3-0.6B\n# Or try larger/alternative models:\n# git clone https://huggingface.co/Qwen/Qwen3-4B\n# git clone https://huggingface.co/Qwen/Qwen3-8B\n# git clone https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B\n```\n\n### 2. Build and run the exporter\n\n```bash\ncargo build --release -p qwen3-cli\n\n# Export a HuggingFace model to quantized checkpoint format\ncargo run --release -p qwen3-cli -- export /path/to/model /path/to/output.bin --group-size 64\n```\n\n### 3. Run inference\n\nIn chat mode with default parameters:\n\n```bash\ncargo run --release -p qwen3-cli -- inference /path/to/output.bin -m chat\n```\n\n## CLI Commands and Options\n\n### `export`\nExports a HuggingFace Qwen3 model to a custom binary format for efficient Rust inference.\n\n**Usage:**\n```bash\nqwen3 export \u003cMODEL_PATH\u003e \u003cOUTPUT_PATH\u003e [--group-size \u003cSIZE\u003e]\n```\n- `MODEL_PATH`: Path to HuggingFace model directory (must contain config.json, *.safetensors, tokenizer.json)\n- `OUTPUT_PATH`: Output path for the binary model file\n- `--group-size`, `-g`: Quantization group size (default: 64)\n\n### `inference`\nRuns inference on a binary Qwen3 model.\n\n**Usage:**\n```bash\nqwen3 inference \u003ccheckpoint\u003e [options]\n```\n**Options:**\n- `--temperature`, `-t \u003cFLOAT\u003e`: Sampling temperature (default: 1.0)\n- `--topp`, `-p \u003cFLOAT\u003e`: Top-p nucleus sampling (default: 0.9)\n- `--seed`, `-s \u003cINT\u003e`: Random seed\n- `--context`, `-c \u003cINT\u003e`: Context window size (default: max_seq_len)\n- `--mode`, `-m \u003cSTRING\u003e`: Mode: `generate` or `chat` (default: chat)\n- `--input`, `-i \u003cSTRING\u003e`: Input prompt\n- `--system`, `-y \u003cSTRING\u003e`: System prompt (for chat mode)\n- `--reasoning`, `-r \u003cINT\u003e`: Reasoning mode: 0=no thinking, 1=thinking (default: 0)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freinterpretcat%2Fqwen3-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freinterpretcat%2Fqwen3-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freinterpretcat%2Fqwen3-rs/lists"}