{"id":44215744,"url":"https://github.com/stark-sim/npu_train","last_synced_at":"2026-02-10T01:36:35.472Z","repository":{"id":334056889,"uuid":"1137444169","full_name":"stark-sim/npu_train","owner":"stark-sim","description":null,"archived":false,"fork":false,"pushed_at":"2026-01-22T14:36:36.000Z","size":105,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-01-23T07:28:20.874Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stark-sim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-19T11:29:25.000Z","updated_at":"2026-01-22T14:36:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/stark-sim/npu_train","commit_stats":null,"previous_names":["stark-sim/npu_train"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/stark-sim/npu_train","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stark-sim%2Fnpu_train","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stark-sim%2Fnpu_train/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stark-sim%2Fnpu_train/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stark-sim%2Fnpu_train/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stark-sim","download_url":"https://codeload.github.com/stark-sim/npu_train/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stark-sim%2Fnpu_train/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29288613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-09T21:57:15.303Z","status":"ssl_error","status_checked_at":"2026-02-09T21:57:11.537Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-10T01:36:32.679Z","updated_at":"2026-02-10T01:36:35.459Z","avatar_url":"https://github.com/stark-sim.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NPU Training Project\n\nLLM training on Huawei Ascend 910A NPU with support for Data Parallel (DDP), Pipeline Parallel (PP), and custom Tensor Parallelism (TP).\n\n## Environment\n\n- Python 3.11\n- PyTorch 2.5.1 + torch-npu 2.5.1\n- transformers 4.57.x\n- CANN 8.1.RC1\n\n## Quick Start\n\n```bash\n# Single NPU training (~30 minutes)\nchmod +x run.sh \u0026\u0026 ./run.sh\n```\n\n## Training Modes\n\n### Single NPU\n```bash\n./run.sh --model_name distilgpt2 --batch_size 16 --epochs 5\n```\n\n### Data Parallel (DDP) - 8 NPUs\nReplicates model across all NPUs, synchronizes gradients.\n```bash\nchmod +x run_ddp.sh \u0026\u0026 ./run_ddp.sh\n```\n\n### Pipeline Parallel (PP) - 8 NPUs\nSplits model layers across NPUs (e.g., 4-stage pipeline).\n```bash\nchmod +x run_pp.sh \u0026\u0026 ./run_pp.sh --pp_size 4\n```\n\n### Tensor Parallel (TP) - Custom Implementation\nSplits weights across NPUs using custom `npu_parallel` module (Megatron-LM style).\n```bash\npython examples/train_tp_custom.py --model_path \"/path/to/model\" --tp_size 4\n```\n\n## npu_parallel Module\n\nCustom Tensor Parallelism implementation for Ascend NPU/HCCL:\n\n- **tp_layers.py**: `ColumnParallelLinear`, `RowParallelLinear`\n- **tp_attention.py**: `TPAttention`, `TPMLP` with SwiGLU\n- **convert_model.py**: Auto-convert HuggingFace models to TP\n\nSupported models: Qwen/Qwen2/Qwen2.5, Llama, Mistral, Gemma, Phi, Yi, DeepSeek, Baichuan, GPT-2.\n\n## Model Management\n\n```bash\n# Download models via ModelScope\npython3 download_models.py\n\n# Download with aria2c (parallel, resume-capable)\npython3 download_with_aria2.py\n\n# Fix corrupted downloads (safetensors integrity)\npython3 fix_downloads.py\n\n# Generate model manifest with PP/TP suggestions\npython3 model_manifest.py\n```\n\n## Verification\n\n```bash\n# Check NPU availability\npython3 -c \"import torch_npu; print(torch.npu.is_available())\"\n\n# Run tests\npython3 tests/test_tp_conversion.py\npython3 tests/test_tp_mlp_only.py\npython3 tests/test_hccl_ops.py\n```\n\n## Configuration\n\n| Setting | Value |\n|---------|-------|\n| Model storage | `/home/sd/npu_train/models` |\n| HTTP proxy | `http://127.0.0.1:7890` |\n| HF mirror | `https://hf-mirror.com` |\n| HCCL timeout | `1200s` |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstark-sim%2Fnpu_train","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstark-sim%2Fnpu_train","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstark-sim%2Fnpu_train/lists"}