{"id":27991764,"url":"https://github.com/mon-ius/xtpu","last_synced_at":"2025-06-25T07:05:08.239Z","repository":{"id":220852207,"uuid":"752762095","full_name":"Mon-ius/XTPU","owner":"Mon-ius","description":"Boost AI application dev on TPU","archived":false,"fork":false,"pushed_at":"2025-06-20T16:59:58.000Z","size":215,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-25T07:05:07.521Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mon-ius.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-02-04T18:26:45.000Z","updated_at":"2025-06-20T17:00:02.000Z","dependencies_parsed_at":"2024-02-25T23:24:04.861Z","dependency_job_id":"b61b8a4a-1dfe-4278-829c-18569452d6ca","html_url":"https://github.com/Mon-ius/XTPU","commit_stats":null,"previous_names":["mon-ius/xtpu"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/Mon-ius/XTPU","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mon-ius%2FXTPU","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mon-ius%2FXTPU/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mon-ius%2FXTPU/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mon-ius%2FXTPU/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mon-ius","download_url":"https://codeload.github.com/Mon-ius/XTPU/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mon-ius%2FXTPU/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261823736,"owners_count":23215141,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-08T17:54:17.920Z","updated_at":"2025-06-25T07:05:08.225Z","avatar_url":"https://github.com/Mon-ius.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# XTPU\n\nBoost AI and LLM application dev on TPU.\n\n## Overview\n\n🚧 Buiding in 2025.. 🚧\n\n---\n\n### 1. ⚙ New vm instance and user\n\n```bash\n# Run on Cloud Shell Terminal\ncurl -fsSL bit.ly/new-gcp-vm-instance | sh\n## Here, USER=m0nius ZONE=asia-east1-b  TEMPLATE=xvm\ncurl -fsSL bit.ly/new-gcp-vm-instance | sh -s -- m0nius asia-east1-b xvm\n\n# Generate new ssh key\ncurl -fsSL bit.ly/ssh-vm-gen | sh\n\n# New ssh server with secure config\ncurl -fsSL bit.ly/create-sshd | sh\n\n# New mamba environment with zsh\ncurl -fsSL bit.ly/create-mamba-zsh | sh\n\n# New rootless mamba environment with zsh\ncurl -fsSL bit.ly/create-mamba-zsh-rootless | sh\n```\n\n### 2. 💽 Attach vm disk\n\n```bash\n# Run on Cloud Shell Terminal\ncurl -fsSL bit.ly/attach-gcp-vm-disk  | sh\n## Here, DISK=disk-1 ZONE=asia-east1-b VM_NAME=xvm-1\ncurl -fsSL bit.ly/attach-gcp-vm-disk | sh -s -- disk-1 asia-east1-b xvm-1\n```\n\n### 3. ⛓ TPUv2, TPUv3, TPUv4, TPUv5 nodes\n\n```bash\n# Clean all queued TPU nodes\ncurl -fsSL bit.ly/clean-tpu-nodes | sh -s -- proj_name asia-east1-b\n# Run on Cloud Shell Terminal, TPUv2\ncurl -fsSL bit.ly/new-tpu-v2-node | sh -s -- -y\n# Run on Cloud Shell Terminal, queued TPUv4\ncurl -fsSL bit.ly/new-tpu-v4-queue | sh -s -- -y\n```\n\n### 4. 🫧 LLM training\n\n#### 4.1 Miniconda Environment\nTPU\n```bash\ncurl -fsSL bit.ly/tpu-torch-xla | sh\n#OR\ncurl -fsSL bit.ly/tpu-rootless-xla | sh\n```\nCUDA\n```bash\ncurl -fsSL bit.ly/cuda-torch-xla | sh\n#OR\ncurl -fsSL bit.ly/cuda-rootless-xla | sh\n```\n#### 4.2 Model Training\n\n```bash\n# Run on Cloud Shell Terminal, TPUv2\ncurl -fsSL bit.ly/new-LLM-TPUv2-train | sh -s -- -y\n# Run on Cloud Shell Terminal, queued TPUv4\ncurl -fsSL bit.ly/new-LLM-TPUv4-train | sh -s -- -y\n```\n\n#### 4.3 Docker Container\n\n```bash\n# Run with Docker Official\ncurl -fsSL bit.ly/create-docker | sh\n\n# Run with Docker THU Mirror\ncurl -fsSL bit.ly/create-docker-mirror | sh\n```\n\n### 5. 🥋 Optimize HW\n\n```bash\n# Replace OS of the VM to Alpine Linux \ncurl -fsSL bit.ly/os-LLM-Alpine-acc | sh -s -- 3.19\n```\n\n### 6. 🪢 Dataset Mount\n\n```bash\n# Mount remote dataset\ncurl -fsSL bit.ly/remote-LLM-dataset-mount | sh -s -- dataset\n```\n\n### 7. API Create\n\n```bash\ncurl -fsSL bit.ly/new-gcp-api | sh -s -- project_name api_num api_target\ncurl -fsSL bit.ly/new-gcp-dns | sh -s -- cf_token_base64 cf_zone\ncurl -fsSL bit.ly/new-gcp-sb | sh -s -- cf_token_base64 cf_zone\ncurl -fsSL bit.ly/new-gcp-sb-hy2 | sh -s -- cf_token_base64 cf_zone\ncurl -fsSL bit.ly/new-gcp-wg | sh -s -- license\ncurl -fsSL bit.ly/create-vm-user | sh -s -- username\ncurl -fsSL bit.ly/create-ssh-tun | sh -s -- username\ncurl -fsSL bit.ly/create-tun-proxy | sh -s -- remote passwd \ncurl -fsSL bit.ly/create-worker | sh -s -- cf_token_base64 my-service worker.js \n```\n\n### 8. API Test\n\n```bash\n\ncurl -fsSL bit.ly/vertex-test | sh -s -- project_name model_name\n```\n\n### 9. Debian System Init\n\nBasic\n\n- bit.ly/create-kmod\n- bit.ly/create-locale\n- bit.ly/create-sshd\n- bit.ly/create-swap\n\nAdd-on\n- bit.ly/create-cron\n- bit.ly/create-docker\n- bit.ly/create-rust\n- bit.ly/create-sbox\n\nAll-in-One\n```sh\ncurl -fsSL bit.ly/create-dev | sh\n```\n\nShell\n```sh\ncurl -fsSL bit.ly/create-host | sh -s -- debian\ncurl -fsSL bit.ly/create-dev-user | sh\ncurl -fsSL bit.ly/create-journald | sh\ncurl -fsSL bit.ly/create-resolv | sh\ncurl -fsSL bit.ly/create-locale | sh\ncurl -fsSL bit.ly/create-kmod | sh\ncurl -fsSL bit.ly/create-swap | sh\ncurl -fsSL bit.ly/create-sshd | sh\n\ncurl -fsSL bit.ly/create-docker | sh\ncurl -fsSL bit.ly/create-cron | sh\ncurl -fsSL bit.ly/create-sbox | sh\ncurl -fsSL bit.ly/create-rust | sh\ncurl -fsSL bit.ly/create-warp | sh\ncurl -fsSL bit.ly/create-golang | sh\ncurl -fsSL bit.ly/create-dart | sh\ncurl -fsSL bit.ly/create-ssl | sh -s -- cf_token cf_subdomain\ncurl -fsSL bit.ly/create-github | sh -s -- cf_token github_name\n```\n\nMirror\n```sh\ncurl -fsSL https://bit.ly/create-dev-user | sh\ncurl -fsSL https://bit.ly/create-apt | sh\ncurl -fsSL https://bit.ly/create-journald | sh\ncurl -fsSL https://bit.ly/create-resolv | sh\ncurl -fsSL https://bit.ly/create-locale | sh\ncurl -fsSL https://bit.ly/create-kmod | sh\ncurl -fsSL https://bit.ly/create-swap | sh\ncurl -fsSL https://bit.ly/create-sshd | sh\ncurl -fsSL https://bit.ly/create-docker-mirror | sh\n```\n\n## Reference\n\n### Basic\n\n1. https://pytorch.org/blog/scaling-pytorch-models-on-cloud-tpus-with-fsdp\n2. https://huggingface.co/blog/accelerate-large-models\n3. https://pytorch.org/blog/path-achieve-low-inference-latency\n\n### SPMD\n\n- https://pytorch.org/blog/high-performance-llama-2\n    1. https://pytorch.org/blog/pytorch-xla-spmd\n    2. https://huggingface.co/blog/llama2\n    3. https://github.com/pytorch/xla/blob/master/docs/spmd.md\n\n- https://github.com/pytorch-tpu/transformers/blob/llama2-google-next-training/SPMD_USER_GUIDE.md\n- https://github.com/pytorch/xla/blob/master/docs/spmd.md#spmd-debugging-tool\n\n### FSDP\n\n1. https://github.com/ronghanghu/vit_10b_fsdp_example\n2. https://pytorch.org/blog/large-scale-training-hugging-face\n3. https://github.com/pytorch/xla/blob/master/docs/fsdp.md\n\n### Finetune\n\n1. https://huggingface.co/blog/gemma-peft","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmon-ius%2Fxtpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmon-ius%2Fxtpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmon-ius%2Fxtpu/lists"}