{"id":20284212,"url":"https://github.com/clebert/llama2.ts","last_synced_at":"2025-08-18T19:34:38.322Z","repository":{"id":206983384,"uuid":"716664518","full_name":"clebert/llama2.ts","owner":"clebert","description":"Inference Llama 2 in pure TypeScript and Zig.","archived":false,"fork":false,"pushed_at":"2023-12-20T21:13:16.000Z","size":56188,"stargazers_count":2,"open_issues_count":6,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-08-16T12:30:25.175Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clebert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-09T15:58:07.000Z","updated_at":"2024-01-18T16:55:43.000Z","dependencies_parsed_at":"2023-12-07T16:46:26.408Z","dependency_job_id":"f0a6d188-c5ea-401c-99c6-de04143ee057","html_url":"https://github.com/clebert/llama2.ts","commit_stats":null,"previous_names":["clebert/llama2.ts"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/clebert/llama2.ts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clebert%2Fllama2.ts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clebert%2Fllama2.ts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clebert%2Fllama2.ts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clebert%2Fllama2.ts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clebert","download_url":"https://codeload.github.com/clebert/llama2.ts/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clebert%2Fllama2.ts/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271049773,"owners_count":24691125,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T14:18:46.319Z","updated_at":"2025-08-18T19:34:38.270Z","avatar_url":"https://github.com/clebert.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Llama2.ts\n\n\u003e Inference Llama 2 in pure TypeScript and Zig.\n\n**NOTE:** This project is currently a Work in Progress (WIP). Please be aware that everything can\nchange at any time.\n\n## Getting Started\n\nInstall dependencies:\n\n```sh\nnpm i\n```\n\n### Node.js\n\nCompile and run the program:\n\n```\nnpm run compile\n```\n\n```\nnpm run start:node -- complete models/tinystories_15m_v1.bin\n```\n\nOutput:\n\n```\nOnce upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red ball in the sky. It was the sun! She thought it was so pretty.\nLily wanted to play with the ball, but it was too high up in the sky. She tried to jump and reach it, but she couldn't. Then, she had an idea. She would use a stick to knock the ball down.\nLily found a stick and tried to hit the ball. But the stick was too short. She tried again and again, but she couldn't reach it. She felt sad.\nSuddenly, a kind man came by and saw Lily. He asked her what was wrong. Lily told him about the ball. The man smiled and said, \"I have a useful idea!\" He took out a long stick and used it to knock the ball down. Lily was so happy! She thanked the man and they played together in the sunshine.\n```\n\n### Browser\n\nBuild the website and start the server:\n\n```\nnpm run build\n```\n\n```\nnpm run start:browser\n```\n\nOpen: http://localhost:3000\n\n## Using Llama 2 Models from Hugging Face\n\nInstall `git-lfs` and clone the\n[TinyLlama-1.1B](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T)\nmodel from Hugging Face:\n\n```sh\ngit lfs install\n```\n\n```sh\ngit clone https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T\n```\n\nInstall the necessary Python packages and convert the Hugging Face model:\n\n```sh\npip3 install -r requirements.txt\n```\n\n```sh\npython3 convert_hf_model.py /path/to/TinyLlama-1.1B models/tiny_llama_v1.bin\n```\n\nCompile and run the program:\n\n```sh\nnpm run compile\n```\n\n```sh\nnpm run start:node -- complete models/tiny_llama_v1.bin --prompt \"The number 42 is\"\n```\n\n## Model Binary Data Format\n\n```\n+--------------------------+\n| Model Config             |\n+--------------------------+\n| Vocab Entry 0            |\n+--------------------------+\n| Vocab Entry ..           |\n+--------------------------+\n| Checkpoint               |\n+--------------------------+\n```\n\n### Model Config (`256` bytes)\n\n| Element               | Size                         | Example |\n| --------------------- | ---------------------------- | ------- |\n| `version`             | `1` × `u8`                   | 2       |\n| `modelTypeByteLength` | `1` × `i32`                  | 5       |\n| `modelType`           | `modelTypeByteLength` × `u8` | llama   |\n| `hiddenSize`          | `1` × `i32`                  | 4096    |\n| `intermediateSize`    | `1` × `i32`                  | 11008   |\n| `maxSequenceLength`   | `1` × `i32`                  | 4096    |\n| `vocabSize`           | `1` × `i32`                  | 32000   |\n| `numLayers`           | `1` × `i32`                  | 32      |\n| `numQueryHeads`       | `1` × `i32`                  | 32      |\n| `numKeyValueHeads`    | `1` × `i32`                  | 32      |\n| `sharedOutputWeight`  | `1` × `u8`                   | 0       |\n\n**Note:** `keyValueSize` = `numKeyValueHeads` × (`hiddenSize` ÷ `numQueryHeads`)\n\n### Vocab Entry (`0` .. `vocabSize`)\n\n| Element           | Size                     |\n| ----------------- | ------------------------ |\n| `score`           | `1` × `f32`              |\n| `tokenByteLength` | `1` × `i32`              |\n| `token`           | `tokenByteLength` × `u8` |\n\n### Checkpoint (`version` = `1`)\n\n| Element                 | Size                                                    | Condition                  |\n| ----------------------- | ------------------------------------------------------- | -------------------------- |\n| `embeddingWeight`       | `vocabSize` × `hiddenSize` × `f32`                      |                            |\n| `attentionNormWeight`   | `numLayers` × `hiddenSize` × `f32`                      |                            |\n| `attentionQueryWeight`  | `numLayers` × `hiddenSize` × `hiddenSize` × `f32`       |                            |\n| `attentionKeyWeight`    | `numLayers` × `keyValueSize` × `hiddenSize` × `f32`     |                            |\n| `attentionValueWeight`  | `numLayers` × `keyValueSize` × `hiddenSize` × `f32`     |                            |\n| `attentionOutputWeight` | `numLayers` × `hiddenSize` × `hiddenSize` × `f32`       |                            |\n| `mlpNormWeight`         | `numLayers` × `hiddenSize` × `f32`                      |                            |\n| `mlpGateWeight`         | `numLayers` × `intermediateSize` × `hiddenSize` × `f32` |                            |\n| `mlpUpWeight`           | `numLayers` × `intermediateSize` × `hiddenSize` × `f32` |                            |\n| `mlpDownWeight`         | `numLayers` × `hiddenSize` × `intermediateSize` × `f32` |                            |\n| `linearNormWeight`      | `hiddenSize` × `f32`                                    |                            |\n| `linearOutputWeight`    | `vocabSize` × `hiddenSize` × `f32`                      | `sharedOutputWeight` = `0` |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclebert%2Fllama2.ts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclebert%2Fllama2.ts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclebert%2Fllama2.ts/lists"}