{"id":14964961,"url":"https://github.com/hyparam/hyllama","last_synced_at":"2025-03-17T15:13:25.857Z","repository":{"id":214392591,"uuid":"736413726","full_name":"hyparam/hyllama","owner":"hyparam","description":"llama.cpp gguf file parser for javascript","archived":false,"fork":false,"pushed_at":"2024-12-11T08:25:41.000Z","size":157,"stargazers_count":31,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-07T04:02:52.419Z","etag":null,"topics":["gguf","javascript","js","llama-cpp","llamacpp","llm","machine-learning","ml","parser"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyparam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-27T20:55:16.000Z","updated_at":"2025-02-03T22:13:52.000Z","dependencies_parsed_at":"2024-08-14T09:06:47.782Z","dependency_job_id":"83e1b20e-809b-4d8b-ba83-36a1a681fef2","html_url":"https://github.com/hyparam/hyllama","commit_stats":{"total_commits":34,"total_committers":2,"mean_commits":17.0,"dds":0.08823529411764708,"last_synced_commit":"f993b59598001aa7b235ae4d2183468d2dc6e326"},"previous_names":["hyparam/hyllama"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhyllama","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhyllama/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhyllama/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyparam%2Fhyllama/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyparam","download_url":"https://codeload.github.com/hyparam/hyllama/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244056424,"owners_count":20390719,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gguf","javascript","js","llama-cpp","llamacpp","llm","machine-learning","ml","parser"],"created_at":"2024-09-24T13:34:01.942Z","updated_at":"2025-03-17T15:13:25.833Z","avatar_url":"https://github.com/hyparam.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hyllama\n\n![hyllama](hyllama.jpg)\n\n[![npm](https://img.shields.io/npm/v/hyllama)](https://www.npmjs.com/package/hyllama)\n[![minzipped](https://img.shields.io/bundlephobia/minzip/hyllama)](https://www.npmjs.com/package/hyllama)\n[![workflow status](https://github.com/hyparam/hyllama/actions/workflows/ci.yml/badge.svg)](https://github.com/hyparam/hyllama/actions)\n[![mit license](https://img.shields.io/badge/License-MIT-orange.svg)](https://opensource.org/licenses/MIT)\n![coverage](https://img.shields.io/badge/Coverage-100-darkred)\n![dependencies](https://img.shields.io/badge/Dependencies-0-blueviolet)\n\nJavascript parser for [llama.cpp](https://github.com/ggerganov/llama.cpp) gguf files.\n\nThis library makes it easy to parse metadata from GGUF files.\n\nllama.cpp was originally an implementation of meta's llama model in C++, particularly on apple m-series chips.\nBut it has quickly evolved into a powerful tool for running various trained LLM models, on cpu or gpu.\nThe runtime has minimal dependencies and so is easy to deploy.\nModel files are frequently distributed as .gguf files which contain all the info needed to run a model including architecture and weights.\n[Hugging Face](https://huggingface.co/models?library=gguf) provides a great collection of serialized gguf model files, at varying levels of quantization.\n\nModel files are often very large.\nA goal of this library is to parse the file efficiently, without loading the entire file.\n\nDependency free since 2023!\n\n## Installation\n\n```bash\nnpm install hyllama\n```\n\n## Usage\n\n### Node.js\n\nIf you're in a node.js environment, you can load a .gguf file with the following example:\n\n```js\nconst { ggufMetadata } = await import('hyllama')\nconst fs = await import('fs')\n\n// Read first 10mb of gguf file\nconst fd = fs.openSync('example.gguf', 'r')\nconst buffer = new Uint8Array(10_000_000)\nfs.readSync(fd, buffer, 0, 10_000_000, 0)\nfs.closeSync(fd)\n\n// Parse metadata and tensor info\nconst { metadata, tensorInfos } = ggufMetadata(buffer.buffer)\n```\n\n### Browser\n\nIf you're in a browser environment, you'll probably get .gguf file data from either a drag-and-dropped file from the user, or downloaded from the web.\n\nTo load .gguf data in the browser from a remote `url`, it is recommended that you use an [HTTP range request](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests) to get just the first bytes:\n\n```js\nimport { ggufMetadata } from 'hyllama'\n\nconst headers = new Headers({ Range: 'bytes=0-10000000' })\nconst res = await fetch(url, { headers })\nconst arrayBuffer = await res.arrayBuffer()\nconst { metadata, tensorInfos } = ggufMetadata(arrayBuffer)\n```\n\nTo parse .gguf files from a user drag-and-drop action, see example in [index.html](index.html).\n\n## File Size\n\nSince .gguf files are typically very large, it is recommended that you only load the start of the file that contains the metadata.\nHow many bytes you need for the metadata depends on the gguf file.\nIn practice, most .gguf files have metadata that takes up a few megabytes.\nIf you get an error \"RangeError: Offset is outside the bounds of the DataView\" then you probably didn't fetch enough bytes.\n\n## References\n\n - https://github.com/ggerganov/llama.cpp\n - https://github.com/ggerganov/ggml/blob/master/docs/gguf.md\n - https://huggingface.co/models?library=gguf\n\n## Contributions\n\nContributions are welcome!\n\nHyparquet development is supported by an open-source grant from Hugging Face :hugs:\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyparam%2Fhyllama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyparam%2Fhyllama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyparam%2Fhyllama/lists"}