{"id":27937555,"url":"https://github.com/alvarobartt/hf-mem","last_synced_at":"2026-04-01T21:45:55.793Z","repository":{"id":332311041,"uuid":"1121301674","full_name":"alvarobartt/hf-mem","owner":"alvarobartt","description":"A CLI to estimate inference memory requirements for Hugging Face models, written in Python.","archived":false,"fork":false,"pushed_at":"2026-03-30T07:11:50.000Z","size":105,"stargazers_count":882,"open_issues_count":7,"forks_count":80,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-03-30T09:05:53.408Z","etag":null,"topics":["gguf","hf-extension","huggingface","safetensors"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/hf-mem/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alvarobartt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-22T19:04:22.000Z","updated_at":"2026-03-30T07:11:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"29e690aa-2944-43d2-8c28-e43daf44ac9c","html_url":"https://github.com/alvarobartt/hf-mem","commit_stats":null,"previous_names":["alvarobartt/hf-mem"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/alvarobartt/hf-mem","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alvarobartt%2Fhf-mem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alvarobartt%2Fhf-mem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alvarobartt%2Fhf-mem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alvarobartt%2Fhf-mem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alvarobartt","download_url":"https://codeload.github.com/alvarobartt/hf-mem/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alvarobartt%2Fhf-mem/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292519,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gguf","hf-extension","huggingface","safetensors"],"created_at":"2025-05-07T07:58:33.290Z","updated_at":"2026-04-01T21:45:55.788Z","avatar_url":"https://github.com/alvarobartt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://github.com/user-attachments/assets/509a8244-8a91-4051-b337-41b7b2fe0e2f\" /\u003e\n\n---\n\n\u003e [!WARNING]\n\u003e `hf-mem` is still experimental and therefore subject to major changes across releases, so please keep in mind that breaking changes may occur until v1.0.0.\n\n`hf-mem` is a CLI to estimate inference memory requirements for Hugging Face models, written in Python. `hf-mem` is lightweight, only depends on `httpx`, as it pulls the [Safetensors](https://github.com/huggingface/safetensors) and / or [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) metadata via [HTTP Range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests). It's recommended to run with [`uv`](https://github.com/astral-sh/uv) for a better experience.\n\n`hf-mem` lets you estimate the inference requirements to run any model from the Hugging Face Hub, including [Transformers](https://github.com/huggingface/transformers), [Diffusers](https://github.com/huggingface/diffusers) and [Sentence Transformers](https://github.com/huggingface/sentence-transformers) models, or really any model as long as it contains any of [Safetensors](https://github.com/huggingface/safetensors) or [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) weights.\n\nRead more information about `hf-mem` in [this short-form post](https://alvarobartt.com/hf-mem), but note it's not up-to-date as it was written in January 2026.\n\n## Usage\n\n### CLI (Recommended)\n\n#### Transformers\n\n```bash\nuvx hf-mem --model-id MiniMaxAI/MiniMax-M2\n```\n\n\u003cimg src=\"https://github.com/user-attachments/assets/545be630-4485-41ac-ba8d-2aedbbed8835\" /\u003e\n\n#### Diffusers\n\n```bash\nuvx hf-mem --model-id Qwen/Qwen-Image\n```\n\n\u003cimg src=\"https://github.com/user-attachments/assets/7a260369-26b2-48e1-a97b-f18cc72f7475\" /\u003e\n\n#### Sentence Transformers\n\n```bash\nuvx hf-mem --model-id google/embeddinggemma-300m\n```\n\n\u003cimg src=\"https://github.com/user-attachments/assets/aef1ddd8-96c2-4944-83e2-57171ff6ac7a\" /\u003e\n\n### Python\n\nYou can also run it programmatically with Python as:\n\n```python\nfrom hf_mem import run\n\nresult = run(model_id=\"MiniMaxAI/MiniMax-M2\", experimental=True)\nprint(result)\n# Result(model_id='MiniMaxAI/MiniMax-M2', revision='main', filename=None, memory=230121630720, kv_cache=24964497408, total_memory=255086128128, details=False)\n```\n\nIf you're already inside an async application, use `arun(...)` instead:\n\n```python\nfrom hf_mem import arun\n\nresult = await arun(model_id=\"MiniMaxAI/MiniMax-M2\", experimental=True)\nprint(result)\n# Result(model_id='MiniMaxAI/MiniMax-M2', revision='main', filename=None, memory=230121630720, kv_cache=24964497408, total_memory=255086128128, details=False)\n```\n\n## Experimental\n\nBy enabling the `--experimental` flag, you can enable the KV Cache memory estimation for LLMs (`...ForCausalLM`) and VLMs (`...ForConditionalGeneration`), even including a custom `--max-model-len` (defaults to the `config.json` default), `--batch-size` (defaults to 1), and the `--kv-cache-dtype` (defaults to `auto` which means it uses the default data type set in `config.json` under `torch_dtype` or `dtype`, or rather from `quantization_config` when applicable).\n\n```bash\nuvx hf-mem --model-id MiniMaxAI/MiniMax-M2 --experimental\n```\n\n\u003cimg src=\"https://github.com/user-attachments/assets/ec0ef39d-0323-4616-bba5-6a18ffee211c\" /\u003e\n\n## GGUF\n\nIf the repository contains GGUF model weights, those will be listed by default (only if there are no Safetensors weights, otherwise the GGUFs will be ignored) and the memory will be estimated for each one of those; whereas if a specific file is provided, then the memory estimation will be targeted for that given file instead.\n\n```bash\nuvx hf-mem --model-id TheBloke/deepseek-llm-7B-chat-GGUF --experimental\n```\n\n\u003cimg src=\"https://github.com/user-attachments/assets/b5514d35-f9c4-4a07-a719-a0185ff1dd9f\" /\u003e\n\nOr if you want to only get the estimation on a given file:\n\n```bash\nuvx hf-mem --model-id TheBloke/deepseek-llm-7B-chat-GGUF --gguf-file deepseek-llm-7b-chat.Q2_K.gguf --experimental\n```\n\n\u003cimg src=\"https://github.com/user-attachments/assets/e32ad635-05e5-4b33-b35b-3e689215dedd\" /\u003e\n\n## (Optional) Agent Skills\n\nOptionally, you can add `hf-mem` as an agent skill, which allows the underlying coding agent to discover and use it when provided as a [`SKILL.md`](skills/hf-mem/SKILL.md), e.g., `.claude/skills/hf-mem/SKILL.md`.\n\nMore information can be found at [Anthropic Agent Skills and how to use them](https://github.com/anthropics/skills).\n\n## References\n\n### Technical\n\n- [Safetensors Metadata parsing](https://huggingface.co/docs/safetensors/en/metadata_parsing)\n- [GGUF File format](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md)\n- [GGUF on the Hugging Face Hub](https://huggingface.co/docs/hub/en/gguf)\n\n### Visual\n\n- [usgraphics - TR-100 Machine Report](https://github.com/usgraphics/usgc-machine-report)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falvarobartt%2Fhf-mem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falvarobartt%2Fhf-mem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falvarobartt%2Fhf-mem/lists"}