{"id":8915955,"url":"https://github.com/Prismadic/magnet","last_synced_at":"2025-08-10T13:30:57.684Z","repository":{"id":212127122,"uuid":"730713448","full_name":"Prismadic/magnet","owner":"Prismadic","description":"the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly","archived":false,"fork":false,"pushed_at":"2024-03-29T02:28:43.000Z","size":9245,"stargazers_count":18,"open_issues_count":5,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-05-02T15:18:57.022Z","etag":null,"topics":["apple-silicon","claude","distributed-computing","distributed-systems","embeddings","fine-tuning","finetuning-llms","gemini","huggingface","inference-api","langchain","llm-training","milvus","mistral","mlx","nats","nats-messaging","nats-streaming","sentence-splitting","tokenizers"],"latest_commit_sha":null,"homepage":"https://prismadic.github.io/magnet/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Prismadic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-12T14:11:21.000Z","updated_at":"2024-07-19T21:40:57.958Z","dependencies_parsed_at":"2023-12-20T15:50:37.872Z","dependency_job_id":"93e786ae-dad1-4a36-a5e4-a67499d172c4","html_url":"https://github.com/Prismadic/magnet","commit_stats":null,"previous_names":["prismadic/magnet"],"tags_count":39,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Prismadic%2Fmagnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Prismadic%2Fmagnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Prismadic%2Fmagnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Prismadic%2Fmagnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Prismadic","download_url":"https://codeload.github.com/Prismadic/magnet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229415820,"owners_count":18069477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apple-silicon","claude","distributed-computing","distributed-systems","embeddings","fine-tuning","finetuning-llms","gemini","huggingface","inference-api","langchain","llm-training","milvus","mistral","mlx","nats","nats-messaging","nats-streaming","sentence-splitting","tokenizers"],"created_at":"2024-05-01T23:10:20.113Z","updated_at":"2024-12-12T16:30:42.698Z","avatar_url":"https://github.com/Prismadic.png","language":"Python","funding_links":[],"categories":["HarmonyOS"],"sub_categories":["Windows Manager"],"readme":"\u003cp align=\"center\"\u003e\n   \u003cimg height=\"300\" width=\"300\" src=\"./magnet.png\"\u003e\n   \u003cbr\u003e\n\n   \u003ch1 align=\"center\"\u003emagnet\u003c/h1\u003e\n\n   \u003ch3 align=\"center\"\u003e\u003ca href=\"https://prismadic.github.io/magnet/\"\u003e📖 docs\u003c/a\u003e | 💻 \u003ca href=\"https://github.com/Prismadic/magnet/tree/main/examples\"\u003eexamples\u003c/a\u003e | 📓 \u003ca href=\"https://prismadic.substack.com\"\u003esubstack\u003c/a\u003e\u003c/h3\u003e \n\n   \u003cp align=\"center\"\u003ethe small distributed language model toolkit\u003c/p\u003e\n   \u003cp align=\"center\"\u003e\u003ci\u003e⚡️ fine-tune state-of-the-art LLMs anywhere, rapidly ⚡️\u003c/i\u003e\u003c/p\u003e\n   \u003cdiv align=\"center\"\u003e\n\u003c/p\u003e\n\n![GitHub release (with filter)](https://img.shields.io/github/v/release/prismadic/magnet)\n![PyPI - Version](https://img.shields.io/pypi/v/llm_magnet)\n![GitHub Workflow Status (with event)](https://img.shields.io/github/actions/workflow/status/prismadic/magnet/python-publish.yml)\n![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/prismadic/magnet)\n![GitHub last commit (branch)](https://img.shields.io/github/last-commit/prismadic/magnet/main)\n![GitHub issues](https://img.shields.io/github/issues/prismadic/magnet)\n![GitHub Repo stars](https://img.shields.io/github/stars/prismadic/magnet)\n![GitHub watchers](https://img.shields.io/github/watchers/prismadic/magnet)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/llm_magnet)\n![PyPI - Wheel](https://img.shields.io/pypi/wheel/llm_magnet)\n![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/prismadic?style=social\u0026link=https%3A%2F%2Fx.com%2Fprismadic)\n\n   \u003c/div\u003e\n\n\u003c/p\u003e\n\n\u003cimg src='./divider.png' style=\"width:100%;height:5px;\"\u003e\n\n## 🧬 Installation\n\n``` bash\npip install llm-magnet\n```\n\nor\n\n``` bash\npython3 setup.py install\n```\n\n\u003cimg src='./divider.png' style=\"width:100%;height:5px;\"\u003e\n\n## 🎉 usage\n\n[check out the example notebooks](./examples/)\n\n\u003csmall\u003ea snippet to get you started\u003c/small\u003e\n\n``` python\nfrom magnet.base import Magnet\nfrom magnet.base import EmbeddedMagnet\n\ncluster = EmbeddedMagnet()\ncluster.start()\nmagnet = cluster.create_magnet()\nawait magnet.align()\n\nconfig = {\n    \"host\": \"127.0.0.1\",\n    \"credentials\": None,\n    \"domain\": None,\n    \"name\": \"my_stream\",\n    \"category\": \"my_category\",\n    \"kv_name\": \"my_kv\",\n    \"session\": \"my_session\",\n    \"os_name\": \"my_object_store\",\n    \"index\": {\n        \"milvus_uri\": \"127.0.0.1\",\n        \"milvus_port\": 19530,\n        \"milvus_user\": \"test\",\n        \"milvus_password\": \"test\",\n        \"dimension\": 1024,\n        \"model\": \"BAAI/bge-large-en-v1.5\",\n        \"name\": \"test\",\n        \"options\": {\n            'metric_type': 'COSINE',\n            'index_type':'HNSW',\n            'params': {\n                \"efConstruction\": 40\n                , \"M\": 48\n            }\n        }\n    }\n}\n\nmagnet = Magnet(config)\nawait magnet.align()\n```\n\n\u003cimg src='./divider.png' style=\"width:100%;height:5px;\"\u003e\n\n## 🔮 features\n\n\u003ccenter\u003e\n\u003cimg src=\"./clustered_bidirectional.png\" style=\"width:50%;transform: rotate(90deg);margin-top:200px;\" align=\"right\"\u003e\n\u003c/center\u003e\n\n- ⚡️ **It's Fast**\n  - \u003csmall\u003efast on consumer hardware\u003c/small\u003e\n  - \u003csmall\u003e_very_ fast on Apple Silicon\u003c/small\u003e\n  - \u003csmall\u003e**extremely** fast on ROCm/CUDA\u003c/small\u003e\n- 🫵 **Automatic or your way**\n  - \u003csmall\u003erely on established transformer patterns to let `magnet` do the work\u003c/small\u003e\n  - \u003csmall\u003ekeep your existing data processing functions, bring them to `magnet`!\u003c/small\u003e\n- 🛰️ **100% Distributed**\n  - \u003csmall\u003eprocessing, embedding, storage, retrieval, querying, or inference from anywhere\u003c/small\u003e\n  - \u003csmall\u003eas much or as little compute as you need\u003c/small\u003e\n- 🧮 **Choose Inference Method**\n  - \u003csmall\u003eHuggingFace\u003c/small\u003e\n  - \u003csmall\u003evLLM node\u003c/small\u003e\n  - \u003csmall\u003eGPU\u003c/small\u003e\n  - \u003csmall\u003emlx\u003c/small\u003e\n- 🌎 **Huge Volumes**\n  - \u003csmall\u003ehandle gigantic amounts of data inexpensively\u003c/small\u003e\n  - \u003csmall\u003efault-tolerant by design\u003c/small\u003e\n  - \u003csmall\u003edecentralized workloads\u003c/small\u003e\n- 🔐 **Secure**\n  - \u003csmall\u003eJWT\u003c/small\u003e\n  - \u003csmall\u003eBasic\u003c/small\u003e\n- 🪵 **World-Class Comprehension**\n  - \u003csmall\u003e`magnet` optionally logs its own code as it's executed (yes, really)\u003c/small\u003e\n  - \u003csmall\u003ebuild a self-aware system and allow it to learn from itself\u003c/small\u003e\n  - \u003csmall\u003eemojis are the future\u003c/small\u003e\n\n\u003cimg src='./divider.png' style=\"width:100%;height:5px;\"\u003e\n\n## 🧲 why\n\n- build a distributed LLM research node with any hardware, from Rasbperry Pi to the expensive cloud\n- Apple silicon first-class citizen with [mlx](https://github.com/ml-explore/mlx)\n- embed \u0026 index to vector db with [milvus](https://milvus.io)\n- distributed processing with [NATS](https://nats.io)\n- upload to S3\n- ideal cyberpunk vision of LLM power users in vectorspace\n\n\u003cimg src='./divider.png' style=\"width:100%;height:5px;\"\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPrismadic%2Fmagnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPrismadic%2Fmagnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPrismadic%2Fmagnet/lists"}