{"id":19664835,"url":"https://github.com/fluxml/safetensors.jl","last_synced_at":"2025-02-27T03:43:23.103Z","repository":{"id":221034999,"uuid":"753202612","full_name":"FluxML/SafeTensors.jl","owner":"FluxML","description":null,"archived":false,"fork":false,"pushed_at":"2024-05-08T17:12:27.000Z","size":53,"stargazers_count":12,"open_issues_count":1,"forks_count":1,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-02-21T03:03:08.941Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FluxML.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-05T17:04:24.000Z","updated_at":"2025-02-02T14:23:17.000Z","dependencies_parsed_at":"2024-04-23T05:17:15.536Z","dependency_job_id":null,"html_url":"https://github.com/FluxML/SafeTensors.jl","commit_stats":null,"previous_names":["fluxml/safetensors.jl"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FluxML%2FSafeTensors.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FluxML%2FSafeTensors.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FluxML%2FSafeTensors.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FluxML%2FSafeTensors.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FluxML","download_url":"https://codeload.github.com/FluxML/SafeTensors.jl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240974461,"owners_count":19887302,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T16:19:13.452Z","updated_at":"2025-02-27T03:43:23.086Z","avatar_url":"https://github.com/FluxML.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SafeTensors.jl\n\n\n[![Build Status](https://github.com/FluxML/SafeTensors.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/FluxML/SafeTensors.jl/actions/workflows/CI.yml?query=branch%3Amain)\n\nThis packages loads data stored in [safetensor format](https://huggingface.co/docs/safetensors/index).\nSince Python is row-major and Julia is column-major, the dimensions are permuted such the tensor has the same shape as in python, but everything is correctly ordered. This includes a performance penalty in sense that we cannot be completely copy-free.\n\nThe main function is `load_safetensors` which returns a `Dict{String,V}` where keys are names of tensors and values are tensors. An example from `runtests` is as follows\n```julia\njulia\u003e using SafeTensors\n\njulia\u003e d = load_safetensors(\"test/model.safetensors\")\nDict{String, Array} with 27 entries:\n  \"int32_357\"   =\u003e Int32[0 7 … 21 28; 35 42 … 56 63; 70 77 … 91 98;;; 1 8 … 22 29…\n  \"uint8_3\"     =\u003e UInt8[0x00, 0x01, 0x02]\n  \"float16_35\"  =\u003e Float16[0.0 1.0 … 3.0 4.0; 5.0 6.0 … 8.0 9.0; 10.0 11.0 … 13.0…\n  \"bool_3\"      =\u003e Bool[0, 1, 0]\n  \"int64_3\"     =\u003e [0, 1, 2]\n  \"int64_35\"    =\u003e [0 1 … 3 4; 5 6 … 8 9; 10 11 … 13 14]\n  \"float32_357\" =\u003e Float32[0.0 7.0 … 21.0 28.0; 35.0 42.0 … 56.0 63.0; 70.0 77.0 …\n  \"bool_35\"     =\u003e Bool[0 1 … 1 0; 1 0 … 0 1; 0 1 … 1 0]\n  \"float32_35\"  =\u003e Float32[0.0 1.0 … 3.0 4.0; 5.0 6.0 … 8.0 9.0; 10.0 11.0 … 13.0…\n  \"float32_3\"   =\u003e Float32[0.0, 1.0, 2.0]\n  \"uint8_35\"    =\u003e UInt8[0x00 0x01 … 0x03 0x04; 0x05 0x06 … 0x08 0x09; 0x0a 0x0b …\n  \"float16_3\"   =\u003e Float16[0.0, 1.0, 2.0]\n  \"int16_357\"   =\u003e Int16[0 7 … 21 28; 35 42 … 56 63; 70 77 … 91 98;;; 1 8 … 22 29…\n  \"int16_3\"     =\u003e Int16[0, 1, 2]\n  \"float64_357\" =\u003e [0.0 7.0 … 21.0 28.0; 35.0 42.0 … 56.0 63.0; 70.0 77.0 … 91.0 …\n  \"uint8_357\"   =\u003e UInt8[0x00 0x07 … 0x15 0x1c; 0x23 0x2a … 0x38 0x3f; 0x46 0x4d …\n  \"float16_357\" =\u003e Float16[0.0 7.0 … 21.0 28.0; 35.0 42.0 … 56.0 63.0; 70.0 77.0 …\n  \"int32_3\"     =\u003e Int32[0, 1, 2]\n  \"int16_35\"    =\u003e Int16[0 1 … 3 4; 5 6 … 8 9; 10 11 … 13 14]\n  \"int8_357\"    =\u003e Int8[0 7 … 21 28; 35 42 … 56 63; 70 77 … 91 98;;; 1 8 … 22 29;…\n  \"int8_35\"     =\u003e Int8[0 1 … 3 4; 5 6 … 8 9; 10 11 … 13 14]\n  \"bool_357\"    =\u003e Bool[0 1 … 1 0; 1 0 … 0 1; 0 1 … 1 0;;; 1 0 … 0 1; 0 1 … 1 0; …\n  \"float64_35\"  =\u003e [0.0 1.0 … 3.0 4.0; 5.0 6.0 … 8.0 9.0; 10.0 11.0 … 13.0 14.0]\n  \"int8_3\"      =\u003e Int8[0, 1, 2]\n  \"int64_357\"   =\u003e [0 7 … 21 28; 35 42 … 56 63; 70 77 … 91 98;;; 1 8 … 22 29; 36 …\n  \"int32_35\"    =\u003e Int32[0 1 … 3 4; 5 6 … 8 9; 10 11 … 13 14]\n  \"float64_3\"   =\u003e [0.0, 1.0, 2.0]\n```\n\nIt can also perform a lazy loading with `SafeTensors.deserialize(\"model.safetensors\")` which `mmap` the file and return a `Dict`-like object:\n```julia\njulia\u003e tensors = SafeTensors.deserialize(\"test/model.safetensors\"; mmap = true #= default to `true`=#);\n\njulia\u003e tensors[\"float32_35\"]\n3×5 mappedarray(ltoh, PermutedDimsArray(reshape(reinterpret(Float32, view(::Vector{UInt8}, 0x0000000000000ef5:0x0000000000000f30)), 5, 3), (2, 1))) with eltype Float32:\n  0.0   1.0   2.0   3.0   4.0\n  5.0   6.0   7.0   8.0   9.0\n 10.0  11.0  12.0  13.0  14.0\n```\n\nSerialization is also supported:\n\n```julia\njulia\u003e using Random, BFloat16s\n\njulia\u003e weights = Dict(\"W\"=\u003erandn(BFloat16, 3, 5), \"b\"=\u003erand(BFloat16, 3))\nDict{String, Array{BFloat16}} with 2 entries:\n  \"W\" =\u003e [0.617188 0.695312 … 0.390625 -2.0; -0.65625 -0.617188 … 0.652344 0.244141; 0.226562 2.70312 … -0.174805 -0.7773…\n  \"b\" =\u003e [0.111816, 0.566406, 0.283203]\n\njulia\u003e f = tempname();\n\njulia\u003e SafeTensors.serialize(f, weights)\n\njulia\u003e loaded = SafeTensors.deserialize(f);\n\njulia\u003e loaded[\"W\"] ≈ weights[\"W\"]\ntrue\n\njulia\u003e SafeTensors.serialize(f, weights, Dict(\"Package\"=\u003e\"SafeTensors.jl\", \"version\"=\u003e\"1\"))\n\njulia\u003e loaded = SafeTensors.deserialize(f);\n\njulia\u003e loaded.metadata\nDict{String, String} with 2 entries:\n  \"Package\" =\u003e \"SafeTensors.jl\"\n  \"version\" =\u003e \"1\"\n```\n\nWorking with gpu:\n```julia\njulia\u003e loaded[\"W\"]\n3×5 mappedarray(ltoh, PermutedDimsArray(reshape(reinterpret(BFloat16, view(::Vector{UInt8}, 0x00000000000000b9:0x00000000000000d6)), 5, 3), (2, 1))) with eltype BFloat16:\n  0.542969    0.201172   1.38281    -0.255859  -1.55469\n  0.172852   -0.949219   0.0561523  -1.34375   -0.206055\n -0.0854492   1.17969   -0.265625   -0.871094   2.25\n\njulia\u003e using CUDA; CUDA.allowscalar(false)\n\njulia\u003e CuArray(loaded[\"W\"])\n3×5 CuArray{BFloat16, 2, CUDA.Mem.DeviceBuffer}:\n  0.542969    0.201172   1.38281    -0.255859  -1.55469\n  0.172852   -0.949219   0.0561523  -1.34375   -0.206055\n -0.0854492   1.17969   -0.265625   -0.871094   2.25\n\njulia\u003e gpu_weights = Dict(\"W\"=\u003eCuArray(loaded[\"W\"]), \"b\"=\u003eCuArray(loaded[\"b\"]))\nDict{String, CuArray{BFloat16, N, CUDA.Mem.DeviceBuffer} where N} with 2 entries:\n  \"W\" =\u003e [0.542969 0.201172 … -0.255859 -1.55469; 0.172852 -0.949219 … -1.34375 -0.206055; -0.0854492 1.17969 … -0.871094…\n  \"b\" =\u003e BFloat16[0.871094, 0.773438, 0.703125]\n\njulia\u003e f = tempname();\n\njulia\u003e SafeTensors.serialize(f, gpu_weights)\n\njulia\u003e SafeTensors.deserialize(f)\nSafeTensors.SafeTensor{SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{UInt64}}, true}} with 2 entries:\n  \"W\" =\u003e BFloat16[0.542969 0.201172 … -0.255859 -1.55469; 0.172852 -0.949219 … -1.34375 -0.206055; -0.0854492 1.17969 … -…\n  \"b\" =\u003e BFloat16[0.871094, 0.773438, 0.703125]\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluxml%2Fsafetensors.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluxml%2Fsafetensors.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluxml%2Fsafetensors.jl/lists"}