https://github.com/murrellgroup/gguf.jl

Read and write GGUF files in Julia
https://github.com/murrellgroup/gguf.jl

ggml gguf tensors

Last synced: about 1 month ago
JSON representation

Read and write GGUF files in Julia

• Host: GitHub
• URL: https://github.com/murrellgroup/gguf.jl
• Owner: MurrellGroup
• License: mit
• Created: 2025-08-20T20:44:20.000Z (about 2 months ago)
• Default Branch: main
• Last Pushed: 2025-08-20T21:13:32.000Z (about 2 months ago)
• Last Synced: 2025-08-20T22:04:03.660Z (about 2 months ago)
• Topics: ggml, gguf, tensors
• Language: Julia
• Homepage:
• Size: 17.6 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# GGUF

[![Build Status](https://github.com/MurrellGroup/GGUF.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/MurrellGroup/GGUF.jl/actions/workflows/CI.yml?query=branch%3Amain)

[![Coverage](https://codecov.io/gh/MurrellGroup/GGUF.jl/branch/main/graph/badge.svg)](https://codecov.io/gh/MurrellGroup/GGUF.jl)

GGUF.jl is a Julia implementation of the [GGUF specification](https://github.com/ggml-org/ggml/blob/323951f1bdcdfbd5b5ff3a9a7c3770e63b1a560e/docs/gguf.md), a file format for quick loading and saving of deep learning models. Unlike tensor-only file formats like [safetensors](https://huggingface.co/docs/safetensors) (see also [SafeTensors.jl](https://github.com/FluxML/SafeTensors.jl)), GGUF encodes both the tensors and a standardized set of metadata.

## Usage

```julia

using GGUF

path = "Qwen3-0.6B-bf16.gguf"

# Load the file with tensors memory-mapped from disk

gguf = GGUF.deserialize(path)

# Pass mmap=false to load tensors into memory

# gguf = GGUF.deserialize(path, mmap=false)

# Get an ordered dict of the metadata

metadata = gguf.metadata

# Get an ordered dict of the tensors

tensors = gguf.tensors

# do a surgery

delete!(tensors, "output.weight")

# GGUFObject need to be ordered collections of string-keyed pairs

new_gguf = GGUFObject(metadata, tensors)

# Serialize to a new file

GGUF.serialize("my_model.gguf", new_gguf)

```

> [!NOTE]

> GGUF.jl does not yet support quantized tensors. Attempting to load a quantized tensor will result in an error.

## Installation

```julia

using Pkg

Pkg.Registry.add(url="https://github.com/MurrellGroup/MurrellGroupRegistry")

Pkg.add("GGUF")

```