https://github.com/abrander/gguf

GGUF parser for Go
https://github.com/abrander/gguf

gguf go machine-learning

Last synced: about 1 month ago
JSON representation

GGUF parser for Go

• Host: GitHub
• URL: https://github.com/abrander/gguf
• Owner: abrander
• License: mit
• Created: 2023-09-14T11:24:36.000Z (over 1 year ago)
• Default Branch: master
• Last Pushed: 2023-12-28T00:45:45.000Z (about 1 year ago)
• Last Synced: 2023-12-28T02:40:39.273Z (about 1 year ago)
• Topics: gguf, go, machine-learning
• Language: Go
• Homepage:
• Size: 33.2 KB
• Stars: 4
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# package gguf

[![Go Reference](https://pkg.go.dev/badge/github.com/abrander/gguf.svg)](https://pkg.go.dev/github.com/abrander/gguf)

This is a Go package for reading GGUF files.

The package is only concerned with reading the metadata and the tensor bytes. It

does not interpret the data in any way.

GGUF versions 1, 2 and 3 are supported.

## Installation

```bash

go get github.com/abrander/gguf

```

## Usage example

```go

package main

import (

	"fmt"

	"os"

	"github.com/abrander/gguf"

)

func readTensor(t gguf.TensorInfo) {

	r, _ := t.Reader()

	// TODO: Read the actual tensor data...

}

func main() {

	g, _ := gguf.OpenFile("llama-2-7b-chat.Q4_0.gguf")

    fmt.Printf("Got GGUF file version: %d\n", g.Version)

	arch, _ := g.Metadata.String("general.architecture")

	if arch != "llama" {

		os.Exit(1)

	}

	contextLength, _ := g.Metadata.Int("llama.context_length")

	fmt.Printf("Context length: %d\n", contextLength)

	for _, t := range g.Tensors {

		readTensor(t)

	}

}

```

## ggufmeta

The package comes with a command line tool for inspecting GGUF files.

```bash

$ go install github.com/abrander/gguf/ggufmeta@latest

$ ggufmeta llama-2-7b-chat.Q4_0.gguf

```