Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/goccy/go-yaml
YAML support for the Go language
https://github.com/goccy/go-yaml
go golang golang-library yaml yaml-parser
Last synced: 3 days ago
JSON representation
YAML support for the Go language
- Host: GitHub
- URL: https://github.com/goccy/go-yaml
- Owner: goccy
- License: mit
- Created: 2019-10-09T07:38:12.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2024-11-17T14:58:38.000Z (25 days ago)
- Last Synced: 2024-11-18T16:33:32.070Z (24 days ago)
- Topics: go, golang, golang-library, yaml, yaml-parser
- Language: Go
- Homepage:
- Size: 931 KB
- Stars: 1,180
- Watchers: 10
- Forks: 132
- Open Issues: 75
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- awesomeLibrary - go-yaml - YAML support for the Go language (语言资源库 / go)
- awesome-ccamel - goccy/go-yaml - YAML support for the Go language (Go)
- awesome-golang-repositories - go-yaml
README
# YAML support for the Go language
[](https://pkg.go.dev/github.com/goccy/go-yaml)
[](https://codecov.io/gh/goccy/go-yaml)
[](https://goreportcard.com/report/github.com/goccy/go-yaml)
## This library has **NO** relation to the go-yaml/yaml library
> [!IMPORTANT]
> This library is developed from scratch to replace [`go-yaml/yaml`](https://github.com/go-yaml/yaml).
> If you're looking for a better YAML library, this one should be helpful.
# Why a new library?
As of this writing, there already exists a de facto standard library for YAML processing for Go: [https://github.com/go-yaml/yaml](https://github.com/go-yaml/yaml). However, we believe that a new YAML library is necessary for the following reasons:
- Not actively maintained
- `go-yaml/yaml` has ported the libyaml written in C to Go, so the source code is not written in Go style
- There is a lot of content that cannot be parsed
- YAML is often used for configuration, and it is common to include validation along with it. However, the errors in `go-yaml/yaml` are not intuitive, and it is difficult to provide meaningful validation errors
- When creating tools that use YAML, there are cases where reversible transformation of YAML is required. However, to perform reversible transformations of content that includes Comments or Anchors/Aliases, manipulating the AST is the only option
- Non-intuitive [Marshaler](https://pkg.go.dev/gopkg.in/yaml.v3#Marshaler) / [Unmarshaler](https://pkg.go.dev/gopkg.in/yaml.v3#Unmarshaler)
By the way, libraries such as [ghodss/yaml](https://github.com/ghodss/yaml) and [sigs.k8s.io/yaml](https://github.com/kubernetes-sigs/yaml) also depend on go-yaml/yaml, so if you are using these libraries, the same issues apply: they cannot parse things that go-yaml/yaml cannot parse, and they inherit many of the problems that go-yaml/yaml has.
# Features
- No dependencies
- A better parser than `go-yaml/yaml`.
- [Support recursive processing](https://github.com/apple/device-management/blob/release/docs/schema.yaml)
- Higher coverage in the [YAML Test Suite](https://github.com/yaml/yaml-test-suite?tab=readme-ov-file)
- Ease and sustainability of maintenance
- The main maintainer is [@goccy](https://github.com/goccy), but we are also building a system to develop as a team with trusted developers
- Since it is written from scratch, the code is easy to read for Gophers
- An API structure that allows the use of not only `Encoder`/`Decoder` but also `Tokenizer` and `Parser` functionalities.
- [lexer.Tokenize](https://pkg.go.dev/github.com/goccy/[email protected]/lexer#Tokenize)
- [parser.Parse](https://pkg.go.dev/github.com/goccy/[email protected]/parser#Parse)
- Filtering, replacing, and merging YAML content using YAML Path
- Reversible transformation without using the AST for YAML that includes Anchors, Aliases, and Comments
- Customize the Marshal/Unmarshal behavior for primitive types and third-party library types ([RegisterCustomMarshaler](https://pkg.go.dev/github.com/goccy/go-yaml#RegisterCustomMarshaler), [RegisterCustomUnmarshaler](https://pkg.go.dev/github.com/goccy/go-yaml#RegisterCustomUnmarshaler))
- Respects `encoding/json` behavior
- Accept the `json` tag. Note that not all options from the `json` tag will have significance when parsing YAML documents. If both tags exist, `yaml` tag will take precedence.
- [json.Marshaler](https://pkg.go.dev/encoding/json#Marshaler) style [marshaler](https://pkg.go.dev/github.com/goccy/go-yaml#BytesMarshaler)
- [json.Unmarshaler](https://pkg.go.dev/encoding/json#Unmarshaler) style [unmarshaler](https://pkg.go.dev/github.com/goccy/go-yaml#BytesUnmarshaler)
- Options for using `MarshalJSON` and `UnmarshalJSON` ([UseJSONMarshaler](https://pkg.go.dev/github.com/goccy/go-yaml#UseJSONMarshaler), [UseJSONUnmarshaler](https://pkg.go.dev/github.com/goccy/go-yaml#UseJSONUnmarshaler))
- Pretty format for error notifications
- Smart validation processing combined with [go-playground/validator](https://github.com/go-playground/validator)
- [example test code is here](https://github.com/goccy/go-yaml/blob/45889c98b0a0967240eb595a1bd6896e2f575106/testdata/validate_test.go#L12)
- Allow referencing elements declared in another file via anchors
# Installation
```sh
go get github.com/goccy/go-yaml
```
# Synopsis
## 1. Simple Encode/Decode
Has an interface like `go-yaml/yaml` using `reflect`
```go
var v struct {
A int
B string
}
v.A = 1
v.B = "hello"
bytes, err := yaml.Marshal(v)
if err != nil {
//...
}
fmt.Println(string(bytes)) // "a: 1\nb: hello\n"
```
```go
yml := `
%YAML 1.2
---
a: 1
b: c
`
var v struct {
A int
B string
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
//...
}
```
To control marshal/unmarshal behavior, you can use the `yaml` tag.
```go
yml := `---
foo: 1
bar: c
`
var v struct {
A int `yaml:"foo"`
B string `yaml:"bar"`
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
//...
}
```
For convenience, we also accept the `json` tag. Note that not all options from
the `json` tag will have significance when parsing YAML documents. If both
tags exist, `yaml` tag will take precedence.
```go
yml := `---
foo: 1
bar: c
`
var v struct {
A int `json:"foo"`
B string `json:"bar"`
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
//...
}
```
For custom marshal/unmarshaling, implement either `Bytes` or `Interface` variant of marshaler/unmarshaler. The difference is that while `BytesMarshaler`/`BytesUnmarshaler` behaves like [`encoding/json`](https://pkg.go.dev/encoding/json) and `InterfaceMarshaler`/`InterfaceUnmarshaler` behaves like [`gopkg.in/yaml.v2`](https://pkg.go.dev/gopkg.in/yaml.v2).
Semantically both are the same, but they differ in performance. Because indentation matters in YAML, you cannot simply accept a valid YAML fragment from a Marshaler, and expect it to work when it is attached to the parent container's serialized form. Therefore when we receive use the `BytesMarshaler`, which returns `[]byte`, we must decode it once to figure out how to make it work in the given context. If you use the `InterfaceMarshaler`, we can skip the decoding.
If you are repeatedly marshaling complex objects, the latter is always better
performance wise. But if you are, for example, just providing a choice between
a config file format that is read only once, the former is probably easier to
code.
## 2. Reference elements declared in another file
`testdata` directory contains `anchor.yml` file:
```shell
├── testdata
└── anchor.yml
```
And `anchor.yml` is defined as follows:
```yaml
a: &a
b: 1
c: hello
```
Then, if `yaml.ReferenceDirs("testdata")` option is passed to `yaml.Decoder`,
`Decoder` tries to find the anchor definition from YAML files the under `testdata` directory.
```go
buf := bytes.NewBufferString("a: *a\n")
dec := yaml.NewDecoder(buf, yaml.ReferenceDirs("testdata"))
var v struct {
A struct {
B int
C string
}
}
if err := dec.Decode(&v); err != nil {
//...
}
fmt.Printf("%+v\n", v) // {A:{B:1 C:hello}}
```
## 3. Encode with `Anchor` and `Alias`
### 3.1. Explicitly declared `Anchor` name and `Alias` name
If you want to use `anchor` or `alias`, you can define it as a struct tag.
```go
type T struct {
A int
B string
}
var v struct {
C *T `yaml:"c,anchor=x"`
D *T `yaml:"d,alias=x"`
}
v.C = &T{A: 1, B: "hello"}
v.D = v.C
bytes, err := yaml.Marshal(v)
if err != nil {
panic(err)
}
fmt.Println(string(bytes))
/*
c: &x
a: 1
b: hello
d: *x
*/
```
### 3.2. Implicitly declared `Anchor` and `Alias` names
If you do not explicitly declare the anchor name, the default behavior is to
use the equivalent of `strings.ToLower($FieldName)` as the name of the anchor.
If you do not explicitly declare the alias name AND the value is a pointer
to another element, we look up the anchor name by finding out which anchor
field the value is assigned to by looking up its pointer address.
```go
type T struct {
I int
S string
}
var v struct {
A *T `yaml:"a,anchor"`
B *T `yaml:"b,anchor"`
C *T `yaml:"c,alias"`
D *T `yaml:"d,alias"`
}
v.A = &T{I: 1, S: "hello"}
v.B = &T{I: 2, S: "world"}
v.C = v.A // C has same pointer address to A
v.D = v.B // D has same pointer address to B
bytes, err := yaml.Marshal(v)
if err != nil {
//...
}
fmt.Println(string(bytes))
/*
a: &a
i: 1
s: hello
b: &b
i: 2
s: world
c: *a
d: *b
*/
```
### 3.3 MergeKey and Alias
Merge key and alias ( `<<: *alias` ) can be used by embedding a structure with the `inline,alias` tag.
```go
type Person struct {
*Person `yaml:",omitempty,inline,alias"` // embed Person type for default value
Name string `yaml:",omitempty"`
Age int `yaml:",omitempty"`
}
defaultPerson := &Person{
Name: "John Smith",
Age: 20,
}
people := []*Person{
{
Person: defaultPerson, // assign default value
Name: "Ken", // override Name property
Age: 10, // override Age property
},
{
Person: defaultPerson, // assign default value only
},
}
var doc struct {
Default *Person `yaml:"default,anchor"`
People []*Person `yaml:"people"`
}
doc.Default = defaultPerson
doc.People = people
bytes, err := yaml.Marshal(doc)
if err != nil {
//...
}
fmt.Println(string(bytes))
/*
default: &default
name: John Smith
age: 20
people:
- <<: *default
name: Ken
age: 10
- <<: *default
*/
```
## 4. Pretty Formatted Errors
Error values produced during parsing have two extra features over regular
error values.
First, by default, they contain extra information on the location of the error
from the source YAML document, to make it easier to find the error location.
Second, the error messages can optionally be colorized.
If you would like to control exactly how the output looks like, consider
using `yaml.FormatError`, which accepts two boolean values to
control turning these features on or off.
## 5. Use YAMLPath
```go
yml := `
store:
book:
- author: john
price: 10
- author: ken
price: 12
bicycle:
color: red
price: 19.95
`
path, err := yaml.PathString("$.store.book[*].author")
if err != nil {
//...
}
var authors []string
if err := path.Read(strings.NewReader(yml), &authors); err != nil {
//...
}
fmt.Println(authors)
// [john ken]
```
### 5.1 Print customized error with YAML source code
```go
package main
import (
"fmt"
"github.com/goccy/go-yaml"
)
func main() {
yml := `
a: 1
b: "hello"
`
var v struct {
A int
B string
}
if err := yaml.Unmarshal([]byte(yml), &v); err != nil {
panic(err)
}
if v.A != 2 {
// output error with YAML source
path, err := yaml.PathString("$.a")
if err != nil {
panic(err)
}
source, err := path.AnnotateSource([]byte(yml), true)
if err != nil {
panic(err)
}
fmt.Printf("a value expected 2 but actual %d:\n%s\n", v.A, string(source))
}
}
```
output result is the following:
# Tools
## ycat
print yaml file with color
### Installation
```sh
git clone https://github.com/goccy/go-yaml.git
cd ./cmd/ycat && go insatll .
```
# For Developers
> [!NOTE]
> In this project, we manage such test code under the `testdata` directory to avoid adding dependencies on libraries that are only needed for testing to the top `go.mod` file. Therefore, if you want to add test cases that use 3rd party libraries, please add the test code to the `testdata` directory.
# Looking for Sponsors
I'm looking for sponsors this library. This library is being developed as a personal project in my spare time. If you want a quick response or problem resolution when using this library in your project, please register as a [sponsor](https://github.com/sponsors/goccy). I will cooperate as much as possible. Of course, this library is developed as an MIT license, so you can use it freely for free.
# License
MIT