https://github.com/mostynb/zstdpool-syncpool
go sync.Pool wrapper for github.com/klauspost/compress/zstd which doesn't leak memory and goroutines.
https://github.com/mostynb/zstdpool-syncpool
compression go zstd
Last synced: 10 days ago
JSON representation
go sync.Pool wrapper for github.com/klauspost/compress/zstd which doesn't leak memory and goroutines.
- Host: GitHub
- URL: https://github.com/mostynb/zstdpool-syncpool
- Owner: mostynb
- License: apache-2.0
- Created: 2020-11-24T17:19:39.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-05-07T10:25:50.000Z (almost 2 years ago)
- Last Synced: 2025-03-28T07:22:44.655Z (28 days ago)
- Topics: compression, go, zstd
- Language: Go
- Homepage:
- Size: 15.6 KB
- Stars: 11
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-golang-repositories - zstdpool-syncpool
README
[](http://godoc.org/github.com/mostynb/zstdpool-syncpool)
# zstdpool-syncpool
github.com/klauspost/compress/zstd is a native Go implementation of Zstandard,
with an API that leaks memory and goroutines if you don't call `Close()`
on every `Encoder` and `Decoder` that you create, and `Decoders` cannot be
reused after they are closed.
zstdpool-syncpool is a Go wrapper for github.com/klauspost/compress/zstd
and sync.Pool, which automatically frees resources if you forget to call
`Close()` and/or when items are dropped from the sync.Pool.
Background: https://github.com/klauspost/compress/issues/264
# Basic usage
https://pkg.go.dev/github.com/mostynb/zstdpool-syncpool
```
import (
"github.com/klauspost/compress/zstd"
syncpool "github.com/mostynb/zstdpool-syncpool"
)
// Create a sync.Pool which returns wrapped *zstd.Decoder's.
decoderPool := syncpool.NewDecoderPool(zstd.WithDecoderConcurrency(1))
// Get a *DecoderWrapper and use it.
decoder := decoderPool.Get().(*syncpool.DecoderWrapper)
decoder.Reset(compressedDataReader)
_, err = io.Copy(uncompressedDataWriter, decoder)
// Return the decoder to the pool. If we forget this, then the zstd.Decoder
// won't leak resources.
decoderPool.Put(decoder)
// Create a sync.Pool which returns wrapped *zstd.Endoder's.
encoderPool := syncpool.NewEncoderPool(zstd.WithEncoderConcurrency(1))
// Get an *EncoderWrapper and use it.
encoder := encoderPool.Get().(*syncpool.EncoderWrapper)
encoder.Reset(compressedDataWriter)
_, err = io.Copy(encoder, uncompressedDataReader)
// Return the encoder to the pool. If we forget this, then the zstd.Encoder
// won't leak resources.
encoderPool.Put(encoder)
```
# Simplified usage
If you would like to avoid type assertions, you can use `NewDecoderPoolWapper`
and `NewEncoderPoolWrapper`:
```
import (
"github.com/klauspost/compress/zstd"
syncpool "github.com/mostynb/zstdpool-syncpool"
)
// Create a sync.Pool which returns wrapped *zstd.Decoder's.
decoderPool := syncpool.NewDecoderPoolWrapper(zstd.WithDecoderConcurrency(1))
// Get a *DecoderWrapper and use it.
decoder := decoderPool.Get(compressedDataReader)
_, err = io.Copy(uncompressedDataWriter, decoder)
// Return the decoder to the pool. If we forget this, then the zstd.Decoder
// won't leak resources.
decoderPool.Put(decoder)
// Create a sync.Pool which returns wrapped *zstd.Endoder's.
encoderPool := syncpool.NewEncoderPoolWrapper(zstd.WithEncoderConcurrency(1))
// Get an EncoderWrapper and use it.
encoder := encoderPool.Get(compressedDataWriter)
_, err = io.Copy(encoder, uncompressedDataReader)
// Return the encoder to the pool. If we forget this, then the zstd.Encoder
// won't leak resources.
encoderPool.Put(encoder)
```
# Contributing
Bug reports, feature requests, PRs welcome.
# License
Licensed under the Apache License, Version 2.0. See the LICENSE file.