https://github.com/bsm/extsort
External merge sort algorithm, implemented in Go
https://github.com/bsm/extsort
external external-sorting golang sort-algorithms sorting-algorithms
Last synced: 7 months ago
JSON representation
External merge sort algorithm, implemented in Go
- Host: GitHub
- URL: https://github.com/bsm/extsort
- Owner: bsm
- License: apache-2.0
- Created: 2019-05-28T07:09:32.000Z (over 6 years ago)
- Default Branch: main
- Last Pushed: 2023-04-11T08:09:32.000Z (over 2 years ago)
- Last Synced: 2025-03-22T19:02:54.428Z (8 months ago)
- Topics: external, external-sorting, golang, sort-algorithms, sorting-algorithms
- Language: Go
- Homepage:
- Size: 43.9 KB
- Stars: 10
- Watchers: 3
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# ExtSort
[](https://pkg.go.dev/github.com/bsm/extsort)
[](https://github.com/bsm/extsort/actions/workflows/test.yml)
[](https://opensource.org/licenses/Apache-2.0)
External merge sort algorithm, implemented in [Go](https://golang.org). Sort arbitrarily large data sets
with a predictable amount of memory using disk.
## Example:
Sorting lines:
```go
import(
"fmt"
"github.com/bsm/extsort"
)
func main() {
// Init sorter.
sorter := extsort.New(nil)
defer sorter.Close()
// Append plain data.
_ = sorter.Append([]byte("foo"))
_ = sorter.Append([]byte("bar"))
_ = sorter.Append([]byte("baz"))
_ = sorter.Append([]byte("dau"))
// Sort and iterate.
iter, err := sorter.Sort()
if err != nil {
panic(err)
}
defer iter.Close()
for iter.Next() {
fmt.Println(string(iter.Data()))
}
if err := iter.Err(); err != nil {
panic(err)
}
}
```
Map-style API with de-duplication:
```go
import(
"fmt"
"github.com/bsm/extsort"
)
func main() {
// Init with de-duplication.
sorter := extsort.New(&extsort.Options{
Dedupe: bytes.Equal,
})
defer sorter.Close()
// Put key/value data.
_ = sorter.Put([]byte("foo"), []byte("v1"))
_ = sorter.Put([]byte("bar"), []byte("v2"))
_ = sorter.Put([]byte("baz"), []byte("v3"))
_ = sorter.Put([]byte("bar"), []byte("v4")) // duplicate
_ = sorter.Put([]byte("dau"), []byte("v5"))
// Sort and iterate.
iter, err := sorter.Sort()
if err != nil {
panic(err)
}
defer iter.Close()
for iter.Next() {
fmt.Println(string(iter.Key()), string(iter.Value()))
}
if err := iter.Err(); err != nil {
panic(err)
}
}
```