{"id":15345375,"url":"https://github.com/fishy/fsdb","last_synced_at":"2025-04-15T03:31:13.364Z","repository":{"id":57560170,"uuid":"115690230","full_name":"fishy/fsdb","owner":"fishy","description":"FSDB is a collection of Go libraries providing a key-value store on your file system","archived":false,"fork":false,"pushed_at":"2020-08-22T23:31:53.000Z","size":108,"stargazers_count":24,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-28T15:11:22.519Z","etag":null,"topics":["fsdb","golang","golang-library","key-value"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fishy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-29T05:33:55.000Z","updated_at":"2021-02-04T18:03:39.000Z","dependencies_parsed_at":"2022-08-26T06:41:09.555Z","dependency_job_id":null,"html_url":"https://github.com/fishy/fsdb","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fishy%2Ffsdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fishy%2Ffsdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fishy%2Ffsdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fishy%2Ffsdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fishy","download_url":"https://codeload.github.com/fishy/fsdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249001404,"owners_count":21196362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fsdb","golang","golang-library","key-value"],"created_at":"2024-10-01T11:13:07.550Z","updated_at":"2025-04-15T03:31:13.336Z","avatar_url":"https://github.com/fishy.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PkgGoDev](https://pkg.go.dev/badge/github.com/fishy/fsdb)](https://pkg.go.dev/github.com/fishy/fsdb)\n[![Go Report Card](https://goreportcard.com/badge/github.com/fishy/fsdb)](https://goreportcard.com/report/github.com/fishy/fsdb)\n\n# FSDB\n\nFSDB is a collection of [Go](https://golang.org) libraries providing a key-value\nstore on your file system.\n\n([Example code](https://pkg.go.dev/github.com/fishy/fsdb/local?tab=doc#example-package))\n\n## Why?\n\ntl;dr: It's for larger (10k+ per entry), less latency sensitive data store.\n\nMost key-value store libraries are optimized for small, in-memory size.\nEven for on-disk libraries, they are usually not optimized for larger (10k+)\nvalues.\nAlso on-disk libraries usually uses write amplify for better performance,\nwhich means they will take more disk space than the actual data stored.\nFSDB store the data as-is or use optional gzip compression,\nmaking it a better solution for companies that need to store huge amount of data\nand is less sensitive to data latency.\n\nFSDB could also be used as the last layer of your layered key-value stores:\nUse in-memory libraries for samllest, most time sensitive data;\nUse other on-disk libraries for larger, less time sensitive data;\nAnd use FSDB for largest, least latency sensitive data.\n\nFurther more, FSDB provided a hybrid implementation,\nwhich allows you to put some of your data on a remote bucket\n(AWS S3, Google Cloud Storage, etc.),\nproviding an exra layer for larger and higher latency data.\n\nIt can also be used to implement mocks of remote libraries.\nFor example, package [bucket](https://pkg.go.dev/github.com/fishy/fsdb/bucket)\nuses local FSDB to implement its\n[mock](https://github.com/fishy/fsdb/blob/master/bucket/mock.go) for testing.\n\n## Highlights\n\nFSDB has a minimal overhead,\nwhich means its performance is almost identical to the disk I/O performance.\nThe local implementation has no locks.\nThe hybrid implementation only has an optional row lock,\nplease refer to the\n[package documentation](https://pkg.go.dev/github.com/fishy/fsdb/hybrid?tab=doc#hdr-Concurrency)\nfor more details.\n\nThe local and hybrid implementation has the same interface,\nwhich means you can use the local implementation first,\nand move the the hybrid implementation later as your data grows.\n\n## Packages Index\n\n* Package [fsdb](https://pkg.go.dev/github.com/fishy/fsdb)\n  defines the interface. It does not provide implementations.\n* Package [local](https://pkg.go.dev/github.com/fishy/fsdb/local)\n  provides the local implementation.\n* Package [hybrid](https://pkg.go.dev/github.com/fishy/fsdb/hybrid)\n  provides the hybrid implementation.\n* Package [bucket](https://pkg.go.dev/github.com/fishy/fsdb/bucket)\n  defines the bucket interface.\n  It does not provide implementations.\n  Implementations for [AWS S3](https://pkg.go.dev/github.com/fishy/s3bucket) and\n  [Google Cloud Storage](https://pkg.go.dev/github.com/fishy/gcsbucket)\n  can be found in external libraries.\n  There's also an\n  [implementation](https://pkg.go.dev/github.com/fishy/blobbucket)\n  based on\n  [Go-Cloud](https://github.com/google/go-cloud)\n  [Blob](https://pkg.go.dev/gocloud.dev/blob)\n\tinterface so you can use any Go-Cloud Blob implementation in hybrid FSDB.\n\n## Test\n\nAll packages provide its own tests can be run with `go test`.\nIf you want to test every package within this repository,\non Go 1.11+ you can use `go test ./...`.\n\nThere are some tests with sleep calls,\nyou can skip them by running `go test -short` instead.\nPackage [local](https://pkg.go.dev/github.com/fishy/fsdb/local)\nprovides a benchmark test can be run with `go test -bench .`.\n\n## License\n\n[BSD License](https://github.com/fishy/fsdb/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffishy%2Ffsdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffishy%2Ffsdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffishy%2Ffsdb/lists"}