{"id":29376493,"url":"https://github.com/twitter/storehaus","last_synced_at":"2025-07-09T22:43:13.810Z","repository":{"id":6521941,"uuid":"7762878","full_name":"twitter/storehaus","owner":"twitter","description":"Storehaus is a library that makes it easy to work with asynchronous key value stores","archived":false,"fork":false,"pushed_at":"2020-07-17T18:18:03.000Z","size":2355,"stargazers_count":465,"open_issues_count":78,"forks_count":92,"subscribers_count":140,"default_branch":"develop","last_synced_at":"2024-06-21T18:15:56.426Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/twitter.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-01-22T22:34:05.000Z","updated_at":"2024-06-05T07:52:55.000Z","dependencies_parsed_at":"2022-09-10T23:41:40.384Z","dependency_job_id":null,"html_url":"https://github.com/twitter/storehaus","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"purl":"pkg:github/twitter/storehaus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Fstorehaus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Fstorehaus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Fstorehaus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Fstorehaus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/twitter","download_url":"https://codeload.github.com/twitter/storehaus/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Fstorehaus/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264504616,"owners_count":23618831,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-09T22:43:13.216Z","updated_at":"2025-07-09T22:43:13.801Z","avatar_url":"https://github.com/twitter.png","language":"Scala","readme":"## Storehaus\n\n[![Build Status](https://secure.travis-ci.org/twitter/storehaus.png)](http://travis-ci.org/twitter/storehaus)\n[![Codecov branch](https://img.shields.io/codecov/c/github/twitter/storehaus/develop.svg?maxAge=3600)](https://codecov.io/github/twitter/storehaus)\n[![Latest version](https://index.scala-lang.org/twitter/storehaus/storehaus-core/latest.svg?color=orange)](https://index.scala-lang.org/twitter/storehaus/storehaus-core)\n[![Chat](https://badges.gitter.im/twitter/storehaus.svg)](https://gitter.im/twitter/storehaus?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\nStorehaus is a library that makes it easy to work with asynchronous key value stores. Storehaus is built on top of Twitter's [Future](https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/Future.scala).\n\n### Storehaus-Core\n\nStorehaus's core module defines three traits; a read-only `ReadableStore` a write-only `WritableStore` and a read-write `Store`. The traits themselves are tiny:\n\n```scala\npackage com.twitter.storehaus\n\nimport com.twitter.util.{ Closable, Future, Time }\n\ntrait ReadableStore[-K, +V] extends Closeable {\n  def get(k: K): Future[Option[V]]\n  def multiGet[K1 \u003c: K](ks: Set[K1]): Map[K1, Future[Option[V]]]\n  override def close(time: Time) = Future.Unit\n}\n\ntrait WritableStore[-K, -V] {\n  def put(kv: (K, V)): Future[Unit] = multiPut(Map(kv)).apply(kv._1)\n  def multiPut[K1 \u003c: K](kvs: Map[K1, V]): Map[K1, Future[Unit]] =\n    kvs.map { kv =\u003e (kv._1, put(kv)) }\n  override def close(time: Time) = Future.Unit\n}\n\ntrait Store[-K, V] extends ReadableStore[K, V] with WritableStore[K, Option[V]]\n```\n\nThe `ReadableStore` trait uses the `Future[Option[V]]` return type to communicate one of three states about each value. A value is either\n\n* definitely present,\n* definitely missing, or\n* unknown due to some error (perhaps a timeout, or a downed host).\n\nThe [`ReadableStore`](http://twitter.github.com/storehaus/#com.twitter.storehaus.ReadableStore$) and [`Store`](http://twitter.github.com/storehaus/#com.twitter.storehaus.Store$) companion objects provide a bunch of ways to create new stores. See the linked API documentation for more information.\n\n### Combinators\n\nCoding with Storehaus's interfaces gives you access to a number of powerful combinators. The easiest way to access these combinators is by wrapping your store in an [`EnrichedReadableStore`](http://twitter.github.com/storehaus/#com.twitter.storehaus.EnrichedReadableStore) or an [`EnrichedStore`](http://twitter.github.com/storehaus/#com.twitter.storehaus.EnrichedStore). Storehaus provides implicit conversions inside of the [`ReadableStore`](http://twitter.github.com/storehaus/#com.twitter.storehaus.ReadableStore$) and [`Store`](http://twitter.github.com/storehaus/#com.twitter.storehaus.Store$) objects.\n\nHere's an example of the `mapValues` combinator, useful for transforming the type of an existing store.\n\n```scala\nimport com.twitter.storehaus.ReadableStore\nimport ReadableStore.enrich\n\n// Create a ReadableStore from Int -\u003e String:\nval store = ReadableStore.fromMap(Map[Int, String](1 -\u003e \"some value\", 2 -\u003e \"other value\"))\n\n// \"get\" behaves as expected:\nstore.get(1).get\n// res5: Option[String] = Some(some value)\n\n// calling \"mapValues\" with a function from V =\u003e NewV returns a new ReadableStore[K, NewV]:\nval countStore: ReadableStore[Int, Int] = store.mapValues { s =\u003e s.size }\n\n// This new store applies the function to every value on the way out:\ncountStore.get(1).get\n// res6: Option[Int] = Some(10)\n```\n\n### Storehaus-Algebra\n\n`storehaus-algebra` module adds the `MergeableStore` trait. If you're using key-value stores for aggregations, you're going to love `MergeableStore`.\n\n```scala\npackage com.twitter.storehaus.algebra\n\ntrait MergeableStore[-K, V] extends Store[K, V] {\n  def monoid: Monoid[V]\n  def merge(kv: (K, V)): Future[Option[V]] = multiMerge(Map(kv)).apply(kv._1)\n  def multiMerge[K1 \u003c: K](kvs: Map[K1, V]): Map[K1, Future[Option[V]]] = kvs.map { kv =\u003e (kv._1, merge(kv)) }\n}\n```\n\n`MergeableStore`'s `merge` and `multiMerge` are similar to `put` and `multiPut`; the difference is that values added with `merge` are added to the store's existing value and the previous value is returned.\nBecause the addition is handled with a `Semigroup[V]` or `Monoid[V]` from Twitter's [Algebird](https://github.com/twitter/algebird) project, it's easy to write stores that aggregate [Lists](http://twitter.github.com/algebird/#com.twitter.algebird.ListMonoid), [decayed values](http://twitter.github.com/algebird/#com.twitter.algebird.DecayedValue), even [HyperLogLog](http://twitter.github.com/algebird/#com.twitter.algebird.HyperLogLog$) instances.\n\nThe [`MergeableStore`](http://twitter.github.com/storehaus/#com.twitter.storehaus.algebra.MergeableStore$) object provides a number of combinators on these stores. For ease of use, Storehaus provides an implicit conversion to an enrichment on `MergeableStore`. Access this by importing `MergeableStore.enrich`.\n\n### Other Modules\n\nStorehaus provides a number of modules wrapping existing key-value stores. Enriching these key-value stores with Storehaus's combinators has been hugely helpful to us here at Twitter. Writing your jobs in terms of Storehaus stores makes it easy to test your jobs; use an in-memory `JMapStore` in testing and a `MemcacheStore` in production.\n\n  * [Storehaus-memcache](http://twitter.github.com/storehaus/#com.twitter.storehaus.memcache.MemcacheStore) (wraps Twitter's [finagle-memcachedx](https://github.com/twitter/finagle/tree/master/finagle-memcachedx) library)\n  * [Storehaus-mysql](http://twitter.github.com/storehaus/#com.twitter.storehaus.mysql.MySqlStore) (wraps Twitter's [finagle-mysql](https://github.com/twitter/finagle/tree/master/finagle-mysql) library)\n  * [Storehaus-redis](http://twitter.github.com/storehaus/#com.twitter.storehaus.redis.RedisStore) (wraps Twitter's [finagle-redis](https://github.com/twitter/finagle/tree/master/finagle-redis) library)\n  * [Storehaus-hbase](http://twitter.github.com/storehaus/#com.twitter.storehaus.hbase.HBaseStore)\n  * [Storehaus-dynamodb](https://github.com/twitter/storehaus/tree/develop/storehaus-dynamodb)\n  * [Storehaus-leveldb](https://github.com/twitter/storehaus/tree/develop/storehaus-leveldb)\n\n#### Planned Modules\n\nHere's a list of modules we plan in implementing, with links to the github issues tracking progress on these modules:\n\n* [storehaus-berkeleydb](https://github.com/twitter/storehaus/issues/52)\n\n## Documentation\n\nTo learn more and find links to tutorials and information around the web, check out the [Storehaus Wiki](https://github.com/twitter/storehaus/wiki).\n\nThe latest ScalaDocs are hosted on Storehaus's [Github Project Page](http://twitter.github.io/storehaus).\n\n## Contact\n\nDiscussion occurs primarily on the [Storehaus mailing list](https://groups.google.com/forum/#!forum/storehaus). Issues should be reported on the [GitHub issue tracker](https://github.com/twitter/storehaus/issues).\n\n## Get Involved + Code of Conduct\nPull requests and bug reports are always welcome!\n\nWe use a lightweight form of project governence inspired by the one used by Apache projects.\nPlease see [Contributing and Committership](https://github.com/twitter/analytics-infra-governance#contributing-and-committership) for our code of conduct and our pull request review process.\nThe TL;DR is send us a pull request, iterate on the feedback + discussion, and get a +1 from a [Committer](COMMITTERS.md) in order to get your PR accepted.\n\nThe current list of active committers (who can +1 a pull request) can be found here: [Committers](COMMITTERS.md)\n\nA list of contributors to the project can be found here: [Contributors](https://github.com/twitter/storehaus/graphs/contributors)\n\n## Maven\n\nStorehaus modules are available on maven central. The current groupid and version for all modules is, respectively, `\"com.twitter\"` and  `0.13.0`.\n\nCurrent published artifacts are\n\n* `storehaus-core_2.11`\n* `storehaus-core_2.10`\n* `storehaus-algebra_2.11`\n* `storehaus-algebra_2.10`\n* `storehaus-memcache_2.11`\n* `storehaus-memcache_2.10`\n* `storehaus-mysql_2.11`\n* `storehaus-mysql_2.10`\n* `storehaus-hbase_2.11`\n* `storehaus-hbase_2.10`\n* `storehaus-redis_2.11`\n* `storehaus-redis_2.10`\n* `storehaus-dynamodb_2.11`\n* `storehaus-dynamodb_2.10`\n* `storehaus-kafka-08_2.11`\n* `storehaus-kafka-08_2.10`\n* `storehaus-mongodb_2.11`\n* `storehaus-mongodb_2.10`\n* `storehaus-elasticsearch_2.11`\n* `storehaus-elasticsearch_2.10`\n* `storehaus-leveldb_2.11`\n* `storehaus-leveldb_2.10`\n* `storehaus-http_2.11`\n* `storehaus-http_2.10`\n* `storehaus-cache_2.11`\n* `storehaus-cache_2.10`\n* `storehaus-testing_2.11`\n* `storehaus-testing_2.10`\n\nThe suffix denotes the scala version.\n\n## Testing notes\n\nWe use `travis-ci` to set up any underlying stores (e.g. MySQL, Redis, Memcached) for the tests. In order for these tests to pass on your local machine, you may need additional setup.\n\n### MySQL tests\n\nYou will need MySQL installed on your local machine.\nOnce installed, run the `mysql` commands listed in [.travis.yml](https://github.com/twitter/storehaus/blob/develop/.travis.yml) file.\n\n### Redis tests\n\nYou will need [redis](http://redis.io/) installed on your local machine. Redis comes bundled with an executable for spinning up a server called `redis-server`. The Storehaus redis tests expect the factory defaults for connecting to one of these redis server instances, resolvable on `localhost` port `6379`.\n\n### Memcached\n\nYou will need [Memcached](http://memcached.org/) installed on your local machine and running on the default port `11211`.\n\n## Authors\n\n* Oscar Boykin \u003chttps://twitter.com/posco\u003e\n* Sam Ritchie \u003chttps://twitter.com/sritchie\u003e\n\n## Contributors\n\nHere are a few that shine among the many:\n\n* Ruban Monu \u003chttps://twitter.com/rubanm\u003e, for `storehaus-mysql`\n* Doug Tangren \u003chttps://twitter.com/softprops\u003e, for `storehaus-redis`\n* Ryan Weald \u003chttps://twitter.com/rweald\u003e, for `storehaus-dynamodb`\n\n## License\n\nCopyright 2013 Twitter, Inc.\n\nLicensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).\n","funding_links":[],"categories":["Key-value Data Model","数据库"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter%2Fstorehaus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftwitter%2Fstorehaus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter%2Fstorehaus/lists"}