{"id":13409137,"url":"https://github.com/pwliwanow/foundationdb4s","last_synced_at":"2025-04-23T14:31:36.229Z","repository":{"id":33685551,"uuid":"148061323","full_name":"pwliwanow/foundationdb4s","owner":"pwliwanow","description":"Type-safe and idiomatic Scala client for FoundationDB","archived":false,"fork":false,"pushed_at":"2024-07-30T00:18:18.000Z","size":212,"stargazers_count":28,"open_issues_count":17,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-07-31T20:34:41.133Z","etag":null,"topics":["akka-streams","cats","database","foundationdb","scala","shapeless"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pwliwanow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-09T20:04:06.000Z","updated_at":"2024-07-21T17:58:43.000Z","dependencies_parsed_at":"2024-10-26T04:53:11.570Z","dependency_job_id":"3c277fbf-4c3d-49d2-9de6-fbc247b8adc8","html_url":"https://github.com/pwliwanow/foundationdb4s","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwliwanow%2Ffoundationdb4s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwliwanow%2Ffoundationdb4s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwliwanow%2Ffoundationdb4s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwliwanow%2Ffoundationdb4s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pwliwanow","download_url":"https://codeload.github.com/pwliwanow/foundationdb4s/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250451775,"owners_count":21432895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["akka-streams","cats","database","foundationdb","scala","shapeless"],"created_at":"2024-07-30T20:00:58.272Z","updated_at":"2025-04-23T14:31:35.891Z","avatar_url":"https://github.com/pwliwanow.png","language":"Scala","funding_links":[],"categories":["Bindings"],"sub_categories":[],"readme":"# foundationdb4s\n\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.github.pwliwanow.foundationdb4s/core_2.12/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.github.pwliwanow.foundationdb4s/core_2.12)\n![Build](https://github.com/pwliwanow/foundationdb4s/actions/workflows/build.yaml/badge.svg)\n[![codecov](https://codecov.io/gh/pwliwanow/foundationdb4s/branch/master/graph/badge.svg)](https://codecov.io/gh/pwliwanow/foundationdb4s)\n[![Scala Steward badge](https://img.shields.io/badge/Scala_Steward-helping-brightgreen.svg?style=flat\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAQCAMAAAARSr4IAAAAVFBMVEUAAACHjojlOy5NWlrKzcYRKjGFjIbp293YycuLa3pYY2LSqql4f3pCUFTgSjNodYRmcXUsPD/NTTbjRS+2jomhgnzNc223cGvZS0HaSD0XLjbaSjElhIr+AAAAAXRSTlMAQObYZgAAAHlJREFUCNdNyosOwyAIhWHAQS1Vt7a77/3fcxxdmv0xwmckutAR1nkm4ggbyEcg/wWmlGLDAA3oL50xi6fk5ffZ3E2E3QfZDCcCN2YtbEWZt+Drc6u6rlqv7Uk0LdKqqr5rk2UCRXOk0vmQKGfc94nOJyQjouF9H/wCc9gECEYfONoAAAAASUVORK5CYII=)](https://scala-steward.org)\n\nfoundationdb4s is a wrapper for [FoundationDB](https://github.com/apple/foundationdb) Java client.\nIt aims to be type-safe and idiomatic for Scala.\n\n```scala\nimplicit val ec = scala.concurrent.ExecutionContext.global\nval database: Database = FDB.selectAPIVersion(630).open(null, ec)\n\nfinal case class Book(isbn: String, title: String, publishedOn: LocalDate)\n\nval booksSubspace = new TypedSubspace[Book, String] {\n  override val subspace: Subspace = new Subspace(Tuple.from(\"books\"))\n  override def toKey(entity: Book): String = entity.isbn\n  override def toRawValue(entity: Book): Array[Byte] = {\n    Tuple.from(entity.title, entity.publishedOn.toString).pack\n  }\n  override def toTupledKey(key: String): Tuple = Tuple.from(key)\n  override def toKey(tupledKey: Tuple): String = tupledKey.getString(0)\n  override def toEntity(key: String, value: Array[Byte]): Book = {\n    val tupledValue = Tuple.fromBytes(value)\n    val publishedOn = LocalDate.parse(tupledValue.getString(1))\n    Book(isbn = key, title = tupledValue.getString(0), publishedOn = publishedOn)\n  }\n}\n\nval dbio: DBIO[Option[Book]] = for {\n  _ \u003c- booksSubspace.set(Book(\"978-0451205766\", \"The Godfather\", LocalDate.parse(\"2002-03-01\")))\n  maybeBook \u003c- booksSubspace.get(\"978-0451205766\").toDBIO\n} yield maybeBook\n\nval maybeBook: Future[Option[Book]] = dbio.transact(database)\n```\n\nIf you:\n- think some functionality is missing\n- find a bug \n- feel that API does not \"feel right\"\n\nplease create an issue.\n\n## Quickstart with sbt\nTo get started you can add the following dependencies to your project:\n```scala\nval fdb4sVersion = \"0.12.0\"\n\nlibraryDependencies ++= Seq(\n  \"com.github.pwliwanow.foundationdb4s\" %% \"core\" % fdb4sVersion,\n  \"com.github.pwliwanow.foundationdb4s\" %% \"schema\" % fdb4sVersion,\n  \"com.github.pwliwanow.foundationdb4s\" %% \"akka-streams\" % fdb4sVersion\n)\n```\nNote that starting from version 0.10.0, modules were renamed \nfrom `foundationdb4s-core` to `core` and from `foundationdb4s-akka-streams` to `akka-streams`.\n\n## Integrations\n- Cats - Monad instances are provided in companion objects for `DBIO` and `ReadDBIO`\n- Akka Streams `Source` implementation (`akka-streams` module)\n\n## Basic abstractions\nModifying data within a `TypedSubspace` (`clear` and `set` operations) returns `DBIO[_]` monad.\n\nReading from a `TypedSubspace` (`get` and `getRange` operations) returns `ReadDBIO[_]` monad.\n`ReadDBIO[_]` provides method `.toDBIO` for converting `ReadDBIO[_]` into `DBIO[_]`.\n\n## Application level schema\nModule `schema` further improves type-safety by requiring schema \n(simply HList from [Shapeless](https://github.com/milessabin/shapeless)) \nfor keys (`KeySchema`) and values (`ValueSchema`) that are to be stored within given `Namespace`.\n\n### Type-safe getRange and clear operations\nHaving schema enables safe `getRange` and `clear` operations - those methods take HList (representing prefix) \nand during compilation it's checked (by requiring implicit parameter) if given prefix starts with the same types \nas `KeySchema` for a given subspace. E.g. given KeySchema: `String :: Int :: HNil`, \nit's possible to call `getRange(String :: HNil)`, but `getRange(Int :: HNil)` will fail to compile.\n\n### Schema evolution and encoders/decoders derivation\nModule `schema` also provides support for automatic derivation of encoders and decoders; \nderived codecs support schema evolution. \n\nE.g. if key was written using `TupleEncoder[String :: HNil]`, it is possible to read the key\n with `TupleDecoder[String :: Option[A] :: HNil]` (where `A` is any type for which `TupleDecoder[A]` exist) or \n with `TupleDecoder[String :: List[A] :: HNil]`.\n\n### Custom encoders/decoders\nImplicit `TupleEncoders` and `TupleDecoders` are provided for basic types, \nsuch as: `Int`, `Long`, `Boolean` and `String`. \nIt also supports encoders and decoders for `Option[A]` and `List[A]`, \ngiven that implicit `TupleEncoder[A]`/`TupleDecoder[A]` exists.\n\nEncoders and decoders can be automatically derived for any case class, given that there exist implicit \nencoders/decoders for all its members.\n\n### Example with schema namespace\n```scala\nimport com.github.pwliwanow.foundationdb4s.schema._\nimport shapeless.{::, HNil}\n\nimplicit val ec = scala.concurrent.ExecutionContext.global\nval database: Database = FDB.selectAPIVersion(630).open(null, ec)\n\nobject Language extends Enumeration {\n  type Language = Value\n  val En, De = Value\n}\ncase class ISBN(value: String) extends AnyVal\nimport Language._\n\nfinal case class Book(language: Language, isbn: ISBN, title: String, publishedOn: LocalDate)\n\nobject Codecs {\n  implicit val localDateEnc = implicitly[TupleEncoder[Long]].contramap[LocalDate](_.toEpochDay)\n  implicit val localDateDec = implicitly[TupleDecoder[Long]].map(LocalDate.ofEpochDay)\n  implicit val languageEnc = implicitly[TupleEncoder[String]].contramap[Language](_.toString)\n  implicit val languageDec = implicitly[TupleDecoder[String]].map(Language.withName)\n}\nimport Codecs._\n\nobject BookSchema extends Schema {\n  type Entity = Book\n  type KeySchema = Language :: LocalDate :: ISBN :: HNil\n  type ValueSchema = String :: HNil\n\n  override def toKey(entity: Book): BookSchema.KeySchema =\n    entity.language :: entity.publishedOn :: entity.isbn :: HNil\n  override def toValue(entity: Book): BookSchema.ValueSchema =\n    entity.title :: HNil\n  override def toEntity(key: BookSchema.KeySchema, valueRepr: BookSchema.ValueSchema): Book = {\n    val language :: publishedOn :: isbn :: HNil = key\n    val title :: HNil = valueRepr\n    Book(language, isbn, title, publishedOn)\n  }\n}\n  \nval booksNamespace = new BookSchema.Namespace(new Subspace(Tuple.from(\"books\")))\n\nval dbio: ReadDBIO[Seq[Book]] = booksNamespace.getRange((Language.En, LocalDate.of(2018, 7, 10)))\n// or booksNamespace.getRange(Language.En :: LocalDate.of(2018, 7, 10) :: HNil)\nval result: Future[Seq[Book]] = dbio.transact(database)\n```\n\n## Parallel requests\nSometimes it may prove useful to perform operations in parallel (e.g. in case of independent `gets`). \nFor that use case `Parallel` type class from [Cats](https://typelevel.org/cats/typeclasses/parallel.html) is provided: \nit allows users to use operations like `parSequence`, `parTraverse` and `parMapN`.\n\n```scala\n// `Parallel` type classes are defined in `DBIO` and `ReadDBIO` companion objects, \n// so there is no need to import them as they are in scope automatically \nimport cats.implicits._\n\nval dbios: List[ReadDBIO[Option[Book]]] = \n  List(booksSubspace.get(\"978-0451205766\"), booksSubspace.get(\"978-1491962299\"))\n// instruct `dbios` to be run in parallel\nval dbio: ReadDBIO[List[Option[Book]]] = dbios.parSequence\nval result: Future[List[Option[Book]]] = dbio.transact(database)\n```\n\n## Versionstamps \nVersionstamp consists of \"transaction\" version and of a user version.\n\nTransaction version is usually assigned by the database in such a way that \nall transactions receive a different version that is consistent with a serialization \norder of the transactions within the database. \nThis also implies that the transaction version of newly committed transactions will \nbe monotonically increasing over time.\nNote that transaction version will be assigned during commit, \nwhich implies that it is not possible to use/get \"current\" transaction version inside the transaction itself. \n\nUser version should be set by the client. \nIt allows the user to use this class to impose a total order of items across multiple \ntransactions in the database in a consistent and conflict-free way.\n\nMore information in [FoundationDB Javadoc](https://apple.github.io/foundationdb/javadoc/com/apple/foundationdb/tuple/Versionstamp.html).\n\n### Working with Versionstamps\n\nfoundationdb4s supports working with keys or values that contain versionstamps \nby providing `SubspaceWithVersionstampedKeys` and `SubspaceWithVersionstampedValues`.\nCompared to `TypedSubspace`, those require additional method to be implemented: `extractVersionstamp`.\n\nTo obtain versionstamp which was used by any versionstamp operations in this `DBIO`, \nuse `transactVersionstamped` instead of `transact`.\n\nNote that if the given `DBIO` did not modify the database, returned `Versionstamp` will be empty.\n\n```scala\nimplicit val ec = scala.concurrent.ExecutionContext.global\nval database: Database = FDB.selectAPIVersion(630).open(null, ec)\n\ncase class EventKey(eventType: String, versionstamp: Versionstamp)\ncase class Event(key: EventKey, content: Array[Byte])\n\nval eventsSubspace = new SubspaceWithVersionstampedKeys[Event, EventKey] {\n  override val subspace: Subspace = new Subspace(Tuple.from(\"events\"))\n  override def toKey(entity: Event): EventKey = entity.key\n  override def toRawValue(entity: Event): Array[Byte] = event.content\n  override def toTupledKey(key: EventKey): Tuple = Tuple.from(key.eventType, key.versiostamp)\n  override def toKey(tupledKey: Tuple): EventKey = {\n    EventKey(tupledKey.getString(0), tupledKey.getVersiostamp(1))\n  }\n  override def toEntity(key: EventKey, value: Array[Byte]): Event = Event(key, value)\n  override def extractVersionstamp(key: EventKey): Versionstamp = key.versionstamp\n}\n\nval event = Event(\n  key = EventKey(\"UserAdded\", Versionstamp.incomplete(0)), \n  content = Tuple.from(\"\"\"{ \"name\": \"John Smith\" }\"\"\").pack)\n\n// save new event\nval setDbio: DBIO[Unit] = eventsSubspace.set(event)\nval completedVersionstamp: Versionstamp = \n  Await.result(\n    setDbio\n      .transactVersionstamped(database, userVersion = 0)\n      .map { case (_, Some(versionstamp)) =\u003e versionstamp },\n    Duration.Inf)\n\n// update previously persisted event\nval updatedEvent = event.copy(key = event.key.copy(versionstamp = completedVersionstamp))\nval updateDbio = eventsSubspace.set(updatedEvent)\n\nupdateDbio.transact(database)\n``` \n\n## Watches\nWhen application needs to monitor changes done to a given key, it can periodically read the key or it can use _watches_.\nWatches are created for a given key, and return a `Promise` that will be completed, once the value for the key changes.\n\nNote that there is limited number of watches that can be active for each database connection (by default 10,000),\nso watches that are no longer needed should be canceled.\nOnce the number is exceeded creating new watches will fail.\n\n```scala\n// reusing first example\nfinal case class Book(isbn: String, title: String, publishedOn: LocalDate)\nval booksSubspace: TypedSubspace[Book, String] = ???\n\nval key = \"978-0451205766\"\nval dbio: DBIO[Promise[Unit]] = booksSubspace.watch(key)\nval futureWatch: Future[Promise[Unit]] = dbio.transact(database)\n```   \n\nMore information about watches can be found in \n[FoundationDB developer guide](https://apple.github.io/foundationdb/developer-guide.html#watches) and\n[FoundationDB Javadoc](https://apple.github.io/foundationdb/javadoc/com/apple/foundationdb/Transaction.html#watch-byte:A-).  \n\n## Reading big amount of data\nIf you want to stream data from a subspace, it can take longer than FoundationDB transaction time limit, your data is immutable and append only, or if approximation is good enough for your use case, \nyou can use either use `SubspaceSource` (from akka-streams module) \nor you can use `RefreshingSubspaceStream` (from core module).\n\nAdvantage of using `SubspaceSource` is that it closes resources automatically and \nexposes easier to use API leveraging Akka Streams.\n\nTo create a source you need at least `subspace: TypedSubspace[Entity, Key]` and `database: Database`: \n```scala\nval source: Source[Entity, _] = SubspaceSource.from(subspace, database)\n```\n\nHowever, if you don't want to add Akka as a dependency or you need more control over streaming the data \nyou can use `RefreshingSubspaceStream`.\n\n### Processing data continuously\nIf a subspace (or part of a subspace) is modeled as a log and one wants to process the data once it arrives, \n`InfinitePollingSubspaceSource` may become useful.\n\n`InfinitePollingSubspaceSource` will stream the data from the subspace (or from part of a subspace). \nOnce it reaches the last element it will try to resume from the last seen value.\n\n```scala\nval processEntityFlow: Flow[Entity, Entity, NotUsed] = ???\nval commitOffsetSink: Sink[Entity, NotUsed] = ???\nval lastSeenKey: Array[Byte] = fetchLastSeenKey()\nval source = Source[Entity, _] = \n  InfinitePollingSubspaceSource.from(\n    typedSubspace, \n    database, \n    pollingInterval = 100.millis, \n    begin = KeySelector.firstGreaterThan(lastSeenKey))\nsource.via(processEntityFlow).to(commitOffsetSink).run()\n```\n\n## Example - class scheduling\nModule `example` contains implementation of [Class Scheduling from FoundationDB website](https://apple.github.io/foundationdb/class-scheduling-java.html).\n\n## Contributing\nContributors and help is always welcome!\n\nPlease make sure that issue exists for the functionality that you want to create (or bug that you want to fix),\nand in the commit message please include issue number and issue title (e.g. \"#1 Incorrect ...\").\n\n## Testing\nTo run tests you'll need a local FoundationDB instance running (here are installation instructions for [Linux](https://apple.github.io/foundationdb/getting-started-linux.html) and [macOS](https://apple.github.io/foundationdb/getting-started-mac.html)).\n\nThen simply execute `sbt test`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpwliwanow%2Ffoundationdb4s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpwliwanow%2Ffoundationdb4s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpwliwanow%2Ffoundationdb4s/lists"}