{"id":22725477,"url":"https://github.com/ovotech/datastore4s","last_synced_at":"2025-03-29T23:42:34.692Z","repository":{"id":135934378,"uuid":"120363856","full_name":"ovotech/datastore4s","owner":"ovotech","description":null,"archived":false,"fork":false,"pushed_at":"2019-04-12T13:04:10.000Z","size":408,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-02-05T01:30:39.824Z","etag":null,"topics":["company-kaluza","datastore","macros","scala"],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ovotech.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-05T21:25:34.000Z","updated_at":"2023-11-24T10:48:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"872ce349-fb41-4aef-b077-6953cbc8fc4f","html_url":"https://github.com/ovotech/datastore4s","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovotech%2Fdatastore4s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovotech%2Fdatastore4s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovotech%2Fdatastore4s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovotech%2Fdatastore4s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ovotech","download_url":"https://codeload.github.com/ovotech/datastore4s/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246258862,"owners_count":20748573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["company-kaluza","datastore","macros","scala"],"created_at":"2024-12-10T16:11:22.048Z","updated_at":"2025-03-29T23:42:34.673Z","avatar_url":"https://github.com/ovotech.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Datastore4s\n[![CircleCI](https://circleci.com/gh/ovotech/datastore4s/tree/master.svg?style=svg)](https://circleci.com/gh/ovotech/datastore4s/tree/master)\n[![Download](https://api.bintray.com/packages/ovotech/maven/datastore4s/images/download.svg) ](https://bintray.com/ovotech/maven/datastore4s/_latestVersion)\n\nDatastore4s is a scala library for [GCP Datastore](https://cloud.google.com/datastore/docs/). Datastore4s\nhides the complexities of the Datastore API and removes boilerplate code making it simpler and less error-prone to use.\n\n## Getting Started\n\nThe library is available in the OVO Bintray repository. Add this snippet to your build.sbt to use it.\n\n```sbtshell\nresolvers += Resolver.bintrayRepo(\"ovotech\", \"maven\")\nlibraryDependencies += \"com.ovoenergy\" %% \"datastore4s\" % \"0.2.2\",\n```\n\n### A Simple Example\n\nHere is a basic example of using datastore4s to persist and list a case class representing a `Person` using their first\nand last name to generate the datastore key.\n\n```scala\nimport com.ovoenergy.datastore4s._\n\ncase class Person(firstName: String, lastName: String, age: Int)\n\nobject PersonRepository extends DatastoreRepository {\n\n  override def datastoreConfiguration = DatastoreConfiguration(\"my-project\", \"my-namespace\")\n\n  implicit val personFormat = EntityFormat[Person, String](\"person-kind\")(p =\u003e p.firstName + p.lastName)\n    \n  def storePerson(person: Person): Either[DatastoreError, Persisted[Person]] = run(put(person))\n    \n  def allPeople: Either[DatastoreError, Seq[Person]] = run(list[Person].sequenced())\n\n}\n```\n\nTo use datastore4s simply extend the `DatastoreRepository` trait and supply the implicit formats needed to persist entities.\n\n## Datastore Operations\n\nDatastore operations do not execute immediately, instead they describe an action to be performed by a `DatastoreService`.\nDatastore Operations can also be combined in for comprehensions e.g:\n\n```scala\nimport com.ovoenergy.datastore4s._\nimport com.ovoenergy.datastore4s.DatastoreService._\n\ncase class Person(firstName: String, lastName: String, age: Int)\nobject ForComprehensionExample {\n  val operation: DatastoreOperation[Seq[Person]] = for {\n    _ \u003c- put(Person(\"oli\", \"boyle\", 26))\n    oli \u003c- findOne[Person, String](\"oliboyle\")\n    _ \u003c- put(Person(\"john\", \"doe\", 27))\n    twentySevenYearOlds \u003c- list[Person].withPropertyEq(\"age\", 27).sequenced()\n  } yield oli.toSeq ++ twentySevenYearOlds\n}\n```\n\nSome simple operations include:\n\n- `put[E](entity: E)` which performs an upsert on the entity passed.\n- `delete[E, K](key: K)` which deletes the entity with the given key if it exists.\n- `findOne[E, K](key: K)` which returns an `Option` of the entity with the given key.\n- `list[E].sequenced()` which returns a `Seq` of all the entities of the given type.\n\nOperations can then be executed synchronously using `run` or asynchronously using `runAsync`. For a full list of operations\nand interpreters see the [Operations](./docs/Operations.md) documentation.\n\n## Entities\n\nEntity (de)serialisation is based on three `Format` traits.\n- `EntityFormat`s which determine how a scala type is turned into a datastore entity. An `Entity` can have many `Field`s.\n- `FieldFormat`s which determine how a field of an entity is stored in datastore. A `Field` can have many `Value`s.\n- `ValueFormat`s which determine the mapping between a scala type and a datastore type.\n\n### Entity Formats\n\nTo be able to persist and read entities from google datastore simply create your case class and use the `EntityFormat` macro.\nThe same macro can be used to create `EntityFormat`s for sealed trait hierarchies that only contain case classes. An additional \nfield `\"type\"` will be used on the entity to determine which subtype in the hierarchy the entity represents.\n\nTo use the macro you need to provide:\n \n- the type of the entity and the type of the key.\n- a string of the kind under which you want your entities to be stored.\n- a function between the entity type and key type which will be used to create the unique key for that entity.\n\nFor example:\n\n`EntityFormat[Person, String](\"person-kind\")(person =\u003e person.name)`\n\n**Warning:** Key types cannot be primitive. Out of the box only `String` and `java.lang.Long` keys are supported, if you \nneed a custom type then see the [Datastore Key Customisation](./docs/CustomKeys.md) documentation. \n\n### Value Formats\n\n`ValueFormat[A]` is used to determine how to store (and retrieve) a type as a datastore value in both persistence and queries. \nThere are multiple `ValueFormat[A]`s already implicitly available for: \n\n- `String` which is stored as a `StringValue`\n- `Long` which is stored as a `LongValue`\n- `Int` which is stored as a `LongValue`\n- `Boolean` which is stored as a `BooleanValue`\n- `Double` which is stored as a `DoubleValue`\n- `Float` which is stored as a `DoubleValue`\n- `Option[A]` for any `[A]` for which a format exists, which is stored as a `NullValue` or the expected value for `A`\n- `Seq[A]` for any `[A]` for which a format exists, which is stored as a `ListValue` of the expected value for `A`\n- `Set[A]` for any `[A]` for which a format exists,  which is stored as a `ListValue` of the expected value for `A`\n- `com.google.cloud.Timestamp`\n- `com.google.cloud.datastore.Blob`\n- `com.google.cloud.datastore.LatLng`\n\nThere are also formats available that can be brought into implicit scope (explicitly or by inheriting the `DefaultFormats` or `DefaultDatastoreRepository` traits) for: \n- `Array[Byte]` in the form of `ValueFormat.byteArrayValueFormat`,  which is stored as a `com.google.cloud.datastore.Blob`\n- `BigDecimal` in the form of `BigDecimalStringValueFormat` (or `ValueFormat.bigDecimalDoubleValueFormat` which is not in the default trait)\n- `java.time.Instant` in the form of `ValueFormat.instantEpochMillisValueFormat`, which is stored as a `LongValue`\n\nThese are not implicit by default to allow your own implementations for those types.\n\nThere is also an implicit value format available for any `E` such that for some `K` there is an instance of `EntityFormat[E,K]`, \n`ToKey[K]` and `DatastoreService` that are implicitly in scope. (**NOTE:** the `DatastoreService` must be implicitly in scope otherwise\nthe `ValueFormat[E]` will not be resolved)\n\n#### Custom Types\n\nThere is a utility function available for creating your own value formats by providing functions to and from a type for which a \nformat already exists in implicit scope:\n\n```scala\nimport com.ovoenergy.datastore4s.ValueFormat\n\ncase class CustomString(innerValue: String)\n\nobject CustomString {\n  implicit val format = ValueFormat.formatFrom(CustomString.apply)(_.innerValue)\n  // DatastoreRepository contains an alias function formatFrom\n}\n```\n\nIn the case where it is possible the creation of your custom type may fail when passed a value from datastore, simply return\nan `Either[String, A]` from your function:\n\n```scala\nimport com.ovoenergy.datastore4s.ValueFormat\n\nclass PositiveInteger(val value: Int)\n\nobject PositiveInteger { \n  def apply(int: Int): Either[String, PositiveInteger] = \n    if(int \u003c= 0) Left(\"whoops not positive\") else Right(new PositiveInteger(int))\n    \n  implicit val format = ValueFormat.failableFormatFrom(PositiveInteger.apply)(_.value)\n  // DatastoreRepository contains an alias function failableFormatFrom\n}\n```\n\n### Field Formats\n\nWhen a field only contains one value a `FieldFormat` will be generated using the `ValueFormat`. For fields of type `Either[L, R]` \na `FieldFormat` is generated using `ValueFormat[L]` and `ValueFormat[R]` and a value `\"either_side\"` of `\"Left\"` or `\"Right\"` is added.\n\n#### Case Classes\n\nIf you have a field that is a custom case class that is comprised of fields for which `FieldFormat`s are already in implicit\nscope there is a macro to generate a `FieldFormat` that will nest the fields of that case class using dots to separate the fields:\n\n```scala\nimport com.ovoenergy.datastore4s.FieldFormat\n\ncase class Employee(name: String, age: Int, department: Department)\ncase class Department(name:String, departmentHead: String)\nobject Department {\n  implicit val format = FieldFormat[Department]\n}\n```\n\nUsing the format above an Employee entity would be serialised to have values:\n\n- name of type `String`\n- age of type `Int`\n- department.name of type `String`\n- department.departmentHead of type `String`\n\n#### Sealed Trait Hierarchies\n\nSimilarly to create a `FieldFormat` for a sealed trait hierarchy composed of only case classes and/or objects simply use the same macro,\nthis will store a nested `fieldname.type` value on the entity to determine what subtype the field is.\n\n### For Those Who Hate Macros\n\nIf you do not want to use the macros you can create the formats yourself, it is however likely you will end up writing \nthe same code that would have been generated by the macro. For example:\n\n```scala\nimport com.ovoenergy.datastore4s._\n\ncase class Person(firstName: String, lastName: String, age: Int, job: Job)\ncase class Job(title: String, wage: BigDecimal)\n\nobject NonMacroExample {\n\n  implicit object JobFormat extends FieldFormat[Job] {\n    override def toEntityField(fieldName: String, value: Job): Field = Field(\n        s\"$fieldName.title\" -\u003e toValue(value.title),\n        s\"$fieldName.wage\" -\u003e toValue(value.wage)\n      )\n\n    override def fromEntityField(fieldName: String, entity: Entity) = for {\n      title \u003c- entity.fieldOfType[String](s\"$fieldName.title\")\n      wage \u003c- entity.fieldOfType[BigDecimal](s\"$fieldName.wage\")\n    } yield Job(title, wage)\n  }\n\n  implicit object PersonFormat extends EntityFormat[Person, String] {\n\n    override def toEntityComponents(person: Person, builder: EntityBuilder): EntityComponents[Person, String] = {\n      val builderFunction = (builder: EntityBuilder) =\u003e\n        builder.add(\"firstName\", person.firstName)\n          .add(\"lastName\", person.lastName)\n          .add(\"age\", person.age)\n          .add(\"job\", person.job)\n          .build()\n      new EntityComponents(Kind(\"person\"), person.firstName + person.lastName, builderFunction)\n    }\n\n    override def fromEntity(entity: Entity): Either[DatastoreError, Person] = for {\n      firstName \u003c- entity.fieldOfType[String](\"firstName\")\n      lastName \u003c- entity.fieldOfType[String](\"lastName\")\n      age \u003c- entity.fieldOfType[Int](\"age\")\n      job \u003c- entity.fieldOfType[Job](\"job\")\n    } yield Person(firstName, lastName, age, job)\n  }\n}\n```\n\nThe above is the same as: \n\n```scala\nimport com.ovoenergy.datastore4s._\n\ncase class Person(firstName: String, lastName: String, age: Int, job: Job)\ncase class Job(title: String, wage: BigDecimal)\n\nobject MacroExample {\n  implicit val jobFormat = FieldFormat[Job]\n  implicit val personFormat = EntityFormat[Person, String](\"person\")(person =\u003e person.firstName + person.lastName)\n}\n```\n\n## Further Documentation\n- [Operations](./docs/Operations.md)\n- [Datastore Key Customisation](./docs/CustomKeys.md)\n- [Configuring Your Repository](./docs/Configuration.md)\n- [Entity Indexes](./docs/Indexes.md)\n- [Examples](./examples/Examples.md)\n- [More Entity Customisation](./docs/MoreCustomisation.md)\n\n## Changes\n\nAll changes will be documented in the [CHANGELOG](./CHANGELOG.md), efforts will be made to ensure backwards compatability.\nWhere possible an API will be deprecated before it is removed, but in some cases breaking changed will occur, in these cases\na migration path will also be documented.\n\n## Feedback And Contribution\n\nFeedback, Issues and PR's are welcome.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fovotech%2Fdatastore4s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fovotech%2Fdatastore4s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fovotech%2Fdatastore4s/lists"}