{"id":13800741,"url":"https://github.com/innFactory/akka-persistence-gcp-datastore","last_synced_at":"2025-05-13T09:31:52.771Z","repository":{"id":38040300,"uuid":"224914339","full_name":"innFactory/akka-persistence-gcp-datastore","owner":"innFactory","description":"akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using google cloud firestore in datastore mode. ","archived":false,"fork":false,"pushed_at":"2025-05-06T00:25:37.000Z","size":237,"stargazers_count":19,"open_issues_count":15,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-11T19:17:50.585Z","etag":null,"topics":["akka","akka-persistence","cloud","cqrs","datastore","event-sourcing","firestore","gcp-datastore","google","java","scala"],"latest_commit_sha":null,"homepage":"https://medium.com/@innFactory/using-google-datastore-with-akka-persistence-1f6f7179b22a","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/innFactory.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-11-29T19:52:00.000Z","updated_at":"2023-08-13T01:15:50.000Z","dependencies_parsed_at":"2025-04-14T19:52:00.923Z","dependency_job_id":null,"html_url":"https://github.com/innFactory/akka-persistence-gcp-datastore","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innFactory%2Fakka-persistence-gcp-datastore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innFactory%2Fakka-persistence-gcp-datastore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innFactory%2Fakka-persistence-gcp-datastore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innFactory%2Fakka-persistence-gcp-datastore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/innFactory","download_url":"https://codeload.github.com/innFactory/akka-persistence-gcp-datastore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253913224,"owners_count":21983277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["akka","akka-persistence","cloud","cqrs","datastore","event-sourcing","firestore","gcp-datastore","google","java","scala"],"created_at":"2024-08-04T00:01:15.774Z","updated_at":"2025-05-13T09:31:51.049Z","avatar_url":"https://github.com/innFactory.png","language":"Scala","funding_links":[],"categories":["Table of Contents"],"sub_categories":["Database"],"readme":"# GCP Datastore Akka Persistence Plugin\n\n![Test](https://github.com/innFactory/akka-persistence-gcp-datastore/workflows/Test/badge.svg?branch=master)\n[ ![Download](https://api.bintray.com/packages/innfactory/sbt-plugins/akka-persistence-gcp-datastore/images/download.svg) ](https://bintray.com/innfactory/sbt-plugins/akka-persistence-gcp-datastore/_latestVersion)\n[![shields.io](http://img.shields.io/badge/license-Apache2-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/e906cfd4a16f46d3b049c404b9d5fa55)](https://www.codacy.com/gh/innFactory/akka-persistence-gcp-datastore?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=innFactory/akka-persistence-gcp-datastore\u0026amp;utm_campaign=Badge_Grade)\n[![Scala Steward badge](https://img.shields.io/badge/Scala_Steward-helping-blue.svg?style=flat\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAQCAMAAAARSr4IAAAAVFBMVEUAAACHjojlOy5NWlrKzcYRKjGFjIbp293YycuLa3pYY2LSqql4f3pCUFTgSjNodYRmcXUsPD/NTTbjRS+2jomhgnzNc223cGvZS0HaSD0XLjbaSjElhIr+AAAAAXRSTlMAQObYZgAAAHlJREFUCNdNyosOwyAIhWHAQS1Vt7a77/3fcxxdmv0xwmckutAR1nkm4ggbyEcg/wWmlGLDAA3oL50xi6fk5ffZ3E2E3QfZDCcCN2YtbEWZt+Drc6u6rlqv7Uk0LdKqqr5rk2UCRXOk0vmQKGfc94nOJyQjouF9H/wCc9gECEYfONoAAAAASUVORK5CYII=)](https://scala-steward.org)\n\nakka-persistence-gcp-datastore is a journal and snapshot store plugin for [akka-persistence](http://doc.akka.io/docs/akka/current/scala/persistence.html) using [Google Cloud Datastore](https://cloud.google.com/datastore).\nIt uses the official Google Java Dependency to talk with the datastore. \n\nScala 2.12 \u0026 2.13, Java 8 \u0026 Java 11, akka 2.6.X are supported.\n\nThe plugin supports the following functionality:\n* serialization of events and snapshots with play-json\n* [peristence-query](http://doc.akka.io/docs/akka/current/scala/persistence-query.html) api\n* custom serialization \n\n## Related Blogposts\n\n- [Using Google Datastore with akka-persistence](https://medium.com/@innFactory/using-google-datastore-with-akka-persistence-1f6f7179b22a)\n- [Was ist Event Sourcing? (de)](https://innfactory.de/softwareentwicklung/scala-akka-play-co/was-ist-event-sourcing-und-wie-funktioniert-es/)\n- [Wie lässt sich das Lesen beim Event Sourcing durch CQRS optimieren? (de)](https://innfactory.de/softwareentwicklung/wie-laesst-sich-das-lesen-beim-event-sourcing-durch-cqrs-optimieren/)\n\n\n## Usage \u0026 Setup \n\nVersions: The table below lists the versions and their main dependencies\n\n| Version to use \t| Scala 2.12 \t| Scala 2.13 \t| Scala 3 / Dotty \t| Akka  \t| play-json \t| google-cloud-datastore \t|\n|----------------\t|------------\t|------------\t|-----------------\t|-------\t|-----------\t|------------------------\t|\n| 1.0.2          \t| ✓          \t| ✓          \t| ?               \t| 2.6.x \t| 2.8.x     \t| 1.102.x                \t|\n\n\n\n### Dependency\n\nYou just need to add the following dependency to you sbt dependencies\n```\nlibraryDependencies += \"de.innfactory\" %% \"akka-persistence-gcp-datastore\" % \"X.Y.Z\"\n```\n\n### Configuration\nTake a look at reference.conf under src/main/resources\nWe forked the cqrs cassandra lightbend example with necessary changes for gcp-datastore (@ Demo Test Project available based on the CQRS Example from Lightbend https://github.com/innFactory/akka-persistence-gcp-datastore-example)\n\nAdd the following to your application.conf for a basic configuration:\n\n```\nakka {\n  # use google cloud datastore as journal and snapshot store\n  persistence {\n\n    journal {\n      plugin = \"gcp-datastore-journal\"\n      auto-start-journals = [\"gcp-datastore-journal\"]\n    }\n\n    snapshot-store {\n      plugin = \"gcp-datastore-snapshot\"\n      auto-start-snapshot-stores = [\"gcp-datastore-snapshot\"]\n    }\n  }\n}\n```\n\n### Datastore Configuration\n\n1. Google Cloud Project with Datastore or FireStore in Datastore mode enabled  \n2. Create a index.yml file with content bolow in the project that will use this plugin:\n\n    ```\n    indexes:\n      - kind: journal\n        properties:\n          - name: persistenceId\n          - name: sequenceNr\n            direction: desc\n    \n      - kind: snapshot\n        properties:\n          - name: persistenceId\n          - name: timestamp\n            direction: desc\n    \n      - kind: snapshot\n        properties:\n          - name: persistenceId\n          - name: timestamp\n    \n      - kind: journal\n        properties:\n        - name: tagsKey\n        - name: timestamp\n    \n      - kind: journal\n        properties:\n        - name: persistenceId\n        - name: sequenceNr\n    ```\n    index.yml\n\n3. Open terminal and execute \n \n    ```\n    gcloud app deploy index.yaml\n    ```\n    This is telling the GCP Datastore to build indexes for the plugin based on the yaml file\n    \n4. Create a service account for read and write to datastore. Download the json and add it to the project\n\n    ```\n    src/main/resources/datastore.json\n    ```\n\n## Persistence query API\n\nThe plugin supports the Persistence query APi, mostly used in CQRS applications to transform/migrate the events from the write side to the read side.\n\nThe ReadJournal is retrieved via the `akka.persistence.datastore.journal.read.DatastoreScaladslReadJournal` and `akka.persistence.datastore.journal.read.DatastoreJavadslReadJournal`. There is also a DatastoreReadJournalProvider.\n\n```scala\nimport akka.persistence.datastore.journal.read.DatastoreScaladslReadJournal\nimport akka.persistence.query.{ EventEnvelope, PersistenceQuery }\n\nval system = ??? //ActorSystem akka-classic or akka-typed then system.toClassic is needed. see the example.\nval readJournal =\n    PersistenceQuery(system).readJournalFor[DatastoreScaladslReadJournal](\"gcp-datastore-query\")\n\n```\n\n#### Supported Queries\nAll queries are live streams and they are not completed when they reaches the end of the currently stored events, but continue to push new events when new events are persisted.\n\n##### eventsByTag\neventsByTags is used for retrieving events that were marked with a given tag.\n\n##### eventsByPersistenceId\neventsByPersistenceId is used for retrieving events for a specific PersistentActor identified by its persistenceId\n    \n## Testing\n\nTo test this plugin \n\n(Source: https://cloud.google.com/datastore/docs/tools/datastore-emulator)\n\n1. ``` gcloud components install cloud-datastore-emulator ```\n\n2. ``` gcloud beta emulators datastore start --no-store-on-disk --consistency=1.0 ```\n\n3. Set Env Variable ```DATASTORE_TESTHOST=http://\u003chost\u003e:\u003cport\u003e``` of datastore emulator\n\n4. Execute ````sbt run````\n\n5. Before executing test reset datastore data: ```curl -X POST http://\u003chost\u003e:\u003cport\u003e/reset```\n\nThere is a shell script under .circle ci which runs all of these tests. cqrs tests are outsourced in the example project.\n\n## Contribution policy\nContributions via GitHub pull requests are gladly accepted from their original author. Along with any pull requests, please state that the contribution is your original work and that you license the work to the project under the project's open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project's open source license and warrant that you have the legal authority to do so.\n\n## Credits\n[innFactory GmbH](https://innfactory.de) is a lightbend partner from germany. We are experts for Apps, BigData \u0026 Cloud Computing. If you need help with your next project, feel free to ask for our support. \n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FinnFactory%2Fakka-persistence-gcp-datastore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FinnFactory%2Fakka-persistence-gcp-datastore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FinnFactory%2Fakka-persistence-gcp-datastore/lists"}