{"id":16445844,"url":"https://github.com/mslinn/quill-cache","last_synced_at":"2025-10-27T05:31:58.544Z","repository":{"id":56532934,"uuid":"94965491","full_name":"mslinn/quill-cache","owner":"mslinn","description":"Caching layer overtop Quill for Scala","archived":true,"fork":false,"pushed_at":"2020-11-02T07:04:34.000Z","size":2006,"stargazers_count":25,"open_issues_count":8,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-28T09:44:17.062Z","etag":null,"topics":["cache-storage","database-access","getquill","scala"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mslinn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-21T05:16:57.000Z","updated_at":"2024-03-17T23:01:12.000Z","dependencies_parsed_at":"2022-08-15T20:30:57.035Z","dependency_job_id":null,"html_url":"https://github.com/mslinn/quill-cache","commit_stats":null,"previous_names":[],"tags_count":44,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mslinn%2Fquill-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mslinn%2Fquill-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mslinn%2Fquill-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mslinn%2Fquill-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mslinn","download_url":"https://codeload.github.com/mslinn/quill-cache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238445871,"owners_count":19473826,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache-storage","database-access","getquill","scala"],"created_at":"2024-10-11T09:45:36.175Z","updated_at":"2025-10-27T05:31:58.130Z","avatar_url":"https://github.com/mslinn.png","language":"Scala","readme":"\u003cimg src='https://raw.githubusercontent.com/mslinn/quill-cache/media/quill-cache.jpg' align='right' width='33%'\u003e\n\n# Cached Persistence for Quill\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![GitHub version](https://badge.fury.io/gh/mslinn%2Fquill-cache.svg)](https://badge.fury.io/gh/mslinn%2Fquill-cache)\n\nPsst: This release dropped async support for MySQL and Postgres.\nMaybe that functionality will be published again in a separate library, no promises.\n\n## Features and Benefits\n  * Dramatically reduces time to fetch results from read-mostly database tables\n  * Database-independent [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) API \n    (`insert`, `deleteById`, `remove`, `update`, `upsert`, `zap`, `findAll`, `findById`, plus application-specific finders)\n  * Thin, light type-safe API\n  * Provides compatible interface to read-write database tables\n  * Multiple databases can be configured, with configurations for development, testing, production, etc.\n  * Choice of caching strategy (strong, soft or none)\n  * Very little boilerplate (convention over configuration)\n  * Switching databases only requires changing one word in a program\n  * Play Framework evolution format support\n  * ScalaTest unit test setup\n  * [DAO code generator available](https://github.com/mslinn/quill-gen)\n\n## Background\nScala uses case classes for modeling domain objects.\n`quill-cache` optimizes database access for read-mostly domain objects by providing a caching layer overtop\n[Quill](https://github.com/getquill/quill).\nThis library depends on [has-id](https://github.com/mslinn/has-id), and case classes that need to be cached must extend\n[HasId](http://mslinn.github.io/has-id/latest/api/#model.persistence.HasId).\n`HasId` is generic and quite flexible, so you are encouraged to subclass all your domain objects from `HasId`,\neven if they do not require database caching.\n\nThe current version of this library has no provision for distributed caches.\nThis could be retrofitted, however the author did not have the need, so the work was not done.\n\n## DAOs\nThe [data access object pattern](https://en.wikipedia.org/wiki/Data_access_object) (DAO) is common across all computer languages.\nDAOs for case classes that require database caching must extend the\n[CachedPersistence](http://mslinn.github.io/quill-cache/latest/api/#model.persistence.CachedPersistence)\nabstract class.\n\nYou are free to name DAOs anything you like; this library does not mandate any naming convention.\nScala DAOs are often given the same name as the class that they persist, but with a suffix indicating plurality.\nFor example, if a case class named `Point` needs to be persisted, the DAO might be called `Points`.\nUnlike some other persistence libraries for Scala, Quill allows you to define your DAO in the case class's companion object,\nso you also have that option when using this library.\n\nThis library can provide each DAO with its own cache.\nDAOs that extend `CachedPersistence` have a method called\n[preload()](http://mslinn.github.io/quill-cache/latest/api/index.html#model.persistence.CacheLike@preload:List[CaseClass])\nwhich your application's initialization must invoke in order to fill that DAO's cache.\nA cache can be flushed by calling the DAO's\n[flushCache()](http://blog.mslinn.com/quill-cache/latest/api/index.html#model.persistence.CacheLike@flushCache():Unit) method.\nBecause `preload()` always flushes the cache before loading it you probably won't ever need to explicitly call `flushCache()`.\n\n## Cache Types\nTwo types of caches are supported:\n\n* [StrongCache](http://mslinn.github.io/scalacourses-utils/latest/api/com/micronautics/cache/StrongCache.html),\n    which is locked into memory until the cache is explicitly flushed.\n    Mix the [StrongCacheLike](http://mslinn.github.io/quill-cache/latest/api/#model.persistence.StrongCacheLike)\n    trait into the DAO to provide this behavior.\n    This type of cache is useful when there is enough memory to hold all instances of the case class.\n* [SoftCache](http://mslinn.github.io/scalacourses-utils/latest/api/com/micronautics/cache/SoftCache.html),\n     which contains \"soft\" values that might expire by timing out or might get bumped if memory fills up.\n     Mix the [SoftCacheLike](http://mslinn.github.io/quill-cache/latest/api/#model.persistence.SoftCacheLike)\n     trait into the DAO to provide this behavior.\n     DAOs that mix in `SoftCacheLike` do not assume that all instances of the case class can fit into memory.\n     `SoftCacheLike` finders that return at most one item from a query the database after every cache miss.\n     These `SoftCacheLike` finders run more slowly than `StrongCacheLike` finders when the cache does not contain the desired value.\n     `SoftCacheLike` finders that return a list of items must always query the database.\n     This trait is experimental, do not use in production.\n\nCaches require an [ExecutionContext](http://www.scala-lang.org/api/current/scala/concurrent/ExecutionContext.html),\nand the unit tests provide one:\n```\npackage model.dao\n\nimport model.persistence.CacheExecutionContext\nimport scala.concurrent.{ExecutionContext, ExecutionContextExecutor}\n\n/** Just delegates to standard Scala ExecutionContext, you can make this do whatever you want */\nobject TestExecutionContext extends CacheExecutionContext {\n  protected val ec: ExecutionContextExecutor = ExecutionContext.Implicits.global\n  override def execute(runnable: Runnable): Unit = ec.execute(runnable)\n\n  override def reportFailure(cause: Throwable): Unit = ec.reportFailure(cause)\n}\n```\n\n## Consistent APIs for Cached and Uncached DAOs\n`CachedPersistence` subclasses\n[UnCachedPersistence](http://mslinn.github.io/quill-cache/latest/api/#model.persistence.UnCachedPersistence),\nwhich you can use to derive DAOs for case classes that must have direct access to the database so the case classes are not cached.\nYou don't have to subclass `UnCachedPersistence` to get this behavior, but if you do then the DAOs for your cached\ndomain objects will have the same interface as the DAOs for your uncached domain objects,\nand your code's structure will be more consistent.\n\n## Installation\nAdd this to your project's `build.sbt`:\n\n    resolvers += \"micronautics/scala on bintray\" at \"http://dl.bintray.com/micronautics/scala\"\n\n    libraryDependencies += \"com.micronautics\" %% \"quill-cache\" % \"3.5.12\"\n\nYou will also need to add a driver for the database you are using.\nQuill only supports H2, MySQL, Postgres and Sqlite.\nFor example, for Postgres, add:\n\n    libraryDependencies += \"org.postgresql\" % \"postgresql\" % \"42.2.5\"\n\nYou will need a logging framework. Logback is a good choice:\n\n    libraryDependencies += \"ch.qos.logback\" % \"logback-classic\" % \"1.2.3\"\n\n## Configuration\nYour database configuration is specified by a HOCON file called `application.conf` on the classpath.\nPlease see `src/main/scala/resources/reference.conf` for an example of how to set that up.\n\nHere is an excerpt showing configuration for H2 and Postgres databases.\nOnly one of these databases can be active per database context:\n\n```\nquill-cache {\n  h2 {\n    dataSourceClassName = org.h2.jdbcx.JdbcDataSource\n    dataSource {\n      url = \"jdbc:h2:tcp://localhost/./h2data;DB_CLOSE_ON_EXIT=FALSE\"\n      url = ${?H2_URL}\n\n      user = sa\n      user = ${?H2_USER}\n\n      password = \"\"\n      password = ${?H2_PASSWORD}\n    }\n  }\n\n  postgres {\n    connectionTimeout = 30000\n    dataSource {\n      databaseName = ${?DB}\n      password = ${?PGPASSWORD}\n\n      portNumber = 5432\n      portNumber = ${?PGPORT}\n\n      serverName = localhost\n      serverName = ${?PGHOST}\n\n      ssl = true\n      sslfactory = \"org.postgresql.ssl.NonValidatingFactory\"\n      #url = \"\"\n\n      user = postgres\n      user = ${?USERID}\n    }\n    dataSourceClassName = \"org.postgresql.ds.PGSimpleDataSource\"\n    maximumPoolSize = 100\n  }\n}\n```\n\nThe `quill-cache` section of the configuration file specifies parameters for this library:\nYou can make up your own subsections and call them whatever you want.\n    The supplied `reference.conf` file also has sample MySQL sections for sync and async, plus an async Postgres section.\nThe contents of the named subsections are database-dependent.\n[Hikari](https://github.com/brettwooldridge/HikariCP#configuration-knobs-baby) interprets the meaning of the `dataSource` sections.\n\nSee also the [Quill application.conf](https://github.com/getquill/quill/blob/master/quill-jdbc/src/test/resources/application.conf),\n[HikariCP initialization docs](https://github.com/brettwooldridge/HikariCP#initialization),\n[HikariConfig source code](https://github.com/brettwooldridge/HikariCP/blob/master/src/main/java/com/zaxxer/hikari/HikariConfig.java#L63-L97),\nand the [Hikari pool sizing docs](https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing#the-formula).\n\n## Working with quill-cache\n### Quill Contexts\nQuill-cache provides many flavors of Quill contexts, one for each type of supported database driver.\nEach context is exposed as an `abstract class`.\nAvailable abstract classes are: `H2Ctx`, `MySqlCtx`, `PostgresCtx`, and `SqliteCtx`.\nSubclass the appropriate `abstract class` for the type of database driver you need, like this:\n\n    class MyClass extends model.persistence.H2Ctx\n\n### Asynchronous Drivers\nAsynchronous drivers are not currently supported by `quill-cache`, but there is an\n[open issue for this enhancement](https://github.com/mslinn/quill-cache/issues/2).\nIf you have need for this, or if you are looking for a fairly easy F/OSS Scala project to burnish your resume with,\nyou might want to submit a pull request for this behavior (it would closely model the asynch code).\nThe database contexts `MysqlAsyncCtx` and `PostgresAsyncCtx` have already been written in anticipation of async support.\n\n### Best Practice\nDefine a trait called `SelectedCtx`, and mix it into all your DAOs.\n`SelectedCtx` merely extends the database context used in your application.\nThe `PersistenceTest` DAO in `test/scala/model/dao` follows this pattern:\n\n    trait SelectedCtx extends model.persistence.H2Ctx\n\nNow define your application's Quill context as a singleton, and mix in the predefined implicits for Quill-cache defined in `QuillCacheImplicits`.\n\n```\npackage model\n\nimport model.dao.SelectedCtx\nimport persistence.QuillCacheImplicits\n\ncase object Ctx extends SelectedCtx with QuillCacheImplicits\n```\n\nIf you have more implicits to mix in, define a trait in the same manner as `QuillCacheImplicits` and mix it in as well:\n\n```\ntrait MyQuillImplicits { ctx: JdbcContext[_, _] =\u003e\n  // define Quill Decoders, Encoders and Mappers here\n}\n```\n\nAfter adding in `MyQuillImplicits`, your revised application Quill context `Ctx` is now:\n\n```\npackage model\n\nimport model.dao.SelectedCtx\nimport persistence.QuillCacheImplicits\n\ncase object Ctx extends SelectedCtx with QuillCacheImplicits with MyQuillImplicits\n```\n\nNow import the Quill context's internally defined implicits into your DAO's scope.\nHere are two examples of how to do that, one for cached and one for uncached persistence.\nNotice that `Users` and `Tokens` are singletons, which makes them easy to work with.\nHere is `Users`, a DAO with a strong cache, which means it needs an `ExecutionContext` like `TestExecutionContext`,\nwhich is in scope because it resides in the same package:\n```\nimport model.{Ctx, User}\nimport model.persistence._\n\nobject Users extends CachedPersistence[Long, Option[Long], User]\n    with StrongCacheLike[Long, Option[Long], User] {\n  import Ctx._\n\n  // DAO code for User goes here\n}\n```\n\nHere is `Tokens`, a DAO without any cache, which means it does not need an `ExecutionContext`:\n```\nimport model.{Ctx, Token}\nimport model.persistence._\n\nobject Tokens extends UnCachedPersistence[Long, Option[Long], Token] {\n  import Ctx._\n\n  // DAO code for Token goes here\n}\n```\n\n### Multiple Database Contexts\nFor circumstances where more than one database contexts need to share the same HikariCP pool, first construct a context,\nthen other contexts can be created from the first context's `dataSource`. In the following example, a context for an H2 database\nis created using the `Ctx.dataSource`:\n\n    case object Ctx2 extends H2Ctx(Ctx.dataSource) with MySpecialImplicits\n\nNote that the new context need not have the same implicit decoders, encoders or mappers as the original context.\nSee the `ContextTest` unit test for a working example.\n\nHere is another variation:\n```\n/** This causes a new Hikari pool to be created */\nobject AuthCtx extends PostgresCtx with QuillCacheImplicits with IdImplicitLike\n\nabstract class DerivedCtx(dataSource: DataSource with Closeable)\n  extends PostgresCtx(dataSource) with QuillCacheImplicits with IdImplicitLike\n\n/** Reuse the HikariCP pool from `AuthCtx` */\nobject Ctx extends DerivedCtx(AuthCtx.dataSource) with MySpecialImplicits\n```\n\n### Working with DAOs\n`Quill-cache` automatically defines a read-only property for each DAO, called `className`.\nThis property is derived from the unqualified name of the case class persisted by the DAO.\nFor example, if `model.User` is being persisted, `className` will be `User`.\n\nEach DAO needs the following CRUD-related functions defined:\n\n  1. `_findAll`     \u0026ndash; Quill query foundation - Encapsulates the Quill query that returns all instances of the case class from the database\n  1. `_deleteById`  \u0026ndash; Encapsulates the Quill query that deletes the instance of the case class with the given `Id` from the database\n  1. `_findById`    \u0026ndash; Encapsulates the Quill query that optionally returns the instance of the case class from the database with the given\n                            `Id`, or `None` if not found.\n  1. `_insert`      \u0026ndash; Encapsulates the Quill query that inserts the given instance of the case class into the database, and returns the\n                            case class as it was stored, including any auto-increment fields.\n  1. `_update`      \u0026ndash; Encapsulates the Quill query that updates the given instance of the case class into the database, and returns the entity.\n                            Throws an Exception if the case class was not previously persisted.\n### DAO CRUD\nHere is an example of the CRUD-related functions, implemented in the DAO for `model.User` in the `quill-cache` unit test suite.\n```\n  @inline def _findAll: List[User] = run { quote { query[User] } }\n\n  val queryById: IdOptionLong =\u003e Quoted[EntityQuery[User]] =\n    (id: IdOptionLong) =\u003e\n      quote { query[User].filter(_.id == lift(id)) }\n\n  val _deleteById: (IdOptionLong) =\u003e Unit =\n    (id: IdOptionLong) =\u003e {\n      run { quote { queryById(id).delete } }\n      ()\n    }\n\n  val _findById: IdOptionLong =\u003e Option[User] =\n    (id: Id[Option[Long]]) =\u003e\n      run { quote { queryById(id) } }.headOption\n\n  val _insert: User =\u003e User =\n    (user: User) =\u003e {\n      val id: Id[Option[Long]] = try {\n        run { quote { query[User].insert(lift(user)) }.returning(_.id) }\n      } catch {\n        case e: Throwable =\u003e\n          logger.error(e.getMessage)\n          throw e\n      }\n      user.setId(id)\n    }\n\n  val _update: User =\u003e User =\n    (user: User) =\u003e {\n      run { queryById(user.id).update(lift(user)) }\n      user\n    }\n```\n\nWith the above defined, `quill-cache` automatically provides the following CRUD-related methods for each DAO:\nOnly finders can take advantage of a cache, if present:\n```\n@inline def deleteById(id: Id[_IdType]): Unit\n@inline override def findAll: List[User]\ndef findById(id: Id[_IdType]): Option[User]\n@inline def insert(user: User): User\n@inline def update(user: User): User\n@inline def remove(user: User): Unit\n@inline def upsert(user: User): User\n@inline def zap(): Unit\n```\n\nSee the unit tests for examples of how to use this library.\n\n## Scaladoc\n[Here](http://mslinn.github.io/quill-cache/latest/api/#model.persistence.package)\n\n## Sponsor\nThis project is sponsored by [Micronautics Research Corporation](http://www.micronauticsresearch.com/),\nthe company that delivers online Scala and Play training via [ScalaCourses.com](http://www.ScalaCourses.com).\nYou can learn how this project works by taking the [Introduction to Scala](http://www.ScalaCourses.com/showCourse/40),\nand [Intermediate Scala](http://www.ScalaCourses.com/showCourse/45) courses.\n\n## License\nThis software is published under the [Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0.html).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmslinn%2Fquill-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmslinn%2Fquill-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmslinn%2Fquill-cache/lists"}