{"id":16661790,"url":"https://github.com/brandonchinn178/persistent-mtl","last_synced_at":"2025-08-03T03:12:46.760Z","repository":{"id":37021974,"uuid":"315160625","full_name":"brandonchinn178/persistent-mtl","owner":"brandonchinn178","description":"Monad transformers for the persistent library","archived":false,"fork":false,"pushed_at":"2025-03-02T02:20:56.000Z","size":271,"stargazers_count":14,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-18T02:13:53.406Z","etag":null,"topics":["database","monad-transformers"],"latest_commit_sha":null,"homepage":"https://hackage.haskell.org/package/persistent-mtl","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brandonchinn178.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-11-23T00:24:07.000Z","updated_at":"2025-03-02T17:16:22.000Z","dependencies_parsed_at":"2023-12-26T22:22:09.414Z","dependency_job_id":"472fc92f-843a-4810-97fb-294a39a2397b","html_url":"https://github.com/brandonchinn178/persistent-mtl","commit_stats":{"total_commits":224,"total_committers":4,"mean_commits":56.0,"dds":0.0580357142857143,"last_synced_commit":"275d55fef44d4d6850a5aadab604f806c9907428"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brandonchinn178%2Fpersistent-mtl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brandonchinn178%2Fpersistent-mtl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brandonchinn178%2Fpersistent-mtl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brandonchinn178%2Fpersistent-mtl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brandonchinn178","download_url":"https://codeload.github.com/brandonchinn178/persistent-mtl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244147342,"owners_count":20405942,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","monad-transformers"],"created_at":"2024-10-12T10:36:08.297Z","updated_at":"2025-03-21T17:31:54.714Z","avatar_url":"https://github.com/brandonchinn178.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `persistent-mtl`\n\n[![GitHub Actions](https://img.shields.io/github/actions/workflow/status/brandonchinn178/persistent-mtl/ci.yml?branch=main)](https://github.com/brandonchinn178/persistent-mtl/actions?query=branch%3Amain)\n[![Hackage](https://img.shields.io/hackage/v/persistent-mtl)](https://hackage.haskell.org/package/persistent-mtl)\n[![Codecov](https://img.shields.io/codecov/c/gh/brandonchinn178/persistent-mtl)](https://app.codecov.io/gh/brandonchinn178/persistent-mtl)\n\nUse the `persistent` API in your monad transformer stack, seamlessly interleaving business logic with database operations by simply dropping `SqlQueryT` into your stack.\n\nFeatures:\n* Easy integration into a monad transformer stack\n* Monad type class to generalize functions that use database operations\n* Simple transaction control\n* Supports mocking database operations in tests\n\n## Quickstart\n\n```hs\n{-# LANGUAGE DerivingStrategies #-}\n{-# LANGUAGE GADTs #-}\n{-# LANGUAGE GeneralizedNewtypeDeriving #-}\n{-# LANGUAGE LambdaCase #-}\n{-# LANGUAGE MultiParamTypeClasses #-}\n{-# LANGUAGE OverloadedStrings #-}\n{-# LANGUAGE QuasiQuotes #-}\n{-# LANGUAGE StandaloneDeriving #-}\n{-# LANGUAGE TemplateHaskell #-}\n{-# LANGUAGE TypeApplications #-}\n{-# LANGUAGE TypeFamilies #-}\n{-# LANGUAGE UndecidableInstances #-}\n\nimport Control.Monad.IO.Class (MonadIO(..))\nimport Control.Monad.Logger (runStderrLoggingT)\nimport Database.Persist.Sql (Entity(..), toSqlKey, (\u003c.))\nimport Database.Persist.Monad\nimport Database.Persist.Sqlite (withSqlitePool)\nimport Database.Persist.TH\nimport UnliftIO (MonadUnliftIO(..), wrappedWithRunInIO)\n\nimport Database.Persist.Monad.TestUtils (runMockSqlQueryT, withRecord)\nimport Test.Tasty.HUnit (Assertion, (@?=))\n\nshare [mkPersist sqlSettings, mkMigrate \"migrate\"] [persistLowerCase|\nPerson\n  name String\n  age Int\n  deriving Show Eq\n|]\n\nnewtype MyApp a = MyApp\n  { unMyApp :: SqlQueryT IO a\n  }\n  deriving (Functor, Applicative, Monad, MonadIO, MonadSqlQuery)\n\ninstance MonadUnliftIO MyApp where\n  withRunInIO = wrappedWithRunInIO MyApp unMyApp\n\ngetYoungPeople :: MonadSqlQuery m =\u003e m [Entity Person]\ngetYoungPeople = selectList [PersonAge \u003c. 18] []\n\nmain :: IO ()\nmain = runStderrLoggingT $ withSqlitePool \"db.sqlite\" 5 $ \\pool -\u003e\n  liftIO $ runSqlQueryT pool $ unMyApp $ do\n    runMigration migrate\n    insert_ $ Person \"Alice\" 25\n    insert_ $ Person \"Bob\" 10\n    youngsters \u003c- getYoungPeople\n    liftIO $ print youngsters\n\n-- unit test with mocks!\nunit_my_function :: Assertion\nunit_my_function = do\n  let person1 = Entity (toSqlKey 1) (Person \"Child1\" 10)\n\n  result \u003c- runMockSqlQueryT getYoungPeople\n    [ withRecord @Person $ \\case\n        SelectList _ _ -\u003e Just [person1]\n        _ -\u003e Nothing\n    ]\n\n  result @?= [person1]\n```\n\n## What's wrong with just using `persistent`?\n\n### Using `persistent` in production code\n\n`persistent` runs all of its functions in `SqlPersistT`, which is an alias for `ReaderT SqlBackend`. Since all functions run in this concrete monad and not a generalized type class, it becomes difficult to integrate database operations into your monad transformer stack. Below are some examples of trying to integrate `persistent` functions into a monad transformer application, and the drawbacks of each option.\n\n#### Option 1: Add `SqlPersistT` to your monad transformer stack\n\nOne might look at the `SqlPersistT` type and think it's a monad transformer, and add it to their monad transformer stack. But since `persistent` functions run in the concrete `SqlPersistT` monad (and not with a type class), you'll need some way of lifting `SqlPersistT` into your application monad.\n\nBefore going further, I do want to point out that `SqlBackend` represents a single database connection, so adding `SqlPersistT` to your monad transformer stack would run your entire application in a single connection (read: single transaction)! So for most applications, this option probably won't work for you, but let's assume you have a use-case where this isn't an issue.\n\nOption 1a is to write `liftSqlPersist` specifically for your application monad:\n\n```hs\nnewtype MyApp a = MyApp (ReaderT MyAppConfig (SqlPersistT (LoggingT IO)) a)\n\n-- Notice the duplication here: anything inside `SqlPersistT` in your stack\n-- needs to go in here.\nliftSqlPersist :: SqlPersistT (LoggingT IO) a -\u003e MyApp a\nliftSqlPersist = MyApp . lift\n```\n\nBut then any function that runs database connections is taken out of mtl-style add needs to be concretely typed to `MyApp`\n\n```hs\n-- you originally had a nice mtl-style function with a generalized monad\nfoo :: MonadReader MyAppConfig m =\u003e m ()\nfoo = do\n  config \u003c- ask\n  _ \u003c- bar config\n  return ()\n\n-- but adding a database operation forces us to remove the generalization\nfoo :: MyApp ()\nfoo = do\n  config \u003c- ask\n  _ \u003c- bar config\n  _ \u003c- liftSqlPersist $ get $ configUserId config\n  return ()\n```\n\nSo then you might try option 1b and write a type class that will lift `SqlPersistT`:\n\n```hs\nclass MonadLiftSqlPersist m where\n  -- Remember how we had to duplicate anything inside `SqlPersistT` in your\n  -- stack? The stack within `SqlPersistT` can be different between monads, so\n  -- you need to define the inner type for each monad\n  type Inner m :: Type -\u003e Type\n\n  liftSqlPersist :: SqlPersistT (Inner m) a -\u003e m a\n\ninstance MonadLiftSqlPersist MyApp where\n  type Inner MyApp = LoggingT IO\n  liftSqlPersist = MyApp . lift\n```\n\nwhich still has the unfortunate problem of copy-pasting whatever is inside `SqlPersistT` into the `Inner` type family instance.\n\nBut the main problem with both of these options is that `liftSqlPersist` will only contain the context you put inside `SqlPersistT`, meaning that within a `liftSqlPersist` action, you can't get access to `MyAppConfig`! Of course, you could always make `SqlPersistT` the very first monad transformer in your stack, but that might not work in another situation. Plus, you'd have even more monad transformers to copy into the type of `liftSqlPersist`.\n\n#### Option 2: Manually run `runSqlPool` every time you run a `persistent` function\n\nHere, you might store the `Pool SqlBackend` in your monad transformer stack and then use `runSqlPool` to immediately unwrap `SqlPersistT`.\n\n```hs\ndata MyAppConfig = MyAppConfig\n  { backendPool :: Pool SqlBackend\n  , ...\n  }\n\nrunQuery :: MonadReader MyAppConfig m =\u003e SqlPersistT m a -\u003e m a\nrunQuery m = do\n  MyAppConfig{backendPool} \u003c- ask\n  runSqlPool m backendPool\n\nfoo :: MonadReader MyAppConfig m =\u003e m ()\nfoo = do\n  config \u003c- ask\n  _ \u003c- bar config\n  _ \u003c- runQuery $ get $ configUserId config\n  return ()\n```\n\nGreat! Let me first say that this is *not a bad solution*. You could even make your own type class like `MonadHasBackendPool` to abstract away monads that contain a `Pool SqlBackend`, not necessarily the whole `MyAppConfig`.\n\nThere are two drawbacks with this approach, one minor drawback and one major drawback. The minor drawback is that you have to put the `Pool SqlBackend` into your environment yourself. It would be great if there could be a monad transformer and type class already made for you to easily plug it in. It's not that much code, so this isn't a big deal, but if you're quickly bootstrapping a new project with persistent, it'd be nice to reach for something already built.\n\nThe major drawback with this approach is transactions and composability. `runSqlPool` (and `runQuery` in this example) runs its action within a single transaction. Say you have two functions that run separate, composable actions that interleave business logic and database operations:\n\n```hs\nfoo :: MonadReader MyAppConfig m =\u003e m ()\nfoo = do\n  -- business logic\n  runQuery $ insert_ $ ...\n  -- more business logic\n\nbar :: MonadReader MyAppConfig m =\u003e m ()\nbar = do\n  -- business logic\n  runQuery $ insert_ $ ...\n  -- more business logic\n```\n\nThere is no way to compose `foo` and `bar` so that it all runs within a single database transaction. You could try\n\n```hs\nfooAndBar :: MonadReader MyAppConfig m =\u003e m ()\nfooAndBar = runQuery $ do\n  lift foo\n  -- something else\n  lift bar\n```\n\nbut `foo` and `bar` each run their own `runQuery` function, so actually, `fooAndBar` uses three connections (i.e. three transactions): one connection from `runQuery` in `fooAndBar` and one connection each from `foo` and `bar`.\n\n#### Option 3: `persistent-mtl`\n\nSo what does `persistent-mtl` do differently?\n\n1. It stores the entire `Pool SqlBackend` in `SqlQueryT`, which means you *can* add `SqlQueryT` to your monad transformer stack. Remember that the problem with adding `SqlPersistT` to your monad transformer stack is that your entire application would run with a single database connection, aka a single database transaction.\n1. It provides a `MonadSqlQuery` type class out of the box and all of `persistent`'s functions lifted to use `MonadSqlQuery`\n1. It provides a `withTransaction` function that runs the given action within a single transaction. For example,\n\n    ```hs\n    foo :: MonadSqlQuery m =\u003e m ()\n    foo = do\n      -- business logic\n      insert_ $ ...\n      -- more business logic\n\n    bar :: MonadSqlQuery m =\u003e m ()\n    bar = do\n      -- business logic\n      insert_ $ ...\n      -- more business logic\n\n    fooAndBar :: MonadSqlQuery m =\u003e m ()\n    fooAndBar = withTransaction $ do\n      foo\n      -- something else\n      bar\n    ```\n\n    `fooAndBar` will run both `foo` and `bar` in the same transaction. Note that `foo` and `bar` themselves don't say anything about transactions. By default, using a `persistent` function without `withTransaction` will run each query in its own transaction. And if `foo` did use `withTransaction`, it would start a transaction within a transaction (if the SQL backend supports it). Now, `foo` and `bar` are composable!\n\nIn summary, `persistent-mtl` takes all the good things about option 2, implements them out of the box (so you don't have to do it yourself), and makes your business logic functions composable with transactions behaving the way YOU want.\n\n### Easy transaction management\n\nSome databases will throw an error if two transactions conflict (e.g. [PostgreSQL](https://www.postgresql.org/docs/9.5/transaction-iso.html)). The client is expected to retry transactions if this error is thrown. `persistent` doesn't easily support this out of the box, but `persistent-mtl` does!\n\n```hs\nimport Database.PostgreSQL.Simple.Errors (isSerializationError)\n\nmain :: IO ()\nmain = withPostgresqlPool \"...\" 5 $ \\pool -\u003e do\n  let env = mkSqlQueryEnv pool $ \\env -\u003e env\n        { retryIf = maybe False isSerializationError . fromException\n        , retryLimit = 100 -- defaults to 10\n        }\n\n  -- in any of the marked transactions below, if someone else is querying\n  -- the postgresql database at the same time with queries that conflict\n  -- with yours, your operations will automatically be retried\n  runSqlQueryTWith env $ do\n    -- transaction 1\n    insert_ $ ...\n\n    -- transaction 2\n    withTransaction $ do\n      insert_ $ ...\n\n      -- transaction 2.5: transaction-within-a-transaction is supported in PostgreSQL\n      withTransaction $ do\n        insert_ $ ...\n\n      insert_ $ ...\n\n    -- transaction 3\n    insert_ $ ...\n```\n\nBecause of this built-in retry support, any IO actions inside `withTransaction` have to be explicitly marked with `rerunnableIO`. If you try to use a function with a `MonadIO m` constraint, you'll get a compile-time error!\n\n```\n.../Foo.hs:100:5: error:\n    • Cannot run arbitrary IO actions within a transaction. If the IO action is rerunnable, use rerunnableIO\n    • In a stmt of a 'do' block: arbitraryIO\n      In the second argument of ‘($)’, namely\n        ‘withTransaction\n           $ do insert_ record1\n                arbitraryIO\n                insert_ record2’\n    |\n100 |     arbitraryIO\n    |     ^^^^^^^^^^^\n```\n\nNote that this **only** applies for transactions, so `MonadIO` and `MonadSqlQuery` constraints can still co-exist (for a function with IO actions that are not rerunnable) as long as the function is never called within `withTransaction`.\n\n### Testing functions that use `persistent` operations\n\nGenerally, I would recommend someone using `persistent` in their application to make a monad type class containing the API for their domain, like\n\n```hs\nclass MonadAppService m where\n  getYoungPeople :: m [Entity Person]\n\ninstance MonadAppService MyApp where\n  getYoungPeople = selectList [PersonAge \u003c. 18] []\n```\n\nso that writing unit tests would mock out domain-level abstractions. I generally wouldn't recommend mocking out the entire database state; if you're testing complex database queries, you should just write integration tests and check that the queries do what you expect on an actual database.\n\nBut maybe you have a small function that uses `selectList` and it's not worth making a whole type class to wrap that call. With `persistent`, `selectList` runs a `SqlPersistT` action, which is completely un-introspectable. Sure, you could pass in a `SqlBackend` that intercepts all queries, but you'd be mocking extremely low level behavior — your mock would need to know the exact `SELECT` query `selectList` sends.\n\n`persistent-mtl`, on the other hand, provides `MockSqlQueryT` which you can use to execute your `MonadSqlQuery` functions with a list of mocks, where a mock intercepts `SqlQueryRep`, a data representation of each `persistent` function, and returns the result. For example, to mock `selectList`, you'd simply do\n\n```hs\nrunMockSqlQueryT getYoungPeople\n  [ withRecord @Person $ \\case\n      SelectList _ _ -\u003e Just mockedPersonList\n      _ -\u003e Nothing\n  ]\n```\n\nand `MockSqlQueryT` would intercept a `selectList` call for a `Person` record and return your mocked result. Each `persistent` function has a corresponding data type constructor (with a few exceptions, such as `selectSource`, which works differently).\n\nIf your function does some complex raw SQL queries, you can intercept those like this:\n\n```hs\ncrazyFunction :: MonadSqlQuery m =\u003e String -\u003e m [Int]\ncrazyFunction postTitle = rawSql\n  \"SELECT age FROM person INNER JOIN post ON person.id = post.author WHERE post.title = ?\"\n  [toPersistValue postTitle]\n\nlet mockRawSql = mockQuery $ \\case\n      RawSql _ [toPersistValue \"foo\"] -\u003e Just [1]\n      RawSql _ [toPersistValue \"bar\"] -\u003e Just [2]\n      _ -\u003e Nothing\n\n-- returns [1]\nrunMockSqlQueryT (crazyFunction \"foo\") [mockRawSql]\n\n-- returns [2]\nrunMockSqlQueryT (crazyFunction \"bar\") [mockRawSql]\n\n-- error: Could not find mock for query\nrunMockSqlQueryT (crazyFunction \"baz\") [mockRawSql]\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrandonchinn178%2Fpersistent-mtl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrandonchinn178%2Fpersistent-mtl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrandonchinn178%2Fpersistent-mtl/lists"}