{"id":37023041,"url":"https://github.com/casehubdk/hxl","last_synced_at":"2026-01-14T02:45:41.425Z","repository":{"id":168721602,"uuid":"644027284","full_name":"casehubdk/hxl","owner":"casehubdk","description":"Pure batching for Scala","archived":false,"fork":false,"pushed_at":"2025-12-13T00:40:52.000Z","size":77,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-14T15:32:55.977Z","etag":null,"topics":["batch-processing","functional-programming","haxl","scala","typelevel"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/casehubdk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-05-22T16:50:13.000Z","updated_at":"2025-12-13T00:40:56.000Z","dependencies_parsed_at":"2025-09-10T20:24:37.983Z","dependency_job_id":"a70e5642-38b2-4f1d-b854-68e2227e3797","html_url":"https://github.com/casehubdk/hxl","commit_stats":null,"previous_names":["casehubdk/hxl"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/casehubdk/hxl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casehubdk%2Fhxl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casehubdk%2Fhxl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casehubdk%2Fhxl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casehubdk%2Fhxl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/casehubdk","download_url":"https://codeload.github.com/casehubdk/hxl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casehubdk%2Fhxl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408737,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T01:52:23.358Z","status":"online","status_checked_at":"2026-01-14T02:00:06.678Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch-processing","functional-programming","haxl","scala","typelevel"],"created_at":"2026-01-14T02:45:40.846Z","updated_at":"2026-01-14T02:45:41.402Z","avatar_url":"https://github.com/casehubdk.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hxl \u003ca href=\"https://typelevel.org/cats/\"\u003e\u003cimg src=\"https://typelevel.org/cats/img/cats-badge.svg\" height=\"40px\" align=\"right\" alt=\"Cats friendly\" /\u003e\u003c/a\u003e\nHxl is a pure applicative batching library for Scala.\n\nHxl is based on the ideas presented in [Haxl](https://simonmar.github.io/bib/papers/haxl-icfp14.pdf), but diverges in in a few ways.\nNotably, Hxl does not perform side effects, but is instead based on a free applicative structure.\n\nHxl is very small (only a couple hundred lines of code) and only depends on cats.\n\nHxl is written in tagless final, which allows for molding the library to your needs.\n\n## Installation\nHxl is available on Maven Central for Scala 2.13 and 3.2.\n```scala\nlibraryDependencies += \"com.github.casehubdk\" %% \"hxl\" % \"0.3.0\"\n```\n\n## Usage\nThere are two primitive structures in Hxl: `DataSource[F, K, V]` and `DSKey[K, V]`.\nA `DataSource[F, K, V]` abstracts over a function `NonEmptyList[K] =\u003e F[Map[K, V]]`.\nA `DataSource` is uniquely (as in Scala universal equals) identified by its `DSKey`.\n```scala\nimport cats._\nimport cats.implicits._\n\nfinal case class MyDSKey(id: String)\ncase object MyDSKey extends DSKey[MyDSKey, String]\n\nval database = Map(\n  \"foo\" -\u003e \"bar\",\n  \"baz\" -\u003e \"qux\"\n)\n\ndef dataSource[F[_]: Applicative] = DataSource.from[F, MyDSKey, String](MyDSKey) { keys =\u003e\n  keys.toList.flatMap(key =\u003e database.get(key.id).toList.tupleLeft(key)).toMap.pure[F]\n}\n\nval fa: Hxl[Id, Option[String]] = Hxl(MyDSKey(\"foo\"), dataSource[Id])\nval fb: Hxl[Id, Option[String]] = Hxl(MyDSKey(\"baz\"), dataSource[Id])\nHxl.runSequential((fa, fb).mapN(_.mkString + \" \" + _.mkString)) // \"bar qux\"\n```\n\nHxl forms an applicative, but sometimes you need a monad.\nHxl is like `Validated` from `cats`, in that it can escape it's applicative nature via a method `andThen`.\nHowever, if you need Hxl to become a monad (like `Either` is to `Validated`), you can use request a monadic view of your effect:\n```scala\nval fa: Hxl[F, String] = ???\n\nval m: HxlM[F, String] = fa.monadic.flatMap{ x =\u003e\n  ???\n}\n\nval back: Hxl[F, String] = m.hxl\n```\n\n## Advanced usage\nSince Hxl is written in tagless final, you can add various behaviors to your data sources.\nFor instance, you can add (pure) caching.\n```scala\nimport cats.data._\nimport cats.implicits._\n\nfinal case class MyDSKey(id: String)\ncase object MyDSKey extends DSKey[MyDSKey, String]\n\nval database = Map(\n  \"foo\" -\u003e \"bar\",\n  \"baz\" -\u003e \"qux\"\n)\n\ntype Cache = Map[String, String]\ntype Effect[A] = State[Cache, A]\n\ndef dataSource: DataSource[Effect, MyDSKey, String] = DataSource.from(MyDSKey) { keys =\u003e\n  State[Cache, Map[MyDSKey, String]] { cache =\u003e\n    val (misses, hits) = keys.toList.partitionEither(k =\u003e cache.get(k.id).tupleLeft(k).toRight(k))\n    val fetched = misses.flatMap(key =\u003e database.get(key.id).toList.map(key -\u003e _)).toMap\n    (cache ++ fetched.map{ case (k, v) =\u003e k.id -\u003e v }, hits.toMap ++ fetched)\n  }\n}\n```\n\n## Extending Hxl\nHxl's interface is public and small, so extension is very possible.\n\nUnder the hood, Hxl compiles your structure into `F[A]` via a natural transformation:\n```scala\ntype Target[F[_], G[_], A] = G[Either[Hxl[F, A], A]]\n\ntype Compiler[F[_], G[_]] = Hxl[F, *] ~\u003e Target[F, G, *]\n```\nHxl repeats your natural transformation until the result becomes `Right`, like `tailRecM`.\n```scala\ndef fa: Hxl[F, A] = ???\n\nfa.foldMap(Hxl.parallelRunner[F]): F[A]\n```\n`Compiler`s can be composed like ordinary functions such that the core of Hxl is exposed for extension.\n\nAs an example, let's add tracing (from `natchez`) to Hxl:\n```scala\nimport _root_.natchez._\nimport cats.data._\nimport cats.implicits._\nimport cats._\nimport Hxl._\n\ndef traceRequests[F[_]: Trace: Applicative, A](req: Requests[F, A]): Requests[F, A] = {\n    def traceSource[K, V](source: DataSource[F, K, V]): DataSource[F, K, V] =\n      DataSource.full[F, K, V](source.key) { ks =\u003e\n        Trace[F].span(s\"datasource.${source.key}\") {\n          Trace[F].put(\"keys\" -\u003e ks.size.toString) *\u003e source.batch(ks)\n        }\n      }(source.optimization)\n\n    req.visit {\n      new Requests.DataSourceVisitor[F] {\n        def visit[K, V](source: DataSource[F, K, V], k: K): (DataSource[F, K, V], K) =\n          (traceSource(source), k)\n      }\n    }\n}\n\ndef composeTracing[F[_]: Trace: Applicative, G[_]: Trace: Applicative](\n    compiler: Compiler[F, G]\n): Compiler[F, StateT[G, Int, *]] = {\n  type Effect[A] = StateT[G, Int, A]\n  new Compiler[F, Effect] {\n    def apply[A](fa: Hxl[F, A]): Hxl.Target[F, Effect, A] =\n      fa match {\n        case Hxl.LiftF(unFetch) =\u003e\n          StateT.liftF {\n            Trace[G].span(\"hxl.fetch\") {\n              compiler(Hxl.LiftF(unFetch))\n            }\n          }\n        case bind: Hxl.Bind[F, a, b] =\u003e\n          StateT { round: Int =\u003e\n            Trace[G]\n              .span(\"hxl.bind\") {\n                Trace[G].put(\"round\" -\u003e round.toString) *\u003e compiler {\n                  Hxl.Bind(traceRequests(bind.requests), bind.f)\n                }\n              }\n              .map(round + 1 -\u003e _)\n          }\n        case other =\u003e StateT.liftF(compiler(other))\n      }\n  }\n}\n\ndef fa: Hxl[F, String] = ???\n\nval result: F[String] = fa.foldMap(composeTracing[F, F](Hxl.parallelRunner)).runA(0)\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcasehubdk%2Fhxl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcasehubdk%2Fhxl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcasehubdk%2Fhxl/lists"}