{"id":13508870,"url":"https://github.com/robconery/moebius","last_synced_at":"2025-05-15T18:06:09.278Z","repository":{"id":40660073,"uuid":"44139879","full_name":"robconery/moebius","owner":"robconery","description":"A functional query tool for Elixir","archived":false,"fork":false,"pushed_at":"2024-10-23T18:55:45.000Z","size":497,"stargazers_count":607,"open_issues_count":3,"forks_count":43,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-05-10T08:05:42.586Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robconery.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-10-12T23:32:35.000Z","updated_at":"2025-04-02T08:27:10.000Z","dependencies_parsed_at":"2024-05-01T17:21:08.346Z","dependency_job_id":"9c0522f2-e536-46c0-94e8-0d3e6d229c1c","html_url":"https://github.com/robconery/moebius","commit_stats":{"total_commits":234,"total_committers":22,"mean_commits":"10.636363636363637","dds":0.3589743589743589,"last_synced_commit":"a1cab32aa57f53849c0a189a9e453b3119ebc4c4"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robconery%2Fmoebius","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robconery%2Fmoebius/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robconery%2Fmoebius/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robconery%2Fmoebius/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robconery","download_url":"https://codeload.github.com/robconery/moebius/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254394719,"owners_count":22063984,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T02:00:59.680Z","updated_at":"2025-05-15T18:06:09.256Z","avatar_url":"https://github.com/robconery.png","language":"Elixir","funding_links":[],"categories":["ORM and Datamapping","Elixir"],"sub_categories":[],"readme":"## A functional query tool for Elixir and PostgreSQL.\n\nOur goal with creating Moebius is to try and keep as close as possible to the functional nature of Elixir and, at the same time, the goodness that is PostgreSQL. We think working with a database should feel like a natural extension of the language, with as little abstraction wonkery as possible.\n\nMoebius is *not* an ORM. There are no mappings, no schemas, no migrations; only queries and data. We embrace PostgreSQL as much as possible, surfacing the goodness so you be a hero.\n\n## Documentation\n\nAPI documentation is available at http://hexdocs.pm/moebius\n\n## Installation\n\nInstalling Moebius involves a few small steps:\n\n  1. Add moebius to your list of dependencies in `mix.exs`:\n\n   ```elixir\n    def deps do\n      [{:moebius, \"~\u003e 4.2.0\"}]\n    end\n   ```\n\n  2. Add the db child process to your `Application` module's supervision tree:\n  \n  ```elixir\n  children = [\n    Moebius.Db\n  ]\n  ```\n\nRun `mix deps.get` and you'll be good to go.\n\n## Connecting to PostgreSQL\n\nThere are various ways to connect to a database with Moebius. You can used a formal, supervised definition or just roll with our default. Either way, you start off by adding connection info in your `config.exs`:\n\n```elixir\nconfig :moebius, connection: [\n  hostname: \"localhost\",\n  username: \"username\",\n  password: \"password\",\n  database: \"my_db\"\n],\nscripts: \"test/db\"\n```\n\nYou can also use a URL if you like:\n\n```elixir\nconfig :moebius, connection: [\n  url: \"postgresql://user:password@host/database\"\n],\nscripts: \"test/db\"\n```\n\nYou can also configure custom [Postgres Extensions](https://hexdocs.pm/postgrex/Postgrex.Extension.html#content):\n\n```elixir\nconfig :moebius,\n  connection: [url: \"postgresql://user:password@host/database\"],\n  types: PostgresTypes\n```\n\nAnd define your custom types in your application under `lib/postgres_types.ex`\n\n```elixir\ntypes = [Geo.PostGIS.Extension, Some.Custom.Extension]\nopts = [json: Jason]\n\nPostgrex.Types.define(PostgresTypes, types, opts)\n```\n\nIf you want to use environment variables, just set things using `System.env`.\n\nUnder the hood, Moebius uses [the Postgrex driver](https://github.com/ericmj/postgrex) to manage connections and connection pooling. Connections are supervised, so if there's an error any transaction pending will be rolled back effectively (more on that later). The settings you provide in `:connection` will be passed directly to Postgrex (aside from `:url`, which we parse).\n\nYou might be wondering what the `scripts` entry is? Moebius can execute SQL files directly for you - we'll get to that in a bit.\n\n## Supervision and Databases\n\nMoebius formalizes the concept of a database connection, so you can supervise each independently, or not at all. This allows for a lot of flexibility. You don't have to do it this way, but it really helps.\n\n**You don't need to do any of this** - we have a default DB setup for you. However, if you want a formalized, supervised module for your database, here's how you do it.\n\nFirst, create a module for your database:\n\n```elixir\ndefmodule MyApp.Db do\n  use Moebius.Database\n\n  # helper/repo methods go here\nend\n```\n\nNext, in your `Application` file, add this new module to your supervision tree:\n\n```elixir\ndef start(_type, _args) do\n  start_db\n  #...\nend\n\ndef start_db do\n  #create a child process\n  children = [\n    {MyApp.Db, [Moebius.get_connection]}\n  ]\n  Supervisor.start_link children, strategy: :one_for_one\nend\n```\n\nThat's it. Now, when your app starts you'll have a supervised database you can use as needed. The function `Moebius.get_connection/0` will look for a key called `:connection` in your `config.exs`. If you want to connect to multiple databases, name these connections something meaningful, then pass that to `Moebius.get_connection/1`.\n\nFor instance, you might have a sales database and an accounting one; or you might have a read-only connection and a write-only one to spread the load. For this, just specify each as needed:\n\n```elixir\nconfig :moebius, read_only: [\n  url: \"postgresql://user:password@host/database\"\n],\nwrite_only: [\n  url: \"postgresql://user:password@host/database\"\n],\nscripts: \"test/db\"\n```\n\nYou can now use these in your database module:\n\n```elixir\ndef start(_type, _args) do\n  start_db\n  #...\nend\n\ndef start_db do\n  #create a worker\n  read_only_db_worker = worker(MyApp.Db, [Moebius.get_connection(:read_only)])\n  write_only_db_worker = worker(MyApp.Db, [Moebius.get_connection(:write_only)])\n  Supervisor.start_link [read_only_db_worker, write_only_db_worker], strategy: :one_for_one\nend\n```\n\nIt bears repeating: *you don't need to do any of this*, we have a default database setup for you. However supporting multiple connections was very high on our list so this is how we chose to do it (with many thanks to [Peter Hamilton](https://github.com/hamiltop) for the idea).\n\nThe rest of the examples you see below use our default database.\n\n## The Basic Query Flow\n\nWhen querying the database (read or write), you construct the query and then pass it to the database you want:\n\n```elixir\n{:ok, result} = Moebius.Query.db(:users) |\u003e Moebius.Db.first\n```\n\nIn this example, `db(:users)` initiates the `QueryCommand`, we can filter it, sort it, do all kinds of things. To run it, however, we need to pass it to the database we want to execute against.\n\nThe default database is `Moebius.Db`, but you can make your own with a dedicated connection as needed (see above).\n\nLet's see some more examples.\n\n## Simple Examples\n\nThe API is built around the concept of transforming raw data from your database into something you need, and we try to make it feel as *functional* as possible. We lean on Elixir's `|\u003e` operator for this, and it's the core of the API.\n\nThis returns a user with the id of 1.\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(name: \"Steve\")\n  |\u003e sort(:city, :desc)\n  |\u003e limit(10)\n  |\u003e offset(2)\n  |\u003e Moebius.Db.run\n```\n\nHopefully it's fairly straightforward what this query returns. All users named Steve sorted by city... skipping the first two, returning the next 10.\n\n### Operators\n\n\nAn \"=\" (Equal) query happens when you pass a column name and a value:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(name: \"mark\")\n  |\u003e Moebius.Db.run\n\n# or, if you want to be more precise, specify the `eq` key:\n\n{:ok, result} =\n  db(:users)\n  |\u003e filter(:name, eq: \"mark\"])\n  |\u003e Moebius.Db.run\n```\n\nA \"!=\" (Not Equal) query happens when you specify the `neq` key:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(:name, neq: \"mark\")\n  |\u003e Moebius.Db.run\n```\n\nA \"\u003e\" (Greater Than) query happens when you specify the `gt` key:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(:order_count, gt: 5)\n  |\u003e Moebius.Db.run\n```\n\nAdditionally, the following comparison operators are available:\n\n- \"\u003c\" (Less Than): `lt`\n- \"\u003e=\" (Greater Than or Equal To): `gte`\n- \"\u003c=\" (Less Than or Equal To) `lte`\n\nAn \"IN\" query happens when you pass an array:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(:name, [\"mark\", \"biff\", \"skip\"])\n  |\u003e Moebius.Db.run\n\n# or, if you want to be more precise, specify the `in` key:\n\n{:ok, result} =\n  db(:users)\n  |\u003e filter(:name, in: [\"mark\", \"biff\", \"skip\"])\n  |\u003e Moebius.Db.run\n```\n\nA \"NOT IN\" query happens when you specify the `not_in` or `nin` key:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(:name, not_in: [\"mark\", \"biff\", \"skip\"])\n  |\u003e Moebius.Db.run\n```\n\nIf you prefer a more SQL-like syntax, you can use the following aliases:\n\n- db: `from`\n- filter: `where`\n- sort: `order_by`\n\n```elixir\n{:ok, result} =\n  from(:users)\n  |\u003e where(name: \"Steve\")\n  |\u003e where(:order_count, gt: 5)\n  |\u003e order_by(id: :asc, name: :desc)\n```\n\nIf you don't want to deal with my abstractions, just use SQL:\n\n```elixir\n{:ok, result} = \"select * from users where id=1 limit 1 offset 1;\" |\u003e Moebius.Db.run\n```\n\n## Full Text indexing\n\nOne of the great features of PostgreSQL is the ability to do intelligent full text searches. We support this functionality directly:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e search(for: \"Mike\", in: [:first, :last, :email])\n  |\u003e Moebius.Db.run\n```\n\nThe `search` function builds a `tsvector` search on the fly for you and executes it over the columns you send in. The results are ordered in descending order using `ts_rank`.\n\n## JSONB Support\n\nMoebius supports using PostgreSQL as a document store in its entirety. Get your project off the ground and don't worry about migrations - just store documents, and you can normalize if you need to later on.\n\nStart by importing `Moebius.DocumentQuery` and saving a document:\n\n```elixir\nimport Moebius.DocumentQuery\n\n{:ok, new_user} =\n  db(:friends)\n  |\u003e Moebius.Db.save(email: \"test@test.com\", name: \"Moe Test\")\n```\n\nTwo things happened for us here. The first is that `friends` did not exist as a document table in our database, but `save/2` did that for us. This is the table that was created on the fly:\n\n```sql\ncreate table NAME(\n  id serial primary key not null,\n  body jsonb not null,\n  search tsvector,\n  created_at timestamptz not null default now(),\n  updated_at timestamptz not null default now()\n);\n\n-- index the search and jsonb fields\ncreate index idx_NAME_search on NAME using GIN(search);\ncreate index idx_NAME on NAME using GIN(body jsonb_path_ops);\n```\n\nThe entire `DocumentQuery` module works off the premise that this is how you will store your JSONB docs. Note the `tsvector` field? That's PostgreSQL's built in full text indexing. We can use that if we want during by adding `searchable/1` to the pipe:\n\n```elixir\nimport Moebius.DocumentQuery\n\n{:ok, new_user} =\n  db(:friends)\n  |\u003e searchable([:name])\n  |\u003e Moebius.Db.save(email: \"test@test.com\", name: \"Moe Test\")\n```\n\nBy specifying the searchable fields, the `search` field will be updated with the values of the name field.\n\nNow, we can query our document using full text indexing which is optimized to use the GIN index created above:\n\n```elixir\n{:ok, user} =\n  db(:friends)\n  |\u003e search(\"test.com\")\n  |\u003e Moebius.Db.run\n```\n\nOr we can do a simple filter:\n\n```elixir\n{:ok, user} =\n  db(:friends)\n  |\u003e contains(email: \"test@test.com\")\n  |\u003e Moebius.Db.run\n```\n\nThis query is optimized to use the `@` (or \"contains\" operator), using the *other* GIN index specified above. There's more we can do...\n\n```elixir\n{:ok, users} =\n  db(:friends)\n  |\u003e filter(:money_spent, \"\u003e\", 100)\n  |\u003e Moebius.Db.run\n```\n\nThis runs a full table scan so is not terribly optimal, but it does work if you need it once in a while. You can also use the existence (`?`) operator, which is very handy for querying arrays. In the library, it is implemented as `exists`:\n\n```elixir\n{:ok, buddies} =\n  db(:friends)\n  |\u003e exists(:tags, \"best\")\n  |\u003e Moebius.Db.run\n```\n\nThis will allow you to query embedded documents and arrays rather easily, but again doesn't use the JSONB-optimized GIN index. You *can* index for using existence, have a look at the PostgreSQL docs.\n\n### Using Structs\n\nIf you're a big fan of structs, you can use them directly on `save` and we'll send that same struct back to you, complete with an `id`:\n\n```elixir\ndefmodule Candy do\n  defstruct [\n    id: nil,\n    sticky: true,\n    chocolate: \"gooey\"\n  ]\nend\n\nyummy = %Candy{}\n{:ok, res} = db(:monkies) |\u003e Moebius.Db.save(yummy)\n#res = %Candy{id: 1, sticky: true, chocolate: \"gooey\"}\n```\n\nI've been using this functionality constantly with another project I'm working on and it's helped me tremendously.\n\n## SQL Files\n\nI built this for [MassiveJS](https://github.com/robconery/massive-js) and I liked the idea, which is this: *some people love SQL*. I'm one of those people. I'd much rather work with a SQL file than muscle through some weird abstraction.\n\nWith this library you can do that. Just create a scripts directory and specify it in the config (see above), then execute your file without an extension. Pass in whatever parameters you need:\n\n```elixir\n{:ok, result} = sql_file(:my_groovy_query, \"a param\") |\u003e Moebius.Db.run\n```\n\nI highly recommend this approach if you have some difficult SQL you want to write (like a windowing query or CTE). We use this approach to build our test database - have a look at our tests and see.\n\n## Adding, Updating, Deleting (Non-Documents)\n\nInserting is pretty straightforward:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e insert(email: \"test@test.com\", first: \"Test\", last: \"User\")\n  |\u003e Moebius.Db.run\n```\n\nUpdating can work over multiple rows, or just one, depending on the filter you use:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(id: 1)\n  |\u003e update(email: \"maggot@test.com\")\n  |\u003e Moebius.Db.run\n```\n\nThe filter can be a single record, or affect multiple records:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(\"id \u003e 100\")\n  |\u003e update(email: \"test@test.com\")\n  |\u003e Moebius.Db.run\n\n{:ok, result} =\n  db(:users)\n  |\u003e filter(\"email LIKE $2\", \"%test\")\n  |\u003e update(email: \"ox@test.com\")\n  |\u003e Moebius.Db.run\n```\n\nDeleting works exactly the same way as `update`, but returns the count of deleted items in the result:\n\n```elixir\n{:ok, result} =\n  db(:users)\n  |\u003e filter(\"email LIKE $2\", \"%test\")\n  |\u003e delete\n  |\u003e Moebius.Db.run\n\n#result.deleted = 10, for instance\n```\n\n## Bulk Inserts\n\nMoebius supports bulk insert operations transactionally. We've fine-tuned this capability quite a lot (thanks to [Jon Atten](https://github.com/xivsolutions)) and, on our local machines, have achieved ~60,000 writes per second. This, of course, will vary by machine, configuration, and use.\n\nBut that's still a pretty good number don't you think?\n\nA bulk insert works by invoking one directly:\n\n```elixir\ndata = [#let's say 10,000 records or so]\n{:ok, result} =\n  db(:people)\n  |\u003e bulk_insert(data)\n  |\u003e Moebius.Db.transact_batch\n```\n\nIf everything works, you'll get back a result indicating the number of records inserted.\n\n## Table Joins\n\nTable joins can be applied for a single join or piped to create multiple joins. The table names can be either atoms or binary strings. There are a number of options to customize your joins:\n\n```elixir\n  :join        # set the type of join. LEFT, RIGHT, FULL, etc. defaults to INNER\n  :on          # specify the table to join on\n  :foreign_key # specify the tables foreign key column\n  :primary_key # specify the joining tables primary key column\n  :using       # used to specify a USING queries list of columns to join on\n```\n\nThe simplest example is a basic join:\n\n```elixir\n{:ok, result} =\n  db(:customer)\n  |\u003e join(:order)\n  |\u003e select\n  |\u003e Moebius.Db.run\n```\n\nFor multiple table joins you can specify the table that you want to join on:\n\n```elixir\n{:ok, result} =\n  db(:customer)\n  |\u003e join(:order, on: :customer)\n  |\u003e join(:item, on: :order)\n  |\u003e select\n  |\u003e Moebius.Db.run\n```\n\n## Transactions\n\nTransactions are facilitated by using a callback that has a `pid` on it, which you'll need to pass along to each query you want to be part of the transaction. The last execution will be returned. If there's an error, an `{:error, message}` will be returned instead and a `ROLLBACK` fired on the transaction. No need to `COMMIT`, it happens automatically:\n\n```elixir\n{:ok, result} = transaction fn(pid) -\u003e\n  new_user =\n    db(:users)\n    |\u003e insert(pid, email: \"frodo@test.com\")\n    |\u003e Moebius.Db.run(pid)\n\n  with(:logs)\n    |\u003e insert(pid, user_id: new_user.id, log: \"Hi Frodo\")\n    |\u003e Moebius.Db.run(pid)\n  new_user\nend\n```\n\nIf you're having any kind of trouble with transactions, I highly recommend you move to a SQL file or a function, which we also support. Abstractions are here to help you, but if we're in your way, by all means shove us (gently) aside.\n\n## Aggregates\n\nAggregates are built with a functional approach in mind. This might seem a bit odd, but when working with any relational database, it's a good idea to think about gathering your data, grouping it, and reducing it. That's what you're doing whenever you run aggregation queries.\n\nSo, to that end, we have:\n\n```elixir\n{:ok, sum} =\n  db(:products)\n  |\u003e map(\"id \u003e 1\")\n  |\u003e group(:sku)\n  |\u003e reduce(:sum, :id)\n  |\u003e Moebius.Db.run\n```\n\nThis might be a bit verbose, but it's also very very clear to whomever is reading it after you move on. You can work with any aggregate function in PostgreSQL this way (AVG, MIN, MAX, etc).\n\nThe interface is designed with *routine* aggregation in mind - meaning that there are some pretty complex things you can do with PostgreSQL queries. If you like doing that, I fully suggest you flex our SQL File functionality and write it out there - or create yourself a cool function and call it with our Function interface.\n\n## Functions\n\nPostgreSQL allows you to do so much, especially with functions. If you want to encapsulate a good time, you can execute it with Moebius:\n\n```elixir\n{:ok, party} = function(:good_time, [me, you]) |\u003e Moebius.Db.run\n```\n\nYou get the idea. If your function only returns one thing, you can specify you don't want an array back:\n\n```elixir\n{:ok, no_party} = function(:bad_time, :single [me]) |\u003e Moebius.Db.run\n```\n\n## Test\n\nYou'll need a local postgres instance running.\n\n```bash\nMIX_ENV=test mix moebius.setup\nMIX_ENV=test mix test\n```\n\n## Help?\n\nI would love to have your help! I do ask that if you do find a bug, please add a test to your PR that shows the bug and how it was fixed.\n\nThanks!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobconery%2Fmoebius","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobconery%2Fmoebius","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobconery%2Fmoebius/lists"}