{"id":17507963,"url":"https://github.com/woylie/flop","last_synced_at":"2025-05-14T01:03:10.708Z","repository":{"id":35169159,"uuid":"216013189","full_name":"woylie/flop","owner":"woylie","description":"Filtering, ordering and pagination for Ecto","archived":false,"fork":false,"pushed_at":"2025-03-29T07:13:11.000Z","size":1649,"stargazers_count":732,"open_issues_count":28,"forks_count":38,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T14:01:30.371Z","etag":null,"topics":["ecto","elixir","filter","ordering","pagination"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/woylie.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"woylie"}},"created_at":"2019-10-18T11:54:40.000Z","updated_at":"2025-04-04T17:19:06.000Z","dependencies_parsed_at":"2024-01-05T07:33:28.349Z","dependency_job_id":"c01f22cd-4fc6-42e3-a26f-7ab3c21fc1d4","html_url":"https://github.com/woylie/flop","commit_stats":{"total_commits":940,"total_committers":21,"mean_commits":44.76190476190476,"dds":0.2468085106382979,"last_synced_commit":"58f5d55ac8e80b8f847ff9a4f36de95d25bc7270"},"previous_names":[],"tags_count":48,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woylie%2Fflop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woylie%2Fflop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woylie%2Fflop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/woylie%2Fflop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/woylie","download_url":"https://codeload.github.com/woylie/flop/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248173816,"owners_count":21059594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ecto","elixir","filter","ordering","pagination"],"created_at":"2024-10-20T04:01:11.325Z","updated_at":"2025-04-12T15:32:12.203Z","avatar_url":"https://github.com/woylie.png","language":"Elixir","funding_links":["https://github.com/sponsors/woylie"],"categories":["Elixir"],"sub_categories":[],"readme":"# Flop\n\n![CI](https://github.com/woylie/flop/workflows/CI/badge.svg) [![Hex](https://img.shields.io/hexpm/v/flop)](https://hex.pm/packages/flop) [![codecov](https://codecov.io/gh/woylie/flop/branch/main/graph/badge.svg?token=32BSY8O2LI)](https://codecov.io/gh/woylie/flop)\n\nFlop is an Elixir library designed to easily apply filtering, ordering, and\npagination to your Ecto queries.\n\n## Features\n\n- **Offset-based pagination:** Allows pagination through `offset`/`limit` or\n  `page`/`page_size` parameters.\n- **Cursor-based pagination:** Also known as key set pagination, provides a more\n  efficient alternative to offset-based pagination. Compatible with Relay\n  pagination arguments.\n- **Sorting:** Applies sort parameters on multiple fields in any direction.\n- **Filtering:** Allows complex data filtering using multiple conditions,\n  operators, and fields.\n- **Parameter validation:** Ensures the validity of provided parameters.\n- **Configurable filterable and sortable fields:** Only applies parameters to\n  the fields that were explicitly configured as filterable or sortable.\n- **Join fields:** Allows the application of pagination, sort, and filter\n  parameters on any named binding. Provides functions to help you to avoid\n  unnecessary join clauses.\n- **Compound fields:** Provides the ability to apply filter parameters on\n  multiple string fields, for example for a full name filter.\n- **Custom fields:** Provides an escape hatch for filters that Flop is not able\n  to build on its own.\n- **Relay connection formatter:** Formats the connection in Relay style,\n  providing edges, nodes, and page info.\n- **UI helpers and URL builders through\n  [Flop Phoenix](https://hex.pm/packages/flop_phoenix):** Pagination, sortable\n  tables and filter forms.\n\n## Installation\n\nTo get started, add `flop` to your dependencies list in your project's `mix.exs`\nfile:\n\n```elixir\ndef deps do\n  [\n    {:flop, \"~\u003e 0.26.1\"}\n  ]\nend\n```\n\nYou can also configure a default repo for Flop by adding the following line to\nyour config file:\n\n```elixir\nconfig :flop, repo: MyApp.Repo\n```\n\nInstead of configuring Flop globally, you can also use a configuration module.\nPlease refer to the Flop module documentation for more information.\n\n## Usage\n\n### Define sortable and filterable fields\n\nTo define sortable and filterable fields in your Ecto schema, you can derive\n`Flop.Schema`. This step is optional but highly recommended, particularly when\nthe parameters passed to Flop's functions are user-provided. Deriving\n`Flop.Schema` ensures that Flop applies filtering and sorting parameters only to\nthe fields you've explicitly configured.\n\n```elixir\ndefmodule MyApp.Pet do\n  use Ecto.Schema\n\n  @derive {\n    Flop.Schema,\n    filterable: [:name, :species],\n    sortable: [:name, :age, :species]\n  }\n\n  schema \"pets\" do\n    field :name, :string\n    field :age, :integer\n    field :species, :string\n    field :social_security_number, :string\n  end\nend\n```\n\nBesides sortable and filterable fields, `Flop.Schema` also allows the definition\nof join fields, compound fields, or custom fields. You can also set maximum or\ndefault limits, among other options. For a comprehensive list of available\noptions, check the `Flop.Schema` documentation.\n\n### Query data\n\nUse the `Flop.validate_and_run/3` or `Flop.validate_and_run!/3` function to both\nvalidate the parameters and fetch data from the database, and acquire pagination\nmetadata in one operation.\n\nHere is an example of how you might use this in your code:\n\n```elixir\ndefmodule MyApp.Pets do\n  import Ecto.Query, warn: false\n\n  alias Ecto.Changeset\n  alias MyApp.{Pet, Repo}\n\n  @spec list_pets(map) ::\n          {:ok, {[Pet.t()], Flop.Meta.t()}} | {:error, Flop.Meta.t()}\n  def list_pets(params \\\\ %{}) do\n    Flop.validate_and_run(Pet, params, for: Pet)\n  end\nend\n```\n\nThe `for` option sets the Ecto schema for which you derived `Flop.Schema`. If\nyou haven't derived `Flop.Schema` as described above, this option can be\nomitted. However, this is not recommended unless all parameters are generated\ninternally and are guaranteed to be safe.\n\nOn success, `Flop.validate_and_run/3` returns an `:ok` tuple. The second element\nof this tuple is another tuple containing the fetched data and metadata.\n\n```elixir\n{:ok, {[%Pet{}], %Flop.Meta{}}}\n```\n\nYou can learn more about the `Flop.Meta` struct in the\n[module documentation](https://hexdocs.pm/flop/Flop.Meta.html).\n\nAlternatively, you may separate parameter validation and data fetching into\ndifferent steps using the Flop.validate/2, Flop.validate!/2, and Flop.run/3\nfunctions. This allows you to manipulate the validated parameters, to modify the\nquery depending on the parameters, or to move the parameter validation to a\ndifferent layer of your application.\n\n```elixir\nwith {:ok, flop} \u003c- Flop.validate(params, for: Pet) do\n  Flop.run(Pet, flop, for: Pet)\nend\n```\n\nThe aforementioned functions internally call the lower-level functions\n`Flop.all/3`, `Flop.meta/3`, and `Flop.count/3`. If you have advanced\nrequirements, you might prefer to use these functions directly. However, it's\nimportant to note that these lower-level functions do not validate the\nparameters. If parameters are generated based on user input, they should always\nbe validated first using `Flop.validate/2` or `Flop.validate!/2` to ensure safe\nexecution.\n\nThe examples above assume that you configured a default repo. However, you can\nalso pass the repo directly to the functions:\n\n```elixir\nFlop.validate_and_run(Pet, flop, repo: MyApp.Repo)\nFlop.all(Pet, flop, repo: MyApp.Repo)\nFlop.meta(Pet, flop, repo: MyApp.Repo)\n```\n\nFor more detailed information, refer the\n[documentation](https://hexdocs.pm/flop/readme.html).\n\n## Parameter format\n\nThe Flop library requires parameters to be provided in a specific format as a\nmap. This map can be translated into a URL query parameter string, typically\nfor use in a web framework like Phoenix.\n\n### Pagination\n\n#### Offset / limit\n\nYou can specify an offset to start from and a limit to the number of results.\n\n```elixir\n%{offset: 20, limit: 10}\n```\n\nThis translates to the following query parameter string:\n\n```html\n?offset=20\u0026limit=10\n```\n\n#### Page / page size\n\nYou can specify the page number and the size of each page.\n\n```elixir\n%{page: 2, page_size: 10}\n```\n\nThis translates to the following query parameter string:\n\n```html\n?page=2\u0026page_size=10\n```\n\n#### Cursor\n\nYou can fetch a specific number of results before or after a given cursor.\n\n```elixir\n%{first: 10, after: \"g3QAAAABZAACaWRiAAACDg==\"}\n%{last: 10, before: \"g3QAAAABZAACaWRiAAACDg==\"}\n```\n\nThese translate to the following query parameter strings:\n\n```html\n?first=10\u0026after=g3QAAAABZAACaWRiAAACDg==\n?last=10\u0026before=g3QAAAABZAACaWRiAAACDg==\n```\n\n### Ordering\n\nTo sort the results, specify fields to order by and the direction of sorting for\neach field.\n\n```elixir\n%{order_by: [:name, :age], order_directions: [:asc, :desc]}\n```\n\nThis translates to the following query parameter string:\n\n```html\n?order_by[]=name\u0026order_by[]=age\u0026order_directions[]=asc\u0026order_directions[]=desc\n```\n\n### Filters\n\nYou can filter the results by providing a field, an operator, and a value. The\noperator is optional and defaults to `==`. Multiple filters are combined with a\nlogical `AND`. At the moment, combining filters with `OR` is not supported.\n\n```elixir\n%{filters: [%{field: :name, op: :ilike_and, value: \"Jane\"}]}\n```\n\nThis translates to the following query parameter string:\n\n```html\n?filters[0][field]=name\u0026filters[0][op]=ilike_and\u0026filters[0][value]=Jane\n```\n\nRefer to the `Flop.Filter` documentation and `t:Flop.t/0` type documentation for\nmore details on using filters.\n\n## Internal parameters\n\nFlop is designed to manage parameters that come from the user side. While it is\npossible to alter those parameters and append extra filters upon receiving them,\nit is advisable to clearly differentiate parameters coming from outside and the\nparameters that your application adds internally.\n\nConsider the scenario where you need to scope a query based on the current user.\nIn this case, it is better to create a separate function that introduces the\nnecessary `WHERE` clauses:\n\n```elixir\ndef list_pets(%{} = params, %User{} = current_user) do\n  Pet\n  |\u003e scope(current_user)\n  |\u003e Flop.validate_and_run(params, for: Pet)\nend\n\ndefp scope(q, %User{role: :admin}), do: q\ndefp scope(q, %User{id: user_id}), do: where(q, user_id: ^user_id)\n```\n\nIf you need to add extra filters that are only used internally and aren't\nexposed to the user, you can pass them as a separate argument. This same\nargument can be used to override certain options depending on the context in\nwhich the function is called.\n\n```elixir\ndef list_pets(%{} = params, opts \\\\ [], %User{} = current_user) do\n  flop_opts =\n    opts\n    |\u003e Keyword.take([\n      :default_limit,\n      :default_pagination_type,\n      :pagination_types\n    ])\n    |\u003e Keyword.put(:for, Pet)\n\n  Pet\n  |\u003e scope(current_user)\n  |\u003e apply_filters(opts)\n  |\u003e Flop.validate_and_run(params, flop_opts)\nend\n\ndefp scope(q, %User{role: :admin}), do: q\ndefp scope(q, %User{id: user_id}), do: where(q, user_id: ^user_id)\n\ndefp apply_filters(q, opts) do\n  Enum.reduce(opts, q, fn\n    {:last_health_check, dt}, q -\u003e where(q, [p], p.last_health_check \u003c ^dt)\n    {:reminder_service, bool}, q -\u003e where(q, [p], p.reminder_service == ^bool)\n    _, q -\u003e q\n  end)\nend\n```\n\nWith this approach, you maintain a clean separation between user-driven\nparameters and system-driven parameters, leading to more maintainable and less\nerror-prone code.\n\n## Relay and Absinthe\n\nThe `Flop.Relay` module is useful if you are using\n[absinthe](https://hex.pm/packages/absinthe) with\n[absinthe_relay](https://hex.pm/packages/absinthe_relay), or if you simply need\nto adhere to the Relay cursor specification. This module provides functions that\nhelp transform query responses into a format compatible with Relay.\n\nConsider the scenario where you have defined node objects for owners and pets,\nalong with a connection field for pets on the owner node object.\n\n```elixir\nnode object(:owner) do\n  field :name, non_null(:string)\n  field :email, non_null(:string)\n\n  connection field :pets, node_type: :pet do\n    resolve \u0026MyAppWeb.Resolvers.Pet.list_pets/2\n  end\nend\n\nnode object(:pet) do\n  field :name, non_null(:string)\n  field :age, non_null(:integer)\n  field :species, non_null(:string)\nend\n\nconnection(node_type: :pet)\n```\n\nAbsinthe Relay will establish the arguments `after`, `before`, `first` and\n`last` on the `pets` field. These argument names align with those used by Flop, facilitating their application.\n\nNext, we'll define a `list_pets_by_owner/2` function in the Pets context.\n\n```elixir\ndefmodule MyApp.Pets do\n  import Ecto.Query\n\n  alias MyApp.{Owner, Pet, Repo}\n\n  @spec list_pets_by_owner(Owner.t(), map) ::\n          {:ok, {[Pet.t()], Flop.Meta.t()}} | {:error, Flop.Meta.t()}\n  def list_pets_by_owner(%Owner{id: owner_id}, params \\\\ %{}) do\n    Pet\n    |\u003e where(owner_id: ^owner_id)\n    |\u003e Flop.validate_and_run(params, for: Pet)\n  end\nend\n```\n\nNow, within your resolver, you merely need to invoke the function and call\n`Flop.Relay.connection_from_result/1`, which transforms the result into a tuple\ncomposed of the edges and the page_info, as required by `absinthe_relay`.\n\n```elixir\ndefmodule MyAppWeb.Resolvers.Pet do\n  alias MyApp.{Owner, Pet}\n\n  def list_pets(args, %{source: %Owner{} = owner} = resolution) do\n    with {:ok, result} \u003c- Pets.list_pets_by_owner(owner, args) do\n      {:ok, Flop.Relay.connection_from_result(result)}\n    end\n  end\nend\n```\n\nIn case you want to introduce additional filter arguments, you can employ\n`Flop.nest_filters/3` to convert simple filter arguments into Flop filters,\nwithout necessitating API users to understand the Flop filter format.\n\nLet's add `name` and `species` filter arguments to the `pets` connection field.\n\n```elixir\nnode object(:owner) do\n  field :name, non_null(:string)\n  field :email, non_null(:string)\n\n  connection field :pets, node_type: :pet do\n    arg :name, :string\n    arg :species, :string\n\n    resolve \u0026MyAppWeb.Resolvers.Pet.list_pets/2\n  end\nend\n```\n\nAssuming that these fields have been already configured as filterable with\n`Flop.Schema`, we can use `Flop.nest_filters/3` to take the filter arguments and\ntransform them into a list of Flop filters.\n\n```elixir\ndefmodule MyAppWeb.Resolvers.Pet do\n  alias MyApp.{Owner, Pet}\n\n  def list_pets(args, %{source: %Owner{} = owner} = resolution) do\n    args = nest_filters(args, [:name, :species])\n\n    with {:ok, result} \u003c- Pets.list_pets_by_owner(owner, args) do\n      {:ok, Flop.Relay.connection_from_result(result)}\n    end\n  end\nend\n```\n\n`Flop.nest_filters/3` uses the equality operator `:==` by default.\nYou can override the default operator per field.\n\n```elixir\nargs = nest_filters(args, [:name, :species], operators: %{name: :ilike_and})\n```\n\n## Flop Phoenix\n\n[Flop Phoenix](https://hex.pm/packages/flop_phoenix) is a companion library that\nprovides Phoenix components for pagination, sortable tables, and filter forms,\nusable with both Phoenix LiveView and in dead views. It also defines helper\nfunctions to build URLs with Flop query parameters.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwoylie%2Fflop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwoylie%2Fflop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwoylie%2Fflop/lists"}