{"id":13508631,"url":"https://github.com/chatgris/funnel","last_synced_at":"2026-02-20T16:01:24.906Z","repository":{"id":11440082,"uuid":"13896750","full_name":"chatgris/funnel","owner":"chatgris","description":"Streaming Elixir API built upon ElasticSearch's percolation.","archived":false,"fork":false,"pushed_at":"2016-06-28T15:13:07.000Z","size":285,"stargazers_count":32,"open_issues_count":0,"forks_count":4,"subscribers_count":31,"default_branch":"master","last_synced_at":"2026-01-14T07:33:17.073Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chatgris.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-10-27T06:12:56.000Z","updated_at":"2024-04-14T04:44:09.000Z","dependencies_parsed_at":"2022-07-17T06:16:03.921Z","dependency_job_id":null,"html_url":"https://github.com/chatgris/funnel","commit_stats":null,"previous_names":["af83/funnel"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/chatgris/funnel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatgris%2Ffunnel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatgris%2Ffunnel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatgris%2Ffunnel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatgris%2Ffunnel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chatgris","download_url":"https://codeload.github.com/chatgris/funnel/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatgris%2Ffunnel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29656589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-20T09:27:29.698Z","status":"ssl_error","status_checked_at":"2026-02-20T09:26:12.373Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T02:00:55.918Z","updated_at":"2026-02-20T16:01:24.830Z","avatar_url":"https://github.com/chatgris.png","language":"Elixir","funding_links":[],"categories":["Miscellaneous"],"sub_categories":[],"readme":"# Funnel\n\n[![Build Status](https://travis-ci.org/chatgris/funnel.png?branch=master)](https://travis-ci.org/chatgris/funnel)\n\nFunnel is for building Streaming APIs build upon ElasticSearch’s\n[percolation](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html).\n\nFunnel supports ElasticSearch \u003e= 1.1.\n\nFunnel allow to register users / devices, associates some queries to user.\n\nThe common usecase is to store a query from a user and notify this user when a\nnew document matching this query is available.\n\nCheck out this [example](https://github.com/AF83/funnel_http) for a complete http api using Funnel dsl.\n\n## Installing things\n\nYou can use Funnel in your projects with the following steps:\n\n1. Adding Funnel to your `mix.exs` dependencies:\n\n  ```elixir\n  def deps do\n    [\n      {:funnel, \"~\u003e 0.1\"}\n    ]\n  end\n  ```\n\n2. List the `:funnel` as your application dependencies:\n\n  ```elixir\n  def application do\n    [applications: [:funnel]]\n  end\n  ```\n\n3. Add this sample configuration to `config/config.exs`.\n\n  ```elixir\n  use Mix.Config\n\n  config :funnel,\n    es_host: \"http://localhost:9200\",\n    percolator_pool_size: 100,\n    percolator_pool_max_overflow: 1000\n  ```\n\n## Testing things\n\n``` shell\nmix test\n```\n\n## Doing things\n\n### Transport\n\nFunnel has this notion of `Funnel.Transport`. Anything implementing the\n`Funnel.Transport` protocol can be a transport.\n\nFor the [Plug](https://github.com/elixir-lang/plug), the protocol would looks\nlike:\n\n```elixir\ndefmodule EventStreamMessage do\n  @moduledoc \"\"\n  This module serialize a given id and body to a ServerSent Events message.\n  \"\"\n  def to_message(id, data) do\n    \"id:#{id}\\ndata: #{data}\\n\\n\"\n  end\nend\n\ndefimpl Funnel.Transport, for: Elixir.Plug.Conn do\n  import Plug.Conn\n\n  def write(conn, %{:id =\u003e id, :item =\u003e item}) do\n    chunk conn, EventStreamMessage.to_message(id, item)\n  end\nend\n```\n\n### Register\n\nA user, or a device, can register to funnel by using the `/register` endpoint.\nThis will return a token. This token must be used in all communications with the\nfunnel's API.\n\nThe first argument is a `transport`, the second one is optionnal, and allow to\nwrite onto the transport the message seen since the `last_id given`.\n\n```elixir\n{:ok, token} = Funnel.register(transport)\n{:ok, \"1a9dc09879374878bd7aab27c7be6bc7\"}\n\n{:ok, token} = Funnel.register(self, \"422f779c759244d4aad45ac94c83b7da\")\n{:ok, \"80cfd5a3e1324db8b076defec2ddc1b2\"}\n```\n\n### Creating Indexes\n\nThis example will create an empty index:\n\n```elixir\n{:ok, status_code, body} = Funnel.Index.create\n{:ok, 200, %{\"body\" =\u003e %{\"acknowledged\" =\u003e true}, \"index_id\" =\u003e \"e4680d88db914ed4854acb8a1a8f317d\"}}\n```\n\nThis example will create an index with specific settings:\n\n``` elixir\nsettings = '{\"settings\" : {\"number_of_shards\" : 1},\"mappings\" : {\"type1\" :{\"_source\" : { \"enabled\" : false },\"properties\" : {\"field1\" : { \"type\" :\"string\", \"index\" : \"not_analyzed\" }}}}}' |\u003e IO.iodata_to_binary\n{:ok, _status_code, body} = Funnel.Index.create(settings)\n{:ok, 200, %{\"body\" =\u003e %{\"acknowledged\" =\u003e true},\"index_id\" =\u003e \"3994bf6c03df412e8b1b05d4aca7a83c\"}}\n```\n\n\n### Deleting an index\n\n``` elixir\n{:ok, status_code, body} = Funnel.Index.destroy(\"3994bf6c03df412e8b1b05d4aca7a83c\")\n{:ok, 200, %{\"acknowledged\" =\u003e true}}\n```\n\n### Creating a query\n\n``` elixir\nquery = '{\"query\" : {\"match\" : {\"message\" : \"funnel\"}}}' |\u003e IO.iodata_to_binary\n{:ok, status_code, body} = Funnel.Query.create(index_id, token, query)\n{:ok, 201, %{\"body\" =\u003e %{\"_id\" =\u003e \"287eae87d5774f2a9d02f5a5dd66856d-0398f4e1a6a34ea4b7ede0c1b7f40f38\", \"_index\" =\u003e \"3994bf6c03df412e8b1b05d4aca7a83c_dev\", \"_type\" =\u003e \".percolator\", \"_version\" =\u003e 1, \"created\" =\u003e true}, \"index_id\" =\u003e \"3994bf6c03df412e8b1b05d4aca7a83c\", \"query_id\" =\u003e \"0398f4e1a6a34ea4b7ede0c1b7f40f38\"}}\n```\n\n### Updating a query\n\n``` elixir\n{:ok, status_code, body} = Funnel.Query.update(index_id, token, query_id, query)\n{:ok, 200, %{\"body\" =\u003e %{\"_id\" =\u003e \"287eae87d5774f2a9d02f5a5dd66856d-0398f4e1a6a34ea4b7ede0c1b7f40f38\", \"_index\" =\u003e \"3994bf6c03df412e8b1b05d4aca7a83c_dev\", \"_type\" =\u003e \".percolator\", \"_version\" =\u003e 2, \"created\" =\u003e false}, \"index_id\" =\u003e \"3994bf6c03df412e8b1b05d4aca7a83c\", \"query_id\" =\u003e \"0398f4e1a6a34ea4b7ede0c1b7f40f38\"}}\n```\n\n### Deleting a query\n\n``` elixir\n{:ok, status_code, body} = Funnel.Query.destroy(index_id, token, query_id)\n{:ok, 200, %{\"_id\" =\u003e \"287eae87d5774f2a9d02f5a5dd66856d-0398f4e1a6a34ea4b7ede0c1b7f40f38\", \"_index\" =\u003e \"3994bf6c03df412e8b1b05d4aca7a83c_dev\", \"_type\" =\u003e \".percolator\", \"_version\" =\u003e 3, \"found\" =\u003e true}}\n```\n\n### Finding queries\n\n``` elixir\n{:ok, status_code, body} = Funnel.Query.find(token)\n{:ok, 200, [%{\"query_id\" =\u003e \"4f122313862e494b8810f073c27cf43d\", \"index_id\" =\u003e \"b79d2e9ff8c949e08ba98c4d8c216547\", \"score\" =\u003e 1.0}]}\n\n{:ok, status_code, body} = Funnel.Query.find(token, %{index_id: index_id})\n{:ok, 200, [%{\"query_id\" =\u003e \"4f122313862e494b8810f073c27cf43d\", \"index_id\" =\u003e \"b79d2e9ff8c949e08ba98c4d8c216547\", \"score\" =\u003e 1.0}]}\n```\n\n### Submitting a document to the percolator\n\n``` elixir\nmessage = '{\"message\" : \"this new elasticsearch percolator feature is nice, borat style\"}' |\u003e IO.iodata_to_binary\nFunnel.percolate(index_id, message)\n{:ok}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchatgris%2Ffunnel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchatgris%2Ffunnel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchatgris%2Ffunnel/lists"}