{"id":15056454,"url":"https://github.com/exosite/ecql2","last_synced_at":"2025-07-25T06:08:07.247Z","repository":{"id":57492552,"uuid":"124202431","full_name":"exosite/ecql2","owner":"exosite","description":null,"archived":false,"fork":false,"pushed_at":"2020-10-22T15:39:26.000Z","size":1968,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-06-27T15:06:07.529Z","etag":null,"topics":["cassandra","cql","database","driver","erlang","mnesia"],"latest_commit_sha":null,"homepage":null,"language":"Io","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/exosite.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-07T08:15:53.000Z","updated_at":"2020-06-11T04:17:01.000Z","dependencies_parsed_at":"2022-08-28T11:51:33.639Z","dependency_job_id":null,"html_url":"https://github.com/exosite/ecql2","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/exosite/ecql2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exosite%2Fecql2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exosite%2Fecql2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exosite%2Fecql2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exosite%2Fecql2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/exosite","download_url":"https://codeload.github.com/exosite/ecql2/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/exosite%2Fecql2/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266963337,"owners_count":24013033,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cassandra","cql","database","driver","erlang","mnesia"],"created_at":"2024-09-24T21:51:38.564Z","updated_at":"2025-07-25T06:08:07.224Z","avatar_url":"https://github.com/exosite.png","language":"Io","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ecql - Cassandra driver for Erlang.\n\nA driver for cassandras native protocol. This driver uses cassandras stream\nfeature to create independent worker processes and avoid deadlocking.\nRather than exposing all features of cassandra ease of use and performance are\nthe main goals of this driver.\nAdditionally this driver includes a drop-in replacement for mnesia to make\na transition from mnesia to cassandra easier.\n\n\n## Usage\n\nTo play with the driver in interactive mode there is a shell script. This script will start Cassandra in a container and run connect erlang to.\n`$ ./interactive.sh`\n\nThen you can play around on the interactive erlang shell:\n```\nTable = \"CREATE TABLE IF NOT EXISTS key_value (key int PRIMARY KEY, value text);\".\necql:execute(Table).\n\n% These three lines:\necql:execute_async(\"INSERT INTO key_value (key, value) VALUES(?, ?)\", [1, \"a number\"]).\necql:execute_async(\"INSERT INTO key_value (key, value) VALUES(?, ?)\", [42, \"The answer to life, the universe and everything\"]).\necql:sync().\n\n% Do the same as this:\necql:execute_batch(\"INSERT INTO key_value (key, value) VALUES(?, ?)\", [\n  [1, \"a number\"],\n  [42, \"The answer to life, the universe and everything\"]\n]).\n```\n\nLet's insert some data and aggregate numbers\n\n```\nTable = \"CREATE TABLE IF NOT EXISTS key_value (key int PRIMARY KEY, value text);\".\necql:execute(Table).\n\nSequence = [[Idx, io_lib:format(\"This is ~p\", [Idx])] || Idx \u003c- lists:seq(1, 50000)].\necql:execute_batch(\"INSERT INTO key_value (key, value) VALUES(?, ?)\", Sequence).\n\n\n% Calculate the sum of all keys on the server side:\nSum = ecql:select_value(\"SELECT SUM(key) FROM key_value\").\n\n% Or the sum of products on the client side:\nProdSum = ecql:foldl(\n  fun([Idx], Acc) -\u003e\n    Acc + (Idx*Idx)\n  end,\n  0,\n  \"SELECT key FROM key_value\"\n).\n```\n\n## ecql:execute/1,2,3\n* `ecql:execute(Cql, Arguments = [], Consistency = local_quorum)`\n\nExecutes a CQL statement and returns the results if any, otherwise ok or an error message. execute/1,2,3 always blocks until the execution is done.\n\nReturns:\n* `ok` - for INSERT and CREATE statements\n* `{[ColumnName], [Row]}` - for select statements\n* `{error, Code, Reason}` - for any error. Code and Reason come from Cassandra error codes.\n\n\n```\n% INSERT\nok = ecql:execute(\"INSERT INTO key_value (key, value) (?, ?)\", [0, \"zero\"]).\n\n% SELECT\n{[key, value], [0, \u003c\u003c\"zero\"\u003e\u003e]} = ecql:execute(\"SELECT * FROM key_value WHERE key = 0\").\n\n% ERROR\n{error,8704,\"Undefined column name a\"} = ecql:execute(\"INSERT INTO key_value (a, b) (?, ?)\", [0, \"zero\"]).\n```\nCQL is any valid Cassandra Query Language statement. See here for CQL documentation: http://cassandra.apache.org/doc/cql3/CQL.html\n\n### Prepared statements\nAll statements with more than one argument are **automatically  prepared internally** and the prepared statements are  cached and reused for following caches.\n\n### Argument casting\nArguments are the arguments to be replaced in the CQL string. Arguments are being cast into the right type as much as possible. Most mentionable the Cassandra type **'text' accepts [iolists](http://www.erlangpatterns.org/iolist.html) is always returned as a erlang binary**\n\nExample:\n```\nok = ecql:execute(\n  \"INSERT INTO key_value (key, value) VALUES(?, ?)\",\n  [0, [\"deep iolists\", [\" can\", $l, $o, $o, $k], \" strange \", [2], \" erlang newcomers\"]]\n).\n\nBinaryString = ecql:select_value(\"SELECT value FROM key_value WHERE key = 0\").\nio:format(\"~s ~n\", [BinaryString]).\n```\n\n\nConsistency is the [Cassandra consistency level](https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlshConsistency.html). The valid atom values are:\n\n* `default` - Alias for local_quorum and default value when no consistency is provied.\n* `one` - Consistency level one.\n* `two` - Consistency level two.\n* `three` - Consistency level three.\n* `quorum` - Consistency level quorum.\n* `all` - Consistency level quorum.\n* `local_quorum` - Consistency level local quorum.\n* `each_quorum` - Consistency level each quorum.\n* `serial` - Consistency level serial.\n* `local_serial` - Consistency level local serial.\n* `local_one` - Consistency level local one.\n\n## ecql:select/1,2,3\n* `ecql:select(Cql, Arguments = [], Consistency = local_quorum)`\n\nAlias for `execute/1,2,3` - there are no behavioural differences.\n\n## ecql:execute_async/1,2,3\n* `ecql:execute_async(Cql, Arguments = [], Consistency = local_quorum)`\n\nReturns: `ok`\n\nSame as `ecql:execute/1,2,3` but asynchronous. It does not wait for the return value. Internally the number of running asynchronous calls is counted and capped to 100 per calling process. If the number exceeds this limit the calling process is blocked until at least one asynchronous query finishes.\n\n## ecql:sync/0\nWaits for all `execute_async` asynchronous queries of the current process to finish. This is useful when placed behind a bulk of queries.\n\n## ecql:batch/2,3,4\n* `execute_batch(Cql, ArgumentListList, Type = unlogged, Consistency = local_quorum)`\n\nType can be any atom of:\n* `unlogged`\n* `logged`\n\nExecutes a [cassandra batch statement](https://docs.datastax.com/en/cql/3.3/cql/cql_using/useBatch.html). The batch is limited to only a single CQL statement but with multiple inputs. This can be used to delete, update or insert many entries at once.\n\nExample:\n```\nArgumentListList = [[Id] || Id \u003c- lists:seq(1, 50000)].\necql:execute_batch(\"DELETE FROM key_value WHERE key = ?\", ArgumentListList).\n```\n\n## ecql:select_value/1,2,3\n* `select_value(Cql, Arguments = [], Consistency = local_quorum)`\n\nReturns: `Value`\n\nSugar around `execute/1,2,3` for select statements. It returns only a single value from the query. Errors in the query or `ok` results from `execute/1,2,3` result in a raised exception.\n\nExample:\n\n```\n62\u003e ecql:select_value(\"SELECT value FROM key_value WHERE key = 0\").\n\u003c\u003c\"deep iolists canlook strange \"...\u003e\u003e\n\n63\u003e ecql:select_value(\"INSERT INTO key_value (key, value) VALUES(-1, 'la')\").\n** exception error: no match of right hand side value ok\n     in function  ecql:select_column/4 (src/ecql.erl, line 467)\n     in call from ecql:select_value/3 (src/ecql.erl, line 420)\n```\n\n## ecql:select_column/1,2,3\n* `select_column(Cql, ColumnNumber = 1, Arguments = [], Consistency = local_quorum)`\n\nReturns: `[Value]`\n\nSugar around `execute/1,2,3` for select statements. It returns only a list with a single value from every row in the query. This is useful when querying for just a single field in a table. Errors in the query or `ok` results from `execute/1,2,3` result in a raised exception.\n\nExample:\n\n```\n68\u003e ecql:select_column(\"SELECT key FROM key_value LIMIT 100\").\n[4317,35262,25269,39433,3372,37032,48451,14340,18417,1584,\n 7034,24299,13909,40239,28386,47076,9892,41114,34323,35243,\n 43690,16096,26713,19221,28459,9640,23912,46348,40244|...]\n```\n\n\n# Streaming functions\nThe driver does use Cassandra result pagination automatically internally and exposes this feature to allow stream processing of large amounts of data without loading them into memory with the following functions:\n\n## ecql:foldl/3,4,5\n* `ecql:foldl(Fun, Acc, Cql, Arguments = [], Consistency = local_quorum)`\n\n`Fun = fun([Column1, Column2], Acc) -\u003e NewAcc`\n\nReturns: `NewAcc`\n\nExtremely scalable method to process huge result sets. Like [lists:foldl](https://www.proctor-it.com/erlang-thursday-lists-foldl-3-and-lists-foldr-3/) the given function `Fun` is executed for each row of the query result. The benefit of using this function in the driver directly instead of loading the whole list first with `ecql:execute/1,2,3` is two fold:\n\n1. The execution can start earlier and finish faster. Cassandra will send the first 1000 rows of a query as soon as they are ready and then continue sending incrementally.\n1. Huge cassandra tables can be processed this way without timeouts or running out of memory on the client side.\n\nExample:\n```\nProdSum = ecql:foldl(\n  fun([Idx], Acc) -\u003e\n    Acc + (Idx*Idx)\n  end,\n  0,\n  \"SELECT key FROM key_value\"\n).\n```\n\n## ecql:foldl_page/3,4,5\n* `ecql:foldl_page(Fun, Acc, Cql, Arguments = [], Consistency = local_quorum)`\n\n`Fun = fun([ColumnName] ,[Row], Acc) -\u003e NewAcc`\n\nLike `ecql:foldl/3,4,5` but passes in a whole page of ~1000 rows (size is not guaranteed) as well as the result header columns. Might be useful when trying to forward larger blocks of data e.g. to sockets or to nifs for further processing.\n\nExample:\n```\nProdSum = ecql:foldl_page(\n  fun([key], Rows, Acc) -\u003e\n    lists:foldl(fun([Idx], A) -\u003e\n      A + (Idx*Idx)\n    end, Acc, Rows)\n  end,\n  0,\n  \"SELECT key FROM key_value\"\n).\n```\n\n## ecql:foreach/2,3,4\n* `ecql:foreach(Fun, Cql, Arguments = [], Consistency = local_quorum)`\n\nReturns: `ok`\n\nExtremely scalable way to iterate all rows of a huge query result. Similiar to `lists:foreach` executes the given Fun for each row of the result.\n\nExample:\n```\necql:foreach(fun([Key, Value]) -\u003e\n  io:format(\"key: '~p' =\u003e '~p'~n\", [Key, Value])\nend, \"SELECT * FROM key_value LIMIT 10\").\n```\n\n## ecql:select_firstpage/1,2,3\n* `ecql:select_firstpage(Cql, Arguments = [], Consistency = local_quorum)`\n\nReturns: `{Page, Continuation}`\n\n`Page = {[ColumnName], Rows}`\n`Continuation = $end_of_table | tuple`\n\nLow level function that is used to implement the other high level streaming functions. The the call returns a first page of results and a continuation. The continuation can be used to fetch further pages using `ecql:select_nextpage/1`.\n\nErlang shell example:\n```\n% Getting the first page\n{Page1, Cont1} = ecql:select_firstpage(\"SELECT * FROM key_value\").\n\ncase Cont1 of\n  '$end_of_table' -\u003e\n    \"one page!\";\n  _ -\u003e\n    {Page2, Cont2} = ecql:select_nextpage(Cont1),\n    case Cont2 of\n      '$end_of_table' -\u003e\n        \"two pages!\";\n      _ -\u003e\n        {Page3, Cont3} = ecql:select_nextpage(Cont2),\n        \"three or more pages!\"\n    end\nend.\n% and so on\n\n```\n\n## ecql:select_nextpage/1,2,3\n* `ecql:select_nextpage(Continuation)`\n\nReturns: `{Page, Continuation}`\n\n`Page = {[ColumnName], Rows}`\n`Continuation = $end_of_table | tuple`\n\nThis function can only be used in conjunction with `ecql:select_firstpage/1,2,3`. Please see it's documentation and example.\n\n# Additional Functions\n\n## term_to_bin/1 \u0026 bin_to_term/1\n* `Binary = ecql:term_to_bin(Term)`\n* `Term = ecql:bin_to_term(Binary)`\n\nThese functions are usefull to store arbitary terms in Cassandra. term_to_bin is using compression and is deterministic - so it can also be used for equals queries.\n\nExample:\n```\nTable = \"CREATE TABLE IF NOT EXISTS term_kv (key blob PRIMARY KEY, value blob);\".\necql:execute(Table).\n\necql:execute_batch(\"INSERT INTO term_kv (key, value) VALUES(?, ?)\", [\n  [ecql:term_to_bin({die_hard, 1}), ecql:term_to_bin(#{good_guy =\u003e john, bad_guy =\u003e hans})],\n  [ecql:term_to_bin({die_hard, 2}), ecql:term_to_bin(#{good_guy =\u003e john, bad_guy =\u003e ramon})],\n  [ecql:term_to_bin({die_hard, 3}), ecql:term_to_bin(#{good_guy =\u003e john, bad_guy =\u003e simon})],\n  [ecql:term_to_bin({die_hard, 4}), ecql:term_to_bin(#{good_guy =\u003e john, bad_guy =\u003e thomas})],\n  [ecql:term_to_bin({die_hard, 5}), ecql:term_to_bin(#{good_guy =\u003e john, bad_guy =\u003e yuri})]\n]).\n\n% Using tuple {die_hard, 4} to search\n#{bad_guy := Name} = ecql:bin_to_term(\n  ecql:select_value(\"SELECT value FROM term_kv WHERE key = ?\",[\n    ecql:term_to_bin({die_hard, 4})\n  ])\n).\n\nio:format(\"The bad guy is: ~p~n\", [Name]).\n```\n\n## ecql:release/0\n* `ecql:release()`\n\nReturns: `ok`\n\nReleases the currently attached stream from the process. Usefull when it's known that this process is not going to do queries in the near future.\n\nBackground: Streams are currently a limited resource in the driver whose number can be configured and defaults to `100` per Cassandra host. Each stream is an independent Erlang process and can operate isolated and asynchronously from all other stream processes. The communication between a stream process and the client making queries is direct with no mediator after the initial handshake. The stream will be automatically released and returned to the pool once the client process terminates.\n\nAll of this usually happens in the background with the initial link between a client process and a stream process being established on the first query. In some cases it might be useful though to release the stream process even though the client process has not yet terminated.\n\n# Configuration Parameters\n\nThese can be configured from the application configuration file following OTP standards or can be set before starting the driver app with `application:set_env(ecql, user, \"rompompel\")`\n\n| Setting                | default            | Description                     |\n| ---------------------- |:------------------:| -------------------------------:|\n| cache_size             | `1000000`          | ecql_mnesia: Number of bytes used for the ecql_mnesia write-through cache |\n| cluster_module         | `erlang`           | ecql_mnesia: Cluster module used for `nodes()` call to find other nodes |\n| log                    | `disabled`         | what?                           |\n| user                   | `\"cassandra\"`      | Cassandra Auth Username         |\n| pass                   | `\"cassandra\"`      | Cassandra Auth Password         |\n| hosts                  | `[{{127,0,0,1},9042}]` | List of hosts to connect to |\n| keyspace               | `\"ecql\"`           | Keyspace to be used by all connections \u0026 streams |\n| replication_strategy   | `\"SimpleStrategy\"` | Replication strategy to be used if keyspace does not yet exists and needs to be created. |\n| replication_factor     | `2`                | Replication factor to be used if keyspace does not yet exists and needs to be created. |\n| streams_per_connection | `100`              | Streams to open to each Server in the cluster |\n\n# Todo List\n\n* Make streams_per_connection dynamic!!\n* Document performance numbers vs. other drivers\n* SSL\n* Replace encode function with nif for performance.\n* Read configuration from env\n* Token aware routing\n* Support multiple keyspaces \u0026 clusters\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexosite%2Fecql2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexosite%2Fecql2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexosite%2Fecql2/lists"}