{"id":13471336,"url":"https://github.com/joergen7/cuneiform","last_synced_at":"2025-05-16T08:04:04.757Z","repository":{"id":17220611,"uuid":"19989395","full_name":"joergen7/cuneiform","owner":"joergen7","description":"Cuneiform distributed programming language","archived":false,"fork":false,"pushed_at":"2025-02-07T12:27:28.000Z","size":2430,"stargazers_count":242,"open_issues_count":8,"forks_count":18,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-05-15T19:03:07.962Z","etag":null,"topics":["bioinformatics","distributed-computing","erlang","functional-programming","machine-learning","otp","parallelization","workflow","workflow-engine"],"latest_commit_sha":null,"homepage":"https://cuneiform-lang.org/","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joergen7.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-05-20T16:58:18.000Z","updated_at":"2025-04-30T05:31:32.000Z","dependencies_parsed_at":"2022-08-31T20:31:14.054Z","dependency_job_id":"87a9365f-47bc-4f06-8d5c-383e2608d378","html_url":"https://github.com/joergen7/cuneiform","commit_stats":{"total_commits":519,"total_committers":8,"mean_commits":64.875,"dds":0.08670520231213874,"last_synced_commit":"f9b5e1c282a6e96c2fee08bd1a39121a8cd28bfe"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joergen7%2Fcuneiform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joergen7%2Fcuneiform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joergen7%2Fcuneiform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joergen7%2Fcuneiform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joergen7","download_url":"https://codeload.github.com/joergen7/cuneiform/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254493379,"owners_count":22080126,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","distributed-computing","erlang","functional-programming","machine-learning","otp","parallelization","workflow","workflow-engine"],"created_at":"2024-07-31T16:00:43.246Z","updated_at":"2025-05-16T08:04:04.735Z","avatar_url":"https://github.com/joergen7.png","language":"Erlang","funding_links":[],"categories":["Erlang"],"sub_categories":[],"readme":"![Cuneiform: Data analysis open and general](priv/cuneiform_title.jpg)\n\n[![hex.pm](https://img.shields.io/hexpm/v/cuneiform.svg?style=flat-square)](https://hex.pm/packages/cuneiform)\n\nCuneiform is a large-scale data analysis functional programming language. It is *open* because it easily integrates foreign tools and libraries, e.g., Python libraries or command line tools. It is *general* because it has the expressive power of a functional programming language while using the independence of sub-expressions to automatically parallelize programs. Cuneiform uses distributed Erlang to scalably run in cluster and cloud environments.\n\n[cuneiform-lang.org](https://www.cuneiform-lang.org/)\n\n## Usage\n\n### Compiling\n\nHaving rebar3 available on your system, compile the project by entering\n\n    rebar3 escriptize\n\n### Displaying Cuneiform Help\n\nCompiling the Cuneiform client using `rebar3 escriptize` creates an Erlang script file `_build/default/bin/cfl` which allows starting Cuneiform via the command line. \n\nTo display a help text enter\n\n    _build/default/bin/cfl --help\n\n\nThis will show the command line synopsis, which looks like the following:\n\n    Usage: cfl [-v] [-h] [-n \u003cn_wrk\u003e] [-w \u003cwrk_dir\u003e] [-r \u003crepo_dir\u003e]\n               [-d \u003cdata_dir\u003e]\n\n      -v, --version   Show cf_worker version.\n      -h, --help      Show command line options.\n      -n, --n_wrk     Number of worker processes to start. 0 means auto-detect \n                      available processors.\n      -w, --wrk_dir   Working directory in which workers store temporary files.\n      -r, --repo_dir  Repository directory for intermediate and output data.\n      -d, --data_dir  Data directory where input data is located.\n\nThis script is self-contained and can be moved around to, e.g., `~/bin/cfl`. From here on we assume that the `cfl` script is accessible in your system path and that we can start it by just entering `cfl` instead of `_build/default/bin/cfl`.\n\n\n#### Starting an Interactive Shell\n\nYou can start a shell and program Cuneiform interactively by starting it without any command line parameters like so:\n\n    cfl\n\nThis will open a shell giving the following initial output, along with a number of status messages:\n\n               @@WB      Cuneiform\n              @@E_____\n         _g@@@@@WWWWWWL  Type help for usage info\n       g@@#*`3@B              quit to exit shell\n      @@P    3@B\n      @N____ 3@B         http://www.cuneiform-lang.org\n      \"W@@@WF3@B         Jorgen Brandt\n\n    1\u003e\n\nNote that starting Cuneiform like that will create a local instance of a Cuneiform that entails the scheduler service, a client service, and as many worker services as CPUs were detected on the host machine. To set up a distributed Cuneiform system, these services need to be started separately on multiple hosts as needed.\n\n\n#### Running a Cuneiform Script\n\nAlternatively, Cuneiform can be started by giving it a source file which will only output the final result of the computation. If your Cuneiform script is stored in `my_script.cfl` start it by entering\n\n    cfl my_script.cfl\n\n## Examples\n\nA collection of self-contained Cuneiform examples is available under [joergen7/cuneiform-examples](https://github.com/joergen7/cuneiform-examples).\n\n### Variable assignment\n\nYou can assign a value to a variable and retrieve a variable's content like so:\n\n    let x : Str =\n      \"foo\";\n\n    x;\n\nIn the first line we assign the value `\"foo\"` to a variable named `x` declaring its type to be `Str`. In the last line we query the variable `x`.\n\n### Booleans and Conditions\n\nWe can branch execution based on conditions using conditional statements. Conditionals are expressions.\n\n    let x : Str =\n      if true\n      then\n        \"bla\"\n      else\n        \"blub\"\n      end;\n\n    x;\n\nThe above command the conditional binding the string `\"bla\"` to the variable `x`. Then, we query the variable.\n\n### Lists\n\nWe can construct list literals by enumerating their elements in square brackets and declaring the type of the list elements.\n\n    let xs : [Bool] =\n      [true, false, true, true : Bool];\n\n    xs;\n\nHere, we define the list `xs` whose elements are of type `Bool` giving four Boolean values of which only the second is `false`.\n\n### Records and Pattern Matching\n\nA record is a collection of fields that can be accessed via their labels. Literal records can be constructed like so:\n\n    let r : \u003ca : Str, b : Bool\u003e =\n      \u003ca = \"blub\", b = false\u003e;\n\n    ( r|a );\n\nWe define a record `r` with two fields `a` and `b`, of types `Str` and `Bool` respectively. The field associated with `a` gets the value `\"blub\"` while the field associated with `b` gets the value `false`. In the last line we access the `a` field of the record `r`.\n\nAlternatively, we can access record fields via pattern matching:\n\n    let \u003ca = z : Str\u003e = r;\n    z;\n\nIn the first line we associate the variable `z` with the field `a` of record `r`. In the second line we query the content of `z`.\n\n### Native Function Definition\n\nDefining native functions in Cuneiform is done by giving the function name, its signature, and a body expression in curly braces:\n\n    def identity( x : Str ) -\u003e Str {\n      x\n    }\n\n    identity( x = \"bar\" );\n\nIn the first line we define the function `identity` which consumes an argument `x` of type `Str` and produces a return value of type `Str`. In the second line, the body expression is just the argument `x`. In the last line we call the function binding the argument `x` to the value `\"bar\"`.\n\n### Foreign Function Definition\n\nDefining foreign functions is done by giving the function name, its signature, the foreign language name, and the function body in mickey-mouse-eared curly braces.\n\n    def greet( person : Str ) -\u003e \u003cout : Str\u003e in Bash *{\n      out=\"Hello $person\"\n    }*\n\n    greet( person = \"Peter\" );\n\nThe first line defines a foreign function `greet` taking one argument `person` of type `Str` and returning a tuple with a single field `out` of type `Str`. The foreign function body is given in Bash code. In the last line we call the foreign function, binding the argument `person` to the string value `\"Peter\"`.\n\n### Iterating over Lists using For\n\nTo perform an operation on each element of a list, one can iterate using for:\n\n    let xs : [Bool] =\n      [true, false, true, true : Bool];\n\n    for x : Bool \u003c- xs do\n      not x\n      : Bool\n    end;\n\nHere, we define a list of four Booleans and negate each element.\n\n### Aggregating Lists using Fold\n\nWe can aggregate over lists using fold:\n\n    def add( a : Str, b : Str ) -\u003e \u003cc : Str\u003e in Python *{\n      c = int( a )+int( b )\n    }*\n\n    let xs : [Str] = [1, 2, 3 : Str];\n\n    let sum : Str =\n      fold acc : Str = 0, x : Str \u003c- xs do\n        ( add( a = acc, b = x )|c )\n      end;\n\n    sum;\n\nHere, we first define the function `add` which lets us add two numbers in Python and then the string list `xs` containing the numbers from one to three. We aggregate the sum of the numbers in `xs` and store it the result in the variable `sum`. Lastly, we query the `sum` variable.\n\n## System Requirements\n\n- [Erlang](https://www.erlang.org) OTP 19.0 or higher\n- [Rebar3](https://www.rebar3.org) 3.0.0 or higher\n\n## Resources\n\n- [cuneiform-lang.org](https://www.cuneiform-lang.org/). Official website of the Cuneiform programming language.\n- [joergen7/cuneiform-examples](https://github.com/joergen7/cuneiform-examples). Collection of small, self-contained Cuneiform code examples.\n- [joergen7/cre](https://github.com/joergen7/cre). A common runtime environment (CRE) for distributed workflow languages.\n- [joergen7/cf_client](https://github.com/joergen7/cf_client). A Cuneiform client implementation.\n- [joergen7/cf_worker](https://github.com/joergen7/cf_worker). A Cuneiform worker implementation.\n\n## License\n\n[Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoergen7%2Fcuneiform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoergen7%2Fcuneiform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoergen7%2Fcuneiform/lists"}