{"id":20456240,"url":"https://github.com/semperos/ari","last_synced_at":"2025-05-08T21:31:43.970Z","repository":{"id":249651732,"uuid":"800715218","full_name":"semperos/ari","owner":"semperos","description":"Ari: Array Relational Interactive Programming","archived":false,"fork":false,"pushed_at":"2024-11-11T17:00:39.000Z","size":718,"stargazers_count":2,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-11T17:35:36.877Z","etag":null,"topics":["array-language","array-programming","duckdb","go-lang","go-language","goal-language","sql"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/semperos/ari","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semperos.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-14T21:23:57.000Z","updated_at":"2024-11-11T17:00:40.000Z","dependencies_parsed_at":"2024-08-27T03:57:06.128Z","dependency_job_id":"92ca0464-37ae-4e55-95e6-4ddaf7007ec0","html_url":"https://github.com/semperos/ari","commit_stats":null,"previous_names":["semperos/ari"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semperos%2Fari","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semperos%2Fari/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semperos%2Fari/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semperos%2Fari/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semperos","download_url":"https://codeload.github.com/semperos/ari/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224771868,"owners_count":17367221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array-language","array-programming","duckdb","go-lang","go-language","goal-language","sql"],"created_at":"2024-11-15T11:21:55.999Z","updated_at":"2024-11-15T11:21:56.469Z","avatar_url":"https://github.com/semperos.png","language":"Go","readme":"# Ari\n\nAri stands for **A**rray **R**elational **I**nteractive programming environment.\n\nAri is a set of extensions to the [Goal] programming language with an extensible CLI and dedicated SQL mode.\n\n## Installation\n\n```shell\ngo install github.com/semperos/ari/cmd/ari@latest\n```\n\nThen run `ari` for a REPL or `ari --help` to see CLI options.\n\n```\nari is an interactive environment for array + relational programming.\n\nIt embeds the Goal array programming language, with extensions for\nworking with SQL and HTTP APIs.\n\nUsage:\n  ari [flags] [source file]\n\nFlags:\n      --config string          ari configuration (default \"$HOME/.config/ari/ari-config.yaml\")\n      --cpu-profile            write CPU profile to file\n  -d, --database string        DuckDB database (default: in-memory)\n      --debug                  enable detailed debugging output on panic\n  -e, --execute string         string of Goal code to execute, last result not printed automatically\n  -h, --help                   help for ari\n      --history string         history of REPL entries (default \"$HOME/.config/ari/ari-history.txt\")\n  -l, --load stringArray       Goal source files to load on startup\n  -m, --mode string            language mode at startup (default \"goal\")\n  -f, --output-format string   evaluation output format (default \"goal\")\n  -p, --println                print final value of the script + newline\n  -r, --raw                    raw REPL w/out history or auto-complete\n  -v, --version                print version info and exit\n```\n\n## Features\n\n- [Goal] is the core language\n  - Goal's `lib` files are loaded by default, with prefix matching their file names (see [vendor-goal](vendor-goal) folder in this repo)\n- Extensible CLI REPL with:\n  - Auto-completion with documentation for:\n    - Built-in keywords\n    - Built-in syntax aliases (e.g., typing \"first\" and TAB will show `*` and `¿` in auto-complete results)\n    - User-defined globals\n  - Runtime configuration:\n    - Configure the REPL prompt by setting string values for the `ari.prompt` and `ari.nextprompt` (for multiline input) globals\n    - Replace default REPL printing by setting a function value for the `ari.print` global (function receives a single Goal value to print)\n    - Configure the output format with `--output-format` or using one of the `)output.` system commands at the REPL. Formats include CSV/TSV, JSON, Markdown, and LaTeX.\n  - `ari.p` is bound to the previous result (value from last evaluation at the REPL)\n  - Alternatively run `ari` with `--raw` for a simpler, raw REPL that lacks line editing, history, and auto-complete, but is better suited for interaction via an editor like (Neo)Vim, or if you prefer rlwrap or another line editor to the one that ships with ari.\n  - `help` based on Goal's, but allows adding help strings when used dyadically (e.g.,`\"sql.q\"help\"Run SQL query\"`)\n- New Goal functions:\n  - `http.` functions for HTTP requests using [Resty]\n  - `ratelimit.new` and `ratelimit.take` for rate limiting (leaky bucket algorithm) using [uber-go/ratelimit]\n  - `sql.` functions for SQL queries and commands\n  - Table-related `csv.tbl` and `json.tbl` to make Goal tables from the output of `csv` and `json` respectively\n  - `tt.` test framework\n  - `time.` functions for more extensive date/time handling\n  - `tui.` functions for basic terminal UI styling (colors, padding/margin, borders)\n- Dedicated SQL mode\n  - The ari CLI uses DuckDB, but the `github.com/semperos/ari` Go package doesn't directly depend on a specific SQL database driver, so you can BYODB.\n  - Activate with `)sql` for read-only, `)sql!` for read/write modes. Execute `)goal` to return to the default Goal mode.\n  - Auto-completion of SQL keywords\n  - Help entries for SQL keywords (shown during auto-complete, still WIP)\n  - Results of the last-run query/command set to the `sql.p` Goal global (for \"SQL previous\" and mirroring `ari.p`), so you can switch between `)sql` and `)goal` at the REPL to run queries via SQL and do data processing via Goal.\n\n## Projects Using Ari\n\n- [Shortcut API Client](https://github.com/semperos/sc-client-goal)\n\nI began building Ari to replicate the experience described in the [Background](#background) section of this README. That code is not publicly available at this time.\n\n## Examples\n\n### Minimal HTTP Server\n\n```\n\"localhost:1234\"http.serve{say x; ..[status:200;bodystring:\"OK\"]}\n```\n\n### Slack API\n\nUse Slack's API to get all messages in a channel for last 30 days. The first five lines set up the HTTP client specifically for Slack; the second block shows a code comment which can be evaluated to see a list of channels from which to pick the ID you need; the third block of code sets up the starting and ending timestamps with which to call the Slack API and a recursive function to fetch messages; the final line defines an `allmsgs` global with all messages from the given channel for the given time range, and then returns `\"ok\"` when it completes.\n\n```\nurl:\"https://slack.com/api/\"; aj:\"application/json\"; tk: 'env\"SLACK_USER_TOKEN\"\nhd:..[Accept:aj;\"Content-Type\":aj;\"Authorization\":\"Bearer $tk\"]; hc:http.client[..[Header:hd]]\nhp:{[f]{[httpf;path]r: 'httpf[hc;url+path]; 'json r\"bodystring\"}[f]}\nhpp:{[f]{[httpf;path;reqopts]r: 'httpf[hc;url+path;reqopts]; 'json r\"bodystring\"}[f]}\nget:hp[http.get]; getq:hpp[http.get]; post:hpp[http.post]\n\n/ convos:get\"conversations.list\"; chans:convos\"channels\"; ^(..name,id)'chans\nchannel:\"\u003cID HERE\u003e\"\n\nday:time.Hour * 24; unow:time.utc time.now@0\noldest:time.unix[time.add[unow;-30 * day]]\nlatest:time.unix[unow]\nmsgs:{[acc;channel;oldest;latest] \\latest\n  hist:post[\"conversations.history\";..[Body:\"\"json..[channel;oldest;latest]]]\n  ms:hist\"messages\"; newlatest:(..ts)@*|ms; acc,:ms\n  ?[hist\"has_more\"\n    o[acc;channel;oldest;newlatest]\n    acc]}\n\nallmsgs:msgs[();channel;oldest;latest]; \"ok\"\n```\n\n## Development\n\nAri is implemented in Go and Goal. See the `script` folder for common development operations.\n\nTo publish a new version of Ari:\n\n```shell\n./script/release vx.y.z\n```\n\n### WASM\n\nUse the `./script/build-wasm` script to generate a `./cmd/wasm/goal.wasm` file from the `./cmd/wasm/main.go` entry-point.\n\nYou then need to locate the `wasm_exec.js` file for your specific Go version and copy that to the `./cmd/wasm` folder. Depending on your system installation, you might find it there under a `lib` or `misc` folder. If not, you can download a source tarball for your specific Go version from the Go downloads which includes this file.\n\nJavaScript that controls the user interface in `./cmd/wasm/index.html` is written in Go in the `./cmd/wasm/main.go` file.\n\nSee [Go Wiki: WebAssembly](https://go.dev/wiki/WebAssembly) for more information.\n\n## Background\n\nI stumbled into a fairly flexible, powerful setup using Julia and DuckDB to do data analysis.\n\nDetails:\n\n- Julia as primary programming language\n- Julia Pluto Notebooks as primary programming environment\n- Notebook One: Data ETL\n  - HTTP: Calling HTTP APIs to fetch JSON data (GitHub, Shortcut)\n  - CSV: Transforming fetched JSON data into CSV\n  - SQL: (Out of band) Defining SQL tables using SQL schema\n  - SQL: (Out of band) Importing CSV into DuckDB using a schema defined in a SQL file\n- Notebook Two: Data Analyses\n  - SQL: DuckDB tables as source of truth\n  - Julia: Wrote trivial utility fn to transform arbitrary SQL query results into Julia DataFrames\n    - `DataFrame(DBInterface.execute(conn, sql))`\n    - Renders as a well-formatted table in the notebook interface\n    - Array-language-like story for interacting with the DataFrame\n    - Fully dynamic: type information from the database schema used to dynamically populate the DataFrame with data of appropriate types.\n  - Julia: Mustache package for templating functions to build a LaTeX report to house all analyses\n  - Julia: Plots package for generating plots as PDF to insert into final LaTeX report\n  - Julia: Statistics, StatsBase packages both for aggregates and data to plot, pull from DataFrames\n  - Julia: Executed external `latexmk` using Julia's `run` to build LaTeX report\n\nWhy move away from this setup?\n\n- Concision and expressive power of array languages.\n- A lot of my code was SQL in Julia.\n\nAri embeds the Goal array programming language. What gaps from my Julia+DuckDB experience need to be filled to use Goal where I used Julia?\n\n- Notebook programming environment\n  - Cell dependencies and automatic re-run\n    - Nice-to-have\n  - Autocomplete\n    - Required\n  - Built-in language/library help documentation\n    - Required\n  - Rich rendering\n    - Tables: Required\n    - Graphics: Nice-to-have\n- HTTP Client\n  - Nice-to-have (can rely on shelling out to `curl` to start).\n- SQL\n  - Query results as Goal tables/dictionaries/arrays\n    - Required\n- Plots\n  - Declarative plot definition producing one of PNG/PDF or HTML/JS output\n    - Required\n    - q tutorials simply leverage JS bridge + Highcharts\n    - Julia Plots package has pluggable backends:\n      - GR (cross-platform, has C bindings)\n      - Plotly (JS)\n      - gnuplot (command-line utility)\n      - HDF5 file format\n    - See GNU Octave plotting\n- See [Gonum](https://github.com/gonum)\n\nGoal already has a number of features, so we don't need to fill these gaps to start (more flexible options may be considered in the future):\n\n- Powerful string API\n- JSON support\n- CSV support\n\n## License\n\n### Ari\n\nOriginal software in this repository is licensed as follows:\n\nCopyright 2024 Daniel Gregoire\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n\n### Goal\n\nSource code copied and/or adapted from the [Goal] project has the following license:\n\nCopyright (c) 2022 Yon \u003canaseto@bardinflor.perso.aquilenet.fr\u003e\n\nPermission to use, copy, modify, and distribute this software for any\npurpose with or without fee is hereby granted, provided that the above\ncopyright notice and this permission notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES\nWITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF\nMERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR\nANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\nWHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\nACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF\nOR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n\n\u003c!-- Links --\u003e\n\n[Goal]: https://codeberg.org/anaseto/goal\n[Resty]: https://github.com/go-resty/resty\n[uber-go/ratelimit]: https://github.com/uber-go/ratelimit\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemperos%2Fari","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemperos%2Fari","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemperos%2Fari/lists"}