{"id":25118969,"url":"https://github.com/dhonysilva/maaite","last_synced_at":"2025-04-02T13:12:57.778Z","repository":{"id":275470818,"uuid":"926126288","full_name":"dhonysilva/maaite","owner":"dhonysilva","description":"Embedded analytics platform for building dashboards utilizing the Elixir, Phoenix, LiveView and Livebook stack 📊","archived":false,"fork":false,"pushed_at":"2025-02-10T16:02:28.000Z","size":86,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-10T17:22:02.918Z","etag":null,"topics":["analytics","elixir","livebook","liveview"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dhonysilva.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-02T16:09:00.000Z","updated_at":"2025-02-10T16:02:32.000Z","dependencies_parsed_at":"2025-02-02T19:23:59.349Z","dependency_job_id":"b4b09226-fb56-4609-95ae-738332bd7dbd","html_url":"https://github.com/dhonysilva/maaite","commit_stats":null,"previous_names":["dhonysilva/maaite"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhonysilva%2Fmaaite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhonysilva%2Fmaaite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhonysilva%2Fmaaite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhonysilva%2Fmaaite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dhonysilva","download_url":"https://codeload.github.com/dhonysilva/maaite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246819786,"owners_count":20839095,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","elixir","livebook","liveview"],"created_at":"2025-02-08T04:23:36.098Z","updated_at":"2025-04-02T13:12:57.771Z","avatar_url":"https://github.com/dhonysilva.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Maaite\n\nA proof of concept of an Embedded analytics platform for building dashboards utilizing the Elixir, Phoenix, LiveView and Livebook technologies.\n\n*Disclaimer*: This is an ongoing projet. I am creating this document to keep track of the development process and to share the knowledge with the community. All the informations here are subject to change.\n\n## References\n\nPlease, check the following references to understand the context of this project.\n\n* [marimo](https://marimo.io)\n\n[GitHub](https://github.com/marimo-team/marimo) repository for the Marimo project.\n\n*[Provide more explanation about it, and their characteristics]*.\n\n* [Observable Framework](https://observablehq.com/platform/framework)\n\n[GitHub](https://github.com/observablehq/framework) repository for the Observable Framework.\n\nFrom the same creator of [d3js](https://d3js.org) JavaScript library for data visualization, the Obserable Framework [more].\n\n* [Evidence](https://evidence.dev)\n\n[GitHub](https://github.com/evidence-dev/evidence)\n\nBusiness intelligence as code: build fast, interactive data visualizations in SQL and markdown.\n\n* Metabase [Embedded Analytics](https://www.metabase.com/product/embedded-analytics)\n\nIt is not open sourced yet. But it is a good reference for the project.\n\n*[Provide more explanation about it, and their characteristics]*.\n\nOthers tools that might be helpfull\n\n[Cloudflare Radar](https://radar.cloudflare.com)\n\n[Cloudfrlare Radar - AI Insights](https://radar.cloudflare.com/ai-insights)\n\n## Objectives\n\nThe aim of this project is to create a platform for building analytical dashboards such as the references prior mentioned utilizing the Elixir, Phoenix, LiveView and Livebook technologies. The purpose is to bring the power of Elixir to the data visualization world.\n\nIt will be a platform to build analytics dashboards as dynamic web pages, utilizing the Livebook engine to create the visualizations, but instead of an interactive document, it will generate web pages with several charts, tables, and texts.\n\nThe users will build different graphics elements and pin them to appear on the central page. When appearing on the central page, it will be possible to resize them and insert text blocks, titles, and other annotations using Markdown.\n\nThey will rely on heavy utilization of SQL queries to fetch data from the database and create the indicators that will feed the visualizations. These SQL statements will be placed inside the markdown blocks.\n\nTaking the Observable Framework as an example, loading an external data source\n\n```yaml\n---\nsql:\n  quakes: https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv\n---\n```\n\nAnd then using the SQL to fetch the data:\n\n```sql\nSELECT * FROM gaia ORDER BY phot_g_mean_mag LIMIT 10\n```\n\nIt will imedialety render the data in a table or any other kind of graphic available.\n\nSee more examples on the [Observable Framework](https://observablehq.com/framework/sql) documentation.\n\n*[provide some more examples that expalins the use of SQL in the platform and why this is important]*\n\nWe can add from basic elements like this:\n\n```html\n\u003cdetails\u003e\n  \u003csummary\u003eClick me\u003c/summary\u003e\n  This text is not visible by default.\n\u003c/details\u003e\n```\n\nTo some more complex like this:\n\n```html\n\u003cdiv class=\"card\" style=\"max-width: 640px;\"\u003e\n  \u003ch2\u003eIt gets hotter during summer\u003c/h2\u003e\n  \u003ch3\u003eAnd months have 28–31 days\u003c/h3\u003e\n  ${Plot.cell(weather.slice(-365), {x: (d) =\u003e d.date.getUTCDate(), y: (d) =\u003e d.date.getUTCMonth(), fill: \"temp_max\", tip: true, inset: 0.5}).plot({marginTop: 0, height: 240, padding: 0})}\n\u003c/div\u003e\n```\n\nFor our purpose, we will create Custom Components that will fit the requirement of create elements for tables, graphs e other elements.\n\n## Who will utilize this platform?\n\nThis is a tool for data analysts, business intelligent developers and data scientists to create dashboards and reports and also might embedded it on their applications.\n\nThis public takes advantage of the features such as window functions, CTEs, and other advanced SQL features to create their indicators.\n\n## Known limitations\n\nAt this point we don't have Livebook as a mix depedency. It could be something like:\n\n\n```elixir\ndefp deps do\n  [\n    {:livebook, \"~\u003e 0.1.0\"}\n  ]\nend\n```\n\nIt demands more research to understand how to transform the Livebook into a `mix deps`. Meanwhile, we will use the standalone Livebook application.\n\n## Database support\n\nDuckDB, SQLite, BigQuery, Parquet Delta Lake, and others. We will provide more information about the database support in the future.\n\n### DuckDB 🦆\n\nWe will utilize the DuckDB engine as the one of the main database for the platform. It is a columnar storage database that is very fast and has a SQL interface. It is possible to load data from CSV, Parquet, and other formats. See the example below:\n\n```sql\ncreate or replace table taxis as\n  from 'https://blobs.duckdb.org/data/taxi_2019_04.parquet'\n```\n\nIt will load the data from the URL and create a table called `taxis` with the data from the Parquet file. It is possible to run SQL queries on this table and create visualizations.\n\n\n![Connecting to and external parquet file with DuckDB!](priv/static/images/connecting_data_with_duckdb.jpg \"Parquet file\")\n\n### DuckDB Wasm\n\nDuring the development, we will learn more about [DuckDB Wasm](https://github.com/duckdb/duckdb-wasm) and about its user cases.\n\n## Running the app\n\nTo start your Phoenix server:\n\n  * Run `mix setup` to install and setup dependencies\n  * Start Phoenix endpoint with `mix phx.server` or inside IEx with `iex -S mix phx.server`\n\nNow you can visit [`localhost:4000`](http://localhost:4000) from your browser.\n\n## Motivation\n\nI have been working on roles such as data analyst, data engineer, business intelligence developer (and some others) for a long time. I have used tools such as Tableau and Power BI. Writing SQL statements and algo some Python code on Jupyter notebook.\nBut I always felt that I was missing something. Aiming to have more control over the data and the visualizations, I wanted to have a tool that could be used by data analysts and data scientists to create dashboards and reports.\n\nSince I started to learning Elixir language and the Phoenix framework, and been using it for some small projects, I am amazed by the power of the language and the framework. That's why I believe this is a great time to start this project, being able to working on the intersection of data and web development with Elixir and its ecosystem.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhonysilva%2Fmaaite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdhonysilva%2Fmaaite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhonysilva%2Fmaaite/lists"}