{"id":13510183,"url":"https://github.com/kuwala-io/kuwala","last_synced_at":"2025-03-30T15:30:52.133Z","repository":{"id":37239015,"uuid":"355909605","full_name":"kuwala-io/kuwala","owner":"kuwala-io","description":"Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data.  Currently, the following data connectors are available worldwide: a) High-resolution demographics data b) Point of Interests from Open Street Map c) Google Popular Times","archived":false,"fork":false,"pushed_at":"2022-08-10T09:00:13.000Z","size":8173,"stargazers_count":786,"open_issues_count":22,"forks_count":52,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-11-01T10:35:15.018Z","etag":null,"topics":["admin-boundaries","data","data-integration","data-science","dbt","elt","google-trends","jupyter","kuwala","no-code","open-data","open-source","population","postgres","pyspark","python","react","react-flow","scraping","spatial-analysis"],"latest_commit_sha":null,"homepage":"https://kuwala.io","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kuwala-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-04-08T13:02:36.000Z","updated_at":"2024-10-28T11:19:19.000Z","dependencies_parsed_at":"2022-07-10T15:00:23.744Z","dependency_job_id":null,"html_url":"https://github.com/kuwala-io/kuwala","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuwala-io%2Fkuwala","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuwala-io%2Fkuwala/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuwala-io%2Fkuwala/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuwala-io%2Fkuwala/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kuwala-io","download_url":"https://codeload.github.com/kuwala-io/kuwala/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246338699,"owners_count":20761421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["admin-boundaries","data","data-integration","data-science","dbt","elt","google-trends","jupyter","kuwala","no-code","open-data","open-source","population","postgres","pyspark","python","react","react-flow","scraping","spatial-analysis"],"created_at":"2024-08-01T02:01:27.485Z","updated_at":"2025-03-30T15:30:47.122Z","avatar_url":"https://github.com/kuwala-io.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","Applications","scraping","data-science","[🧑‍💻 dev](https://github.com/stars/ketsapiwiq/lists/dev)","data"],"sub_categories":["Workflow \u0026 Automation"],"readme":"[![Slack](https://img.shields.io/badge/slack-chat-orange.svg)](https://join.slack.com/t/kuwala-community/shared_invite/zt-l5b2yjfp-2cdhReNACXDkLhuh0jFzZw)\n![License](https://img.shields.io/github/license/kuwala-io/kuwala)\n\n\u003cimg src=\"./docs/images/kuwala_header_github.png\" width=\"100%\" /\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://join.slack.com/t/kuwala-community/shared_invite/zt-l5b2yjfp-2cdhReNACXDkLhuh0jFzZw\"\u003e\u003cimg src=\"./docs/images/slack_join_button_github.png\" height=\"96\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/orgs/kuwala-io/projects/3/views/6\"\u003e\u003cimg src=\"./docs/images/roadmap_button_github.png\" height=\"96\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nKuwala is **the data workspace for BI analysts and engineers enabling you to build powerful analytics workflows together.** \nWe are set out to bring **state-of-the-art data engineering tools** you love, such as \n[Airbyte](https://github.com/airbytehq/airbyte), [dbt](https://github.com/dbt-labs/dbt-core) and \n[Prefect](https://github.com/prefecthq/prefect) together in one intuitive interface built with \n[React Flow](https://github.com/wbkd/react-flow).\n\nDo you want to **discuss your first contribution**, want to **learn more in general**, or \n**discuss your specific use-case** for Kuwala? Just **book a digital coffee session** with the core team \n[here](https://calendly.com/kuwala_io/kuwala-community-chat).\n\n![Collaboration between BI analysts and engineers](./docs/images/kuwala_collaborative_cycle.png)\n\nKuwala stands for **extendability, reproducibility, and enablement**. Small data teams build data products fastly and \ncollaboratively. Analysts and engineers stay with their strengths. Kuwala  is the tool that makes it possible to keep a \ndata project within scope while having fun again.\n\n- **Kuwala Canvas runs directly on a data warehouse** = Maximum flexibility and no lock-in effect\n- **Engineers enable their analysts** by adding transformations and models via dbt or new data sources through Airbyte\n- **The node-based editor enables analyst** to build advanced data workflows with many data sources and transformations through simple drag-and-drop\n- **With models-as-a-block** the BI analyst can launch advanced Marketing Mix Models and attributions without knowing R or Python\n\n\u003cimg src=\"./docs/images/tool_description_github.png\" width=\"100%\" style=\"background-color: white\" /\u003e\n\n### Extract and Load with Airbyte\n\nCurrently we support the following databases and data warehouses\n\n- [x] Postgres\n- [x] BigQuery\n- [x] Snowflake\n\n For connecting and loading all your tooling data into a data warehouse, we are integrating with Airbyte connectors. \n For everything related to third-party data, such as POI and demographics data, we are building separate data pipelines.\n\n### Transform with dbt\n\nTo apply transformations on your data, we are integrating dbt which is running on top of your data warehouses. \nEngineers can easily create dbt models and make them reusable for the frontend. We have already a catalog of several \ntransformations that you can use on the canvas. The complete documentation can be found here: \n[https://docs.kuwala.io/](https://docs.kuwala.io/transformation-catalog)\n\n### Run a Data Science Model\n\nWe are going to include open-source data science and AI models as blocks (e.g., \n[Meta's Robyn Marketing Mix Modeling](https://github.com/facebookexperimental/Robyn)).\n\n### Report\n\nYou can easily connect your preferred visualization tool and connect it to a saved table on the canvas in the future. \nWe will make the results exportable to Google Sheets and also available in a Medium-style markdown editor.\n\n---\n\n# How can I use Kuwala?\n\n### Canvas\n\nWith the canvas you can connect to your data warehouse and start building data pipelines. To start the canvas, simply\nrun the following command from inside the root directory:\n\n```zsh\ndocker-compose --profile kuwala up\n```\n\nNow open http://localhost:3000 in your browser, and you are good to go. 🚀\n\n### Third-party data connectors\n\nWe currently have five pipelines for different third-party data sources which can easily be imported into a Postgres \ndatabase. The following pipelines are integrated:\n\n - [Admin Boundaries](https://github.com/kuwala-io/kuwala/tree/master/kuwala/pipelines/admin-boundaries/README.md)\n - [Google POIs](https://github.com/kuwala-io/kuwala/tree/master/kuwala/pipelines/google-poi/README.md)\n - [Google Trends](https://github.com/kuwala-io/kuwala/tree/master/kuwala/pipelines/google-trends/README.md)\n - [OSM POIs](https://github.com/kuwala-io/kuwala/tree/master/kuwala/pipelines/osm-poi/README.md)\n - [Population Density](https://github.com/kuwala-io/kuwala/tree/master/kuwala/pipelines/population-density/README.md)\n\n### Using Kuwala components individually\n\nTo use Kuwala's components, such as the data pipelines or the Jupyter environment, individually, please refer to the\n[instructions under `/kuwala`](https://github.com/kuwala-io/kuwala/blob/master/kuwala/README.md).\n\n---\n\n# Use cases\n\n- [How to build an Uber-like analytics system with Kuwala](https://kuwala.io/data-pipelines/how-to-build-an-uber-like-intelligence-system-for-your-new-mobility-startup-without-a-big-data-team/)\n- [Perform location analytics for a grocery store with Kuwala](https://kuwala.io/case-studies/why-instant-grocery-delivery-should-follow-a-data-driven-path-like-uber-to-survive-part-1/)\n- [Querying the most granular demographics data set with Kuwala](https://kuwala.io/data-pipelines/querying-the-most-granular-demographics-dataset/)\n\n---\n\n# How can I contribute?\n\nEvery new issue, question, or comment is a contribution and very welcome! This project lives from your feedback and\ninvolvement!\n\n## Be part of our community\n\nThe best first step to get involved is to join the \n[Kuwala Community on Slack](https://join.slack.com/t/kuwala-community/shared_invite/zt-l5b2yjfp-2cdhReNACXDkLhuh0jFzZw). \nThere we discuss everything related to our roadmap, development, and support. \n\n## Contribute to the project\n\nPlease refer to our [contribution guidelines](https://github.com/kuwala-io/kuwala/tree/master/CONTRIBUTING.md) for further information on how to get involved.\n\n---\n\n# Get more content about Kuwala\n  Link  | Description    \n-------------|-------------\n[Blog](https://kuwala.io/blog/) | Read all our blog articles related to the stuff we are doing here. \n[Join Slack](https://join.slack.com/t/kuwala-community/shared_invite/zt-l5b2yjfp-2cdhReNACXDkLhuh0jFzZw) | Our Slack channel with over 250 data engineers and many discussions.\n[Jupyter notebook - Popularity correlation](https://mybinder.org/v2/gh/kuwala-io/kuwala_examples/1def41ded01dbb67ecada9dc227c834cd93d0721?urlpath=lab%2Ftree%2Fnotebooks%2F1.%20Popularity%20Correlation.ipynb) | Open a Jupyter notebook on Binder and merge external popularity data with Uber traversals by making use of convenient dbt functions. \n[Podcast](https://anchor.fm/kuwala-io) | Listen to our community podcast and maybe join us on the next show.\n[Digital coffee break](https://calendly.com/kuwala_io/kuwala-community-chat?month=2022-02) | Are you looking for new inspiring tech talks? Book a digital coffee chit-chat with one member of the core team.\n[Our roadmap](https://github.com/orgs/kuwala-io/projects/3/views/6) | See our upcoming milestones and sprint planing.\n[Contribution guidelines](https://github.com/kuwala-io/kuwala/tree/master/CONTRIBUTING.md) | Further information on how to get involved.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuwala-io%2Fkuwala","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuwala-io%2Fkuwala","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuwala-io%2Fkuwala/lists"}