{"id":13494209,"url":"https://github.com/MaterializeInc/materialize","last_synced_at":"2025-03-28T13:32:35.105Z","repository":{"id":36950349,"uuid":"172104891","full_name":"MaterializeInc/materialize","owner":"MaterializeInc","description":"The Cloud Operational Data Store: use SQL to transform, deliver, and act on fast-changing data.","archived":false,"fork":false,"pushed_at":"2024-10-29T09:56:20.000Z","size":240955,"stargazers_count":5782,"open_issues_count":352,"forks_count":467,"subscribers_count":75,"default_branch":"main","last_synced_at":"2024-10-29T10:06:23.668Z","etag":null,"topics":["data-warehouse","database","distributed-systems","kafka","materialized-view","operational-data-warehouse","postgresql","postgresql-dialect","rust","sql","stream-processing","streaming","streaming-data"],"latest_commit_sha":null,"homepage":"https://materialize.com","language":"Rust","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MaterializeInc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-22T17:15:15.000Z","updated_at":"2024-10-29T09:56:25.000Z","dependencies_parsed_at":"2023-01-17T07:31:20.738Z","dependency_job_id":"b9062835-a506-4bec-b72a-e4855dcabf84","html_url":"https://github.com/MaterializeInc/materialize","commit_stats":{"total_commits":16922,"total_committers":164,"mean_commits":103.1829268292683,"dds":0.8028010873419218,"last_synced_commit":"43f2d7a109a90bb7004a8331c378ed0a02108a09"},"previous_names":[],"tags_count":489,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fmaterialize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fmaterialize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fmaterialize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fmaterialize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MaterializeInc","download_url":"https://codeload.github.com/MaterializeInc/materialize/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222382532,"owners_count":16975382,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-warehouse","database","distributed-systems","kafka","materialized-view","operational-data-warehouse","postgresql","postgresql-dialect","rust","sql","stream-processing","streaming","streaming-data"],"created_at":"2024-07-31T19:01:22.895Z","updated_at":"2024-10-31T08:32:02.100Z","avatar_url":"https://github.com/MaterializeInc.png","language":"Rust","readme":"[![Build status](https://badge.buildkite.com/97d6604e015bf633d1c2a12d166bb46f3b43a927d3952c999a.svg?branch=main)](https://buildkite.com/materialize/test)\n[![Doc reference](https://img.shields.io/badge/doc-reference-orange)](https://materialize.com/docs)\n[![Chat on Slack](https://img.shields.io/badge/chat-on%20slack-purple)](https://materialize.com/s/chat)\n\n[\u003cimg src=\"https://github.com/MaterializeInc/materialize/assets/23521087/39270ecb-7ac4-4829-b98b-c5b5699a16b8\" width=35%\u003e](https://materialize.com)\n\nMaterialize is the **cloud operational data store** purpose-built to use SQL to\ntransform, deliver, and act on fast-changing data.\n\nUsing SQL and common tools in the wider data ecosystem, Materialize allows you\nto build real-time automation, engaging customer experiences, and interactive\ndata products that drive value for your business while reducing the cost of\n[data freshness](https://materialize.com/blog/decouple-cost-and-freshness/).\n\n## Get started\n\nReady to try out Materialize? [Sign up](https://materialize.com/register/) to get started! 🚀\n\nHave questions? We'd love to hear from you:\n  * [Join our Slack](https://materialize.com/s/chat)\n  * [Send us mail](https://materialize.com/contact/)\n\n## About\n\nMaterialize is designed to help you interactively explore your streaming data, perform analytics against live relational data, or increase data freshness while reducing the load of your dashboard and monitoring tasks. The moment you need a refreshed answer, you can get it in milliseconds.\n\nIt focuses on providing correct and [consistent](https://materialize.com/docs/overview/isolation-level/) answers with minimal latency, and does not ask you to accept either approximate answers or eventual consistency. Whenever Materialize answers a query, that answer is the correct result on some specific (and recent) version of your data. Materialize does all of this by recasting your SQL queries as *dataflows*, which can react efficiently to changes in your data as they happen.\n\nOur fully managed service is cloud native, featuring **high availability**, via\nmulti-active replication, **horizontal scalability**, by seamlessly scaling\ndataflows across multiple machines, and **near infinite storage**, by\nleveraging cloud object storage (e.g., Amazon S3).\n\nWe support a large fraction of PostgreSQL features, and are actively working on supporting more built-in PostgreSQL functions. Please file an issue if something doesn't work as expected!\n\n## Get data in\n\nMaterialize can read data from [Kafka](https://materialize.com/docs/sql/create-source/kafka/) (and other Kafka API-compatible systems like [Redpanda](https://materialize.com/docs/integrations/redpanda/)), directly from a [PostgreSQL](https://materialize.com/docs/sql/create-source/postgres/) or [MySQL](https://materialize.com/docs/sql/create-source/mysql/) replication stream, or from SaaS applications [via webhooks](https://materialize.com/docs/sql/create-source/webhook/). It also supports regular database tables to which you can insert, update, and delete rows.\n\n## Transform, manipulate, and read your data\n\nOnce you've got the data in, define views and perform reads via the PostgreSQL protocol. Use your favorite SQL client, including the `psql` you probably already have on your system.\n\nMaterialize supports a comprehensive variety of SQL features, all using the PostgreSQL dialect and protocol:\n\n-   Joins, joins, joins! Materialize supports multi-column join conditions, multi-way joins, self-joins, cross-joins, inner joins, outer joins, etc.\n-   Delta-joins avoid intermediate state blowup compared to systems that can only plan nested binary joins - tested on joins of up to 64 relations.\n-   Support for subqueries. Materialize's SQL optimizer performs subquery decorrelation out-of-the-box, avoiding the need to manually rewrite subqueries into joins.\n-   Materialize can incrementally maintain views in the presence of arbitrary inserts, updates, and deletes. No asterisks.\n-   All the aggregations: `min`, `max`, `count`, `sum`, `stddev`, etc.\n-   `HAVING`\n-   `ORDER BY`\n-   `LIMIT`\n-   `DISTINCT`\n-   JSON support in the PostgreSQL dialect including operators and functions like `-\u003e`, `-\u003e\u003e`, `@\u003e`, `?`, `jsonb_array_element`, `jsonb_each`. Materialize automatically plans lateral joins for efficient `jsonb_each` support.\n-   Nest views on views on views!\n-   Multiple views that have overlapping subplans can share underlying indices for space and compute efficiency, so just declaratively define _what you want_, and we'll worry about how to efficiently maintain them.\n\n### Just show us what it can do!\n\nHere's an example join query that works fine in Materialize, `TPC-H` query 15:\n\n```sql\nCREATE SOURCE tpch\n  FROM LOAD GENERATOR TPCH (SCALE FACTOR 1)\n  FOR ALL TABLES;\n\n-- Views define commonly reused subqueries.\nCREATE VIEW revenue (supplier_no, total_revenue) AS\n    SELECT\n        l_suppkey,\n        SUM(l_extendedprice * (1 - l_discount))\n    FROM\n        lineitem\n    WHERE\n        l_shipdate \u003e= DATE '1996-01-01'\n        AND l_shipdate \u003c DATE '1996-01-01' + INTERVAL '3' month\n    GROUP BY\n        l_suppkey;\n\n-- The MATERIALIZED keyword is the trigger to begin\n-- eagerly, consistently, and incrementally maintaining\n-- results that are stored directly in durable storage.\nCREATE MATERIALIZED VIEW tpch_q15 AS\n  SELECT\n    s_suppkey,\n    s_name,\n    s_address,\n    s_phone,\n    total_revenue\nFROM\n    supplier,\n    revenue\nWHERE\n    s_suppkey = supplier_no\n    AND total_revenue = (\n        SELECT\n            max(total_revenue)\n        FROM\n            revenue\n    )\nORDER BY\n    s_suppkey;\n\n-- Creating an index keeps results always up to date and in memory.\n-- In this example, the index will allow for fast point lookups of\n-- individual supply keys.\nCREATE INDEX tpch_q15_idx ON tpch_q15 (s_suppkey);\n```\n\nStream inserts, updates, and deletes on the underlying tables (`lineitem` and `supplier`), and Materialize keeps the materialized view incrementally updated. You can type `SELECT * FROM tpch_q15` and expect to see the current results immediately!\n\n## Get data out\n\n**Pull based**: Use any PostgreSQL-compatible driver in any language/environment to make `SELECT` queries against your views. Tell them they're talking to a PostgreSQL database, they don't ever need to know otherwise.\n\n**Push based**: Listen to changes directly using `SUBSCRIBE` or configure Materialize to stream results to a Kafka topic as soon as the views change.\n\n## Documentation\n\nCheck out [our documentation](https://materialize.com/docs/).\n\n## License\n\nMaterialize is provided primarily as a fully managed cloud service with\n[credit-based pricing](https://materialize.com/pricing/). Included in the price\nare proprietary cloud-native features like horizontal scalability, high\navailability, and a web management console.\n\nHowever, we're big believers in advancing the frontier of human knowledge. To\nthat end, the source code of the standalone database engine is publicly\navailable, in this repository, and [licensed](LICENSE) under the BSL 1.1,\nconverting to the open-source Apache 2.0 license after 4 years. As stated in the\nBSL, use of the standalone database engine on a single node is free forever.\nPlease be warned that this deployment model is *not* suitable for production use\nand we cannot offer support for it.\n\nMaterialize depends upon many open source Rust crates. We maintain a [list of\nthese crates and their licenses](https://dev.materialize.com/licenses.html),\nincluding links to their source repositories.\n\n## For developers\n\nMaterialize is primarily written in Rust.\n\nDevelopers can find docs at [doc/developer](doc/developer), and Rust API documentation is hosted at \u003chttps://dev.materialize.com/api/rust/\u003e.\n\nContributions are welcome. Prospective code contributors might find the [D-good\nfor external\ncontributors](https://github.com/MaterializeInc/materialize/discussions/categories/contribute-to-materialize?discussions_q=is%3Aopen+category%3A%22Contribute+to+Materialize%22+label%3A%22D-good+for+external+contributors%22)\ndiscussion label useful. See\n[CONTRIBUTING.md](https://github.com/MaterializeInc/materialize/blob/main/CONTRIBUTING.md)\nfor additional guidance.\n\n## Credits\n\nMaterialize is lovingly crafted by [a team of developers](https://github.com/MaterializeInc/materialize/graphs/contributors) and one bot. [Join us](https://materialize.com/careers/).\n","funding_links":[],"categories":["Rust","Applications","📂 Additional Sections","应用程序 Applications","streaming","Stream Processing","Databases","软件"],"sub_categories":["Database","3. Open-source Projects","数据库 Database","Streaming Databases","Materialize"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMaterializeInc%2Fmaterialize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMaterializeInc%2Fmaterialize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMaterializeInc%2Fmaterialize/lists"}