{"id":13409382,"url":"https://github.com/foundationdb/fdb-record-layer","last_synced_at":"2026-01-11T16:54:50.500Z","repository":{"id":37412718,"uuid":"143344440","full_name":"FoundationDB/fdb-record-layer","owner":"FoundationDB","description":"A record-oriented store built on FoundationDB","archived":false,"fork":false,"pushed_at":"2024-04-12T17:51:52.000Z","size":23317,"stargazers_count":563,"open_issues_count":369,"forks_count":99,"subscribers_count":32,"default_branch":"main","last_synced_at":"2024-04-12T21:33:40.915Z","etag":null,"topics":["foundationdb","relational-database"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FoundationDB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2018-08-02T20:59:44.000Z","updated_at":"2024-04-15T14:54:43.196Z","dependencies_parsed_at":"2024-02-19T21:34:29.366Z","dependency_job_id":"f9ac44e1-58a7-4ba5-8442-444954bdea8c","html_url":"https://github.com/FoundationDB/fdb-record-layer","commit_stats":null,"previous_names":[],"tags_count":545,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FoundationDB%2Ffdb-record-layer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FoundationDB%2Ffdb-record-layer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FoundationDB%2Ffdb-record-layer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FoundationDB%2Ffdb-record-layer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FoundationDB","download_url":"https://codeload.github.com/FoundationDB/fdb-record-layer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243593366,"owners_count":20316174,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["foundationdb","relational-database"],"created_at":"2024-07-30T20:01:00.324Z","updated_at":"2026-01-11T16:54:50.495Z","avatar_url":"https://github.com/FoundationDB.png","language":"Java","funding_links":[],"categories":["Layers"],"sub_categories":[],"readme":"\u003cimg alt=\"FoundationDB logo\" src=\"docs/sphinx/source/FDB_logo.png?raw=true\" width=\"400\"\u003e\n\nFoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as an ordered key-value store and employs ACID transactions for all operations. It is especially well-suited for read/write workloads but also has excellent performance for write-intensive workloads. Users interact with the database using API language binding.\n\nTo learn more about FoundationDB, visit [foundationdb.org](https://www.foundationdb.org/)\n\n# FoundationDB Record Layer (FRL)\n\nFRL provides a **relational database with SQL support** built on top of FoundationDB, featuring:\n\n* **SQL Database** - SQL support with JDBC connectivity for defining schemas,\n  querying data, and managing tables using familiar SQL syntax. The SQL API is under\n  active development with frequent enhancements.\n* **Advanced Data Types** - Beyond standard SQL types (STRING, INTEGER, FLOAT, BOOLEAN),\n  FRL supports:\n  * **Nested Structures** - User-defined struct types that can be nested arbitrarily deep\n  * **Arrays** - Collections of primitives or complex types\n  * **Vectors** - Fixed-dimension numerical vectors for ML embeddings and similarity search\n* **Schema Templates** - Reusable schema definitions that enable multi-tenant architectures\n  where each tenant gets their own database instance with a shared, evolvable schema.\n* **Intelligent Query Planning** - Automatic index selection and query optimization with\n  support for JOINs, aggregations (COUNT, SUM, etc.), GROUP BY, and ORDER BY.\n  Queries are efficiently executed using index-backed operations without in-memory sorting.\n* **Indexes** - Rich indexing capabilities including value indexes, rank indexes, aggregate\n  indexes, and indexes on nested fields. Indexes are materialized views that update incrementally.\n* **Scalable Architecture** - Designed for distributed, stateless environments with\n  millisecond-level store initialization. Perfect for applications managing thousands\n  of discrete database instances.\n* **ACID Transactions** - Full transactional semantics inherited from FoundationDB,\n  with support for continuations for efficiently paging through large result sets.\n\n## Quick Start with SQL\n\n```java\n// Connect via JDBC\nString url = \"jdbc:embed:/__SYS?schema=CATALOG\";\nConnection conn = DriverManager.getConnection(url);\n\n// Define a schema template with tables and indexes\nconn.createStatement().execute(\"\"\"\n    CREATE SCHEMA TEMPLATE my_template\n        CREATE TABLE customers (\n            customer_id BIGINT,\n            name STRING,\n            email STRING,\n            PRIMARY KEY(customer_id)\n        )\n        CREATE INDEX email_idx AS\n            SELECT email FROM customers ORDER BY email\n    \"\"\");\n\n// Create a database and schema\nconn.createStatement().execute(\n    \"CREATE DATABASE /my_app/production\");\nconn.createStatement().execute(\n    \"CREATE SCHEMA /my_app/production/main WITH TEMPLATE my_template\");\n\n// Insert and query data\nPreparedStatement insert = conn.prepareStatement(\n    \"INSERT INTO customers VALUES (?, ?, ?)\");\ninsert.setLong(1, 1);\ninsert.setString(2, \"Alice\");\ninsert.setString(3, \"alice@example.com\");\ninsert.executeUpdate();\n\nResultSet rs = conn.createStatement().executeQuery(\n    \"SELECT * FROM customers WHERE email = 'alice@example.com'\");\n```\n\n## Key Features\n\n### Multi-Tenant Schema Templates\n\nSchema templates enable efficient multi-tenant architectures:\n\n```sql\n-- Define the template once\nCREATE SCHEMA TEMPLATE user_data_template\n    CREATE TABLE documents (id BIGINT, content STRING, PRIMARY KEY(id))\n    CREATE INDEX content_idx AS SELECT content FROM documents ORDER BY content;\n\n-- Create separate database instances for each tenant\nCREATE DATABASE /tenant/user_1;\nCREATE SCHEMA /tenant/user_1/data WITH TEMPLATE user_data_template;\n\nCREATE DATABASE /tenant/user_2;\nCREATE SCHEMA /tenant/user_2/data WITH TEMPLATE user_data_template;\n```\n\nEach tenant's data is completely isolated with its own database, yet all share\nthe same schema definition for easy management and evolution.\n\n### Advanced Type System\n\nDefine complex, nested data structures:\n\n```sql\n-- Define custom struct types\nCREATE TYPE AS STRUCT address (\n    street STRING,\n    city STRING,\n    postal_code STRING\n)\n\nCREATE TYPE AS STRUCT contact_info (\n    email STRING,\n    phone STRING,\n    mailing_address address\n)\n\n-- Use in tables with arrays and nesting\nCREATE TABLE users (\n    user_id BIGINT,\n    name STRING,\n    contacts contact_info ARRAY,\n    PRIMARY KEY(user_id)\n)\n```\n\n### Vector Support for ML Applications\n\nStore and query high-dimensional vectors for embeddings and similarity search:\n\n```sql\nCREATE TABLE embeddings (\n    doc_id BIGINT,\n    content STRING,\n    embedding_half VECTOR(128, HALF),     -- 16-bit precision\n    embedding_float VECTOR(768, FLOAT),   -- 32-bit precision\n    embedding_double VECTOR(1024, DOUBLE), -- 64-bit precision\n    PRIMARY KEY(doc_id)\n)\n```\n\nVectors are inserted via JDBC PreparedStatements and can be efficiently stored\nand retrieved using the FoundationDB backend.\n\n### Index-Backed Query Execution\n\nFRL's query planner intelligently selects indexes to execute\nqueries efficiently:\n\n```sql\n-- Queries use indexes automatically\nSELECT name FROM customers WHERE email = 'alice@example.com';\n-- Uses email_idx if available\n\n-- JOINs using comma-separated FROM clause\nSELECT c.name, o.order_id\nFROM customers c, orders o\nWHERE c.customer_id = o.customer_id;\n\n-- Aggregations backed by indexes\nSELECT category, COUNT(*)\nFROM products\nGROUP BY category;\n-- Requires ordered index or primary key on category for streaming aggregate, or a aggregate index for direct retrieval\n\n-- ORDER BY requires index or primary key order\nSELECT * FROM customers ORDER BY email;\n-- Requires index on email (like email_idx above)\n```\n\n**Important**: FRL does not perform in-memory sorting or aggregation. Operations like ORDER BY, GROUP BY,\nand aggregates require underlying indexes to provide the required ordering.\n\n## Architecture Notes\n\nFRL is designed for:\n* **Horizontal scalability** - Thousands of independent database instances\n* **Low latency** - Millisecond-level initialization and query execution\n* **Stateless services** - No server-side state; all data in FoundationDB\n* **Schema evolution** - Templates can evolve over time (template evolution features\n  coming to relational layer; currently available via advanced Record Layer API)\n\n## Advanced: Direct Record Layer API\n\nFor applications requiring fine-grained control over storage layout, index\nmaintenance, or features not yet available in the SQL Relational layer, the Record Layer\nprovides a low-level Java API using Protocol Buffers.\n\n**Note**: This API is maintained for advanced use cases but is being positioned\nas a lower-level alternative to the SQL interface. Features available only\nthrough this API will migrate to the SQL layer over time. Long-term support of this lower-level API is not guaranteed once equivalent features are available at the Relational Layer.\n\nKey Record Layer API features:\n* **Protobuf-based schema definition** - Define records using `.proto` files\n* **Programmatic index management** - `IndexMaintainer` extension points\n* **Custom query components** - Extend the query planner\n* **Schema evolution** - `MetaDataEvolutionValidator` for safe schema changes\n* **Low-level control** - Direct access to FoundationDB operations\n\nSee [Record Layer Documentation](https://foundationdb.github.io/fdb-record-layer/Overview.html) for details.\n\n## Documentation\n\n* **Getting Started** - [SQL Quick Start](https://foundationdb.github.io/fdb-record-layer/GettingStarted.html)\n* **SQL Reference** - [SQL Commands and Data Types](https://foundationdb.github.io/fdb-record-layer/SQL_Reference.html)\n* **Schema Templates** - [Databases, Schemas, and Templates](https://foundationdb.github.io/fdb-record-layer/reference/Databases_Schemas_SchemaTemplates.html)\n* **Advanced: Record Layer API** - [Record Layer Overview](https://foundationdb.github.io/fdb-record-layer/Overview.html)\n* [Documentation Home](https://foundationdb.github.io/fdb-record-layer/)\n* [Contributing](CONTRIBUTING.md)\n* [Code of Conduct](CODE_OF_CONDUCT.md)\n* [License](LICENSE)\n\n## Getting Help\n\n* **Bugs \u0026 Feature Requests**: [GitHub Issues](https://github.com/FoundationDB/fdb-record-layer/issues)\n* **Community**: [FoundationDB Community Forums](https://forums.foundationdb.org/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoundationdb%2Ffdb-record-layer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffoundationdb%2Ffdb-record-layer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoundationdb%2Ffdb-record-layer/lists"}