{"id":16689592,"url":"https://github.com/sigpwned/jdbq","last_synced_at":"2025-05-15T10:33:32.738Z","repository":{"id":148223531,"uuid":"556910778","full_name":"sigpwned/jdbq","owner":"sigpwned","description":"JDBI-inspired Database Access Framework for Java + BigQuery","archived":false,"fork":false,"pushed_at":"2025-04-01T00:40:54.000Z","size":211,"stargazers_count":3,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-03T08:04:46.761Z","etag":null,"topics":["bigquery","data-access-framework","data-access-layer","data-access-library","data-lake","java","persistence","persistence-framework","persistence-layer"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sigpwned.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-24T18:47:05.000Z","updated_at":"2025-03-21T23:40:36.000Z","dependencies_parsed_at":"2023-05-03T03:00:49.510Z","dependency_job_id":"d44ba131-eff2-4fed-8ca1-b5bb4ab52a7a","html_url":"https://github.com/sigpwned/jdbq","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigpwned%2Fjdbq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigpwned%2Fjdbq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigpwned%2Fjdbq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigpwned%2Fjdbq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sigpwned","download_url":"https://codeload.github.com/sigpwned/jdbq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254323332,"owners_count":22051773,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","data-access-framework","data-access-layer","data-access-library","data-lake","java","persistence","persistence-framework","persistence-layer"],"created_at":"2024-10-12T15:48:41.797Z","updated_at":"2025-05-15T10:33:32.358Z","avatar_url":"https://github.com/sigpwned.png","language":"Java","readme":"# jdbq [![tests](https://github.com/sigpwned/jdbq/actions/workflows/tests.yml/badge.svg)](https://github.com/sigpwned/jdbq/actions/workflows/tests.yml)  ![Maven Central](https://img.shields.io/maven-central/v/com.sigpwned/jdbq)\n\nA [JDBI](https://jdbi.org/)-inspired database access framework for Java 8+ and [Google BigQuery](https://cloud.google.com/bigquery).\n\n## Motivation\n\nBigQuery is one of the best data lake implementations available on the market today. While its official Java client supports several useful features for working with data stores, such as parameter bindings, there are many important patterns it does not support, such as modular row and column mappers. JDBI is the state-of-the-art tool for working with persistence stores. JDBQ is a library for using BigQuery that makes the best features of JDBI available to BigQuery users.\n\n## Goals\n\n* To work directly with the official Java client\n* To provide the most important features from the JDBI framework\n* To improve the QOL and productivity of Java developers using BigQuery\n\n## Non-Goals\n\n* To provide all features from the JDBI framework\n\n## Example Usage\n\nFor the below examples, imagine a table with the following structure:\n\n    CREATE TABLE sales(\n        buyer STRING NOT NULL,\n        invoice STRING NOT NULL,\n        sku STRING NOT NULL,\n        quantity INT64 NOT NULL,\n        timestamp TIMESTAMP NOT NULL\n    );\n    \n### Initial Setup\n\nFirst, the user should create a `Jdbq` instance:\n\n    BigQuery client=createBigQueryClient();\n    Jdbq jdbq=new Jdbq(client);\n    \nUsers may also find it useful to add a customizer to set default dataset, SQL dialect, etc.\n\n    jdbq.getConfig().get(SqlStatements.class).addCustomizer(new StatementCustomizer() {\n        @Override\n        public void beforeExecution(QueryJobConfiguration.Builder stmt, StatementContext ctx) {\n            stmt.setDefaultDataset(DEFAULT_DATASET_NAME).setUseLegacySql(false);\n        }\n    });\n    \n### Column Mapped Results\n\nConsider the following query:\n\n    long quantity=jdbq.createQuery(\"\"\"\n            SELECT\n                SUM(quantity) AS quantity\n            FROM sales\n            WHERE sku=:sku\n                AND timestamp BETWEEN :since AND :until\"\"\")\n        .bind(\"sku\", \"abcd1234\")\n        .bind(\"since\", LocalDate.of(2023, 1, 1))\n        .bind(\"until\", LocalDate.of(2023, 3, 31))\n        .mapTo(Long.class)\n        .one();\n        \nThis query computes how many units of SKU `abcd1234` were sold in Q1 2023.\n\nIn this example, we see that we have named parameters in the query (e.g., `:sku`) with values provided using the `bind` method later. Next, the result is mapped to `Long` values, and then exactly one value is retrieved, otherwise an exception is thrown. This works because there are built-in `ColumnMapper` classes for most builtin types, such as `Integer`, `Long`, `Double`, `String`, `LocalDate`, and so on.\n\n### Row Mapped Results\n\nNow consider this code:\n\n    Jdbq jdbq=createJdbq();\n    \n    record SkuSales(String sku, long quantity) {}\n    \n    jdbq.getConfig(RowMappers.class).register(new RowMapper\u003cSkuSales\u003e() {\n        @Override\n        public SkuSales map(FieldValueList fvs, StatementContext ctx) {\n            String sku=fvs.get(\"sku\").getStringValue();\n            long quantity=fvs.get(\"quantity\").getLongValue();\n            return new SkuSales(sku, quantity);\n        }\n    });\n\n    List\u003cSkuSales\u003e sales=jdbq.createQuery(\"\"\"\n            SELECT\n                sku AS sku,\n                SUM(quantity) AS quantity\n            FROM sales\n            WHERE timestamp BETWEEN :since AND :until\n            GROUP BY 1\n            ORDER BY 2 DESC\n            LIMIT 10\"\"\")\n        .bind(\"since\", LocalDate.of(2023, 1, 1))\n        .bind(\"until\", LocalDate.of(2023, 3, 31))\n        .mapTo(SkuSales.class)\n        .list();\n\nThis query computes the top 10 SKUs with the most sales in Q1 2023.\n\nIn this example, we see our first `RowMapper`, which is custom code used to map a SQL query result row to a Java bean. In this case, each row is mapped to a `SkuSales` object. Note that the registering the `RowMapper` for the `SkuSales` class during initialization effectively decouples the serialization of records from business logic.\n\n### DML\n\nThe library also supports [DML operations](https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax).\n\n    Jdbq jdbq=createJdbq();\n\n    long deleted=jdbq.createUpdate(\"\"\"\n            DELETE FROM sales\n            WHERE sku=:sku AND quantity=0\"\"\")\n        .bind(\"sku\", \"1234\")\n        .execute();\n\nThis query deletes all sales records with sku `1234` and quantity `0`.\n\n### QueryFragment\n\nJDBQ does have one important innovation over the rote JDBI feature set: the `QueryFragment`. A `QueryFragment` allows users to bundle SQL along with attributes and arguments for use in a query, which may contain other `QueryFragment` instances, and so on. For example:\n\n    Jdbq jdbq=createJdbq();\n    jdbq.get(SqlStatements.class).setTemplateEngine(new QueryFragmentTemplateEngine());\n    \n    record SkuSales(String sku, long quantity) {}\n    \n    jdbq.getConfig(RowMappers.class).register(new RowMapper\u003cSkuSales\u003e() {\n        @Override\n        public SkuSales map(FieldValueList fvs, StatementContext ctx) {\n            String sku=fvs.get(\"sku\").getStringValue();\n            long quantity=fvs.get(\"quantity\").getLongValue();\n            return new SkuSales(sku, quantity);\n        }\n    });\n    \n    QueryFragment buyerPredicate;\n    if(filterToBuyer != null) {\n        buyerPredicate = new QueryPredicate(\"buyer=:buyer\").bind(\"buyer\", filterToBuyer);\n    }\n    else {\n        buyerPredicate = new QueryPredicate(\"TRUE\");\n    }\n\n    List\u003cSkuSales\u003e sales=jdbq.createQuery(\"\"\"\n            SELECT\n                sku AS sku,\n                SUM(quantity) AS quantity\n            FROM sales\n            WHERE timestamp BETWEEN :since AND :until\n                AND (\u003cBUYER\u003e)\n            GROUP BY 1\n            ORDER BY 2 DESC\n            LIMIT 10\"\"\")\n        .define(\"BUYER\", buyerPredicate)\n        .bind(\"since\", LocalDate.of(2023, 1, 1))\n        .bind(\"until\", LocalDate.of(2023, 3, 31))\n        .mapTo(SkuSales.class)\n        .list();\n\nThis query computes the top 10 SKUs with the most sales in Q1 2023 from the given optional buyer.\n\nNote that the buyer predicate includes an argument. Using a `QueryFragment`, the entire predicate is self-contained because it supports not only SQL but also attributes and arguments, and is therefore reusable. If this were handled without `QueryFragment`, then the builder of the overall query would have to know about how the predicate works, which violates encapsulation and reduces reusability.\n\nThe `QueryFragment` feature allows users to divide and conquer query generation, as well as to reuse components of query generation more freely. This style of query generation is sometimes referred to as the [specification pattern](https://en.wikipedia.org/wiki/Specification_pattern).\n\nEach `QueryFragment` has its own logical \"namespace,\" which means that users don't have to worry about attribute or argument name overlap between `QueryFragment` instances, even when used in the same query.\n\n## Extensibility\n\nThe following key features have been brought over from JDBI:\n\n* Column mappers\n* Row mappers\n* Query customizers\n* Pluggable template engines\n* Custom arguments\n\n## FAQ\n\n### Why not just use JDBI with the Simba JDBC driver?\n\nThat is a fine option, and may work for many businesses. However, not all BigQuery features are available through the JDBC driver, and not all users are comfortable using a closed-source JDBC driver for their data lake. In short: feel free to use the JDBC option if it works for you, but it does not work for everyone.\n\n## Roadmap\n\nMore features, such as [JDBI-style annotated methods](https://jdbi.org/#_annotated_methods), may be added to the library if there is demand.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsigpwned%2Fjdbq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsigpwned%2Fjdbq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsigpwned%2Fjdbq/lists"}