{"id":49045730,"url":"https://github.com/pbosetti/mdbplyr","last_synced_at":"2026-04-19T16:10:12.577Z","repository":{"id":352002704,"uuid":"1213390337","full_name":"pbosetti/mdbplyr","owner":"pbosetti","description":"dplyr-style interface to MongoDB database","archived":false,"fork":false,"pushed_at":"2026-04-17T12:57:22.000Z","size":1855,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-17T13:29:05.622Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pbosetti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-17T10:28:08.000Z","updated_at":"2026-04-17T12:13:46.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/pbosetti/mdbplyr","commit_stats":null,"previous_names":["pbosetti/mdbplyr"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/pbosetti/mdbplyr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbosetti%2Fmdbplyr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbosetti%2Fmdbplyr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbosetti%2Fmdbplyr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbosetti%2Fmdbplyr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pbosetti","download_url":"https://codeload.github.com/pbosetti/mdbplyr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbosetti%2Fmdbplyr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32012953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-19T16:10:11.963Z","updated_at":"2026-04-19T16:10:12.569Z","avatar_url":"https://github.com/pbosetti.png","language":"R","funding_links":[],"categories":["Libraries"],"sub_categories":["R"],"readme":"# mdbplyr\n\n[![CI](https://github.com/pbosetti/mdbplyr/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/pbosetti/mdbplyr/actions/workflows/ci.yaml)\n\n\u003cimg src=\"media/mdbplyr.png\" width=\"200\" /\u003e\n\nA native tidy lazy backend for MongoDB in R.\n\n## Overview\n\n`mdbplyr` provides a disciplined `dplyr`-style interface for read-only analytical MongoDB queries. Queries stay lazy, compile into MongoDB aggregation pipelines, and only execute at `collect()`.\n\nThe package is intentionally conservative:\n\n- it targets MongoDB aggregation pipelines directly,\n- it does not emulate SQL or extend `dbplyr`,\n- it fails explicitly for unsupported semantics,\n- it avoids silent client-side fallback.\n\n## Install\n\nThe package is still in its initial development phase. While it is under testing and not available on CRAN, you can install it with:\n\n```R\ninstall.package(\"pak\")\npak::pak(\"pbosetti/mdbplyr\")\n```\n\n## Current implemented subset\n\n### Core objects and terminals\n\n- `mongo_src()`\n- `tbl_mongo()`\n- `collect()`\n- `cursor()`\n- `show_query()`\n- `schema_fields()`\n- `append_stage()`\n\n### Supported verbs\n\n- `filter()`\n- `select()`\n- `rename()`\n- `mutate()`\n- `transmute()`\n- `arrange()`\n- `group_by()`\n- `summarise()`\n- `slice_head()`\n- `head()`\n- `inner_join()`\n- `left_join()`\n- `semi_join()`\n- `anti_join()`\n\n### Supported expressions\n\n- field references, including backticked dot paths such as `` `user.age` ``,\n- scalar literals,\n- comparison operators, including `%in%`,\n- boolean operators,\n- arithmetic operators, including `%%` and `^`,\n- `abs()`, `sqrt()`, `log()`, `log10()`, `exp()`, `floor()`, `ceiling()`, `trunc()`, `round()`,\n- `sin()`, `cos()`, `tan()`, `asin()`, `acos()`, `atan()`, `atan2()`,\n- `sinh()`, `cosh()`, `tanh()`, `asinh()`, `acosh()`, `atanh()`,\n- `pmin()`, `pmax()`,\n- `tolower()`, `toupper()`, `nchar()`, `paste()`, `paste0()`, `substr()`, `substring()`,\n- `if_else()`,\n- `case_when()`,\n- `is.na()`,\n- `n()`, `sum()`, `mean()`, `min()`, `max()`.\n\n### Current limits\n\n- `select()` and `rename()` currently support only explicit bare field names,\n- `mutate()` and `transmute()` require named expressions,\n- `group_by()` supports bare field names only,\n- `summarise()` supports only the documented aggregate functions,\n- joins require both tables to be `tbl_mongo` objects backed by collections in the same MongoDB database,\n- `right_join()` and `full_join()` are not supported (MongoDB has no native full outer join stage starting from the left collection),\n- window functions, `across()`, reshaping, and write operations are out of scope.\n\n## Example\n\n```r\nlibrary(mdbplyr)\nlibrary(dplyr)\n\norders \u003c- tbl_mongo(\n  collection = mongolite::mongo(collection = \"orders\", db = \"analytics\"),\n  schema = c(\"customer\", \"amount\", \"status\")\n)\n\nquery \u003c- orders |\u003e\n  filter(status == \"paid\", amount \u003e 0) |\u003e\n  mutate(double_amount = amount * 2) |\u003e\n  group_by(customer) |\u003e\n  summarise(total = sum(double_amount), n = n()) |\u003e\n  arrange(desc(total)) |\u003e\n  slice_head(n = 10)\n\nshow_query(query)\nresult \u003c- collect(query)\n\niter \u003c- cursor(query)\nfirst_page \u003c- iter$page(5)\n```\n\nWhen field metadata is not discoverable from the collection object, pass `schema = ...` to `tbl_mongo()` so that projection and rename operations can stay explicit and lazy.\n\n### Join example\n\n`inner_join`, `left_join`, `semi_join`, and `anti_join` compile lazily to\nMongoDB `$lookup` aggregation stages. Both tables must be `tbl_mongo` objects\nbacked by collections in the **same** MongoDB database.\n\n```r\norders    \u003c- tbl_mongo(mongolite::mongo(\"orders\",    db = \"shop\"),\n                       schema = c(\"order_id\", \"customer_id\", \"amount\"))\ncustomers \u003c- tbl_mongo(mongolite::mongo(\"customers\", db = \"shop\"),\n                       schema = c(\"customer_id\", \"name\", \"country\"))\n\n# Inner join: only orders with a matching customer record\nresult \u003c- inner_join(orders, customers, by = \"customer_id\") |\u003e\n  select(order_id, name, amount) |\u003e\n  collect()\n\n# Left join: all orders, NA for fields of unmatched customers\nresult \u003c- left_join(orders, customers, by = \"customer_id\") |\u003e collect()\n\n# Semi join: orders that have at least one matching customer (no extra columns)\nresult \u003c- semi_join(orders, customers, by = \"customer_id\") |\u003e collect()\n\n# Anti join: orders with no matching customer\nresult \u003c- anti_join(orders, customers, by = \"customer_id\") |\u003e collect()\n\n# Multi-key join\ninner_join(sales, budgets, by = c(\"year\", \"region\")) |\u003e collect()\n\n# Join with a pre-filtered right table (filter is embedded in $lookup)\nactive_customers \u003c- filter(customers, country == \"Italy\")\ninner_join(orders, active_customers, by = \"customer_id\") |\u003e collect()\n\n# Inspect the generated pipeline without executing\nshow_query(inner_join(orders, customers, by = \"customer_id\"))\n```\n\n---\n\n## Project Goal\n\nThe package should provide:\n\n- lazy query composition,\n- tidy evaluation,\n- translation of supported verbs into MongoDB aggregation stages,\n- query inspection,\n- explicit and predictable failure for unsupported operations.\n\nThe package should **not** aim to deliver full `dplyr` compatibility over arbitrary MongoDB collections.\n\n---\n\n## Design Position\n\nThis project should be framed as:\n\n\u003e a native tidy lazy analytical backend for MongoDB.\n\nIt should **not** be framed as:\n\n\u003e a complete `dbplyr` equivalent for MongoDB.\n\nMongoDB documents are not rectangular SQL tables. Nested fields, arrays, missing keys, heterogeneous schemas, and document-oriented semantics require a backend that is native to MongoDB rather than adapted from SQL assumptions.\n\n---\n\n## Support matrix\n\n| Capability | Status | Notes |\n| --- | --- | --- |\n| Lazy query state | Supported | Verbs update internal IR only |\n| Pipeline inspection | Supported | `show_query()` renders compiled JSON |\n| Flat-field filters | Supported | Uses `$match` + `$expr` |\n| Projection and rename | Supported with caveats | Explicit bare field names only |\n| Scalar mutation | Supported | Conservative expression subset |\n| Grouped summaries | Supported | `n()`, `sum()`, `mean()`, `min()`, `max()` |\n| Dot-path fields | Supported with caveats | Use backticked names such as `` `user.age` `` |\n| Manual pipeline stage append | Supported with caveats | `append_stage()` appends raw JSON after generated stages and does not infer schema changes |\n| `inner_join` / `left_join` | Supported | Compiles to `$lookup` (correlated pipeline form, MongoDB 3.6+) |\n| `semi_join` / `anti_join` | Supported | Compiles to `$lookup` + `$match` |\n| `right_join` / `full_join` | Not supported | No native MongoDB equivalent; `right_join` can be rewritten as `left_join` with swapped tables |\n| Window functions | Not supported | Explicitly out of scope |\n| Client-side fallback | Not supported | Unsupported features error clearly |\n\n---\n\n## Core Technical Principles\n\n### 1. Native MongoDB translation\nDo not generate SQL. Do not emulate SQL. Translate directly into MongoDB aggregation pipelines.\n\n### 2. Internal intermediate representation\nIntroduce a package-specific internal query representation between the user API and the pipeline compiler.\n\nThis is a core design requirement. It allows:\n\n- better testing,\n- better diagnostics,\n- cleaner compiler logic,\n- easier future extension.\n\n### 3. Lazy semantics\nAll supported verbs should update query state, not execute immediately.\n\nExecution should occur only at terminal steps such as `collect()`.\n\n### 4. Explicit failure\nUnsupported operations should fail with precise diagnostics. The package should not silently pull data locally and continue computation unless that behavior is deliberately introduced later as an opt-in mode.\n\n### 5. Conservative semantics\nAmbiguous cases should be handled conservatively and documented explicitly, especially for:\n\n- missing fields,\n- `NULL` / `NA` behavior,\n- heterogeneous field types,\n- nested document paths,\n- ordering assumptions.\n\n---\n\n\n## Expected Translation Model\n\nTypical verb mappings are:\n\n- `filter()` -\u003e `$match`\n- `select()` -\u003e `$project`\n- `mutate()` -\u003e `$addFields` or `$project`\n- `arrange()` -\u003e `$sort`\n- `group_by()` + `summarise()` -\u003e `$group`\n- `slice_head()` / `head()` -\u003e `$limit`\n- `inner_join()` / `left_join()` -\u003e `$lookup` + `$unwind` + `$replaceRoot` + `$project`\n- `semi_join()` / `anti_join()` -\u003e `$lookup` + `$match` + `$project`\n\nThis mapping should be documented, inspectable, and testable.\n\n---\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpbosetti%2Fmdbplyr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpbosetti%2Fmdbplyr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpbosetti%2Fmdbplyr/lists"}