{"id":13480838,"url":"https://github.com/EnterpriseDB/mongo_fdw","last_synced_at":"2025-03-27T11:30:58.877Z","repository":{"id":17599581,"uuid":"20403299","full_name":"EnterpriseDB/mongo_fdw","owner":"EnterpriseDB","description":"PostgreSQL foreign data wrapper for MongoDB","archived":false,"fork":true,"pushed_at":"2025-03-18T03:24:40.000Z","size":1058,"stargazers_count":331,"open_issues_count":58,"forks_count":72,"subscribers_count":42,"default_branch":"master","last_synced_at":"2025-03-18T04:27:44.657Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"citusdata/mongo_fdw","license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EnterpriseDB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-06-02T12:37:43.000Z","updated_at":"2025-03-18T03:24:44.000Z","dependencies_parsed_at":"2023-02-12T09:15:51.807Z","dependency_job_id":null,"html_url":"https://github.com/EnterpriseDB/mongo_fdw","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EnterpriseDB%2Fmongo_fdw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EnterpriseDB%2Fmongo_fdw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EnterpriseDB%2Fmongo_fdw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EnterpriseDB%2Fmongo_fdw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EnterpriseDB","download_url":"https://codeload.github.com/EnterpriseDB/mongo_fdw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245835979,"owners_count":20680303,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:00:45.687Z","updated_at":"2025-03-27T11:30:58.867Z","avatar_url":"https://github.com/EnterpriseDB.png","language":"C","funding_links":[],"categories":["C","Tools"],"sub_categories":["Data"],"readme":"MongoDB Foreign Data Wrapper for PostgreSQL\n============================================\n\nThis PostgreSQL extension implements a Foreign Data Wrapper (FDW) for\n[MongoDB][1].\n\nPlease note that this version of mongo_fdw works with PostgreSQL and EDB\nPostgres Advanced Server 12, 13, 14, 15, 16 and 17.\n\nContents\n--------\n\n1. [Features](#features)\n2. [Supported platforms](#supported-platforms)\n3. [Installation](#installation)\n4. [Usage](#usage)\n5. [Functions](#functions)\n6. [Character set handling](#character-set-handling)\n7. [Examples](#examples)\n8. [Limitations](#limitations)\n9. [Contributing](#contributing)\n10. [Support](#support)\n11. [Useful links](#useful-links)\n12. [License](#license)\n\n\nFeatures\n--------\n\nThe following enhancements are added to the latest version of `mongo_fdw`:\n\n#### Write-able FDW\nThe previous version was only read-only, the latest version provides the\nwrite capability. The user can now issue an insert, update, and delete\nstatements for the foreign tables using the `mongo_fdw`.\n\n#### Connection Pooling\nThe latest version comes with a connection pooler that utilizes the\nsame MongoDB database connection for all the queries in the same session.\nThe previous version would open a new [MongoDB][1] connection for every\nquery. This is a performance enhancement.\n\n#### JOIN push-down\n`mongo_fdw` now also supports join push-down. The joins between two\nforeign tables from the same remote MongoDB server are pushed to a remote\nserver, instead of fetching all the rows for both the tables and\nperforming a join locally, thereby may enhance the performance. Currently,\njoins involving only relational and arithmetic operators in join-clauses\nare pushed down to avoid any potential join failure. Also, only the\nINNER and LEFT/RIGHT OUTER joins are supported, and not the FULL OUTER,\nSEMI, and ANTI join. Moreover, only joins between two tables are pushed\ndown and not when either inner or outer relation is the join itself.\n\n#### AGGREGATE push-down\n`mongo_fdw` now also supports aggregate push-down. Push aggregates to the\nremote MongoDB server instead of fetching all of the rows and aggregating\nthem locally. This gives a very good performance boost for the cases\nwhere aggregates can be pushed down. The push-down is currently limited\nto aggregate functions min, max, sum, avg, and count, to avoid pushing\ndown the functions that are not present on the MongoDB server. The\naggregate filters, orders, variadic and distinct are not pushed down.\n\n#### ORDER BY push-down\n`mongo_fdw` now also supports order by push-down. If possible, push order\nby clause to the remote server so that we get the ordered result set from\nthe foreign server itself. It might help us to have an efficient merge\njoin. NULLs behavior is opposite on the MongoDB server. Thus to get an\nequivalent result, we can only push-down ORDER BY with either\nASC NULLS FIRST or DESC NULLS LAST. Moreover, as MongoDB sorts only on\nfields, only column names in ORDER BY expressions are pushed down.\n\n#### LIMIT OFFSET push-down\n`mongo_fdw` now also supports limit offset push-down. Wherever possible,\nperform LIMIT and OFFSET operations on the remote server. This reduces\nnetwork traffic between local PostgreSQL and remote MongoDB servers.\n\n#### GUC variables:\n\n  * `mongo_fdw.enable_join_pushdown`: If `true`, pushes the join between two\n    foreign tables from the same foreign server, instead of fetching all the\n    rows for both the tables and performing a join locally. Default is `true`.\n  * `mongo_fdw.enable_aggregate_pushdown`: If `true`, pushes aggregate\n\toperations to the foreign server, instead of fetching rows from the\n\tforeign server and performing the operations locally. Default is `true`.\n  * `mongo_fdw.enable_order_by_pushdown`: If `true`, pushes the order by\n\toperation to the foreign server, instead of fetching rows from the\n\tforeign server and performing the sort locally. Default is `true`.\n\nSupported platforms\n-------------------\n\n`mongo_fdw` was developed on Linux, and should run on any\nreasonably POSIX-compliant system.\n\nInstallation\n------------\n\nAbout script or manual installation, `mongo-c` driver please read the following [instructions in INSTALL.md](INSTALL.md).\n\nIf you run into any issues, please [let us know][2].\n\nUsage\n-----\n\n## CREATE SERVER options\n\n`mongo_fdw` accepts the following options via the `CREATE SERVER` command:\n\n- **address** as *string*, optional, default `127.0.0.1`\n\n  Address or hostname of the MongoDB server.\n\n- **port** as *integer*, optional, default `27017`.\n\n  Port number of the MongoDB server.\n\n- **use_remote_estimate** as *boolean*, optional, default `false`\n\n  Controls whether `mongo_fdw` uses exact rows from\n    remote collection to obtain cost estimates.\n\n- **authentication_database** as *string*, optional\n\n  Database against which user will be\n    authenticated against. Only valid with password based authentication.\n\n- **replica_set** as *string*, optional\n\n  Replica set the server is member of. If set,\n    driver will auto-connect to correct primary in the replica set when\n    writing.\n\n- **read_preference** as *string*, optional, default `primary`\n\n  `primary`, `secondary`, `primaryPreferred`,\n    `secondaryPreferred`, or `nearest`.\n\n- **ssl** as *boolean*, optional, default `false`\n\n  Enable ssl. See http://mongoc.org/libmongoc/current/mongoc_ssl_opt_t.html to\n    understand the options.\n\n- **pem_file** as *string*, optional\n\n  The .pem file that contains both the TLS/SSL certificate and\n    key.\n\n- **pem_pwd** as *string*, optional\n\n  The password to decrypt the certificate key file(i.e. pem_file)\n\n- **ca_file** as *string*, optional\n\n  The .pem file that contains the root certificate chain from the\n    Certificate Authority.\n\n- **ca_dir** as *string*, optional\n\n  The absolute path to the `ca_file`.\n\n- **crl_file** as *string*, optional\n\n  The .pem file that contains the Certificate Revocation List.\n\n- **weak_cert_validation** as *boolean*, optional, default `false`\n\n    Enable the validation checks for TLS/SSL certificates and allows the use of invalid\n    certificates to connect if set to `true`.\n\n- **enable_join_pushdown** as *boolean*, optional, default `true`\n\n  If `true`, pushes the join between two foreign\n\ttables from the same foreign server, instead of fetching all the rows\n\tfor both the tables and performing a join locally. This option can also\n\tbe set for an individual table, and if any of the tables involved in the\n\tjoin has set it to false then the join will not be pushed down. The\n\ttable-level value of the option takes precedence over the server-level\n\toption value.\n\n- **enable_aggregate_pushdown** as *boolean*, optional, default `true`\n\n  If `true`, push aggregates to the remote\n\tMongoDB server instead of fetching all of the rows and aggregating them\n\tlocally. This option can also be set for an individual table. The\n\ttable-level value of the option takes precedence over the server-level\n\toption value.\n\n- **enable_order_by_pushdown** as *boolean*, optional, default `true`\n\n  If `true`, pushes the ORDER BY clause to the foreign server instead of\n    performing a sort locally. This option can also be set for an individual\n    table, and if any of the tables involved in the query has set it to\n    false then the ORDER BY will not be pushed down. The table-level value\n    of the option takes precedence over the server-level option value.\n\n## CREATE USER MAPPING options\n\n`mongo_fdw` accepts the following options via the `CREATE USER MAPPING`\ncommand:\n\n- **username** as *string*, optional\n\n  Username to use when connecting to MongoDB.\n\n- **password** as *string*, optional\n\n  Password to authenticate to the MongoDB server.\n\n## CREATE FOREIGN TABLE options\n\n`mongo_fdw` accepts the following table-level options via the\n`CREATE FOREIGN TABLE` command:\n\n- **database** as *string*, optional, default `test`\n\n  Name of the MongoDB database to query.\n\n- **collection** as *string*, optional, default name of foreign table\n\n  Name of the MongoDB collection to query.\n\n- **enable_join_pushdown** as *boolean*, optional, default `true`\n\n  Similar to the server-level option, but can be\n    configured at table level as well.\n\n- **enable_aggregate_pushdown** as *boolean*, optional, default `true`\n\n  Similar to the server-level option, but can be configured at table level as well.\n\n- **enable_order_by_pushdown** as *boolean*, optional, default `true`\n\n  Similar to the server-level option, but can be configured at table level as well.\n\nNo column-level options are available.\n\n## IMPORT FOREIGN SCHEMA options\n\n`mongo_fdw` don't supports [IMPORT FOREIGN SCHEMA](https://www.postgresql.org/docs/current/sql-importforeignschema.html)\nbecause MongoDB is schemaless.\n\n## TRUNCATE support\n\n`mongo_fdw` don't implements the foreign data wrapper `TRUNCATE` API, available\nfrom PostgreSQL 14, because MongoDB is schemaless.\n\nFunctions\n---------\n\nAs well as the standard `mongo_fdw_handler()` and `mongo_fdw_validator()`\nfunctions, `mongo_fdw` provides the following user-callable utility functions:\n\n- **mongo_fdw_version()**\n\n  Returns the version number as an integer.\n\nCharacter set handling\n----------------------\n\n`BSON` in MongoDB can only be encoded in `UTF-8`. Also `UTF-8` is recommended and\nde-facto most popular PostgreSQL server encoding.\n\nExamples\n--------\n\nAs an example, the following commands demonstrate loading the\n`mongo_fdw` wrapper, creating a server, and then creating a foreign\ntable associated with a MongoDB collection. The commands also show\nspecifying option values in the `OPTIONS` clause. If an option value\nisn't provided, the wrapper uses the default value mentioned above.\n\n`mongo_fdw` can collect data distribution statistics will incorporate\nthem when estimating costs for the query execution plan. To see selected\nexecution plans for a query, just run `EXPLAIN`.\n\n### Install the extension:\n\nOnce for a database you need, as PostgreSQL superuser.\n\n```sql\nCREATE EXTENSION mongo_fdw;\n```\n\n### Create a foreign server with appropriate configuration:\n\nOnce for a foreign data source you need, as PostgreSQL superuser.\n\n```sql\nCREATE SERVER \"MongoDB server\" FOREIGN DATA WRAPPER mongo_fdw OPTIONS (\n  address '127.0.0.1',\n  port '27017'\n);\n```\n\n### Grant usage on foreign server to normal user in PostgreSQL:\n\nOnce for a normal user (non-superuser) in PostgreSQL, as PostgreSQL superuser. It is a good idea to use a superuser only where really necessary, so let's allow a normal user to use the foreign server (this is not required for the example to work, but it's security recommendation).\n\n```sql\nGRANT USAGE ON FOREIGN SERVER \"MongoDB server\" TO pguser;\n```\nWhere `pguser` is a sample user for works with foreign server (and foreign tables).\n\n### User mapping\n\nCreate an appropriate user mapping:\n```sql\nCREATE USER MAPPING FOR pguser SERVER \"MongoDB server\" OPTIONS (\n  username 'mongo_user',\n  password 'mongo_pass'\n);\n```\nWhere `pguser` is a sample user for works with foreign server (and foreign tables).\n\n### Create foreign table\nAll `CREATE FOREIGN TABLE` SQL commands can be executed as a normal PostgreSQL user if there were correct `GRANT USAGE ON FOREIGN SERVER`. No need of PostgreSQL supersuer for security reasons but also works with PostgreSQL supersuer.\n\nCreate a foreign table referencing the MongoDB collection:\n\n```sql\n-- Note: first column of the table must be \"_id\" of type \"name\".\nCREATE FOREIGN TABLE warehouse (\n  _id name,\n  warehouse_id int,\n  warehouse_name text,\n  warehouse_created timestamptz\n) SERVER \"MongoDB server\" OPTIONS (\n    database 'db',\n\tcollection 'warehouse'\n);\n```\n\n### Typical examples with [MongoDB][1]'s equivalent statements.\n\n#### `SELECT`\n```sql\nSELECT * FROM warehouse WHERE warehouse_id = 1;\n```\n```\n           _id            | warehouse_id | warehouse_name |     warehouse_created\n--------------------------+--------------+----------------+---------------------------\n 53720b1904864dc1f5a571a0 |            1 | UPS            | 2014-12-12 12:42:10+05:30\n(1 row)\n```\n```\ndb.warehouse.find\n(\n\t{\n\t\t\"warehouse_id\" : 1\n\t}\n).pretty()\n{\n\t\"_id\" : ObjectId(\"53720b1904864dc1f5a571a0\"),\n\t\"warehouse_id\" : 1,\n\t\"warehouse_name\" : \"UPS\",\n\t\"warehouse_created\" : ISODate(\"2014-12-12T07:12:10Z\")\n}\n```\n#### `INSERT`\n```sql\nINSERT INTO warehouse VALUES (0, 2, 'Laptop', '2015-11-11T08:13:10Z');\n-- Note: The given value for \"_id\" column will be ignored and allows MongoDB to\n-- insert the unique value for the \"_id\" column.\n```\n```\ndb.warehouse.insert\n(\n\t{\n\t\t\"warehouse_id\" : NumberInt(2),\n\t\t\"warehouse_name\" : \"Laptop\",\n\t\t\"warehouse_created\" : ISODate(\"2015-11-11T08:13:10Z\")\n\t}\n)\n```\n#### `DELETE`\n```sql\nDELETE FROM warehouse WHERE warehouse_id = 2;\n```\n```\ndb.warehouse.remove\n(\n\t{\n\t\t\"warehouse_id\" : 2\n\t}\n)\n```\n#### `UPDATE`\n```sql\nUPDATE warehouse SET warehouse_name = 'UPS_NEW' WHERE warehouse_id = 1;\n```\n```\ndb.warehouse.update\n(\n\t{\n\t\t\"warehouse_id\" : 1\n\t},\n\t{\n\t\t\"warehouse_id\" : 1,\n\t\t\"warehouse_name\" : \"UPS_NEW\",\n\t\t\"warehouse_created\" : ISODate(\"2014-12-12T07:12:10Z\")\n\t}\n)\n```\n#### `EXPLAIN`, `ANALYZE`\n```sql\nEXPLAIN SELECT * FROM warehouse WHERE warehouse_id = 1;\n```\n```\n                           QUERY PLAN\n-----------------------------------------------------------------\n Foreign Scan on warehouse  (cost=0.00..0.00 rows=1000 width=84)\n   Filter: (warehouse_id = 1)\n   Foreign Namespace: db.warehouse\n(3 rows)\n```\n```\nANALYZE warehouse;\n```\n\nLimitations\n-----------\n\n  - If the BSON document key contains uppercase letters or occurs within\n    a nested document, ``mongo_fdw`` requires the corresponding column names\n    to be declared in double quotes.\n\n  - Note that PostgreSQL limits column names to 63 characters by\n    default. If you need column names that are longer, you can increase the\n    `NAMEDATALEN` constant in `src/include/pg_config_manual.h`, compile,\n    and re-install.\n\nContributing\n------------\n\nHave a fix for a bug or an idea for a great new feature? Great! Check\nout the contribution guidelines [here][3].\n\nSupport\n-------\nThis project will be modified to maintain compatibility with new\nPostgreSQL and EDB Postgres Advanced Server releases.\n\nIf you need commercial support, please contact the EnterpriseDB sales\nteam, or check whether your existing PostgreSQL support provider can\nalso support `mongo_fdw`.\n\n\nUseful links\n------------\n\n### Documentation\n\n - For details, please refer to [mongo_fdw documentation][5].\n\n### Source code\n\nReference FDW realization, `postgres_fdw`\n - https://git.postgresql.org/gitweb/?p=postgresql.git;a=tree;f=contrib/postgres_fdw;hb=HEAD\n\n### General FDW Documentation\n\n - https://www.postgresql.org/docs/current/ddl-foreign-data.html\n - https://www.postgresql.org/docs/current/sql-createforeigndatawrapper.html\n - https://www.postgresql.org/docs/current/sql-createforeigntable.html\n - https://www.postgresql.org/docs/current/sql-importforeignschema.html\n - https://www.postgresql.org/docs/current/fdwhandler.html\n - https://www.postgresql.org/docs/current/postgres-fdw.html\n\n### Other FDWs\n\n - https://wiki.postgresql.org/wiki/Fdw\n - https://pgxn.org/tag/fdw/\n\n\nLicense\n-------\nPortions Copyright (c) 2004-2025, EnterpriseDB Corporation.\nPortions Copyright © 2012–2014 Citus Data, Inc.\n\nThis program is free software: you can redistribute it and/or modify it\nunder the terms of the GNU Lesser General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or (at\nyour option) any later version.\n\nSee the [`LICENSE`][4] file for full details.\n\n[1]: http://www.mongodb.com\n[2]: https://github.com/enterprisedb/mongo_fdw/issues/new\n[3]: CONTRIBUTING.md\n[4]: LICENSE\n[5]: https://www.enterprisedb.com/docs/mongo_data_adapter/latest/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEnterpriseDB%2Fmongo_fdw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FEnterpriseDB%2Fmongo_fdw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEnterpriseDB%2Fmongo_fdw/lists"}