{"id":25615226,"url":"https://github.com/wednesday-solutions/node-mongo-express","last_synced_at":"2025-04-13T21:13:15.985Z","repository":{"id":38148089,"uuid":"444838540","full_name":"wednesday-solutions/node-mongo-express","owner":"wednesday-solutions","description":"An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.","archived":false,"fork":false,"pushed_at":"2024-07-03T11:55:20.000Z","size":2038,"stargazers_count":27,"open_issues_count":1,"forks_count":11,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-13T21:13:08.093Z","etag":null,"topics":["authorization","circuitbreaker","express","mongodb","mongoose","node-api","node-i18n","node-mongo-api","node-mongo-express","node-mongo-express-api","nodejs","oauth2","rbac","rest-api","sharding"],"latest_commit_sha":null,"homepage":"https://wednesday.is/building-products/?utm_source=github\u0026utm_medium=node-mongo-express","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wednesday-solutions.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-05T14:41:44.000Z","updated_at":"2025-01-17T21:15:35.000Z","dependencies_parsed_at":"2024-05-14T11:53:17.578Z","dependency_job_id":"e9bdb33e-36b1-4aa3-9f53-7fc7aad52efa","html_url":"https://github.com/wednesday-solutions/node-mongo-express","commit_stats":null,"previous_names":[],"tags_count":45,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wednesday-solutions%2Fnode-mongo-express","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wednesday-solutions%2Fnode-mongo-express/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wednesday-solutions%2Fnode-mongo-express/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wednesday-solutions%2Fnode-mongo-express/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wednesday-solutions","download_url":"https://codeload.github.com/wednesday-solutions/node-mongo-express/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248782260,"owners_count":21160717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["authorization","circuitbreaker","express","mongodb","mongoose","node-api","node-i18n","node-mongo-api","node-mongo-express","node-mongo-express-api","nodejs","oauth2","rbac","rest-api","sharding"],"created_at":"2025-02-22T03:18:59.625Z","updated_at":"2025-04-13T21:13:15.916Z","avatar_url":"https://github.com/wednesday-solutions.png","language":"JavaScript","readme":"\u003cimg align=\"left\" src=\"./node-mongo-express.svg\" width=\"480\"  /\u003e\n\n\u003cdiv\u003e\n  \u003ca href=\"https://www.wednesday.is/?utm_source=github\u0026utm_medium=node-mongo-express\" align=\"left\" style=\"margin-left: 0;\"\u003e\n    \u003cimg src=\"https://uploads-ssl.webflow.com/5ee36ce1473112550f1e1739/5f5879492fafecdb3e5b0e75_wednesday_logo.svg\"\u003e\n  \u003c/a\u003e\n  \u003cp\u003e\n    \u003ch1 align=\"left\"\u003eNode Mongo Express\n    \u003c/h1\u003e\n  \u003c/p\u003e\n\n  \u003cp\u003e\nAn enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.\n  \u003c/p\u003e\n\n---\n\n  \u003cp\u003e\n    \u003ch4\u003e\n      Expert teams of digital product strategists, developers, and designers.\n    \u003c/h4\u003e\n  \u003c/p\u003e\n\n  \u003cdiv\u003e\n    \u003ca href=\"https://www.wednesday.is/contact-us/?utm_source=github\u0026utm_medium=node-mongo-express\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://uploads-ssl.webflow.com/5ee36ce1473112550f1e1739/5f6ae88b9005f9ed382fb2a5_button_get_in_touch.svg\" width=\"121\" height=\"34\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/wednesday-solutions/\" target=\"_blank\"\u003e\n      \u003cimg src=\"https://uploads-ssl.webflow.com/5ee36ce1473112550f1e1739/5f6ae88bb1958c3253756c39_button_follow_on_github.svg\" width=\"168\" height=\"34\"\u003e\n    \u003c/a\u003e\n  \u003c/div\u003e\n\n---\n\n\u003cspan\u003eWe’re always looking for people who value their work, so come and join us. \u003ca href=\"https://www.wednesday.is/hiring/?utm_source=github\u0026utm_medium=node-mongo-express\"\u003eWe are hiring!\u003c/a\u003e\u003c/span\u003e\n\n\u003c/div\u003e\n\n\u003c!-- # node-mongo-express\n\n\nA basic starter for a web app with node, express and mongoose --\u003e\n\n[![Node Mongo Express CI](https://github.com/wednesday-solutions/node-mongo-express/actions/workflows/ci.yml/badge.svg)](https://github.com/wednesday-solutions/node-mongo-express/actions/workflows/ci.yml)\n\n[![Node Mongo Express CD](https://github.com/wednesday-solutions/node-mongo-express/actions/workflows/cd.yml/badge.svg)](https://github.com/wednesday-solutions/node-mongo-express/actions/workflows/cd.yml)\n\n---\n\n\u003cdiv\u003e\n\u003cimg src='./badges/badge-statements.svg' height=\"20\"/\u003e\n\u003cimg src='./badges/badge-branches.svg' height=\"20\"/\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n\u003cimg src='./badges/badge-lines.svg'  height=\"20\"/\u003e\n\u003cimg src='./badges/badge-functions.svg' height=\"20\"/\u003e\n\u003c/div\u003e\n\n---\n\n## Pre-requisites\n\n    - yarn\n    - docker\n\n## Features\n\n    - Mongo support\n    - Docker support\n    - Rate limited APIs\n    - RBAC middleware using Auth0\n    - Sharding mongoDB collection support\n    - Paginated APIs\n    - Autogenerated APIs from mongoose models\n    - Built in slack alerting mechanism\n    - Suport for redis cache\n    - Support for aggregate caching\n    - Support for batch jobs in multi-container environment\n    - Support for circuit breakers\n    - Autogenerated swagger documentation\n    - Load testing using k6\n    - Support for i18n\n\n## Running Load tests\n\n- [Install](https://k6.io/docs/getting-started/installation/) k6\n- Execute the following command: `k6 run __tests__/__load__/script.js`\n## Build and run docker container locally\n\n    - docker-compose down\n    - docker-compose build\n    - docker-compose up\n\n# Shard setup\n\nRun the following script\n\n```\n./setup-shards/scripts/setup/base.sh\n```\n\nTake a look at [this](./setup-shards/README.md) to create shards and replica sets.\n\n## Seeders\n\nRun the following command to begin seeding\n\n```\n./seeders/seed.sh\n```\n\n## How to start\n\n    - cd `node-mongo-express`\n    - yarn\n    - ./setup-shards/scripts/setup/base.sh\n    - cp .env.example .env.local\n    - ./seeders/seed.sh\n    - yarn start\n    - open browser to `localhost:9000` (port default to 9000)\n\n## API Documentation\n\nOnce you've to the server started check out the api documentation at [/api-docs](http://localhost:9000/api-docs)\n\n## Navigating the code base\n\n-   The entry point of the application is the [server/index.js](./server/index.js)\n-   The server/app.js imports the APIs from [server/api/index.js](./server/api/index.js)\n-   All the different APIs in the [server/api](./server/api) are registered [here](./server/api/index.js)\n-   MongoDB is used to store data \u0026 mongoose is used as the ORM\n    -   [mongo](./server/database/mongo.js)\n    -   [models](./server/database/models/)\n-   The template has support for the following middlewares\n    -   [auth](./server/middlewares/auth/)\n    -   [injectRequestId](./server/middlewares/injectRequestId)\n    -   [rateLimiter](./server/middlewares/rateLimiter)\n-   The template has inbuilt support for\n    -   [redis](./server/services/redis.js)\n    -   [circuitBreakers](./server/services/circuitBreaker.js)\n    -   [slack alerts](./server/utils/slackNotify.js)\n    -   [docker](./Dockerfile)\n    -   [docker-compose](./docker-compose.yml)\n    -   [auto generated apis](./server/api/requestGenerators.js)\n    -   [sharding of collections](./setup-shards)\n    -   [aggregate caching](./server/api/aggregate/)\n\n## Philosophy\n\nWhen using NoSQLs you are optimising for read performance. We're doing this by denormalising data. There are multiple copies of the same data. For example\n\n-   Orders contains purchasedProducts which contains Products. Instead of referencing here we embed\n-   SupplierProducts contains embedded objects for both Suppliers and Products\n-   StoreProducts contains embedded objects for both Stores and Products\n\nThis makes our application write heavy. Every time there is a change to a product we need to make a change to\n\n-   SupplierProducts\n-   StoreProducts\n-   Products\n\nOrders is not impacted since a change in the product after purchase will not affect the order.\n\nHowever the application is able to perform extremely fast reads. 2 reasons for better performance is\n\n-   shards\n-   document embedding\n\nNoSQLs are also good for handling large volumes of data. This is supported due to its ability to have shards. In this application we create 4 shards and the data is distributed amongst these shards.\n\nThese are the shard keys that we use\n\n-   \\_id\n    -   Order\n-   name\n    -   Products\n    -   Suppliers\n    -   Stores\n        \u003cbr/\u003eWe got really good distribution across shards(24-26%) per shard after seeding 4 million records. It's possible to get a hot shard due to this but we're yet to see that.\n-   productId\n    -   SupplierProducts\n    -   StoreProducts\n        \u003cbr/\u003eproductId is chosen as the shard key since we anticipate that the queries for fetching all suppliers/stores that sell a particular product will be much more than fetching all products of a supplier/store.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwednesday-solutions%2Fnode-mongo-express","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwednesday-solutions%2Fnode-mongo-express","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwednesday-solutions%2Fnode-mongo-express/lists"}