https://github.com/wednesday-solutions/node-mongo-express
An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.
https://github.com/wednesday-solutions/node-mongo-express
authorization circuitbreaker express mongodb mongoose node-api node-i18n node-mongo-api node-mongo-express node-mongo-express-api nodejs oauth2 rbac rest-api sharding
Last synced: about 2 months ago
JSON representation
An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.
- Host: GitHub
- URL: https://github.com/wednesday-solutions/node-mongo-express
- Owner: wednesday-solutions
- License: mit
- Created: 2022-01-05T14:41:44.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-07-03T11:55:20.000Z (11 months ago)
- Last Synced: 2025-04-13T21:13:08.093Z (about 2 months ago)
- Topics: authorization, circuitbreaker, express, mongodb, mongoose, node-api, node-i18n, node-mongo-api, node-mongo-express, node-mongo-express-api, nodejs, oauth2, rbac, rest-api, sharding
- Language: JavaScript
- Homepage: https://wednesday.is/building-products/?utm_source=github&utm_medium=node-mongo-express
- Size: 1.94 MB
- Stars: 27
- Watchers: 3
- Forks: 11
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
![]()
Node Mongo Express
An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.
---
Expert teams of digital product strategists, developers, and designers.
---
We’re always looking for people who value their work, so come and join us. We are hiring!
[](https://github.com/wednesday-solutions/node-mongo-express/actions/workflows/ci.yml)
[](https://github.com/wednesday-solutions/node-mongo-express/actions/workflows/cd.yml)
---
![]()
![]()
![]()
![]()
---
## Pre-requisites
- yarn
- docker## Features
- Mongo support
- Docker support
- Rate limited APIs
- RBAC middleware using Auth0
- Sharding mongoDB collection support
- Paginated APIs
- Autogenerated APIs from mongoose models
- Built in slack alerting mechanism
- Suport for redis cache
- Support for aggregate caching
- Support for batch jobs in multi-container environment
- Support for circuit breakers
- Autogenerated swagger documentation
- Load testing using k6
- Support for i18n## Running Load tests
- [Install](https://k6.io/docs/getting-started/installation/) k6
- Execute the following command: `k6 run __tests__/__load__/script.js`
## Build and run docker container locally- docker-compose down
- docker-compose build
- docker-compose up# Shard setup
Run the following script
```
./setup-shards/scripts/setup/base.sh
```Take a look at [this](./setup-shards/README.md) to create shards and replica sets.
## Seeders
Run the following command to begin seeding
```
./seeders/seed.sh
```## How to start
- cd `node-mongo-express`
- yarn
- ./setup-shards/scripts/setup/base.sh
- cp .env.example .env.local
- ./seeders/seed.sh
- yarn start
- open browser to `localhost:9000` (port default to 9000)## API Documentation
Once you've to the server started check out the api documentation at [/api-docs](http://localhost:9000/api-docs)
## Navigating the code base
- The entry point of the application is the [server/index.js](./server/index.js)
- The server/app.js imports the APIs from [server/api/index.js](./server/api/index.js)
- All the different APIs in the [server/api](./server/api) are registered [here](./server/api/index.js)
- MongoDB is used to store data & mongoose is used as the ORM
- [mongo](./server/database/mongo.js)
- [models](./server/database/models/)
- The template has support for the following middlewares
- [auth](./server/middlewares/auth/)
- [injectRequestId](./server/middlewares/injectRequestId)
- [rateLimiter](./server/middlewares/rateLimiter)
- The template has inbuilt support for
- [redis](./server/services/redis.js)
- [circuitBreakers](./server/services/circuitBreaker.js)
- [slack alerts](./server/utils/slackNotify.js)
- [docker](./Dockerfile)
- [docker-compose](./docker-compose.yml)
- [auto generated apis](./server/api/requestGenerators.js)
- [sharding of collections](./setup-shards)
- [aggregate caching](./server/api/aggregate/)## Philosophy
When using NoSQLs you are optimising for read performance. We're doing this by denormalising data. There are multiple copies of the same data. For example
- Orders contains purchasedProducts which contains Products. Instead of referencing here we embed
- SupplierProducts contains embedded objects for both Suppliers and Products
- StoreProducts contains embedded objects for both Stores and ProductsThis makes our application write heavy. Every time there is a change to a product we need to make a change to
- SupplierProducts
- StoreProducts
- ProductsOrders is not impacted since a change in the product after purchase will not affect the order.
However the application is able to perform extremely fast reads. 2 reasons for better performance is
- shards
- document embeddingNoSQLs are also good for handling large volumes of data. This is supported due to its ability to have shards. In this application we create 4 shards and the data is distributed amongst these shards.
These are the shard keys that we use
- \_id
- Order
- name
- Products
- Suppliers
- Stores
We got really good distribution across shards(24-26%) per shard after seeding 4 million records. It's possible to get a hot shard due to this but we're yet to see that.
- productId
- SupplierProducts
- StoreProducts
productId is chosen as the shard key since we anticipate that the queries for fetching all suppliers/stores that sell a particular product will be much more than fetching all products of a supplier/store.