https://github.com/andreibesleaga/nestjs-mstv

NestJs/Node.js backend template, multi-protocol and microservice design patterns support.
https://github.com/andreibesleaga/nestjs-mstv

backend fastify microservice nestjs nestjs-backend nestjs-boilerplate nodejs template

Last synced: about 2 months ago
JSON representation

NestJs/Node.js backend template, multi-protocol and microservice design patterns support.

Host: GitHub
URL: https://github.com/andreibesleaga/nestjs-mstv
Owner: andreibesleaga
License: mit
Created: 2025-08-10T21:29:37.000Z (11 months ago)
Default Branch: main
Last Pushed: 2026-03-27T07:13:08.000Z (3 months ago)
Last Synced: 2026-03-27T16:48:49.430Z (3 months ago)
Topics: backend, fastify, microservice, nestjs, nestjs-backend, nestjs-boilerplate, nodejs, template
Language: TypeScript
Homepage:
Size: 2.28 MB
Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# Microservice Template Variant

A clean, extensible, maintainable, Node.js based, NestJs extended, microservice template implementing essential microservice design patterns with optional advanced features. Designed for rapid development with production-ready defaults, for single microservice development and multi microservices design architectures.

You can view the [ARCHITECTURE.md](ARCHITECTURE.md) file for more details.

Simple by default, scalable by choice. Enable only what you need.

## ⚡ **Core Features**

- **�️ Feature Flags System** - Flags to enable/disable services as needed
- **🚀 Fast Startup** - Lightweight by default, everything optional
- **� Modular Design** - Focused managers for transport, streaming, scheduling, caching
- **🔧 Type Safety** - Comprehensive TypeScript interfaces eliminate runtime errors
- **🚀 Fast Startup** - Minimal resource usage with selective service loading
- **📊 Performance Monitoring** - Optional performance interceptors and metrics
- **🔧 Environment-Specific Configs** - Different feature sets for dev/test/prod

## 🏗️ **Architecture**

- **Clean Services** - Single responsibility focused managers
- **Protocol Support** - HTTP/REST, GraphQL, WebSocket, MQTT (gRPC optional)
- **Microservice Ready** - Optional TCP, Redis, NATS, RabbitMQ transports
- **Configuration Diet** - Essential configs only, no complexity bloat
- **SOLID Principles** - Maintainable and extensible codebase with basic DDD concept
- **Event-Driven Architecture** - Kafka integration for scalable messaging

## 🗄️ **Database & Persistence**

- **Database Support** - PostgreSQL (via Prisma) or MongoDB with runtime selection
- **Simple Repository Pattern** - Clean data access abstraction
- **Connection Management** - Health checks and proper connection handling
- **Database Migrations** - Prisma/MongoDB migrations and seed scripts
- **Connection Management** - Proper connection pooling and health checks

## 🔐 **Authentication & Security**

- **JWT Authentication** - Access and refresh tokens
- **CASL Authorization** - Role-based access control
- **Password Security** - bcrypt hashing
- **Security Headers** - Helmet, CORS, rate limiting
- **Input Validation** - class-validator integration

## 🚀 **APIs & Communication**

- **REST API** - Fastify Clean HTTP endpoints
- **GraphQL API** - Optional GraphQL schema with resolvers
- **OpenAPI/Swagger** - Interactive API documentation
- **Event Streaming** - Kafka producer/consumer for user lifecycle events
- **Background Jobs** - BullMQ for async tasks

## 🔧 Microservice features (details)

- **Service Discovery** - Consul integration for service registration
- **Distributed Tracing** - Jaeger integration for request tracing
- **Health Checks** - Comprehensive health monitoring endpoints
- **Configuration Management** - Environment-based configuration
- **Graceful Shutdown** - Proper resource cleanup on termination
- **Message Queues** - Optional Kafka/BullMQ for async tasks
- **Real-time** - WebSocket support for live updates

## 🔧 **Optional Advanced Features**

All features disabled by default - enable what you need:

- **Microservice Transports** - TCP, Redis, NATS, RabbitMQ (set `ENABLE_*_MICROSERVICE=true`)
- **Service Discovery** - Consul integration (set `ENABLE_CONSUL_DISCOVERY=true`)
- **Distributed Tracing** - Jaeger integration (set `ENABLE_JAEGER_TRACING=true`)
- **Circuit Breakers** - Resilience patterns (set `ENABLE_CIRCUIT_BREAKER=true`)
- **Performance Monitoring** - Metrics collection (set `ENABLE_PERFORMANCE_MONITORING=true`)
- **Advanced Caching** - Redis cache (set `ENABLE_REDIS_CACHE=true`)

**Philosophy:** Template includes everything but enables only essentials by default.

## 🧪 **Testing & Quality**

- **Unit Tests** - Comprehensive test coverage with mocking
- **Integration Tests** - Real database testing capabilities
- **E2E Tests** - Full application flow testing
- **Performance Tests** - Load testing with concurrent requests
- **Code Quality** - ESLint, Prettier, and pre-commit hooks

## 🐳 **DevOps & Deployment**

- **Docker Support** - Multi-stage builds and compose files
- **Kubernetes Ready** - Helm charts for K8s deployment
- **CI/CD Pipeline** - GitHub Actions with automated testing
- **Production Monitoring** - Health checks and observability
- **Environment Management** - Development, staging, and production configs

## 🚀 **Quick Start**

### 1. Required Environment Setup

```bash
# Copy environment template (simplified configuration)
cp .env.example .env

# For advanced features (MQTT, NATS, gRPC, advanced observability):
# cp .env.example.legacy .env

# Configure database type (postgresql, mysql/mariadb or mongodb)
DATABASE_TYPE=postgresql # or mysql | mariadb | mongodb
DATABASE_URL=postgresql://dev:dev@localhost:5432/dev
# For MySQL/MariaDB:
# MYSQL_URL=mysql://dev:dev@localhost:3306/dev
# MONGODB_URL=mongodb://dev:dev@localhost:27017/nestjs-app

# Security (REQUIRED)
JWT_SECRET=your-strong-secret-key-change-in-production
ALLOWED_ORIGINS=https://yourdomain.com,https://api.yourdomain.com

# Services
REDIS_URL=redis://localhost:6379
KAFKA_BROKERS=localhost:9092

# Microservices (Optional)
CONSUL_HOST=localhost
CONSUL_PORT=8500
JAEGER_ENDPOINT=http://localhost:14268/api/traces

# Protocols (Optional)
SSL_CERT_PATH=/path/to/cert.pem
SSL_KEY_PATH=/path/to/key.pem
MQTT_BROKER_URL=mqtt://localhost:1883
GRPC_PORT=5000

# Security Settings
RATE_LIMIT_MAX=100
RATE_LIMIT_WINDOW=60000
BODY_LIMIT=1048576

# Email/SMTP Configuration
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password
SMTP_FROM=your-email@gmail.com

# Feature Flags - Protocols
ENABLE_WEBSOCKET=true
ENABLE_MQTT=false
ENABLE_HTTPS=true
ENABLE_GRPC=false

# Feature Flags - Services
ENABLE_JAEGER_TRACING=false
ENABLE_REDIS_CACHE=true
ENABLE_CONSUL_DISCOVERY=false
ENABLE_CIRCUIT_BREAKER=false
ENABLE_PERFORMANCE_MONITORING=false
ENABLE_EMAIL_SERVICE=true

```

## Environment Variables Reference

### Database Configuration

- `DATABASE_TYPE`: Database type selection (`postgresql`, `mysql`, `mariadb` or `mongodb`)
- `DATABASE_URL`: PostgreSQL connection string
- `MYSQL_URL`: MySQL/MariaDB connection string (used when `DATABASE_TYPE` is `mysql` or `mariadb`)
- `MONGODB_URL`: MongoDB connection string

### Authentication & Security

- `JWT_SECRET`: Secret key for JWT token signing (required in production)
- `ACCESS_TOKEN_EXP`: Access token expiration time (default: 15m)
- `REFRESH_TOKEN_EXP`: Refresh token expiration time (default: 7d)
- `ALLOWED_ORIGINS`: CORS allowed origins (comma-separated)
- `RATE_LIMIT_MAX`: Maximum requests per window (default: 100)
- `RATE_LIMIT_WINDOW`: Rate limit window in ms (default: 60000)
- `BODY_LIMIT`: Request body size limit in bytes (default: 1048576)

### Service Configuration

- `SERVICE_NAME`: Microservice identifier
- `SERVICE_VERSION`: Service version for discovery
- `SERVICE_HOST`: Service host binding
- `PORT`: HTTP server port (default: 3000)

### External Services

- `REDIS_URL`: Redis connection string for caching and sessions
- `KAFKA_BROKERS`: Kafka broker addresses (comma-separated)
- `CONSUL_HOST`: Consul server host for service discovery
- `CONSUL_PORT`: Consul server port (default: 8500)
- `JAEGER_ENDPOINT`: Jaeger tracing endpoint URL

### Protocol Configuration

- `SSL_CERT_PATH`: SSL certificate file path for HTTPS
- `SSL_KEY_PATH`: SSL private key file path for HTTPS
- `MQTT_BROKER_URL`: MQTT broker connection URL
- `MQTT_USERNAME`: MQTT authentication username
- `MQTT_PASSWORD`: MQTT authentication password
- `WS_PORT`: WebSocket server port (default: 3001)
- `GRPC_PORT`: gRPC server port (default: 5000)

### Email Configuration

- `SMTP_HOST`: SMTP server hostname
- `SMTP_PORT`: SMTP server port (default: 587)
- `SMTP_SECURE`: Use secure connection (default: false)
- `SMTP_USER`: SMTP authentication username
- `SMTP_PASS`: SMTP authentication password
- `SMTP_FROM`: Default sender email address

### Application Settings

- `NODE_ENV`: Environment mode (`development`, `production`, `test`)
- `LOG_LEVEL`: Logging level (`info`, `debug`, `warn`, `error`)
- `APP_URL`: Application base URL for links
- `API_BASE_URL`: API base URL for external references

### 2. Install Dependencies

```bash
pnpm install
```

### 3. Start Services

```bash
# Basic services (PostgreSQL, Redis, Kafka)
pnpm docker:up

# Full stack with MongoDB, Consul, Jaeger (OTel enabled)
pnpm docker:full
```

### 4. Database Setup

**PostgreSQL:**

```bash
pnpm prisma:generate
pnpm prisma:migrate:dev
pnpm prisma:seed
```

**MongoDB:**

```bash
pnpm mongodb:migrate
MONGODB_URL=mongodb://user:pass@localhost:27017/db
```

### 5. Start Application

```bash
pnpm start:dev
```

### 6. Run Tests

```bash
# Unit tests only
pnpm test:unit

# All tests (unit + e2e with mocked DB)
pnpm test:all

# Full test suite (unit + integration + E2E mocked DB)
pnpm test:full

# Full REAL E2E (hits Postgres/Redis/Kafka from docker:full)
pnpm test:e2e:real
```

## 🗄️ **Database Configuration**

### PostgreSQL (Default)

```bash
DATABASE_TYPE=postgresql
DATABASE_URL=postgresql://user:pass@localhost:5432/db
```

### MySQL / MariaDB

Set the database type and URL, then generate Prisma client and run migrations:

```bash
DATABASE_TYPE=mysql # or mariadb
MYSQL_URL=mysql://user:pass@localhost:3306/db
```

Notes:

- Prisma schema provider must match the selected engine. This repo includes a small helper script that switches the Prisma datasource `provider` based on `DATABASE_TYPE` automatically before `prisma generate` and `start:dev`.
- For a fresh MySQL/MariaDB setup, run:
- pnpm prisma:generate
- pnpm prisma:migrate:dev --name init
- pnpm prisma:seed (optional)
- Existing PostgreSQL migrations won’t apply to MySQL. Create new migrations when switching engines in development.

### MongoDB

```bash
DATABASE_TYPE=mongodb
```

The application automatically selects the appropriate repository implementation based on `DATABASE_TYPE`.

- Prisma (PostgreSQL): Prisma manages connections internally. In production, it’s recommended to front Postgres with PgBouncer and point `DATABASE_URL` to the pooler for robust pooling at scale.
- You can override the datasource URL at runtime via `DATABASE_URL` (already wired in `PrismaService`).
- MongoDB: The driver’s pool is enabled by default. You can tune with optional envs:
- `MONGODB_MAX_POOL_SIZE`, `MONGODB_MIN_POOL_SIZE`, `MONGODB_MAX_IDLE_TIME_MS`, `MONGODB_WAIT_QUEUE_TIMEOUT_MS`

Docker full stack includes a PgBouncer service. The app is preconfigured to use it in `docker/docker-compose.full.yml`:

```yaml
DATABASE_URL: 'postgresql://dev:dev@pgbouncer:6432/dev?pgbouncer=true'
```

## 🔧 **Microservice Features**

### Service Discovery (Consul)

```bash
CONSUL_HOST=localhost
CONSUL_PORT=8500
SERVICE_NAME=nestjs-api
```

### Distributed Tracing (Jaeger)

```bash
JAEGER_ENDPOINT=http://localhost:14268/api/traces
```

### Event Streaming (Kafka)

```bash
KAFKA_BROKERS=localhost:9092
```

### Background Jobs (BullMQ + Redis)

```bash
REDIS_URL=redis://localhost:6379
```

### Circuit Breaker (Fastify + programmatic)

Enable a service-wide circuit breaker to protect endpoints and wrap outbound calls.

Env flags:

```env
ENABLE_CIRCUIT_BREAKER=true
# Trip circuit after N consecutive failures
CB_THRESHOLD=5
# Per-request timeout (ms) before considering a failure
CB_TIMEOUT=2000
# Time (ms) to stay open before trying a half-open probe
CB_RESET_TIMEOUT=10000
```

Behavior:

- When the threshold is reached, requests get HTTP 503 with message "Circuit open" until reset timeout elapses.
- After `CB_RESET_TIMEOUT`, one request is allowed in half-open state to probe recovery.
- The Fastify plugin is applied globally to all routes when enabled.

Programmatic usage for outbound HTTP calls (via `HttpClientService`):

```ts
// Example
await httpClient.fetch('https://api.example.com', { timeoutMs: 1500, cbName: 'upstreamA' });
```

Introspection endpoint:

```bash
curl http://localhost:3000/health/circuit/global
```

Notes: Uses @fastify/circuit-breaker (Fastify v4 compatible). E2E tests cover open/half-open/close semantics.

## Storage (S3 / Azure Blob / GCS / In-memory)

A pluggable storage service with a simple interface for upload/download/list/delete and signed URLs. By default, an in-memory adapter is used (great for tests). Cloud adapters are wired to their real SDKs but are disabled by default behind a feature flag.

Env flags:

```env
# Master on/off switch (defaults to false → memory adapter)
ENABLE_STORAGE=false

# Select provider: aws | azure | gcp | none (defaults to none → in-memory)
STORAGE_PROVIDER=none

# AWS S3 (when STORAGE_PROVIDER=aws)
S3_BUCKET=your-bucket
S3_REGION=us-east-1
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
# Optional for S3-compatible stores (e.g., MinIO):
S3_ENDPOINT=http://localhost:9000

# Azure Blob (when STORAGE_PROVIDER=azure)
AZURE_BLOB_CONNECTION_STRING=...
AZURE_BLOB_CONTAINER=your-container

# Google Cloud Storage (when STORAGE_PROVIDER=gcp)
GCP_PROJECT_ID=your-project
GCS_BUCKET=your-bucket
# One of these (standard GCP ways to auth):
GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
# or
GCP_APPLICATION_CREDENTIALS=/path/to/key.json
```

Streaming usage (large files) via demo endpoints in development/test environments:

- Upload (optionally gzip on the fly):
- curl -X POST --data-binary @bigfile.bin ""
- Download (optionally gunzip on the fly):
- curl -L "" -o out.bin

Programmatic usage example:

```ts
import { Controller, Post, Get, Param, Body } from '@nestjs/common';
import { StorageService } from '@/common/storage/storage.service';

@Controller('files')
export class FilesController {
constructor(private readonly storage: StorageService) {}

@Post()
async upload(@Body() body: { key: string; content: string }) {
await this.storage.upload(body.key, Buffer.from(body.content, 'utf8'), {
contentType: 'text/plain',
});
return { ok: true };
}

@Get(':key')
async get(@Param('key') key: string) {
const buf = await this.storage.download(key);
return buf.toString('utf8');
}
}
```

## �🛠️ **Development Workflow**

### Code Quality

- **Linting**: `pnpm lint` - ESLint with TypeScript support
- **Formatting**: `pnpm format` - Prettier code formatting
- **Pre-commit hooks**: Automatically run linting, formatting, and unit tests
- **Pre-push hooks**: Run full test suite before pushing

### Testing

- **Unit tests**: `pnpm test:unit` - 90 tests with full mocking
- **E2E tests**: `pnpm test:e2e` - Application startup and endpoint tests
- **All tests**: `pnpm test:all` - Test suite for CI/CD
- **Coverage**: Add `--coverage` flag to any test command for coverage reports

All tests use comprehensive mocking (Prisma, Redis, bcrypt) and run without external dependencies.

## 🔐 **Authentication & Authorization**

### Auth Endpoints

- **Register**: `POST /auth/register` - Create new user account
- **Login**: `POST /auth/login` - Authenticate and get tokens
- **Refresh**: `POST /auth/refresh` - Refresh access token
- **Logout**: `POST /auth/logout` - Revoke refresh token
- **Profile**: `GET /auth/profile` - Get current user (protected)
- **Users**: `GET /auth/users` - List all users (admin only)

### Security Features

- **JWT Tokens**: Short-lived access tokens (15m) + refresh tokens (7d)
- **Token Revocation**: Redis-based blacklist for immediate logout
- **Password Security**: bcrypt hashing with configurable rounds
- **Rate Limiting**: Configurable request limits per IP
- **CORS Protection**: Environment-based origin restrictions
- **Security Headers**: Helmet integration with CSP
- **Input Validation**: Comprehensive validation with sanitization

### Role-Based Access Control (RBAC)

- **CASL Integration**: Fine-grained permission system
- **Policy Guards**: Declarative permission checks
- **Role Management**: User and admin roles with different capabilities
- **Resource Protection**: Method-level authorization

## 🌐 **API Access**

### REST API

- **Base URL**: `http://localhost:3000`
- **Documentation**: `http://localhost:3000/api` (Swagger UI)
- **OpenAPI Spec**: `http://localhost:3000/api-json`
- **Health Check**: `http://localhost:3000/health`

### GraphQL API

- **Endpoint**: `http://localhost:3000/graphql`
- **Playground**: `http://localhost:3000/graphql` (development mode)
- **Schema**: Auto-generated from resolvers
- **Introspection**: Enabled in development

### Schema Documentation

- **GraphQL Schema**: `http://localhost:3000/schemas/graphql`
- **Kafka Events**: `http://localhost:3000/schemas/kafka`
- **All Schemas**: `http://localhost:3000/schemas`

### Monitoring & Observability

- **Health Checks**: `http://localhost:3000/health`
- **Readiness**: `http://localhost:3000/health/ready`
- **Liveness**: `http://localhost:3000/health/live`
- **Jaeger UI**: `http://localhost:16686` (when enabled)
- **Consul UI**: `http://localhost:8500` (when enabled)

## Observability (OpenTelemetry + Logging)

This template includes OpenTelemetry (OTel) support for distributed tracing and metrics, plus structured logs enriched with correlation metadata.

### Enable OTel

Set the following in your `.env`:

```env
ENABLE_OPENTELEMETRY=true
SERVICE_NAME=nestjs-mstv

# Traces (OTLP over HTTP)
OTEL_TRACES_EXPORTER=otlp
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
# Optional auth headers (JSON or comma-separated key=value pairs)
# OTEL_EXPORTER_OTLP_HEADERS=api-key=xxx

# Metrics (choose one)
OTEL_METRICS_EXPORTER=none # prometheus | otlp | none
ENABLE_PROMETHEUS_METRICS=false
PROMETHEUS_HOST=0.0.0.0
PROMETHEUS_PORT=9464

# Convenience endpoints (OTLP-compatible)
# SigNoz
ENABLE_SIGNOZ_TRACING=false
SIGNOZ_ENDPOINT=http://localhost:4318/v1/traces

# Datadog
ENABLE_DATADOG_TRACING=false
DATADOG_OTLP_ENDPOINT=http://localhost:4318/v1/traces
```

The app auto-initializes OTel at startup when `ENABLE_OPENTELEMETRY=true`.

### Exporters

- Jaeger (via OTLP): set `OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger-collector:4318`
- SigNoz: set `SIGNOZ_ENDPOINT=http://signoz-otel-collector:4318/v1/traces` (or use `OTEL_EXPORTER_OTLP_ENDPOINT`)
- Datadog: set `DATADOG_OTLP_ENDPOINT=http://datadog-agent:4318/v1/traces` (or use `OTEL_EXPORTER_OTLP_ENDPOINT` and headers)
- Prometheus metrics: set `ENABLE_PROMETHEUS_METRICS=true` (scrape `http://host:9464/metrics`)
- OTLP metrics: set `OTEL_METRICS_EXPORTER=otlp` and `OTEL_EXPORTER_OTLP_ENDPOINT`

Notes

- All three backends (Jaeger, SigNoz, Datadog) can ingest OTLP; pick the URL that matches your setup.
- Optional headers for auth can be provided via `OTEL_EXPORTER_OTLP_HEADERS`.

### Log Correlation

Pino logs automatically include these fields per request:

- `traceId`, `requestId`, `userId`, `method`, `url`, `ip`, plus `service` and `env`.

Every HTTP response also includes the headers:

- `x-trace-id` and `x-request-id` for cross-system correlation.

If OpenTelemetry is enabled, the logger will reuse the active OTel trace id, aligning logs with traces.

## API Examples

### REST Endpoints

```bash
# Register user
curl -X POST http://localhost:3000/auth/register \
-H "Content-Type: application/json" \
-d '{"email":"user@example.com","password":"password123","name":"John Doe"}'

# Login
curl -X POST http://localhost:3000/auth/login \
-H "Content-Type: application/json" \
-d '{"email":"user@example.com","password":"password123"}'

# Get profile (with JWT token)
curl -X GET http://localhost:3000/auth/profile \
-H "Authorization: Bearer YOUR_JWT_TOKEN"
```

### GraphQL Queries

```graphql
# Register user
mutation {
register(input: { email: "user@example.com", password: "password123", name: "John Doe" }) {
id
email
name
role
}
}

# Login
mutation {
login(input: { email: "user@example.com", password: "password123" }) {
access_token
refresh_token
user {
id
email
name
role
}
}
}

# Get current user
query {
me {
id
email
name
role
}
}

# Get all users (admin only)
query {
getAllUsers {
id
email
name
role
createdAt
}
}
```

## 📨 **Event-Driven Architecture**

### Kafka Events

- **User Events**: `user.registered`, `user.updated`, `user.deleted`
- **Auth Events**: `user.logged_in`, `user.logged_out`, `token.refreshed`
- **Email Events**: `email.welcome`, `email.password_reset`, `email.verification`

### Background Jobs (BullMQ)

- **Email Processing**: Welcome emails, password resets, verification (provider not implemented)
- **Queue Management**: Job retry logic and dead letter queues
- **Monitoring**: Queue statistics and job status tracking

### Event Schema

All events follow standardized schemas defined in `/src/schemas/kafka.schemas.ts`:

```typescript
{
event: 'user.registered',
userId: 'cuid123',
email: 'user@example.com',
timestamp: '2023-01-01T00:00:00.000Z'
}
```

## 🐳 **Docker & Deployment**

### Development

```bash
# Basic services
docker-compose -f docker/docker-compose.yml up -d

# Full microservice stack
docker-compose -f docker/docker-compose.full.yml up -d
```

### Production (Kubernetes)

```bash
# Deploy with Helm
helm upgrade --install nestjs-api ./helm/nest-ddd-chart

# With custom values
helm upgrade --install nestjs-api ./helm/nest-ddd-chart -f values.prod.yaml
```

### CI/CD Pipeline

- **GitHub Actions**: Automated testing and deployment
- **Multi-stage builds**: Optimized Docker images
- **Health checks**: Kubernetes readiness and liveness probes
- **Rolling updates**: Zero-downtime deployments

## 🧪 **Testing Guide**

Comprehensive testing strategy with multiple test types and environments.

## 🧪 **Test Types**

### Unit Tests

- **Command**: `pnpm test:unit`
- **Coverage**: 90 tests covering core business logic
- **Mocks**: All external dependencies (Prisma, Redis, Kafka, BullMQ)
- **Speed**: Fast execution for development feedback

### Integration Tests

- **Command**: `pnpm test:integration`
- **Purpose**: Test with real database connections
- **Setup**: Requires running database services
- **Scope**: Database operations and service interactions

### E2E Tests (Mock)

- **Command**: `pnpm test:e2e`
- **Purpose**: Application startup and endpoint availability
- **Environment**: No external dependencies required
- **CI/CD**: Suitable for continuous integration

### E2E Tests (Full)

- **Command**: `pnpm test:e2e:full`
- **Purpose**: Application flows with real services
- **Environment**: Full Docker stack required
- **Scope**: End-to-end user journeys

### Performance Tests

- **Command**: `pnpm test:performance`
- **Purpose**: Load testing and concurrent request handling
- **Metrics**: Response times and throughput measurement
- **Location**: `test/performance/load.test.ts`

### Test Commands

```bash
# Quick feedback loop
pnpm test:unit

# CI/CD pipeline
pnpm test:all

# Full validation
pnpm test:full

# With coverage
pnpm test:unit --coverage
```

## Mock Configuration

### Global Mocks (test/setup.ts)

- Prisma Client with CRUD operations
- Redis client with get/set/del operations
- bcrypt for password hashing
- Kafka producer/consumer
- BullMQ queue and worker

### E2E Mocks (test/setup.e2e.mock.ts)

- Enhanced Prisma mocks with conditional responses
- Redis mocks for token management
- bcrypt mocks for authentication

## Test Environment

- **NODE_ENV**: Set to 'test' automatically
- **JWT_SECRET**: Test-specific secret
- **DATABASE_URL**: Mock URL to prevent real connections
- **REDIS_URL**: Mock URL for Redis operations

## Authentication in Tests

The `PoliciesGuard` includes special handling for test environments:

- Automatically provides mock admin user when `NODE_ENV=test`
- Allows testing protected endpoints without JWT setup
- Gracefully handles missing context methods in test mocks

## Running Tests

```bash
# Run all tests (recommended for CI)
pnpm test:all

# Run specific test types
pnpm test:unit
pnpm test:e2e
pnpm test:e2e:full

# Run with coverage
pnpm test:unit --coverage

# Run specific test file
pnpm test:unit test/auth.service.spec.ts
```

## Test Structure

```text
test/
├── e2e/ # End-to-end tests
│ ├── mock.e2e.spec.ts # Basic app startup tests
│ ├── auth.e2e.spec.ts # Authentication flow tests
│ ├── graphql.e2e.spec.ts # GraphQL API tests
│ └── users.e2e.spec.ts # User management tests
├── setup.ts # Global test setup with mocks
├── setup.e2e.mock.ts # E2E test setup with enhanced mocks
└── *.spec.ts # Unit test files
```

## CI/CD Integration

The project is configured for GitHub Actions with:

- Pre-commit hooks running unit tests
- Pre-push hooks running all tests
- CI pipeline running `pnpm test:all`

All tests run without external dependencies, making them suitable for any CI environment.

## 🐳 **Docker Testing**

### **Container E2E Tests**

```bash
# Run tests in Docker container (no external dependencies)
docker-compose -f docker/docker-compose.simple-test.yml up --build --abort-on-container-exit

# Run tests with real database services
docker-compose -f docker/docker-compose.test.yml up --build --abort-on-container-exit

# Clean up after tests
docker-compose -f docker/docker-compose.test.yml down --volumes
```

### **Docker Build Testing**

```bash
# Test Docker build process
docker build -f docker/Dockerfile -t nestjs-test .

# Validate Docker Compose configurations
docker-compose -f docker/docker-compose.yml config
docker-compose -f docker/docker-compose.test.yml config
```

### **Available Docker Configurations**

- **docker-compose.yml** - Development with PostgreSQL, Redis, Kafka
- **docker-compose.full.yml** - Full stack with MongoDB, Consul, Jaeger
- **docker-compose.test.yml** - Testing with real database services
- **docker-compose.simple-test.yml** - Testing without external dependencies

## 🏢 **CQRS Architecture**

## 📝 **CQRS Implementation**

### **Command Query Responsibility Segregation**

- **Write Operations** - Commands handled by CommandBus
- **Read Operations** - Queries handled by QueryBus
- **Event Processing** - Events handled by EventBus
- **Auto-discovery** - Handlers registered automatically via decorators

### **Architecture Flow**

```text
Commands → CommandBus → CommandHandlers → Repository → Events
Queries → QueryBus → QueryHandlers → Repository → Results
Events → EventBus → EventHandlers → Side Effects
```

### **Available Commands**

- **CreateUserCommand** - Create new user with validation
- **UpdateUserCommand** - Update user information
- **DeleteUserCommand** - Remove user from system

### **Available Queries**

- **GetUserQuery** - Retrieve user by ID
- **GetAllUsersQuery** - List all users with pagination

### **Available Events**

- **UserCreatedEvent** - Published when user is created
- **UserUpdatedEvent** - Published when user is modified
- **UserDeletedEvent** - Published when user is removed

## 💻 **CQRS Usage Examples**

### **Command Usage**

```typescript
// In your service
const user = await this.commandBus.execute(
new CreateUserCommand('user@example.com', 'password123', 'John Doe')
);
```

### **Query Usage**

```typescript
// In your service
const user = await this.queryBus.execute(new GetUserQuery('user-id-123'));

const users = await this.queryBus.execute(
new GetAllUsersQuery(10, 0) // limit, offset
);
```

### **Event Handling**

```typescript
@EventHandler(UserCreatedEvent)
export class UserCreatedHandler implements IEventHandler {
async handle(event: UserCreatedEvent): Promise {
// Send welcome email, update analytics, etc.
}
}
```

### **Testing CQRS**

```bash
# Run CQRS-specific tests
pnpm test test/cqrs.spec.ts

# All CQRS tests are included in unit tests
pnpm test:unit
```

## 🔧 **Creating Custom CQRS Components**

### **1. Create a Command**

```typescript
export class YourCommand implements ICommand {
readonly type = 'YourCommand';
constructor(public readonly data: any) {}
}
```

### **2. Create a Command Handler**

```typescript
@Injectable()
@CommandHandler(YourCommand)
export class YourCommandHandler implements ICommandHandler {
async execute(command: YourCommand): Promise {
// Handle command logic
}
}
```

### **3. Register in Module**

```typescript
@Module({
imports: [CqrsModule],
providers: [YourCommandHandler],
})
export class YourModule {}
```

## 🌐 **Multi-Protocol Examples**

## **Protocol Usage Examples**

### **HTTPS Secure Requests**

```typescript
// Configure SSL certificates
SSL_CERT_PATH = /path/ot / cert.pem;
SSL_KEY_PATH = /path/ot / key.pem;

// Make secure API calls
const httpsService = new HttpsService(new FeatureFlagsService());
const response = await httpsService.makeSecureRequest('https://api.example.com/data');
```

### **WebSocket Real-time Communication**

```javascript
// Client-side WebSocket connection
const socket = io('ws://localhost:3000/ws');

// Send message
socket.emit('message', { text: 'Hello WebSocket!' });

// Join room for targeted messaging
socket.emit('join-room', 'user-notifications');

// Listen for responses
socket.on('response', (data) => console.log(data));
```

### **MQTT IoT Messaging**

```bash
# Configure MQTT broker
MQTT_BROKER_URL=mqtt://localhost:1883
MQTT_USERNAME=your_username
MQTT_PASSWORD=your_password
```

```ts
// Publish user events
mqttService.publishUserEvent('user123', 'login', { ip: '192.168.1.1' });

// Publish system alerts
mqttService.publishSystemAlert('error', 'Database connection failed');

// Subscribe to topics
mqttService.subscribe('sensors/temperature');
```

### **gRPC High-Performance RPC**

```bash
# Configure gRPC
GRPC_PORT=5000
```

```ts
// gRPC client usage
const client = new UserServiceClient('localhost:5000');

// Create user via gRPC
const user = await client.createUser({
email: 'user@example.com',
name: 'John Doe',
password: 'secure123',
});

// Get user via gRPC
const userData = await client.getUser({ id: 'user123' });
```

### **Testing Protocols**

```bash
# Test all protocol implementations
pnpm test test/protocols.spec.ts

# Protocol tests are included in unit tests
pnpm test:unit
```

## 🔒 **Security Implementation**

## 🛡️ **Multi-Layer Security**

### 1. Helmet (@fastify/helmet)

- **Content Security Policy (CSP)**: Prevents XSS attacks
- **X-Frame-Options**: Prevents clickjacking (set to DENY)
- **X-Content-Type-Options**: Prevents MIME type sniffing
- **Referrer-Policy**: Controls referrer information

### 2. Rate Limiting (@fastify/rate-limit)

- **Default**: 100 requests per minute per IP
- **Configurable**: Via `RATE_LIMIT_MAX` and `RATE_LIMIT_WINDOW` env vars
- **IP-based**: Uses client IP for rate limiting
- **Graceful**: Skips on error to maintain availability

### 3. Compression (@fastify/compress)

- **Encodings**: gzip, deflate
- **Performance**: Reduces bandwidth usage

### 4. Sensible Defaults (@fastify/sensible)

- **HTTP errors**: Standardized error responses
- **Utilities**: Common HTTP status codes and helpers

## Security Headers

### Custom Security Headers (via Fastify hooks)

- **X-Frame-Options**: DENY
- **X-Content-Type-Options**: nosniff
- **Referrer-Policy**: strict-origin-when-cross-origin
- **Permissions-Policy**: Restricts geolocation, microphone, camera

### Cache Control for Sensitive Endpoints

- **Auth endpoints** (`/auth/*`): No caching
- **User endpoints** (`/users/*`): No caching
- **Headers**: Cache-Control, Pragma, Expires

## CORS Configuration

### Development (CORS)

- **Origin**: Allow all origins
- **Credentials**: Enabled

### Production (CORS)

- **Origin**: Restricted to `ALLOWED_ORIGINS` env var
- **Methods**: GET, POST, PUT, DELETE, OPTIONS only
- **Credentials**: Enabled for authenticated requests

## Input Validation & Sanitization

### Global Validation Pipe

- **Whitelist**: Only allow defined properties
- **Transform**: Auto-transform input types
- **Forbid non-whitelisted**: Reject unknown properties
- **Production**: Disable detailed error messages

### Body Limits

- **Default**: 1MB request body limit
- **Configurable**: Via `BODY_LIMIT` env var
- **Protection**: Prevents DoS via large payloads

## Authentication & Authorization

### JWT Security

- **Secret**: Configurable via `JWT_SECRET` env var
- **Expiration**: Short-lived access tokens (15m default)
- **Refresh tokens**: Longer-lived, stored securely

### Token Revocation

- **Redis-based**: Revoked tokens stored in Redis
- **Logout**: Immediate token invalidation
- **Security**: Prevents token reuse after logout

### Role-based Access Control (RBAC)

- **CASL integration**: Fine-grained permissions
- **Guards**: Protect sensitive endpoints
- **Policies**: Declarative permission checks

## Production Security

### Environment-based Configuration

- **Swagger UI**: Disabled in production
- **Error messages**: Sanitized in production
- **Host binding**: Secure host binding for production

### Trust Proxy

- **Enabled**: For proper IP detection behind proxies
- **Rate limiting**: Accurate IP-based limiting
- **Security headers**: Proper forwarded headers handling

## Security Environment Variables

### Required Security Variables

```env
JWT_SECRET=your-strong-secret-key
ALLOWED_ORIGINS=https://yourdomain.com,https://api.yourdomain.com
RATE_LIMIT_MAX=100
RATE_LIMIT_WINDOW=60000
BODY_LIMIT=1048576
```

## Security Best Practices Implemented

1. **Defense in Depth**: Multiple security layers
2. **Principle of Least Privilege**: Minimal permissions by default
3. **Input Validation**: All inputs validated and sanitized
4. **Secure Headers**: Comprehensive security headers
5. **Rate Limiting**: Protection against abuse
6. **HTTPS Ready**: Secure transport layer support
7. **Error Handling**: No sensitive information in errors
8. **Token Security**: Secure JWT implementation with revocation

## Security Testing

All security measures are tested in the test suite:

- Unit tests for authentication logic
- E2E tests for endpoint security
- Validation tests for input sanitization
- Authorization tests for access control

## Monitoring & Logging

- **Request logging**: All requests logged with IP and user agent
- **Error logging**: Security-relevant errors logged
- **Rate limit logging**: Abuse attempts logged
- **Authentication logging**: Login/logout events logged

## Security Updates

Regular updates of dependencies and security patches:

- Automated dependency scanning
- Security vulnerability monitoring
- Regular security audits

---

## 🧭 Coordination with Temporal (Cron + Orchestration + Signals)

This template includes an optional hybrid coordination package using Temporal to support:

- Cron schedules (Temporal Schedules)
- Orchestration (Saga-style workflows + compensations)
- Choreography via Signals (external services signaling progress)

Docs: see `coordination/README.md`.

Quick commands:

```bash
# Start local Temporal and UI
npm run coordination:temporal:up

# Run the coordination worker
npm run coordination:worker

# Run the hybrid smoke (starts workflow and signals it)
npm run coordination:smoke

# One-step orchestrated smoke (brings up Temporal, starts worker, runs smoke)
npm run coordination:smoke:local

# Tear down local Temporal
npm run coordination:temporal:down
```

## 📐 Architecture Diagrams

These diagrams illustrate how this template can be used in different setups. They are examples; tailor to your needs.

### 1) Core architecture (single service)

![Core architecture](diagrams/svg/core_architecture.svg)

### 2) Multi-microservice with API Gateway

![Multi-microservice with API Gateway](diagrams/svg/multi_microservice_gateway.svg)

### 3) Choreography (event-driven) between services

![Choreography (event-driven)](diagrams/svg/choreography_kafka.svg)

### 4) Orchestration (Temporal) across services

![Orchestration (Temporal)](diagrams/svg/orchestration_temporal.svg)

## 🔧 **Microservice Implementation Summary**

### Overview

Comprehensive microservice service in `/src/common/microservice/` enables all NestJS microservice protocols and implementations including RPC, Pub/Sub, AMQP, cron scheduling, and streaming.

#### Core Service Files

- **`microservice.service.ts`** - Main service implementing all microservice protocols
- **`microservice-config.service.ts`** - Configuration management for all transports
- **`microservice.controller.ts`** - HTTP API endpoints for microservice operations
- **`microservice.module.ts`** - NestJS module wiring everything together
- **`example-usage.service.ts`** - Comprehensive examples of service usage
- **`index.ts`** - Export declarations for clean imports
- **`README.md`** - Comprehensive documentation

#### Test Files

- **`test/microservice.integration.spec.ts`** - Integration tests with 29 passing test cases

### Microservice Features Implemented

#### Supported Transports

✅ **TCP** - Simple TCP-based communication
✅ **Redis** - Redis as message broker
✅ **NATS** - NATS messaging system
✅ **RabbitMQ (AMQP)** - Advanced Message Queuing Protocol
✅ **gRPC** - High-performance RPC framework
✅ **Kafka** - Distributed streaming platform (integrated with existing KafkaService)
✅ **MQTT** - Lightweight messaging protocol (integrated with existing MqttService)
✅ **BullMQ** - Job queue system (integrated with existing BullMQService)

#### Additional Features

✅ **Real-time Streaming** - Server-Sent Events with multiple channels
✅ **Cron Scheduling** - Full cron job management with @nestjs/schedule
✅ **Health Monitoring** - Automatic health checks for all transports
✅ **Circuit Breaker** - Fault tolerance configuration
✅ **Retry Logic** - Configurable retry mechanisms
✅ **Comprehensive Configuration** - Environment-based configuration for all features

### Microservice API Endpoints

#### Status & Health

- `GET /microservice/status` - Service status and available transports
- `GET /microservice/health` - Comprehensive health check
- `GET /microservice/metrics` - System and service metrics
- `GET /microservice/config` - Configuration overview

#### Messaging Operations

- `POST /microservice/message` - Send message via any transport
- `POST /microservice/event` - Emit event via any transport
- `POST /microservice/kafka/message` - Send Kafka message
- `POST /microservice/mqtt/message` - Publish MQTT message
- `POST /microservice/queue/job` - Add job to BullMQ queue

#### Streaming Operations

- `POST /microservice/stream` - Stream data to channel
- `GET /microservice/stream/channels` - List available channels
- `GET /microservice/stream/:channel` - Subscribe to stream (SSE)
- `GET /microservice/status/live` - Live status updates (SSE)

#### Scheduling Operations

- `POST /microservice/cron` - Add cron job
- `POST /microservice/cron/:name/remove` - Remove cron job

#### Testing

- `POST /microservice/test/:transport` - Test transport connectivity

### Microservice Configuration

#### Environment Variables Added

```bash
# Microservice Transports
ENABLE_TCP_MICROSERVICE=false
ENABLE_REDIS_MICROSERVICE=true
ENABLE_NATS_MICROSERVICE=false
ENABLE_RABBITMQ_MICROSERVICE=false
ENABLE_KAFKA=true
ENABLE_BULLMQ=true
ENABLE_STREAMING=true

# Transport-specific configurations
TCP_HOST=localhost
TCP_PORT=3001
NATS_SERVERS=nats://localhost:4222
RABBITMQ_URL=amqp://localhost:5672
STREAMING_CHANNELS=user-events,system-metrics,audit-logs,notifications,real-time-data

# Scheduler/Cron configurations
ENABLE_HEALTH_CHECK_CRON=true
HEALTH_CHECK_CRON=*/30 * * * *
ENABLE_METRICS_COLLECTION_CRON=false
METRICS_COLLECTION_CRON=*/5 * * * *

# Plus many more detailed configuration options...
```

### Microservice Integration

#### Module Integration

- Added `MicroserviceModule` to `src/apps/api-gateway/app.module.ts`
- Added `MicroserviceController` to expose HTTP endpoints
- Integrated with existing `MessagingModule` and `ProtocolsModule`

#### Dependencies Added

- `@nestjs/schedule` - For cron job management
- `cron` - For cron expression parsing

### Microservice Usage Examples

#### Programmatic Usage

```typescript
// Inject the service
constructor(private readonly microserviceService: MicroserviceService) {}

// Send messages
await microserviceService.sendMessage('redis', 'user.create', userData);

// Stream data
microserviceService.streamData('user-events', eventData);

// Add cron jobs
microserviceService.addCronJob('backup', '0 2 * * *', backupCallback);

// Subscribe to streams
microserviceService.subscribeToStream('notifications').subscribe(msg => {
console.log('Received:', msg);
});
```

#### HTTP API Usage

```bash
# Send a Redis message
curl -X POST http://localhost:3000/microservice/message
-H "Content-Type: application/json"
-d '{"transport": "redis", "pattern": "user.create", "data": {"name": "John"}}'

# Stream data
curl -X POST http://localhost:3000/microservice/stream
-H "Content-Type: application/json"
-d '{"channel": "user-events", "data": {"event": "user_login"}}'

# Subscribe to stream (SSE)
curl http://localhost:3000/microservice/stream/user-events
```

### Microservice Testing

✅ **29 passing test cases** covering:

- Service initialization and configuration
- All transport configurations
- Streaming functionality
- External service integration (Kafka, BullMQ, MQTT)
- Service status and health monitoring
- Cron job management
- Error handling scenarios

### Microservice Architecture

The implementation follows a clean layered architecture:

1. **Controller Layer** - HTTP API endpoints with proper error handling
2. **Service Layer** - Core business logic and transport management
3. **Configuration Layer** - Environment-based configuration management
4. **Integration Layer** - Seamless integration with existing services

### Key Benefits

1. **Unified Interface** - Single service to manage all microservice protocols
2. **Feature Flag Support** - Individual transports can be enabled/disabled
3. **Production Ready** - Comprehensive error handling, health checks, and monitoring
4. **Extensible** - Easy to add new transports or features
5. **Well Tested** - Full test coverage with integration tests
6. **Documentation** - Comprehensive README and inline documentation

### Next Steps

The microservice service is ready for production use. To get started:

1. Configure desired transports via environment variables
2. Start the application: `pnpm start:dev`
3. Access the API at `http://localhost:3000/microservice/status`
4. Use the HTTP endpoints or inject the service programmatically

The implementation provides a solid foundation for building scalable microservice architectures with NestJS.

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/andreibesleaga/nestjs-mstv

Awesome Lists containing this project

README