{"id":23268541,"url":"https://github.com/dubems/outbox-nestjs","last_synced_at":"2025-04-06T08:42:01.306Z","repository":{"id":259777199,"uuid":"871008990","full_name":"dubems/outbox-nestjs","owner":"dubems","description":"An outbox Library for solving dual write problem using NestJs and Typeorm ","archived":false,"fork":false,"pushed_at":"2024-11-12T07:11:39.000Z","size":6396,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-12T14:17:50.858Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dubems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-11T04:58:33.000Z","updated_at":"2024-11-12T07:11:43.000Z","dependencies_parsed_at":"2024-11-12T08:28:20.283Z","dependency_job_id":null,"html_url":"https://github.com/dubems/outbox-nestjs","commit_stats":null,"previous_names":["dubems/outbox-nestjs"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dubems%2Foutbox-nestjs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dubems%2Foutbox-nestjs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dubems%2Foutbox-nestjs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dubems%2Foutbox-nestjs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dubems","download_url":"https://codeload.github.com/dubems/outbox-nestjs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247457741,"owners_count":20941906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-19T17:19:24.576Z","updated_at":"2025-04-06T08:42:01.275Z","avatar_url":"https://github.com/dubems.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Transactional Outbox NestJs library\n\nThis library implements the transactional outbox pattern using Nestjs and Typeorm. For more information\non transactional outbox patter, check [here](https://microservices.io/patterns/data/transactional-outbox.html)\n\nCurrently, the library supports event publishing to **Kafka** and **RabbitMQ**\n\n## Installation\n\n````\nnpm install outbox-nestjs -s\n\n````\n\n### Context\n\nOutbox pattern guarantees data consistency for services that, as part of a business process, needs to persist an entity\nand also publish and event/message to a broker.\n\nTo achieve this, the message is stored in the database as part of the\ntransaction that updates the business entities. A separate process then sends the messages to the message broker. This\nlibrary implements this using the **polling publisher pattern**\n\nThe message that is persisted in the database is called the `Outbox` entity which looks like below\n\n#### Outbox Entity\n\n```typescript\n@Entity()\nexport class Outbox {\n\n   @PrimaryGeneratedColumn(`uuid`)\n   id: string;\n\n   @Column()\n   aggregateId: string\n\n   @Column()\n   messagePayload: string\n\n   @Column()\n   eventType: string\n\n   @CreateDateColumn()\n   createdAt: Date\n}\n\n```\n\n| Outbox Entity Properties | Meaning                                                                                       |\n|--------------------------|-----------------------------------------------------------------------------------------------|\n| `id`                     | UUID primary key for the outbox entity                                                        |\n| `aggregateId`            | This represents the Id of the aggregate. i:e entity that is being saved along with the outbox |\n| `messagePayload`         | Message payload to be published. Should be in JSON string format                              |\n| `eventType`              | This represents the type of event. i:e `OrderProcessedEvent`                                  |\n| `createdAt`              | When the event was created. Event are published in reverse chronological order. i:e FIFO      |\n\n### Configuration\n\n#### App Module configuration\n\nIn your NestJs application, navigate the `app.module.ts` and make the following configuration. The configuration can\neither\none of the two below\n\n* Kafka Publisher configuration\n* RabbitMQ configuration\n\nHowever, regardless of the event queue, there are two common configuration properties as can be seen below\n\n| OutboxModule Properties | Meaning                                                                                          |\n|-------------------------|--------------------------------------------------------------------------------------------------|\n| `queueType`             | The type of message queue to publish the message. This can either **KAFKA** or **RABBITMQ**      |\n| `datasource`            | This is the datasource configured for your application for connecting to the underlying database |\n\n#### Kafka Publisher Configuration\n\nIf you will be publishing to Kafka, your configuration should look something like this below. We are going to add the\n`OutboxModule` in the imports section of the` @Module` decorator\n\n````typescript\nimport {MessageQueueType, OutboxModule} from \"outbox-nestjs\";\n\nimports: [\n  OutboxModule.forRoot({\n    kafkaOptions: {\n      clientId: `test-application`,\n      brokers: [`localhost:9092`],\n      topic: `outbox-topic`\n    },\n    queueType: MessageQueueType.KAFKA,\n    datasource: connectionSource\n  }),\n]\n````\n\n| Kafka Options Properties | Meaning                                                                   |\n|--------------------------|---------------------------------------------------------------------------|\n| `clientId`               | This is the name of your service or whatever you want to call your client |\n| `brokers`                | This the broker url(s). Supports an array for multiple brokers            |\n| `topic`                  | The topic name to send your messages to                                   |\n\n#### RabbitMQ Configuration\n\nIf you are using RabbitMQ, your configuration should look something like this below. We are going to add the\n`OutboxModule` in the imports section of the `@Module` decorator\n\n````typescript\nimport {MessageQueueType, OutboxModule} from \"outbox-nestjs\";\n\nimports: [\n  OutboxModule.forRoot({\n    rabbitOptions: {\n      exchange: `sample-exchange`,\n      connectionString: `amqp://guest:guest@localhost:5672`\n    },\n    queueType: MessageQueueType.RABBIT_MQ,\n    datasource: connectionSource\n  }),\n]\n````\n\n| Kafka Options Properties | Meaning                                                  |\n|--------------------------|----------------------------------------------------------|\n| `exchange`               | This is the rabbitMQ exchange to message publishing      |\n| `connectionString`       | This is the connection string for connecting to rabbitMQ |\n\n#### Datasource Configuration\n\nThis library requires a `datasource` in order to fetch data from the outbox table. Typically, your datasource configuration\nwould look something like below\n\n```typescript\nimport { Outbox } from \"outbox-nestjs\";\n\n\nconst config: TypeOrmModuleOptions = {\n   type: \"xxxx\",\n   host: `xxx`,\n   port: xxxx,\n   username: `xxxx`,\n   password: `xxxx`,\n   database: `xxxx`,\n   entities: [`xxx`, Outbox],\n   migrationsRun: true,\n   migrations: [\"dist/domain/migrations/*{.ts,.js}\"],\n   autoLoadEntities: true,\n   synchronize: true,\n   logger: \"advanced-console\"\n};\n\n```\n\n**The important part of the above configuration is that we have to add the `Outbox` entity to the entities array**\n\n### Polling publisher\n\nThe library uses nestjs `ScheduleModule` to poll and publish to the message broker. You need to initiate the module.\nIn `app.module.ts`, add the following\n\n````typescript\n@Module({\n   imports: [\n      ScheduleModule.forRoot()\n   ],\n})\n````\n\n### Database Migrations\n\nThe Typeorm configuration from above has `synchronize:true`, which would create the `Outbox` table in the database,\nhowever, **DO NOT USE THIS IN PRODUCTION!!!**\n\n_You should create the Outbox table in your database, choosing the right data type for each property_\n\n## Other  Considerations\n\nThere are some details to be aware of when using this library\n\n* The default polling interval is **1 Second** for now\n* This library would try to publish the events at least once. However, there can be cases where an event is published\n  more than once.  \n  Your clients should be **Idempotent**\n* While the implementation tries to publish the events in order which they arrive, the order may not be guaranteed ;)\n\n### Author\n\nNriagu Chidubem\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdubems%2Foutbox-nestjs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdubems%2Foutbox-nestjs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdubems%2Foutbox-nestjs/lists"}