awesome-integration

A curated list of awesome system integration software and resources.
https://github.com/stn1slv/awesome-integration

Last synced: 2 days ago
JSON representation

About the author
- Structure and Validation
  - Stanislav Deviatov - relation/newsletter-follow?entityUrn=7263963549411004416), a LinkedIn newsletter with news on API management, iPaaS, ESB, integration frameworks, message brokers, and related topics.
Integration Patterns
- Enterprise Integration Patterns
  - Aggregator - How do we combine the results of individual, but related messages so that they can be processed as a whole?
  - Canonical Data Model - How can you minimize dependencies when integrating applications that use different data formats?
  - Channel Adapter - How can you connect an application to the messaging system so that it can send and receive messages?
  - Channel Purger - How can you keep 'left-over' messages on a channel from disturbing tests or running systems?
  - Claim Check - How can we reduce the data volume of message sent across the system without sacrificing information content?
  - Command Message - How can messaging be used to invoke a procedure in another application?
  - Competing Consumers - How can a messaging client process multiple messages concurrently?
  - Composed Message Processor - How can you maintain the overall message flow when processing a message consisting of multiple elements, each of which may require different processing?
  - Content Enricher - How do we communicate with another system if the message originator does not have all the required data items available?
  - Content Filter - How do you simplify dealing with a large message, when you are interested only in a few data items?
  - Content-Based Router - How do we handle a situation where the implementation of a single logical function (e.g., inventory check) is spread across multiple physical systems?
  - Control Bus - How can we effectively administer a messaging system that is distributed across multiple platforms and a wide geographic area?
  - Correlation Identifier - How does a requestor that has received a reply know which request this is the reply for?
  - Datatype Channel - How can the application send a data item such that the receiver will know how to process it?
  - Dead Letter Channel - What will the messaging system do with a message it cannot deliver?
  - Detour - How can you route a message through intermediate steps to perform validation, testing or debugging functions?
  - Document Message - How can messaging be used to transfer data between applications?
  - Durable Subscriber - How can a subscriber avoid missing messages while it is not listening for them?
  - Dynamic Router - How can you avoid the dependency of the router on all possible destinations while maintaining its efficiency?
  - Envelope Wrapper - How can existing systems participate in a messaging exchange that places specific requirements on the message format, such as message header fields or encryption?
  - Event Message - How can messaging be used to transmit events from one application to another?
  - Event-Driven Consumer - How can an application automatically consume messages as they become available?
  - Format Indicator - How can a message`s data format be designed to allow for possible future changes?
  - Guaranteed Delivery - How can the sender make sure that a message will be delivered, even if the messaging system fails?
  - Idempotent Receiver - How can a message receiver deal with duplicate messages?
  - Invalid Message Channel - How can a messaging receiver gracefully handle receiving a message that makes no sense?
  - Message - How can two applications connected by a message channel exchange a piece of information?
  - Message Dispatcher - How can multiple consumers on a single channel coordinate their message processing?
  - Message Expiration - How can a sender indicate when a message should be considered stale and thus shouldn`t be processed?
  - Message Translator - How can systems using different data formats communicate with each other using messaging?
  - Message Broker - How can you decouple the destination of a message from the sender and maintain central control over the flow of messages?
  - Message Bus - What is an architecture that enables separate applications to work together, but in a decoupled fashion such that applications can be easily added or removed without affecting the others?
  - Message Channel - How does one application communicate with another using messaging?
  - Message Endpoint - How does an application connect to a messaging channel to send and receive messages?
  - Message Filter - How can a component avoid receiving uninteresting messages?
  - Message History - How can we effectively analyze and debug the flow of messages in a loosely coupled system?
  - Message Router - How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions?
  - Message Sequence - How can messaging transmit an arbitrarily large amount of data?
  - Message Store - How can we report against message information without disturbing the loosely coupled and transient nature of a messaging system?
  - Messaging Bridge - How can multiple messaging systems be connected so that messages available on one are also available on the others?
  - Messaging Gateway - How do you encapsulate access to the messaging system from the rest of the application?
  - Messaging Mapper - How do you move data between domain objects and the messaging infrastructure while keeping the two independent of each other?
  - Normalizer - How do you process messages that are semantically equivalent, but arrive in a different format?
  - Pipes and Filters - How can we perform complex processing on a message while maintaining independence and flexibility?
  - Point-to-Point Channel - How can the caller be sure that exactly one receiver will receive the document or perform the call?
  - Polling Consumer - How can an application consume a message when the application is ready?
  - Process Manager - How do we route a message through multiple processing steps when the required steps may not be known at design-time and may not be sequential?
  - Publish-Subscribe Channel - How can the sender broadcast an event to all interested receivers?
  - Recipient List - How do we route a message to a list of dynamically specified recipients?
  - Request-Reply - When an application sends a message, how can it get a response from the receiver?
  - Resequencer - How can we get a stream of related but out-of-sequence messages back into the correct order?
  - Return Address - How does a replier know where to send the reply?
  - Routing Slip - How do we route a message consecutively through a series of processing steps when the sequence of steps is not known at design-time and may vary for each message?
  - Scatter-Gather - How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?
  - Selective Consumer - How can a message consumer select which messages it wishes to receive?
  - Service Activator - How can an application design a service to be invoked both via various messaging technologies and via non-messaging techniques?
  - Smart Proxy - How can you track messages on a service that publishes reply messages to the Return Address specified by the requestor?
  - Splitter - How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?
  - Test Message - What happens, though, if a component is actively processing messages, but garbles outgoing messages due to an internal fault?
  - Transactional Client - How can a client control its transactions with the messaging system?
  - Wire Tap - How do you inspect messages that travel on a point-to-point channel?
- Integration Architecture Patterns
  - API-led Connectivity pattern - Use APIs to connect different systems and applications.
  - Anti Corruption Layer Pattern - Add a layer to isolate and transform data between systems.
  - Change Data Capture Pattern - Capture and propagate changes made to a database or data source in real-time.
  - Hybrid API Management pattern - Manage APIs that span both cloud and on-premises environments using a central control plane.
  - Hybrid Integration pattern - Integrate systems and applications that are deployed both on-premises and in the cloud using a combination of integration technologies.
- Microservice API Patterns
  - Frontend Integration - How can client-side end-user interfaces that are physically separated from server-side business logic and data storage be populated and updated with computing results, result sets from searches in data sources, and detailed information about data entities? How can application frontends invoke activities in a backend or upload data to it?
  - Backend Integration - How can distributed applications and their parts, which have been built independently and are deployed separately, exchange data and trigger mutual activity while preserving system-internal conceptual integrity without introducing undesired coupling?
  - Public API - How can an API be made available to an unlimited and/or unknown number of API clients outside the organization that are globally, nationally, and/or regionally distributed?
  - Community API - How can the visibility of and the access to an API be restricted to a closed user group that does not work for a single organizational unit but for multiple legal entities (such as companies, nonprofit/nongovernment organizations, and governments)?
  - Solution-Internal API - How can access to and usage of an API be limited to an application, for instance, components in the same or another logical layer and/or physical tier?
  - API Description - Which knowledge should be shared between an API provider and its clients? How should this knowledge be documented?
  - Processing Resource - How can an API provider allow its clients to trigger an action in it?
  - Information Holder Resource - How can domain data be exposed in an API, but its implementation still be hidden? How can an API expose data entities so that API clients can access and/or modify these entities concurrently without compromising data integrity and quality?
  - State Creation Operation - How can an API provider allow its clients to report that something has happened that the provider needs to know about, for instance, to trigger instant or later processing?
  - Retrieval Operation - How can information available from a remote party (the API provider, that is) be retrieved to satisfy an information need of an end user or to allow further client-side processing?
  - State Transition Operation - How can a client initiate a processing action that causes the provider-side application state to change? How can API clients and API providers share the responsibilities required to execute and control business processes and their activities?
  - Computation Function - How can a client invoke side-effect-free remote processing on the provider side to have a result calculated from its input?
  - Operational Data Holder - How can an API support clients that want to create, read, update, and/or delete instances of domain entities that represent operational data: data that is rather short-lived, changes often during daily business operations, and has many outgoing relations?
  - Master Data Holder - How can I design an API that provides access to master data that lives for a long time, does not change frequently, and will be referenced from many clients?
  - Reference Data Holder - How should data that is referenced in many places, lives long, and is immutable for clients be treated in API endpoints? How can such reference data be used in requests to and responses from Processing Resources or Information Holder Resources?
  - Link Lookup Resource - How can message representations refer to other, possibly many and frequently changing, API endpoints and operations without binding the message recipient to the actual addresses of these endpoints?
  - Data Transfer Resource - How can two or more communication participants exchange data without knowing each other, without being available at the same time, and even if the data has already been sent before its recipients became known?
  - Atomic Parameter - How can simple, unstructured data (such as a number, a string, a Boolean value, or a block of binary data) be exchanged between API client and API provider?
  - Atomic Parameter List - How can multiple related Atomic Parameters be combined in a representation element so that each of them stays simple, but their relatedness becomes explicit in the API Description and the runtime message exchanges?
  - Parameter Tree - How can containment relationships be expressed when defining complex representation elements and exchanging such related elements at runtime?
  - Parameter Forest - How can multiple Parameter Trees be exposed as request or response payload of an API operation?
  - Data Element - How can domain/application-level information be exchanged between API clients and API providers without exposing provider-internal data definitions in the API? How can API client and API provider be decoupled from a data management point of view?
  - Metadata Element - How can messages be enriched with additional information so that receivers can interpret the message content correctly, without having to hardcode assumptions about the data semantics?
  - Id Element - How can API elements be distinguished from each other at design time and at runtime? When applying domain-driven design, how can elements of the Published Language be identified?
  - Link Element - How can API endpoints and operations be referenced in request and response message payloads so that they can be called remotely?
  - API Key - How can an API provider identify and authenticate clients and their requests?
  - Error Report - How can an API provider inform its clients about communication and processing faults? How can this information be made independent of the underlying communication technologies and platforms (for example, protocol-level headers representing status codes)?
  - Context Representation - How can API consumers and providers exchange context information without relying on any particular remoting protocols? How can identity information and quality properties in a request be made visible to related subsequent ones in conversations?
  - Embedded Entity - How can one avoid sending multiple messages when their receivers require insights about multiple related information elements?
  - Linked Information Holder - How can messages be kept small even when an API deals with multiple information elements that reference each other?
  - Pagination - How can an API provider deliver large sequences of structured data without overwhelming clients?
  - Wish List - How can an API client inform the API provider at runtime about the data it is interested in?
  - Wish Template - How can an API client inform the API provider about nested data that it is interested in? How can such preferences be expressed flexibly and dynamically?
  - Conditional Request - How can unnecessary server-side processing and bandwidth usage be avoided when frequently invoking API operations that return rarely changing data?
  - Request Bundle - How can the number of requests and responses be reduced to increase communication efficiency?
  - Pricing Plan - How can the API provider meter API service consumption and charge for it?
  - Rate Limit - How can the API provider prevent API clients from excessive API usage?
  - Service Level Agreement - How can an API client learn about the specific quality-of-service characteristics of an API and its endpoint operations? How can these characteristics, and the consequences of not meeting them, be defined and communicated in a measurable way?
  - Version Identifier - How can an API provider indicate its current capabilities as well as the existence of possibly incompatible changes to clients in order to prevent malfunctioning of clients due to undiscovered interpretation errors?
  - Semantic Versioning - How can stakeholders compare API versions to detect immediately whether they are compatible?
  - Two In Production - How can a provider gradually update an API without breaking existing clients but also without having to maintain a large number of API versions in production?
  - Aggressive Obsolescence - How can API providers reduce the effort for maintaining an entire API or its parts (such as endpoints, operations, or message representations) with guaranteed service quality levels?
  - Experimental Preview - How can providers make the introduction of a new API, or new API version, less risky for their clients and obtain early adopter feedback without having to freeze the API design prematurely?
  - Limited Lifetime Guarantee - How can a provider let clients know for how long they can rely on the published version of an API?
  - Eternal Lifetime Guarantee - How can a provider support clients that are unable or unwilling to migrate to newer API versions at all?
- SOA Patterns
  - Canonical Protocol - Defines a common communication protocol between services to enable interoperability and decoupling.
  - Canonical Schema - Defines a standard data model and format for exchanging data between services.
  - Domain Inventory - Identifies and categorizes the types of services within a particular domain.
  - Enterprise Inventory - Establishes a single, enterprise-wide inventory of services to maximize reuse and recomposition.
  - Logic Centralization - Centralizes business logic within a service layer to reduce redundancy and promote consistency.
  - Service Layers - Organizes services into logical layers based on the type of logic they encapsulate.
  - Service Normalization - Ensures service boundaries within an inventory do not overlap, avoiding redundant functionality.
  - Entity Abstraction - Abstracts data entities to simplify data access and reduce coupling.
  - Process Abstraction - Abstracts processes to improve reusability and maintainability.
  - Utility Abstraction - Abstracts common utilities to reduce duplication and promote consistency.
  - Micro Task Abstraction - Breaks down tasks into smaller, more granular tasks for easier management.
  - Policy Centralization - Centralizes policies to reduce duplication and improve consistency.
  - Process Centralization - Centralizes processes to improve reusability and maintainability.
  - Rules Centralization - Centralizes business rules to reduce duplication and promote consistency.
  - Schema Centralization - Centralizes data schemas to reduce duplication and improve consistency.
  - Canonical Resources - Defines a standard set of resources for common functionality across services.
  - Cross-Domain Utility Layer - Establishes a common utility services layer shared across multiple domain inventories.
  - Dual Protocols - Allows the service inventory to support two communication protocols to balance interoperability and performance.
  - Inventory Endpoint - Defines a standard endpoint for accessing inventory resources.
  - Service Grid - Provides a framework for managing and scaling services.
  - State Repository - Stores and manages service state information.
  - Stateful Services - Maintains state information across service invocations.
  - Augmented Protocols - Enhances protocols with additional features.
  - Canonical Expression - Defines a standard expression language for inventory data.
  - Canonical Versioning - Defines a standard versioning scheme for inventory resources.
  - Metadata Centralization - Centralizes metadata to reduce duplication and improve consistency.
  - Agnostic Capability - Abstracts service capabilities for improved flexibility.
  - Agnostic Context - Abstracts service context for improved flexibility.
  - Functional Decomposition - Breaks down services into smaller, more manageable components.
  - Non-Agnostic Context - Encapsulates single-purpose, non-reusable logic in a dedicated service so it can still be governed within the inventory.
  - Service Encapsulation - Encapsulates service functionality for improved maintainability.
  - Partial State Deferral - Temporarily defers a portion of service state data to optimize memory and resource consumption.
  - Partial Validation - Validates only relevant data for improved performance.
  - Redundant Implementation - Implements multiple versions of a service for improved flexibility.
  - Service Data Replication - Replicates data across multiple services for improved performance.
  - Service Façade - Provides a simplified interface to a complex service for improved usability.
  - UI Mediator - Mediates between the user interface and underlying services for improved usability.
  - Reference Data Centralization - Centralizes reference data to reduce duplication and improve consistency.
  - Microservice Deployment - Deploys services as independent, autonomous units for improved scalability and resilience.
  - Containerization - How can an environment be provided with maximum support for services with high-performance recovery and scalability requirements?
  - Exception Shielding - Protects against security vulnerabilities through proper handling of exceptions.
  - Message Screening - Screens messages for malicious content.
  - Service Perimeter Guard - Guards the service perimeter to prevent unauthorized access.
  - Trusted Subsystem - Establishes trust between subsystems for improved security.
  - Concurrent Contracts - Allows multiple versions of a service contract to coexist simultaneously.
  - Contract Centralization - Centralizes service contracts to improve consistency and reduce duplication.
  - Contract Denormalization - Allows a service contract to include redundant capabilities to accommodate consumers with different data exchange requirements.
  - Decoupled Contract - Decouples service contracts from their implementation for improved flexibility.
  - Validation Abstraction - Abstracts validation logic to improve reusability and maintainability.
  - File Gateway - Provides a gateway to access legacy file-based systems.
  - Legacy Wrapper - Wraps legacy systems to expose them as services.
  - Multi-Channel Endpoint - Provides multiple communication channels to access legacy systems.
  - Compatible Change - Enables changes to a service without breaking existing clients.
  - Decomposed Capability - Breaks down a service capability into smaller, more manageable pieces.
  - Distributed Capability - Breaks down a service capability into smaller, more manageable pieces.
  - Proxy Capability - Provides a proxy to a remote service capability to improve performance and reduce network overhead.
  - Service Decomposition - Breaks down a monolithic service into smaller, more manageable pieces.
  - Service Refactoring - Refactors a service to improve its design and performance.
  - Termination Notification - Notifies clients of a service's termination.
  - Version Identification - Identifies the version of a service.
  - Capability Composition - Combines multiple service capabilities to create a new capability.
  - Capability Recomposition - Recombines existing service capabilities to create a new capability.
  - Asynchronous Queuing - Uses a message queue to decouple services and improve scalability and reliability.
  - Event-Driven Messaging - Uses events to trigger service invocations and reduce coupling.
  - Intermediate Routing - Uses intermediate routing nodes to improve performance and flexibility.
  - Messaging Metadata - Uses metadata to describe and manage service messages.
  - Reliable Messaging - Ensures message delivery and reliability in a distributed environment.
  - Service Agent - Acts as a proxy for a remote service to improve performance and reduce network overhead.
  - Service Callback - Uses callbacks to communicate between services.
  - Service Instance Routing - Routes messages based on the service instance to improve performance and scalability.
  - Service Messaging - Describes the communication between services.
  - State Messaging - Uses messages to manage state information in a distributed environment.
  - Agnostic Sub-Controller - Separates sub-controllers from the main controller to improve reusability and maintainability.
  - Atomic Service Transaction - Uses transactions to ensure atomicity and consistency across multiple service invocations.
  - Compensating Service Transaction - Reverses the effects of a failed transaction.
  - Composition Autonomy - Enables services to act autonomously within a composition.
  - Brokered Authentication - Authenticates clients through a broker.
  - Data Confidentiality - Ensures data confidentiality in a distributed environment.
  - Data Origin Authentication - Authenticates the origin of a message.
  - Direct Authentication - Authenticates clients directly.
  - Data Format Transformation - Transforms data formats to enable interoperability.
  - Data Model Transformation - Transforms data models to enable interoperability.
  - Protocol Bridging - Bridges between different protocols to enable interoperability.
  - Entity Linking - Links related resources to enable navigation and discovery.
  - Lightweight Endpoint - Provides a lightweight endpoint for resource access.
  - Reusable Contract - Reuses common contracts to improve consistency and reduce duplication.
  - Content Negotiation - Negotiates the content format between clients and servers.
  - Endpoint Redirection - Redirects clients to alternative endpoints.

Programming Languages

Java 65 Go 37 JavaScript 21 TypeScript 19 Python 17 Rust 10 C++ 9 C# 9 Kotlin 3 Scala 3

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

awesome-integration

About the author

Structure and Validation

Integration Patterns

Enterprise Integration Patterns

Integration Architecture Patterns

Microservice API Patterns

SOA Patterns