Skip to content
/ awesome-integration Public
forked from stn1slv/awesome-integration

A curated list of awesome system integration software and resources.

License

CC0-1.0 license
0 stars 45 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Mahendra0001/awesome-integration

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

132 Commits
.github/workflows
.github/workflows
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 

Repository files navigation

Awesome Integration Awesome

A curated list of awesome system integration software, patterns and resources.

System integration is the process of linking together different IT systems (components) to functionally cooperate as a whole.

Contents

Projects

API Management

API Management solutions.

API Design

API design, documentation and life-cycle automation tools.

  • OpenAPI Diff - Compare two OpenAPI specifications (3.x) and render the difference to HTML plaintext, or Markdown files.
  • OpenAPI Generator - Allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (both 2.0 and 3.0 are supported).
  • OpenAPI Style Validator - A customizable style validator to make sure your OpenAPI spec follows your organization's standards.
  • Swagger Editor - Design, describe, and document your API on the first open source editor fully dedicated to OpenAPI-based APIs.
  • Zally - A linter for OpenAPI specifications.

API Gateway

API Gateways.

  • Ambassador Edge Stack - Kubernetes-native API Gateway.
  • Apache APISIX - Dynamic, real-time, high-performance API gateway.
  • Gloo Edge - An Envoy Proxy-based API Gateway.
  • Goku API Gateway - Golang-based microservice gateway that enables high-performance dynamic routing,service orchestration, multi-tenancy management, API access control, etc.
  • Kong API Gateway - Cloud-native, platform-agnostic, scalable API Gateway distinguished for its high performance and extensibility via plugins.
  • KrakenD API Gateway - Ultra-High performance API Gateway with middlewares.
  • Ocelot - .NET API Gateway.
  • Spring Cloud Gateway - API Gateway built on top of the Spring Ecosystem, including: Spring 5, Spring Boot 2 and Project Reactor.
  • Tyk API Gateway - An open source Enterprise API Gateway, supporting REST, GraphQL, TCP and gRPC protocols.

API Testing

API testing tools.

  • API clients
    • Advanced REST Client - An open-source and free for everyone API tool out of a passion for giving the developer community tools they need.
    • curl - Command line tool and library for transferring data with URLs.
    • curlie - A frontend to curl that adds the ease of use of httpie, without compromising on features and performance.
    • HTTPie - Command line HTTP client, far more dev-friendly than curl.
    • Insomnia - Cross-platform HTTP and GraphQL Client.
    • Postman - A tool to make HTTP requests, generate mock servers and make API documentation.
    • SoapUI - An open-source web service testing application for SOAP and REST.
  • MQ clients
    • JMSToolBox - An universal JMS Client able to interact with the greatest number of JMS based message brokers.
    • kcat - A generic non-JVM producer and consumer for Apache Kafka, think of it as a netcat for Kafka. Formerly known as kafkacat.
    • Offset Explorer - A GUI application for managing and using Apache Kafka clusters. Formerly known as Kafka Tool.
  • Mocking tools
    • Microcks - Open source Kubernetes Native tool for API Mocking and Testing. Supports AsyncAPI, OpenAPI, Postman Collections etc.
    • Mockable - A simple configurable service to mock out RESTful API or SOAP web-services.
    • Mockoon - The easiest and quickest way to design and run mock REST APIs.
    • MockServer - Easy mocking of any system you integrate with via HTTP or HTTPS.
    • Mocky - A free and unlimited online service which allows to generate custom HTTP responses.
    • Prism - An open-source HTTP mock server that can mimic your API`s behavior as if you already built it.
    • WireMock - Mock your APIs for fast, robust and comprehensive testing.
  • Testing tools and frameworks
    • Apache JMeter - An open source java based load testing tool.
    • Karate - Open-source tool to combine API test-automation, mocks, performance-testing and even UI automation into a single, unified framework.
    • Pyresttest - A REST testing and API microbenchmarking tool.
    • REST Assured - Java DSL for easy testing of REST services.
    • Schemathesis - Property-based testing for API schemas.

BRE

Business Rules Engines and Business Rules Management Systems.

  • Drools - Open source tool with full runtime support for Decision Model and Notation (DMN) models at Conformance level 3 and an Eclipse IDE plugin for core development.
  • Easy Rules - A simple Java rules engine.
  • NxBRE - Open-source rule engine for the .NET platform.
  • OpenL Tablets - An open source business rules engine (BRE, BRMS) and decision management system.

BPM

Business Process Management solutions.

  • Appian BPM Suite - Platform combines low-code development with process management to allow both IT and citizen developers to construct process-centric and case-centric applications.
  • Genpack Cora SeQuence - Orchestrate a more effective flow, generate growth, improve cost efficiency, and drive business agility.
  • IBM Business Automation Workflow - A comprehensive business process management platform.
  • Oracle BPM Suite - An integrated environment for developing, administering, and using business applications centered around business processes.
  • Pega Platform - A combined business process management and robotic process automation (RPA) platform with advanced workforce analytics from Pegasystems.
  • Red Hat Process Automation Manager - An application development platform that enables developers and business experts to create cloud-native applications that automate business operations.
  • SAP Process Orchestration - A business process management offering and application infrastructure.
  • TIBCO BPM Enterprise - A business process management platform with capabilities for process automation, process documentation, human capital management, process patterns, and predictive operations analytics.

Data Mapping Solution

Data Mapping solutions.

  • AltasMap - Data mapping solution with interactive web based user interface, that simplifies configuring integrations between Java, XML, CSV and JSON data sources.
  • Altova MapForce - A graphical data mapping tool for any-to-any conversion and integration.
  • JOLT - JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document.
  • JSLT - A complete query and transformation language for JSON. The language design is inspired by jq, XPath, and XQuery.

ESB

Enterprise service bus solutions.

  • Anypoint Platform - Combines API management and integration capabilities in a single platform from Mulesoft.
  • Apache ServiceMix - A flexible, open-source integration container that unifies the features and functionality of Apache ActiveMQ, Camel, CXF, and Karaf.
  • ArcESB - Synchronize data across applications, seamlessly integrate with partners, or make data publicly accessible.
  • IBM App Connect - Connect applications together, regardless of the message formats or protocols that they support. Formerly known as IBM Integration Bus.
  • NServiceBus - The most developer-friendly service bus for .NET.
  • Oracle Service Bus - Transforms complex and brittle architectures into agile integration networks by connecting, virtualizing, and managing interactions between services and applications.
  • Oracle SOA Suite - Enables system developers to set up and manage services and to orchestrate them into composite applications and business processes.
  • Red Hat Fuse - Distributed, cloud-native integration platform.
  • Software AG webMethods Integration Server - Integrate anything faster.
  • TIBCO BusinessWorks - Implement enterprise patterns for hybrid integrations.
  • UltraESB - The only ESB that supports Zero-Copy proxying for extreme levels of performance utilizing Direct Memory Access (DMA) and the sendfile system call along with Non-Blocking IO.
  • WSO2 Enterprise Integrator - An API-centric, cloud-native, and distributed integration platform.

ETL

ETL/ELT and data integration tools.

  • Apache NiFi - Apache NiFi is an integrated data logistics platform for automating the movement of data between disparate systems.
  • CloverETL - A data integration software suite for data migration and data warehousing, and for feeding data into business intelligence and reporting applications.
  • IBM DataStage - A data integration platform that helps businesses understand, cleanse, transform, and deliver trusted information.
  • Informatica PowerCenter - Delivers enterprise data integration and management software powering analytics for big data and cloud.
  • Microsoft SSIS - SQL Server Integration Services is a platform for building enterprise-level data integration and data transformations solutions.
  • Oracle Data Integrator - A comprehensive data integration platform that covers all data integration requirements, including batch loads, integration processes, and SOA-enabled data services.
  • Pentaho Data Integration - Enables users to ingest, blend, cleanse, and prepare diverse data from any source with visual tools to eliminate coding and complexity.
  • SAS Data Management - Helps transform, integrate, govern, and secure data while improving its overall quality and reliability.
  • Stitch - A simple, powerful ETL service for businesses of all sizes, up to and including the enterprise. Running on a scalable, fault-tolerant cloud platform, Stitch integrates data from dozens of different sources.
  • Talend Data Integration - Bring data together from multiple sources to power all your business decisions.

Integration Frameworks

Integration frameworks based on known Enterprise Integration Patterns.

  • Apache Camel - Glues together different transport APIs via Enterprise Integration Patterns.
  • Ballerina - An open-source programming language that makes it easier to use, combine, and create network services.
  • Spring Integration - Extends the Spring programming model to support the well-known Enterprise Integration Patterns.

iPaaS

Integration Platform as a Service.

MaaS

Cloud messaging as a service (MaaS).

Managed File Transfer

Managed file transfer (MFT) solutions.

Master Data Management

Master Data Management solutions.

  • IBM InfoSphere Master Data Management - Comprehensive master data management.
  • Informatica Multidomain MDM - All-in-one master data management.
  • Oracle Enterprise Data Management - Adapt and respond to change faster and more effectively by managing your master data.
  • SAP Master Data Governance - Improve the quality and consistency of information across your organization by consolidating and centrally governing the master data lifecycle.
  • SAS MDM - Provides a single, accurate and unified view of corporate data, integrating information from various data sources into one master record.
  • Software AG OneData for MDM - Provides “one version of the truth” when it comes to data, ensuring that only reliable, accurate and approved information is flowing through your systems, processes and applications.
  • Teradata MDM - Drives significant ROI using consistent reference data to establish an accurate analytical data foundation.
  • TIBCO EBX - A single solution to govern, manage, and consume all shared data assets.

Messaging

Message brokers.

  • Apache ActiveMQ - Message broker that implements JMS and converts synchronous to asynchronous communication.
  • Apache Kafka - High-throughput distributed messaging system.
  • Apache Pulsar - Distributed pub/sub-messaging system.
  • Apache RocketMQ - Fast, reliable, and scalable distributed messaging platform.
  • Apache Qpid - Apache Qpid makes messaging tools that speak AMQP and support many languages and platforms.
  • Eclipse Mosquitto - An lightweight open source message broker that implements the MQTT protocol.
  • IBM MQ - Enterprise-grade messaging capabilities that skillfully and safely move information between applications.
  • KubeMQ - A Kubernetes Message Queue Broker Enterprise-grade message broker and message queue, scalable, high available and secured.
  • NATS - Lightweight, high performance messaging system for microservices, IoT, and cloud native systems.
  • Oracle AQ - Oracle Advanced Queuing (AQ) provides enterprise message functionality across many industries and is widely used within the database itself.
  • RabbitMQ - An open-source message-broker that originally implemented the Advanced Message Queuing Protocol (AMQP).
  • Redpanda - A streaming platform for mission critical workloads. Kafka® compatible, No Zookeeper®, no JVM, and no code changes required.
  • Red Hat AMQ - Red Hat AMQ-based on open source communities like Apache ActiveMQ and Apache Kafka.
  • TIBCO Enterprise Message Service - TIBCO EMS is standards-based Java™ Message Service (JMS) implementation allows any application that supports JMS, whether home grown or third-party, to quickly and easily exchange messages.
  • VerneMQ - A high-performance, distributed MQTT message broker.

RPA

Robotic Process Automation (RPA) solutions.

  • Automation Anywhere - Intelligent Automation Ecosystem.
  • Blue Prism - Intelligent automation - business-developed and no-code automation RPA platform.
  • UiPath - Automattion repetitive digital tasks normally performed by people.
  • WorkFusion - Automation Platform for the Enterprise.

Self-Service Integration

Any self-service and citizen integrator tools.

  • IFTTT - Allows creating chains of conditional statements between web services in order to make the web work for you and boost your productivity.
  • Oracle Self-Service Integration - Automates tasks between cloud applications.
  • Zapier - Automation tool that allows you to connect hundreds of web services and create automations between the processes.

Workflow engine

Workflow and orchestration engines.

  • Activiti - A lightweight and java-centric open-source BPMN engine.
  • Apache Airflow - A platform to programmatically author, schedule, and monitor workflows.
  • Argo Workflows - An open source container-native workflow engine for orchestrating parallel jobs on Kubernetes.
  • Azkaban - A distributed Workflow Manager, implemented at LinkedIn to solve the problem of Hadoop job dependencies.
  • Bonita - BPMN engine that comes with an optional development environment, a designer, an optional user interface and administrative tools.
  • Cadence - Fault-Tolerant Stateful Code Platform.
  • Camunda - An open-source workflow and decision automation platform.
  • Conductor - Netflix Conductor is an orchestration engine that runs in the cloud.
  • Elsa Core - A workflows library that enables workflow execution in any .NET Core application.
  • Flowable - Provides a core set of open source business process engines that are compact and highly efficient.
  • jBPM - A toolkit for building business applications to help automate business processes and decisions.
  • StackStorm - Robust Automation Engine providing Sensors, Triggers, Rules, Workflows, and Actions. StackStorm is how you “glue” your applications together.

Resources

Some useful specifications, patterns, articles and documentation.

API Specification

  • AsyncAPI - The AsyncAPI specification settles the base for a greater and better tooling ecosystem for Event-Driven Architectures (EDA).
  • GraphQL - A query language for APIs and a runtime for fulfilling those queries with your existing data.
  • OpenAPI (ex.Swagger) - Defines a standard, language-agnostic interface to RESTful APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection.
  • RAML - RESTful API Modeling Language.
  • WSDL - XML-based interface description language for SOAP based services.

Articles

  • Enterprise Integration Using REST - Discusses the constraints and flexibility that you have with nonpublic APIs, and lessons learned from doing large scale RESTful integration across multiple teams.
  • Richardson Maturity Model - Explained by Martin Fowler, originally presented by Leonard Richardson.

Connectors

  • JCA - Defines a standard architecture for Jakarta EE application components to connect to Enterprise Information Systems. Formerly Java EE Connector Architecture and J2EE Connector Architecture.
  • Kafka Connect - A tool for scalably and reliably streaming data between Apache Kafka and other systems.

Data Formats

  • Apache Avro - Data serialization system.
  • BSON - Short for Binary JSON, is a binary-encoded serialization of JSON-like documents.
  • CSV - A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values.
  • JSON - The JavaScript Object Notation (JSON) Data Interchange Format.
  • NDJSON - A standard for delimiting JSON in stream protocols.
  • Protocol Buffers - A language-neutral, platform-neutral extensible mechanism for serializing structured data.
  • XML - Extensible Markup Language (XML).
  • YAML - YAML is a human friendly data serialization standard for all programming languages.

Enterprise Integration Patterns

Patterns from a book by Gregor Hohpe and Bobby Woolf.

  • Aggregator - How do we combine the results of individual, but related messages so that they can be processed as a whole?
  • Canonical Data Model - How can you minimize dependencies when integrating applications that use different data formats?
  • Channel Adapter - How can you connect an application to the messaging system so that it can send and receive messages?
  • Channel Purger - How can you keep 'left-over' messages on a channel from disturbing tests or running systems?
  • Claim Check - How can we reduce the data volume of message sent across the system without sacrificing information content?
  • Command Message - How can messaging be used to invoke a procedure in another application?
  • Competing Consumers - How can a messaging client process multiple messages concurrently?
  • Composed Message Processor - How can you maintain the overall message flow when processing a message consisting of multiple elements, each of which may require different processing?
  • Content Enricher - How do we communicate with another system if the message originator does not have all the required data items available?
  • Content Filter - How do you simplify dealing with a large message, when you are interested only in a few data items?
  • Content-Based Router - How do we handle a situation where the implementation of a single logical function (e.g., inventory check) is spread across multiple physical systems?
  • Control Bus - How can we effectively administer a messaging system that is distributed across multiple platforms and a wide geographic area?
  • Correlation Identifier - How does a requestor that has received a reply know which request this is the reply for?
  • Datatype Channel - How can the application send a data item such that the receiver will know how to process it?
  • Dead Letter Channel - What will the messaging system do with a message it cannot deliver?
  • Detour - How can you route a message through intermediate steps to perform validation, testing or debugging functions?
  • Document Message - How can messaging be used to transfer data between applications?
  • Durable Subscriber - How can a subscriber avoid missing messages while it is not listening for them?
  • Dynamic Router - How can you avoid the dependency of the router on all possible destinations while maintaining its efficiency?
  • Envelope Wrapper - How can existing systems participate in a messaging exchange that places specific requirements on the message format, such as message header fields or encryption?
  • Event Message - How can messaging be used to transmit events from one application to another?
  • Event-Driven Consumer - How can an application automatically consume messages as they become available?
  • Format Indicator - How can a message`s data format be designed to allow for possible future changes?
  • Guaranteed Delivery - How can the sender make sure that a message will be delivered, even if the messaging system fails?
  • Idempotent Receiver - How can a message receiver deal with duplicate messages?
  • Invalid Message Channel - How can a messaging receiver gracefully handle receiving a message that makes no sense?
  • Message - How can two applications connected by a message channel exchange a piece of information?
  • Message Dispatcher - How can multiple consumers on a single channel coordinate their message processing?
  • Message Expiration - How can a sender indicate when a message should be considered stale and thus shouldn`t be processed?
  • Message Translator - How can systems using different data formats communicate with each other using messaging?
  • Message Broker - How can you decouple the destination of a message from the sender and maintain central control over the flow of messages?
  • Message Bus - What is an architecture that enables separate applications to work together, but in a decoupled fashion such that applications can be easily added or removed without affecting the others?
  • Message Channel - How does one application communicate with another using messaging?
  • Message Endpoint - How does an application connect to a messaging channel to send and receive messages?
  • Message Filter - How can a component avoid receiving uninteresting messages?
  • Message History - How can we effectively analyze and debug the flow of messages in a loosely coupled system?
  • Message Router - How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions?
  • Message Sequence - How can messaging transmit an arbitrarily large amount of data?
  • Message Store - How can we report against message information without disturbing the loosely coupled and transient nature of a messaging system?
  • Messaging Bridge - How can multiple messaging systems be connected so that messages available on one are also available on the others?
  • Messaging Gateway - How do you encapsulate access to the messaging system from the rest of the application?
  • Messaging Mapper - How do you move data between domain objects and the messaging infrastructure while keeping the two independent of each other?
  • Normalizer - How do you process messages that are semantically equivalent, but arrive in a different format?
  • Pipes and Filters - How can we perform complex processing on a message while maintaining independence and flexibility?
  • Point-to-Point Channel - How can the caller be sure that exactly one receiver will receive the document or perform the call?
  • Polling Consumer - How can an application consume a message when the application is ready?
  • Process Manager - How do we route a message through multiple processing steps when the required steps may not be known at design-time and may not be sequential?
  • Publish-Subscribe Channel - How can the sender broadcast an event to all interested receivers?
  • Recipient List - How do we route a message to a list of dynamically specified recipients?
  • Request-Reply - When an application sends a message, how can it get a response from the receiver?
  • Resequencer - How can we get a stream of related but out-of-sequence messages back into the correct order?
  • Return Address - How does a replier know where to send the reply?
  • Routing Slip - How do we route a message consecutively through a series of processing steps when the sequence of steps is not known at design-time and may vary for each message?
  • Scatter-Gather - How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?
  • Selective Consumer - How can a message consumer select which messages it wishes to receive?
  • Service Activator - How can an application design a service to be invoked both via various messaging technologies and via non-messaging techniques?
  • Smart Proxy - How can you track messages on a service that publishes reply messages to the Return Address specified by the requestor?
  • Splitter - How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?
  • Test Message - What happens, though, if a component is actively processing messages, but garbles outgoing messages due to an internal fault?
  • Transactional Client - How can a client control its transactions with the messaging system?
  • Wire Tap - How do you inspect messages that travel on a point-to-point channel?

Integration Styles

Market Analysis

Protocols

  • AMQP 0-9-1 - Advanced Message Queuing Protocol (AMQP) Version 0-9-1.
  • AMQP 1.0 - Advanced Message Queuing Protocol (AMQP) Version 1.0.
  • HTTP - The Hypertext Transfer Protocol (HTTP) is an application layer protocol for distributed, collaborative, hypermedia information systems.
  • JSON-RPC - Designed to be simple, it's a stateless, lightweight remote procedure call (RPC) protocol.
  • MQTT - The Message Queuing Telemetry Transport (MQTT) is a lightweight, publish-subscribe network protocol that transports messages between devices.
  • SOAP - Simple Object Access Protocol is XML-based messaging protocol.
  • STOMP - Simple (or Streaming) Text Orientated Messaging Protocol.

Standard APIs

  • JDBC - Java Database Connectivity (JDBC) API provides universal data access from the Java programming language.
  • JMS - Java Message Service (JMS) API.
  • ODBC - Open Database Connectivity (ODBC) is a widely accepted application programming interface (API) for database access.
  • OData - An open protocol that allows the creation and consumption of queryable and interoperable REST APIs in a simple and standard way.

Structure and Validation

  • JSON Schema - A powerful tool for validating the structure of JSON data.
  • XML Schema - Offers facilities for describing the structure and constraining the contents of XML documents.

Contributing

Your contributions are always welcome! Please take a look at the contribution guidelines first.

About

A curated list of awesome system integration software and resources.

Resources

Readme

License

CC0-1.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published