Overview – Event-Driven Architecture for Cloud-Native in Kubernetes
Event-Driven Architecture (EDA) is a model or an architectural paradigm for software that supports the production, detection, consumption of, and reaction to the event or a significant change in system state.
This structure consists of event creators and event consumers. Creators are the ones who produce events and have knowledge about what events occurred on the other hand’s consumers are the ones who become affected by events occurring and they are also involved in processing events.
The event-driven architecture is also sometimes related to stream processing architecture or message-driven architecture based on its fluctuations. Message Driven Architecture centre is to combine the categorized systems by transferring messages from one module to another module of the system using a standard Message Broker. Event stream processing runs by controlling a data set by one data point at a time. Instead of view data as a whole set, event stream processing is about
Characteristics & Considerations for choosing EDA
The central Characteristics & Considerations of an Event-Driven architecture :-
Multicast communications: The publishers can send events to various systems that have subscribed to it. This is opposite to unicast communication in which publishers can send the information only to one subscriber.
Real-time transmission: Publishers generate the events and when the event occurs in real-time to the subscribers. The mode of processing is real-time rather than batch processing.
Asynchronous communication: The publisher doesn’t wait for the receiver to prepare an event before sending the new event. Event-Driven Architecture can perform asynchronous operations in correspondence. This results in more reliable performance of the architecture, irrespective of the time stay involved in queuing and dequeuing messages.
Fine-grained communication: Publishers keep publishing small and specific events instead of waiting for a single aggregated event.
Event Ontology: Event-Driven Architecture systems always have a technique to classify events in terms of group/hierarchy based on their general characteristics which give flexibility to the subscribers to subscribe to a specific event and subscribers to subscribe to a specific category
Deployment and Development: In Event-Driven Architecture pattern elements are loosely coupled, and development is also easy because of the asynchronous nature of the pattern.
Event-driven Architecture in Cloud-Native World
Event-driven architectures and reactive programming models aren’t new ideas. But, during cloud-native architectures with microservices, container-based workloads, and serverless computing, you can revisit event-driven approaches in the cloud-native context.
Two phases of a cloud-native architecture are necessary for developing an event-driven architecture:
Micro-services implement the loosely coupled application architecture that allows deployment in highly distributed patterns for flexibility, agility, and scale.
Cloud-native platforms with containers and serverless deployments present the application platform and tools that realize the flexibility, agility, and scalability of the microservices architecture.
Implementing Event Source in Kubernetes
Event Sources are Kubernetes Custom Resources which implement a tool for expressing interest in a collection of events from a particular software system. For different events, sources may be defined by different CustomResources.
Kubernetes events are a source type in Kubernetes that are automatically generated when other sources have state changes, errors, or other messages that should be declared to the system.
While there is not a lot of documentation available for events in Kubernetes, they are an invaluable source when debugging issues in the Kubernetes cluster.
It is necessary to understand that events do not develop in a way that you might expect. For example, if the container is weak to start because the docker image cannot be pulled the events are only visible on the pod, not on any Replica-set, StatefulSet, Deployment, or any other resource that generated the pod.
Tools to Implement Event-Driven Architecture
A message broker is a middle-ware software, tool or SaaS used to send events between event producers and consumers in a publish-subscribe pattern. All implementations of event-driven architecture need to use some technology in the role of an event broker or message broker. there are various Tools for message broker but top three tools which are used:-
KubeMQ is a Cloud Native, enterprise-grade, message broker for shared services architecture. KubeMQ is given as a small, lightweight Docker container, created for any workload and architecture working in Kubernetes or any new container orchestration system which supports Docker. KubeMQ is also Lightweight and Small Footprint Container. The capacity of each container is 30MB, which gives it natural to perform in a microservice architecture. The capacity of a cluster of three containers is 90 MB. KubeMQ was developing in GO to maximize computer resources and to work smoothly with Kubernetes. According to Kubernetes experts when they benchmarked KubeMQ using a cluster with 1k message size and sent one million messages with the same hardware specs. KubeMQ beat another message broker by 20% in speed.
Apache Kafka is a very successful publish/subscribe system, which can be used probably to process a stream of data. The central concept in Kafka is a topic, which can be replicated across a cluster giving a safe data warehouse. By performing processed message offsets following to Kafka, it is nearly straightforward to perform guaranteed “at-least-once” processing. Kafka is a stateful service, and this does make the Kubernetes configuration more difficult than stateless micro-services. Zookeeper is a centralized set of services for managing configuration data, labelling, providing shared synchronization, and providing group services. To manage a cluster, both Kafka and Zookeeper should be deployed. Each Kafka container weighs about 600 MB, and each Zookeeper container weighs about 100MB, totalling 2.1GB (three Kafka containers + three Zookeeper containers). Kafka is heavy and manages large microservice without any problem.
NATS is a simple, secure and high-performance, open-source messaging system for cloud-native applications, IoT messaging, and microservices architectures. The NATS server develops in the Go language. The centre design systems of NATS are performance, ease of use and scalability. A data streaming system powered by NATS that combines persistence, message replay, durable subscriptions to NATS.NATS supports four main patterns for communicating messages across entities. Those are Subject-Based messaging, Publish-Subscribe, Request-Reply, and Queue Group.
Challenges in Event-Driven Architecture
The key difficulties in an EDA are Guaranteed delivery, Processing events in order or exactly once and managing consistency across service boundaries.
Almost once: The producer transfers a message, and the consumer demand may / may not receive it.
At least once: The producer transfers a message, and the consumer may prepare duplicate occurrences of the message.
Exactly once: The producer transfers a message exactly once, and the consumer processes it exactly once in EDA.
The event-driven architecture enables real-time, complex transactions among IoT devices, data streams and workflow systems. Every enterprise needs to respond quickly and effectively to evolving markets or circumstances. Greater operating efficiencies: a loose combining of the EDA without disrupting the layering, which allows for smooth, cost-effective service gives enterprises the competitive advantage they need.
You may like to read this Cloud-Native Application Architecture Pattern and Design