XenonStack Recommends

DataOps

Big Data Analytics on Kubernetes for Streaming App

Chandan Gaur | 07 December 2022

Big Data Analytics on Kubernetes for Streaming App

Building Analytics Stack for Streaming Applications

  • SKACK Stack is an open source Full-Stack platform for Real-Time analysis of Big Data. It consists of Apache Spark, Kubernetes, Akka, Apache Cassandra, and Apache Kafka.
  • GCP & GlusterFS acts a storage solution as it supports multi-mount and data remains on all nodes of GlusterFS & GCP.

Common challenge for Multi Node cluster on SKACK

  • Set up a multi-node cluster for SKACK Stack with a document on Kubernetes.
  • Container environment is not persistent by default, so application in it needs Persistent storage to store data.
  • Using it to scale up Spark.
  • Using it to scale up Cassandra
  • Using it to scale up Kafka

Solution for on Premises Kubernetes Cluster

To overcome the challenges mentioned above, set up a three-node on premises Kubernetes cluster in which one will as a master and the other two workers. The Cluster includes-
  • It Master
  • Kubernetes Scheduler
  • It Controller Manager
Setup for analyzing the cluster and reporting to the API server to store metrics that contains resource utilization, availability, and performance.

Guide to Reactive Application Development and Akka

Reactive Application Development builds reliable enterprise applications using reactive design patterns. It designs elastic domain models, builds fault-tolerant systems, efficient handling of large data volumes. Akka is a toolkit to build highly concurrent, distributed, and fault tolerant event-driven applications on the JVM. Reactive Design Patterns are event-based architectures for asynchronous handling of the large volume of concurrent service requests from single or multiple service handlers building responsive applications on JVM.

Features of Reactive Design Patterns

  • Domain Driven Design
  • Reactive Microservices
  • Consistent, Availability and Scalable
  • Message and Coordination
  • Logistics
  • Responsiveness
  • Elasticity
  • Resilience

Real-Time Applications of Reactive Design Patterns

  • Design and deploy scalable, resilient, responsive and elastic systems.
  • Design Reactive Microservices.
  • Reactive techniques and patterns.
  • Online Shopping Application.
  • Huge transaction processing services of the Banking sector.
  • Share trade business.
captcha text
Refresh Icon

Thanks for submitting the form.