Building Samsara-Analytics Platform for Shyrne Limited
XenonStack helped Shyrne Limited Enterprises to build an analytics platform to archive data from Social Media account. It creates a scrapbook for all memories and keeps them in a single place to archive every interaction into one feed scrolled through mobile app, web app or desktop app.
Samsara-Analytics uses third-party components Apache ZooKeeper, Apache Kafka, ElasticSearch, Kibana. It provides us –
- A fast, scalable solution to ingest user/machine generated events.
- A Real-Time processing pipeline with a collection of common processing tools.
- An interactive frontend user interfaces to explore your data-set in Real Time.
Samsara-Analytics supports the following types of deployments and AWS Components-
- Bare metal with Docker
- Amazon EC2
- Amazon ECS
Samsara has four major components: the ingestion APIs, Real-Time processing pipeline, the live index and query APIs, and the frontend data exploration tool.
Challenge for Building the Processing and Messaging Platform for Shyrne Limited
- Stream processing is always a challenge for any Real-Time analytics platform.
- End to End Solution from Ingestion to Indexing.
- Full Stack Solutions are not available and are expensive.
- Integrate and Translate Big Data into useful insight.
Solution Offerings for Building Stream Processing and Analytics Platform for Shyrne Limited
Samsara used for Stream Processing and storing the data into Elasticsearch. REST services use Samsara ingestion API for ingesting the events to the Samsara Analytics platform. The events ingested using ingestion-API and Samsara Analytics platform does further processing.
Deploy Full Stack on Kubernetes, use multiple pods for ingestion API and a load balancer to manage those ingestion API from a single endpoint. It gives high ingestion speed of the events from the different sources.
One of the Samsara component ‘QANAL’ responsible for the indexing of the events to the Elasticsearch. At this point, use multiple QANALs and assign the Kafka partitions to them. After indexing into the Elasticsearch further perform analytics operation on the events.