Ecommerce Analytics Platform and Solutions

Introduction to Analytics Platform

Challenges for Building eCommerce Analytics Platform

Solution and Tools for Building the eCommerce Analytics Platform

• Apache Spark - A fast engine for large scale data processing.
• Scala language - A dynamic, interpreted general-purpose programming language.
• Kafka - A distributed streaming platform, publish-subscribe messaging system.
• Node.js - Open source, cross-platform javascript runtime environment.
• Socket.io - Real-time engine for Node.js, is a javascript library for near real-time web applications.
• HighCharts - JavaScripting charting framework which provides interactive charts for web pages.
• CloudxLab - It provides a real cloud-based environment for learning various tools.

Building Near Real Time Analytics Platform

• When a user buys a product, the id related to the particular product with the order status and time gets passed to the Kafka topic.
• Spark streaming code takes the data from Kafka topic in a window of a few seconds, processes it as to count each different order status in that few seconds of an open window.
• As soon as Spark stream processes the total count of each unique order, the state gets pushed to new Kafka topic.
• Now a Node.js server will start consuming messages as soon as new messages are available in the one minute Kafka topic, and then the consumed message gets emitted to the browser via Socket.io.
• As socket.io-client in the browser receives a new “message” event, data in the event start getting processed.
• If the order status of the buyer is “shipped” in the received data, then it gets attached to HighCharts series and gets presented on the browser of the buyer.