IoT analytics is an application that helps to understand the huge volume of data generated connected IoT devices. In other words, ‘Internet of Things’ is coined by Kevin Ashton co-founder of the Auto-ID Center at the Massachusetts Institute of Technology (MIT). In other words, the Internet of Things is just an ecosystem, the physical things which are connected to the internet and we see results through IoT analytics platform. You may also explore more about Real-Time Anomaly Detection for Cognitive Intelligence in this blog. For instance, these physical things include
This IoT analytics has the ability to transfer data over the network. Learn more about Learning Analytics in this insight.
There are 4 layers in IoT architecture are highlighted below:
Hardware/Sensing Layer - This is the very first layer of the IoT architecture. Also, it consists of the hardware device which is connected to the Network layer. Wireless Sensor Networks (WSN) and radio-frequency identification (RFID) are considered as the two main building blocks of the IoT. The Arduino Microcontrollers are directly connected with the sensors. The Arduino microcontrollers are also connected to the Raspberry Pi which is also connected to the internet using Ethernet or WiFi. It transmits the data collected from the sensor in real-time to the server.
Network/Gateway Layer - This layer acts as the bridge in between Hardware /Sensing Layer and Management Layer. This layer receives the digitized data and routes it over wired LANs, Wi-Fi, or the Internet for further processing. There is a different protocol which can be used to communicate between IoT gateway and servers like MQTT, AMQP, COAP, and HTTP. We have discussed different protocols used in the next section.
Management Layer - This is the layer which is responsible for data modeling and security control. This is the layer where all the data handling or data processing operation will be done. At this layer, the necessary data is extracted from the data transferred by the sensor.
Application Layer - This is the final layer of the IoT analytics architecture. This layer uses the data processed by the management layer.
What is Real-Time Stream Processing for IoT?
Real-Time Stream processing refers to the data processing with the data stream collected from the IoT device in Real-Time. Learn more about Open Source tools for real-time Analytics Platform. These are the tasks which can be included in this processing -
Transformation - It includes the conversion of the data which is collected from the IoT device. After this conversion, the resulting data is transferred for further analytics.
Data Enrichment - Data enrichment process is the operation in which the sensor collected raw data is combined with the other dataset to get the results.
Storing Data - This task includes storing the data at the required storage location.
The below mentioned protocol are most widely used:
MQTT - MQTT protocol uses a publish/subscribe architecture. The central communication point of this protocol is MQTT broker. Every client includes a topic name while publishing data to the broker. Topics are responsible for routing information for the broker. Each client that wants to receive messages subscribes to a particular topic and the broker delivers all messages with the matching topic to the client.
COAP - COAP Protocol (Constrained Application Protocol) is a web-based protocol that has been designed to connect with lightweight devices to the Internet of things(IoT). Like HTTP protocols, COAP is also used Request-Response model. It also allows to make API calls GET, PUT, POST, DELETE data via URL.
AMQP - AMQP (Advanced Messaging Queuing Protocol) is an open standard for passing messages in between applications and organizations. It connects system, provides business processes with the information they need.
HTTP - This is the standard protocol for web services and still is used in IoT analytics solutions. The most popular architectural style called RESTFul is widely used on mobile and web application and must be considered on IoT Solutions.
DDS - DDS stands for Data Distribution Service which is a standard for real-time IoT analytics, scalable and high-performance machine-to-machine communication. DDS can be deployed in both low footprint devices and on the cloud as well.
Xenonify IoT Architecture
Data Collection and Ingestion
This will be the first step in data handling from the IoT device. We have different option to collect data from the sensors. In the upcoming paragraph, we are going to discuss the data ingestion using a different platform.
Using Apache NiFi
Using Apache MiNiFi on Raspberry - We can use Apache MiNiFi which is a sub-project of Apache NiFi on the Raspberry Pi. This a very lightweight and low resource consuming agent. From this, we can collect data from the sensor and route it the Apache NiFi. From here we can route the data to multiple required destinations.
Direct ingestion from MQTT Broker - The second way is to ingest data directly from the MQTT broker and routing it to the multiple required destinations. For this, we need to design the data flow pipeline using different Apache NiFi processors.
Using Stream sets
By Using StreamSets data collector Edge - StreamSets data collector Edge is an ultralight agent for the IoT. Which collects the data sends it to the stream sets.
By Direct ingestion from MQTT broker - The second way is to ingest data directly from the MQTT broker and routing it to the multiple required destinations. For this, we need to design the data flow pipeline using different Streamsets processors.
Xenonify provides you with a platform where you can use to build your own IoT analytics platform for the data collected from the sensor. Apache Spark is being used by Xenonify at the data processing stage. The speed of the data stream can also vary according to the time. So, Our Data Pipeline should be able to handle data of any size and at any velocity. The platform should be intelligent enough to scale up and down automatically according to load on the data pipeline.
Xenonify offers you multiple data storage option which includes HBase, HDFS, ElasticSearch, Druid as the Time-Series database.
After the process now the most important part is the visualization of the data. Xenonify offers the custom dashboard for the visualization of the data. We can build our dashboard by using Kibana and Grafana. We can also build our custom dashboard using React JS and D3.JS.
Google IoT Architecture
In Google, IoT analytics architecture Cloud Pub/Sub is used to listen to the data from MQTT broker. Also, Cloud Pub/Sub is a messaging system that can handle a real-time data stream. Cloud Pub/Sub also natively connects to other Cloud Platform services, gluing together data import, data pipelines, and storage systems. After ingesting the data from the IoT device Cloud Pub/Sub further sends the data to the Cloud Dataflow for the data processing. Cloud Dataflow is used to create the data pipeline to perform some data transformations. Cloud Dataflow sends the processed data to the BigQuery. BigQuery provides a fully managed data warehouse with a familiar, SQL-like interface. And further Cloud DataLab is used for the data representation.
Microsoft Azure IoT Architecture
Microsoft Azure Events Hub connects to MQTT via Cloud Gateway and consumes the data published by Raspberry Pi on the MQTT broker. Azure provides Stream Analytics to data processing. Above all, we can use the data stream from Events Hub and process the real-time data stream. After processing, the data is sent to Azure CosmoDB and can be further visualized on Power BI by using the CosmoDB connector.
AWS IoT Architecture
Events Hub connects to MQTT via Cloud Gateway and consumes the data published by Raspberry Pi on the MQTT broker. Azure provides Stream Analytics to data processing. Above all, we can use the data stream from Events Hub and process the real-time data stream. After processing, the data is sent to Azure CosmoDB and can be further visualized on Power BI by using the CosmoDB connector.
Configuring Sensor DHT22 with Raspberry Pi 3
Requirements for Configuring Sensor
Raspberry Pi 3
DHT 22 or AM2302 temperature and humidity sensor
DHT22 is a 4 pin sensor. It can be operated on 3.3V which makes it compatible with Raspberry Pi 3.
Pin 1 is the VCC pin
Pin 2 is the data pin
Pin 3 is the No connection pin
Pin 4 is the ground pin
Pin 1 requires being connected with a 3.3V power source. Pin 2 to the GPIO(General purpose input-output) on the Raspberry Pi. Pin 4 to the Ground(GND) and Pin 3 remains disconnected. A 10kΩ resistor is connected between pin 1 and pin 2.
Configuring Raspberry Pi 3
Firstly, we need to make sure that SSH login is enabled on our Raspberry Pi. Then we need to connect to Raspberry Pi using ssh.
Updating Raspberry Pi Packages
It is always necessary to ensure that your installed packages are up to date. First of all, we need to run sudo apt-get update to update the list of available packages and versions. Now we can update our installed packages using sudo apt-get dist-upgrade.
Install additional Raspberry Pi packages
Now, we are using sensor DHT22 which needs some additional packages. DHT22 is accessed by using the python library. So we need to install python libraries using this - sudo apt-get install python
Installing DHT22 Sensor Libraries
Now, we are going to download the libraries which we will use for reading data from the sensor. To add more, we will use the libraries provided by Adafruit. So, we will clone this repository into our Raspberry Pi - https://github.com/adafruit/Adafruit_Python_DHT.git Furthermore, all the installations have been completed and we can test the readings from the sensor. For this, we will go the below folder -
cd /root/iot/Adafruit_Python_DHT/examples Now, we need to know on which GPIO pin the data pin of the sensor is connected and which sensor we are using. This is the pin diagram for Raspberry Pi 3 - Now, we are using the sensor DHT22 and GPIO pin is GPIO22. So, we will execute this - ./AdafruitDHT.py 22 22 This will give results as below - So, we can modify the AdafruitDHT.py script and enable that to push data on the MQTT broker. This is all we need to set up to on the Raspberry Pi 3 and push data on the MQTT broker.
What are the key use cases of IoT Analytics?
The best use cases of IoT Analytics are listed below:
Healthcare sector is one of the fastest sectors which is adopting the Internet of Things. There are a lot of sensors coming into the scenario and the healthcare sector is adopting it very fast. These devices are allowing the doctor to track their patients whether they are stuck to the treatment or not.
IoT enabled machinery will enable operation managers to manage the factory units remotely. Hence, the use of IoT sensors in manufacturing equipment enables condition-based alerts that are designed to function within specific temperature and vibration range.
Keep vehicles on the road by predicting maintenance need and Streamlining logistics using real-time data and alerts to optimize delivery routes, monitor performance, and quickly respond to delays or issues as they happen. Therefore, we can ease traffic congestion with the help of using real-time analytics.
Predictive equipment maintenance is used for managing energy, predicting equipment failure or detecting other issues. Hence, IoT analytics in retail helps to enable precise inventory management, and most importantly enhancing the consumer’s shopping experience.
No doubt, IoT helps to Connect, reduce management and utility costs by learning from data you collected. For example, IoT analytics for smart buildings helps to personalize and automate your building’s heating and cooling, room utilization, which further helps create a more comfortable and productive work environment.
Smart Agriculture is growing rapidly in the field of agriculture. Therefore, farmers can use meaningful insights from the IoT driven data for sensing soil moisture and nutrients, controlling water usage for plant growth to get better outcomes.
Real-Time Streaming Analytics Solutions for Enterprises & Startups helps them to accelerate their business growth for enhanced productivity. To know more about enterprise IoT Analytics platform, you are advised to go through with the below steps: