In the era of the Internet of Things (IoT), an enormous amount of sensing devices collect and generate a multitude of data over time for a wide range of fields and applications. Based on the nature of the application, these devices will result in big or near real-time data streams. 

In my current position in this past year, I have been working on a Big Data Analytics project for a Malaysian GLC and with this project, it brought me back to a time more than 10 years ago when I 1st joined Shell in Miri, Sarawak as a Upstream Production Project Manager.

While working in the Oil & Gas industry for close to 10 years, I became quite familiar with real-time or near real-time data through OSIsoft PI system. It is somewhat not much different from what is being done now with IoT and Big Data Analytics for other industries.

PI was used to capture, process, analyze, and store any form of real-time data. The PI System is a suite of software products that are used for data collection, historicizing, finding, analyzing, delivering, and visualizing. It is an enterprise solution for management of real-time data and events. 

PI uses more than 450 off-the-shelf proprietary interfaces to capture, collect, connect and centralize information with its unique ability to collect high-fidelity time-series data from a diversity of sources. It is designed to store millions of data points. 

PI could instantly compare historical and real-time information and is optimized for fast and efficient data delivery.

Through PI, you could visualize and bring information to life – on any device. 

You could foster collaboration within and across operations boundaries, empower people and build a more creative and innovative organization. 

Applying analytics over such data streams to discover new information, predict future insights, and make control decisions is a crucial process that makes IoT a worthy paradigm for businesses and a quality-of-life improving technology.

Add that with some advanced machine learning techniques to facilitate the analytics and learning in the IoT domain and Wallah - we can start articulating IoT data characteristics and identifying two major treatments for IoT data from a machine learning perspective, namely IoT big data analytics and IoT streaming data analytics.