Although the terms are often used interchangeably real-time analytics and streaming analytics are not the same. Real-time analytics generally refer to analysis that happens literally in real time without human intervention. They could be executed by a complex event processing engine or a computer that’s engaged in machine learning. But whatever the case, they’re happening at speeds beyond those at which humans operate. Another example would be the high-frequency trading (HFT) platforms investment firms use to gain micro-second advantages in the market. These platforms execute trades much faster than the blink of an eye — 300 to 400 milliseconds — with no human interaction.

What are Streaming Analytics?

Organizations adopt streaming data feeds so they can react to events in near real time (NRT). In near real-time analytics there’s a slight delay between the occurrence of an event and the use of the processed data for display purposes.

It’s still very fast. And it’s the very opposite of batch processed data. If you’re looking to visualize and analyze streaming data, you know there’s immense value in acting on the freshest data. You also know that the interpretation of real-time events often needs a look back in history for context. And you can stream historical data as well.

Today, enterprises reap value from high velocity, streaming data such as device logs, sensor readings, social media feeds, IoT data, and more. Streaming analytics mean a paradigm shift from batch-oriented architectures. Recent technologies such as Apache Kafka, Spark Streaming, Apache Storm, Apache Apex, Apache Nifi, and Amazon Kinesis have emerged to manage the velocity of big data.

While much of the discussion on real-time data focuses on machine processing, helping users see these streams through near real-time analytics and visualization is just as important.

Real-Time Analytics

Working with Streaming Data

Streaming Analytics for IoT

IoT data processing brings several challenges to data science and data analytics platforms. First, there’s simply a lot of it and the volume of IoT data will balloon in the next decade as many millions of IoT devices come online. In fact, industry analysts including Gartner predict 25 billion IoT devices by 2020. Even if that prediction turns out to be wrong by half, that’s still a virtually unimaginable amount of data.

The promise of streaming analytics for IoT is that it presents a chance to gather and analyze real-time information about every aspect of a business. Although IoT data is real-time data, the speed at which it can be processed and ingested—speeds and feeds—is another matter. That’s what the architects of streaming analytics platforms are facing.

What are the Advantages of Streaming Analytics?

The point of any analytics is to gain deeper, real-time insights into a situation or problem with an eye towards solving it or producing a better outcome. Because streaming analytics gives visibility into what’s happening right now, it can:

  • Enable decision makers to act sooner
  • Manage key performance indicators (KPIs) daily rather weekly or monthly
  • Understand the root cause of problems more quickly
  • Detect patterns that emerge across diverse data sets such as sales figures and weather variations

A good example of the power of streaming analytics would be a company like GuestDNA, which helps many of the largest retailers transform huge volumes of transaction and sentiment data into a consistent and pleasant experience for their guests, employees, and franchisees. To do this, the company built a data lake on AWS and chose Zoomdata for its visual analytics tool. GuestDNA updates data in real time, 24 hours a day from thousands of sites handling millions of transactions.

Streaming Analytics Common Questions

What are Streaming Analytics?

Organizations adopt streaming data feeds so they can react to events in near real time (NRT). In near real-time analytics there’s a slight delay between the occurrence of an event and the use of the processed data for display purposes.

It’s still very fast. And it’s the very opposite of batch processed data. If you’re looking to visualize and analyze streaming data, you know there’s immense value in acting on the freshest data. You also know that the interpretation of real-time events often needs a look back in history for context. And you can stream historical data as well.

Are Real-Time and Streaming Analytics the Same?

Although the terms are often used interchangeably real-time analytics and streaming analytics are not the same. Real-time analytics generally refer to analysis that happens literally in real time without human intervention. They could be executed by a complex event processing engine or a computer that’s engaged in machine learning. But whatever the case, they’re happening at speeds beyond those at which humans operate. Another example would be the high-frequency trading (HFT) platforms investment firms use to gain micro-second advantages in the market. These platforms execute trades much faster than the blink of an eye — 300 to 400 milliseconds — with no human interaction.

What is IoT analytics?

Internet of Things (IoT) analytics refers to analyzing the date from devices other than computers that are connected to the Internet and send and receive data. For example, it could be data streams from smart appliances in the home or sensor data from an automobile. Many organizations are integrating their IoT streaming analytics with cloud platforms which take advantage of cloud distributed architectures.

What are some of the most compelling streaming analytics use cases?

Streaming data sources and analytics use cases go hand-in-hand. As data sources increase so do use cases. Right now, on the consumer side, personal fitness applications that track and analyze real-time heart rate, blood pressure, sleep quality, etc. are very popular. But it's easy to see how streaming data analytics might predict the imminent breakdown of automobile components or manufacturing equipment failures. In fact, predictive maintenance is a big area of research for large manufacturers. Likewise, financial services applications may help people and institutions make smarter decisions. Look for a spate of real-time streaming analytics use cases to emerge once the revised Payment Services Directive (PSD2) takes effect in Europe in January 2018.

According to a survey by Markets and Markets, the market for streaming analytics is expected to grow to almost $2 billion by 2020.

What is edge analytics?

In edge analytics, data collection and analysis occur automatically on data at a sensor, network switch, or other device instead of having data sent to a central data store.

What is sensor data?

Sensor data is just the output of a device that detects and responds to some type of activity in the physical environment. Sensors can respond to nearly anything: heat, moisture, movement, pressure, velocity, etc.

What makes a modern data framework modern?

Without going into the weeds, two things separate modern and legacy data frameworks. The first is the ability to handle huge volumes of data without "breaking." Hadoop is what comes to mind first in most discussions around this topic, but there are many other options. It seems like a new big data processing framework pops up almost every week. The second is the type of data that the framework can manage. Legacy data frameworks are typically relational -- and they can only process structured data. Most modern data frameworks can manage structured, unstructured, and semi-structured data. If you have an analytics platform that can handle any data type, you need a data store that can as well. Many modern data frameworks have open source software roots.

Featured Resources

Real-Time & Streaming Analytics

Zoomdata enables users to execute  streaming analytics against real-time, historical, and asynchronous data sources.

Contact

401 E. 3rd Avenue, Second Floor
San Mateo, CA 94401
(650) 399-0024

11921 Freedom Drive, Suite 750
Reston, VA 20190
(571) 279-6166