Does Your Analytics Tool Support Batch and Streaming Data?
Watch this video, to learn why it’s important that your analytics tool supports batch and streaming data.
Part of any big data strategy includes determining the role of batch and streaming data. To succeed with big data, sometimes you need to dig deep into massive historical data sets. For that, you need a tool that can analyze batch data. Other times, you need to analyze what's happening as it happens. That's a job for streaming data analysis. That’s why you need a tool that can do both. Both are valuable depending on the particular application or solution you're building.
Hi, everyone. I want to talk to you today about an important characteristic of a big data analytics tool. And that is its ability to support both batch and streaming data.
Analyzing Data at Rest
Now, traditionally analytical tools, BI tools, have analyzed data at rest. And that data has been moved from various business systems using ETL tools into a data warehouse of some sort. And that’s been great, it allows us to view integrated historical data for strategic and tactical purposes. In other words, let’s find out what’s happened and what to do about it.
Analyzing Data In Flight
Now, today this increasing need to analyze not data at rest, but data in flight, typically streaming data that’s coming in off of sensor fields or POS systems or customer data records that are coming off of telephone networks, large volumes of data that we don’t have enough time in our nightly batch windows to load and then analyze. And also, the business increasingly doesn’t want to analyze stale old data, they want to analyze the real-time data.
The Ideal Analytics Tool
So, ideally an analytics tool can analyze both data at rest and data in flight. And ideally they can virtualize between the two. In other words, get me the data that I need whether it’s data in a warehouse or data that’s coming in through some kind of a streaming interface.
This is particularly important for the real time data except for a few operational administrators who really need to look at the current data as it’s coming in. Most of us want to view that data in the context of historical patterns and trends so that we can tell whether what we’re seeing is normal or out of whack and we need to take action. So, a great big data analytics tool needs to support both batch and streaming data.