Streaming architecture is fundamentally different from that used by traditional business intelligence (BI) systems. A streaming analytics architecture expects data to be in motion and flowing from the original source to the end user. Zoomdata is built for stream processing and consists of a set of data sources, the Zoomdata server, and clients that present visual analytics to end users. Starting from the source, new data can appear at any time via Smart Connectors. The stream processing engine assembles results from multiple input streams. For the user, a WebSockets-based UI pushes visualization updates from the Zoomdata server to the browser or mobile device.
Within the Zoomdata server, its stream processing engine treats all data as fast streams. Of course, streaming analytics are not limited to real-time data. A good analogy is streaming video: just because you are streaming video doesn’t mean it’s live--you can also stream historical video, like a movie. Through its stream processing engine, Zoomdata streams real-time and historical data from the source to the user.
For historical data, the effect of this streaming architecture is easiest to see through Data Sharpening. As soon as the user creates a visualization, Zoomdata instantly streams an initial set of results. The visualization sharpens with data updates as the rest of the query completes and becomes available.
Zoomdata pushes query processing to the source as much as possible. For sources like Impala that speak SQL, Zoomdata generates SQL queries and sends them to the source. For sources like Elasticsearch and Solr that speak search, Zoomdata generates search queries and sends them to the source. The “heavy lifting,” such as aggregation, filtering, and calculations, involved with resolving a query is performed by the source system where the data resides. Only the final result set is transferred from the source to the Zoomdata server. Avoiding unnecessary data movement is essential for big data scalability.
Zoomdata leverages Apache Spark as a complementary processing layer within the Zoomdata server. As we look under the covers of Zoomdata, you’ll see how we leverage Spark to provide the fastest visual analytics at a high scale.
Zoomdata’s microservices architecture makes it easy to deploy in the cloud or on-prem on modern data center infrastructure.
With a streaming analytics architecture, Zoomdata is built for stream processing. See what Zoomdata can do for you!