Within the Zoomdata server, its stream processing engine treats all data as fast streams. Of course, streaming analytics are not limited to real-time data. A good analogy is streaming video: just because you are streaming video doesn’t mean it’s live -- you can also stream historical video like a movie. Through its stream processing engine, Zoomdata streams real-time and historical data from any database or modern big data source to the user.
For historical data, the effect of this streaming architecture is easiest to see through Data Sharpening. As soon as the user creates a visualization, Zoomdata instantly streams an initial set of results. The visualization sharpens with data updates as the rest of the query completes and becomes available.
Zoomdata pushes query processing to the source as much as possible. For sources like Impala that speak SQL, Zoomdata generates SQL queries and sends them to the source. For sources like Elasticsearch, Cloudera Search, and Solr that speak search, Zoomdata generates search queries and, again, sends them to the source. The “heavy lifting” involved with resolving a query, such as aggregation, filtering and calculations, is performed by the system where the data resides. Only the final result set is transferred from the source to the Zoomdata server. Avoiding unnecessary data movement is essential for big data scalability.
Zoomdata leverages Apache Spark as a complementary processing layer within the Zoomdata server. As we look under the hood of Zoomdata, you’ll see how we leverage Spark to provide the fastest visual analytics at a high scale.
With recent advances in cloud computing, virtualization, containerization, continuous integration, and the DevOps movement, deploying software solutions today is very different from even just a few years ago. Today’s software adheres to a set of principles that deliver modern distributed applications.
Modern distributed applications are built as a set of independently deployable microservices distributed over clusters of commodity hardware. Why is this good?
Modern distributed applications also provide flexibility:
Zoomdata is built with all these principles in mind. If you’re building a software-as-a-service (SaaS) application that includes visual analytics, using Zoomdata ensures that your application architecture is scalable and flexible. Even if you’re an enterprise deploying visual analytics for internal use cases, these same principles apply as you deploy on a hybrid mix of cloud and on-premise infrastructure.
Zoomdata’s microservices architecture enables the software to scale out on commodity hardware. You can scale out additional Zoomdata server nodes to meet greater user demand. Or add Spark nodes as data processing demand grows. And scale out nodes of the original data sources as data volumes increase. Zoomdata is also available in a containerized format, so it’s easy to deploy in the cloud, on premise, or in a hybrid environment.
Learn More About Zoomdata in the Cloud
No big data solution is complete without security. Our data architects have built security in to Zoomdata's architecture. Regardless of whether Zoomdata is used as a standalone BI platform or used to visually analyze data in an embedded application, it ensures adherence to the "three As" of security -- proper authentication, authorization, and auditing of the visual analytics environment.
Streaming architecture is fundamentally different from that used by traditional business intelligence (BI) systems. A streaming analytics architecture expects data to be in motion and flowing from the original source to the end user. Zoomdata is built for stream processing and consists of a set of data sources, the Zoomdata server, and clients that present visual analytics to end users.