Modern Big Data Analytics Architecture

Modernize Your Big Data Architecture for Real-Time Analytics

Deliver Analytics at Scale with Microservices

Zoomdata is a modern microservices based platform that scales and offers a whole host of other benefits.

What are microservices? 

Microservices are loosely-coupled mini application servers that self-register with each other. Zoomdata microservices are written in Java.

What are the benefits of using microservices for a modern BI platform?

  • The architecture scales, period. Add more computing nodes as needed to meet demand.
  • It scales efficiently and modularly. Scale and optimize memory allocation and runtime attributes for each microservice based on need. For example, you can scale for more users, or apply more nodes and computing power to a big data Smart Data Connector relative to other connectors.
  • You maximize uptime and recover faster in failure scenarios. Zoomdata microservices can be deployed or restarted on-the-fly while the rest of the application is running. You don't have to redeploy the whole application if a new data source comes online, or interrupt users if a single connector fails. 
  • Modularity eliminates system conflicts. Since each microservice is self-contained, you don't have to worry about system conflicts with different versions of Java or any other dependency.
  • Modern distributed applications also provide flexibility:
    • Flexibility of choice for deployment -- deploy the same set of microservices on-premise, in the cloud, or across hybrid infrastructures.
    • Flexibility for future evolution -- swap out or upgrade components without affecting other components of the system.
    • Flexibility for integration -- embed and extend components, such as the Query Engine and Smart Data Connectors, with other services to build integrated applications.

What about high availability and load balancing?

Glad you asked! When done right, a platform built using microservices can be highly fault tolerant to maximize uptime and performance for mission-critical applications. Learn more about our journey towards simple scalability in this blog post: Microservices Architecture: A Brief History At Zoomdata.

A Streaming Analytics Architecture for Real-Time and Big Data

Treating data as streams, rather than batches or sets, is fundamentally different from how traditional BI works. Zoomdata’s unique point of view that data can be represented in streams is baked into the web application, Query Engine, and Smart Data Connectors.

Zoomdata's streaming data analytics architecture uniquely supports:

  • Real-time data analytics through Live Mode
  • Big data analytics through our patented Data Sharpening™ functionality
  • Historical data playback using the Data DVR
  • High-touch interactive data visualization through dynamic query cancellation, time windowing, and other optimizations

Real-Time Communication with WebSockets 

One of the technologies that enable a streaming data analytics platform is the WebSockets bidirectional communication protocol. The Zoomdata web application and any other client application that uses the Zoomdata JavaScript  APIs establishes a two-way WebSocket communication channel between itself and the Zoomdata Query Engine. WebSockets are what allows data to stream in one direction while user requests stream in the other, allowing a dynamic and interactive end-user experience.

 

Stream Real-Time Data in Live Mode

Empower your whole organization to visualize and analyze streaming data for powerful real-time insights. 

 

Big Data Visualization with Data Sharpening™

Work with data that's too big move using Zoomdata's unique Data Sharpening functionality.

 

Stream Data to Custom Applications

Application developers can use simple JavaScript commands to embed charts and populate them with fresh streaming data -- direct from almost any source.

Data Sharpening™ and Microqueries

Microqueries and Data Sharpening™ are patented technologies that work together to provide the most sophisticated end user experience for analyzing big data. There is literally nothing else like it available on the market today.

If enabled for a data source, the Query Engine determines whether to invoke microqueries. Microqueries repeatedly sample data across partitions and return the sample data to the Query Engine for processing.

 

Zoomdata Architecture

 

Data Sharpening is the complementary process of analyzing sample data returned by microqueries, and streaming estimated results to the user’s browser (or other client). Data Sharpening’s estimated results may fluctuate a bit up or down until the final query resolves. However, the relative values of each group usually remain consistent as the data is sharpened. For example, the tallest bar in the chart at 10% completion will almost always remain the tallest bar at 100% completion. This means that users can be confident exploring data even as it streams live to the dashboard.

When the user zooms in, filters, changes metrics or groupings, or any other action that would change data values, Zoomdata cancels any active queries. Canceling active queries, however, is not trivial, and many JDBC drivers do not support it. In these cases, Zoomdata’s smart data connectors issue native API calls to complete the task. 

Security

Watch our master class with Zoomdata CTO Ruhollah Farchtchi, Securing Data in a Distributed Data Ecosystem.

No big data solution is complete without security. Our data architects have built security in to Zoomdata's architecture. Regardless of whether Zoomdata is used as a standalone BI platform or used to visually analyze data in an embedded application, it ensures adherence to the "three As" of security -- proper authentication, authorization, and auditing of the visual analytics environment.

Modern Microservices Architecture for Scale and Speed

Microservices Architecture
  1. Zoomdata Application Server
  2. Modern Query Engine
  3. Smart Data Connectors
  1. JavaScript SDK and RESTful APIs
  2. Microservices architecture -- you are here
  3. Adaptable security model

 

Featured Resources

Zoomdata Architecture

Streaming architecture is fundamentally different from that used by traditional business intelligence (BI) systems. A streaming analytics architecture expects data to be in motion and flowing from the original source to the end user. Zoomdata is built for stream processing and consists of a set of data sources, the Zoomdata server, and clients that present visual analytics to end users.

Contact

Sales: +1 888-564-4965