One of Zoomdata’s customers, a brand analytics software provider, embedded Zoomdata into its software to enable its customers to see and interact with analytics in new ways. The company’s customers are primarily in the retail, transportation and financial services markets. They need to visualize in near real-time sales transactions, social media mentions, and other factors that influence their brands. By embedding Zoomdata in their software, their customers can:
There’s more! Learn how Cielo S.A. uses Zoomdata in Live Mode to:
A: Zoomdata keeps two-way WebSockets connections open between the users’ web browsers and the Zoomdata Query Engine. When in Live Mode, Zoomdata polls the data source for data that arrives within a configurable time window (e.g., a few seconds, or a minute), processes it, and then pushes the aggregated and calculated results to the users’ dashboards.
A: Any supported data source that includes some sort of query and processing engine, such as traditional databases and high-performance query engines for big data such as Apache Impala and Presto, can be used in Live Mode. Plain file data sources such as Amazon S3 and HDFS need a query engine in order to work in Live Mode. Zoomdata recommends using modern data sources that operate as “fast data sinks” for Live Mode.
A: A “fast data sink” is a fancy term for any database or data platform that can be configured to handle fast writes and many concurrent reads (queries).
A: Some good examples of fast data sinks are Apache Impala on top of Kudu or Parquet files in HDFS; search-engine databases such as Elasticsearch, Cloudera Search, and Solr; MemSQL; Snowflake, and the like. You can use a traditional database, but test first to make sure it can handle the data quantity and refresh rates so they are acceptable for your user community.
A: Data is “landed” in fast data sinks using any number of stream processors, such as Kafka Connect, Google Cloud Dataflow, NiFi, Spark Streaming, SQLStream, StreamSets, and Zoomdata’s built-in Stream Writer Service. Some stream processors can rapidly clean, enrich, and transform data before it is landed in the fast data sink.
A: The business requirements for near-real-time business intelligence demand a complete data platform that go beyond visualizing an event stream. Fast data sinks with “Live Mode” enabled can scale to handle ad hoc end-user exploration of potentially massive amounts of live, historical, and other related data.
A: Lambda architectures attempt to combine batch and stream processing, and are tricky to set up and expensive to maintain. Instead, Zoomdata recommends landing and keeping data in a fast data sink, and using Live Mode to get near-real-time updates. If old data needs to be rolled off the fast data sink into warm or cold data storage, you can use one of Zoomdata Multisource Many Ways™ techniques to work concurrently with live and historical data.
A: Zoomdata does not ingest or store data, but does offer a Stream Writer Service that can land data from a stream to a database.
A: The data must be indexed or partitioned by a timestamp field.