Making a Trade: Faster, or Bigger?
If you’re in search of a BI or analytics tool, know that you’ll have to make a choice between speed and scalability. As you review your prospective solutions, be sure to determine how well each can balance scalability and performance—there’s bound to be a sacrifice for one or the other.
Scale vs. Speed
Your decision may depend on what you’re trying to accomplish with your solution. If fast queries are your priority, then speed is the way to go. The good news is, you’ll be able to model data, put it in memory, and receive fast queries in alignment with the demands of your users. The bad news is you’ll only get a subset of data—usually the summary data, which is only as fresh as it was when extracted.
If going big is more important than going fast, then scalability is what matters. This approach will give users direct access to data by querying source data directly. The unfortunate part of opting for scalability is that query performance can suffer, particularly when it comes to larger amounts of data.
How to Make Up for Lack of Scalability or Weak Performance
No matter your priority, there are ways to find a balance between the two. Every BI tool, no matter which side of the spectrum it’s on, tries to compensate for inherent weaknesses.
A few ways to compensate when emphasizing speed:
- Increase the hardware, size, or CPU power of your hardware overall
- Extract the summary data that users are going to query most of the time
A few ways to compensate when emphasizing scalability:
- Craft interfaces to the native API of the database to extract all of the performance possible that the database has to offer
- Use caching
- Use projections, which are regressions that estimate result sets in real-time while your query is running
At Zoomdata, we aim to bend the rules of nature and give you the best of both worlds, allowing you to do fast queries against huge volumes of data through data sharpening.