Picking the Right Data Store: What is Spark?
In this video, you’ll learn about another modern data store -- Spark.
With Spark’s in-memory processing, you can process workloads you couldn’t with traditional map bridge data access. For example, you can put machine learning algorithms in Spark. You can also use Spark as the query engine on top of Hadoop. That way you can have a very fast SQL layer that enables very fast queries. The power of Spark makes it suitable for many types of analytic tasks.
Transcript
So, what is Spark? Spark kind of continues on in that vein, which is open source software to serve analytic type problems. Spark is all about in memory. Spark is all about being able to process workloads which go beyond the workloads that you could do with traditional map bridges.
For example, you can put your own code in Spark. You can put machine learning algorithms in Spark. You can use Spark to actually be the query engine on top of a storage layer in Hadoop. So, your traditional Hadoop storage file system HDFS can be accessed by Spark to create a SQL layer so you can ask fast questions.
But, Spark is a very powerful engine, and lots of companies have used it for lots of different things.
