High-Performance Virtualized Spark Clusters on Kubernetes for Deep Learning
Publisher : VMware
Latest Version : November 18, 2019
A virtualized cluster was set up with both Spark Standalone worker nodes and Kubernetes worker nodes running on the same VMware vSphere virtual machines. The same Spark image classification workload was run on both Spark Standalone and Spark on Kubernetes with very small (~1%) performance differences, demonstrating that Spark users can achieve all the benefits of Kubernetes without sacrificing performance.