How QDS Improves Computing Efficiency

Qubole provides advanced capabilities that automatically optimize computing efficiency and help save you money. These include automating cluster sizing and lifecycle management so as to match computing resources with actual usage. Qubole also provides automatic and intelligent purchasing options across different tiers of computing resources.

Cluster Lifecycle Management

QDS automatically shuts down a cluster when there are no more active jobs running on it. You can always shut down the cluster manually, but in practice Qubole has found that the vast majority of clusters are auto-terminated. Automated cluster lifecycle management protects you against accidentally leaving an idle cluster running, saving unnecessary compute charges.

For more information about lifecycle management, see Cluster Termination under Introduction to Qubole Clusters.

Autoscaling

QDS dynamically adds or removes cluster nodes to match the cluster workload; this is called autoscaling. Autoscaling is particularly effective for workloads that are less predictable and which involve many users running jobs on the cluster concurrently. For such workloads, Qubole has seen autoscaling provide substantial savings compared to a static-sized cluster. (see Case Study TubeMogul).

Autoscaling is available for Hadoop (Hive), Presto, and Spark clusters.