An Overview of Heterogeneous Nodes in Clusters

QDS supports heterogeneous Spark and Hadoop 2 clusters; this means that the worker nodes comprising the cluster can be of different instance types.

Configuring Heterogeneous Worker Nodes in the QDS UI

Managing Clusters describes how to edit the cluster configuration through the QDS UI.

Select Use Multiple Worker Node Types to configure heterogeneous worker nodes. The UI displays worker node type and weight.

Select the worker node type; its weight’s predetermined value is populated.

The default node weight is calculated as (memory of the node type / memory of the primary worker type).

Note

You must carefully pick instance types that have similar CPU and memory capacity. Choosing instances types with significantly different CPU and memory capacity may lead to degraded performance and increased query failures as the weakest configuration instance would be the bottleneck during query execution.

You can edit the worker node’s weight. Override the default weight if you want to base it on the number of CPUs, cost, or any other parameter.

The order of preference among worker nodes is set to the order in which worker node types are selected.

Click Add worker node type to add another worker node type. You can select a maximum of 10 worker node types.

Note

In a heterogeneous cluster, upscaling can cause the actual number of nodes running in the cluster to exceed the configured Maximum Worker Nodes. See Why is my cluster scaling beyond the configured maximum number of nodes?.