Hadoop

Gracefully Terminating Shell CLI Commands

HADTWO-2522: Qubole plans to gracefully terminate shellcli commands if the connection to the coordinator node fails. Gradual Rollout | Cluster Restart Required

After the feature is enabled, Qubole waits for a fixed timeout (120 seconds) to connect with the cluster logs’ location. Qubole gracefully terminates the command only when:

1. The connection to the cluster logs location fails.
2. The running application is stopped.

End of Life for Hadoop 2.8

• HADTWO-2375: Hadoop 2.8 has been removed from the cluster AMI.

Bug Fixes

• HADTWO-2365: QDS now marks a Hadoop worker node as unhealthy if its root disk is full. This unhealthy node is eventually removed from the cluster.
• HADTWO-2490: Fixes a problem that caused Spark shell commands to fail with the following error when the cluster was being downscaled: Unable to close file because the last block does not have enough number of replicas.
• HADTWO-2506: Eliminates a race condition which caused data to be written on decommissioned HDFS Datanodes.