Cluster Management

New Feature

  • ACM-4294: A new cluster-health tile card appears in Clusters section of the QDS UI, on the details page tor each cluster.

Enhancements

  • ACM-5555: Qubole supports using shared VPCs for GCP clusters.
  • ACM-5529: Users can now create Presto clusters from the UI on GCP.
  • ACM-5237: Cluster creation now accepts 2.3 Beta as a Hive Version in GCP. You can configure it from the Configuration tab of the Clusters UI while creating a cluster.
  • ACM-5253: Information about which user started or terminated a specific cluster will now be available in the Cluster State API.
  • ACM-5240: Autoscaling logs now contain additional details when Qubole falls back to on-demand VMs due to unavailability of preemptible VMs.
  • ACM-5050: Cluster creation now accepts Use Qubole Placement Policy in GCP. You can use it by checking the Use Qubole Placement Policy checkbox in the Composition tab of the Clusters UI while creating a cluster.
  • ACM-4997: Qubole proactively replaces existing preemptible VMs with new ones before the interruption time of 24 hours, thereby preventing the duration limit from having an adverse effect on running queries.
  • ACM-4955: Service accounts created by manual scripts in GCP now allow BigQuery storage reads.
  • ACM-4884: Cluster creation in GCP now accepts the Node Cooldown Period option from both the UI and the API. Users can configure it from the Composition page in the UI while creating or updating clusters.
  • ACM-515: QDS will use account credentials if cluster credentials do not work to terminate the cluster. You can also contact Qubole Support to force-terminate a cluster that is stuck in the terminating state.
  • ACM-1493: The email address of a user who has manually terminated a cluster is now visible on the Clusters page of the QDS UI.
  • ACM-4323: QDS supports these query runtime configurations for a given cluster: Via Support
    • Set the query execution timeout in minutes for a cluster. QDS auto-terminates a query if its runtime exceeds the timeout.
    • Set a warning about the query runtime in minutes for a cluster. QDS notifies the user of the account via email if a query’s runtime exceeds the configured time.
  • ACM-4801: A QDS API provides the details of the last cleanup activity for a given cluster. The API provides the reason why the cluster was selected or skipped for auto termination. For more information, see get-cleanup-information.
  • TOOLS-1178: QDS has removed the deprecated pycrypto package as of R57. Pycryptodome 3.0, a replacement for pycrypto, is included as of QDS R53.
  • ACM-5515: The cluster console index page now displays the latest cleanup status. For example this will show why a cluster was not terminated.

Bug Fixes

  • ACM-5237: Cluster creation now accepts 2.3 Beta as a Hive Version in GCP. You can configure it from the Configuration tab of the Clusters UI while creating a cluster.
  • ACM-5221: If enable-oslogin was set to true at the project level, Qubole cluster start would fail. This issue has been fixed by setting enable-oslogin as false at the instance level to override the project-level setting.
  • ACM-5192: Configuring a bastion node in a GCP cluster worked only when the user called “ec2-user.” Any valid username can now be used.