What’s New

Important new features and improvements are as follows.

Note

A link in blue text next to a description in these Release Notes indicates the launch state, availability, and default state of the item (for example, Beta). The link provides more information. Unless otherwise stated, features are generally available, available as self-service (without intervention by Qubole support), and enabled by default.

Administration
Cluster Management
Data Analytics
Hadoop
Hive
Notebooks and Dashboards
Presto
Spark
  • The Account Level Concurrent Command Limit (shown under Account Settings in the QDS UI) has increased from 20 to 100. Gradual Rollout. Learn more.
  • The Clusters page of the QDS UI displays a new cluster health tile card with metrics. Learn more.
  • Other enhancements and bug fixes.
  • A new version of the Analyze page, previously released as New Analyze, is now called Workbench.
  • Cluster monitoring, includng daemon status, heap usage and coordinator node metrics, is available in Workbench and a REST API.
  • QDS now allows you to configure buffer capacity in Hadoop and Spark clusters. Learn more.
  • QDS Hadoop now allows more containers per node, improving memory management in YARN.
  • QDS supports enterprise installations of Github and Gitlab. Via Support. Learn more.
  • The Environments UI is now available in the Control Panel by default for new users. Beta.
  • Added a new scheduler to optimally schedule tasks based on locality of data cached with Rubix. See `https://www.qubole.com/blog/presto-rubix-scheduler-improves-cache-reads/`__.
  • Added call hive.default.clear_cache() procedure call to clear stale Hive metastore caches. Useful when metastore updates might have occurred from outside the Presto cluster.
  • Made performance improvement in queries involving IN and NOT IN over a subquery. See `https://prestosql.io/blog/2019/05/30/semijoin-precomputed-hasd.html`__.
  • Improved smart query retry to support INSERT OVERWRITE TABLE, CREATE TABLE AS and SELECT queries which failed without returning any data. Tracking of query retries has been improved in command logs with Query Tracker links for retries.
  • Qubole supports Apache Ranger integration with Spark on Spark 2.4.0 and later versions. Beta, Via Support. Learn more.
  • Spark 2.4.3 is generally available. Learn more.