Migrating Data from Qubole Hive Metastore to a Custom Hive Metastore

By default, a QDS account comes with the Qubole-managed Hive metastore.

Qubole provides you an option to switch to a custom Hive metastore as described in Connecting to a Custom Hive Metastore.

Creating a Custom Hive Metastore describes how to create a custom Hive metastore from the very beginning.

To migrate the data from the Qubole-managed Hive metastore when you decide to a custom Hive metastore, the following section provides the steps.

Prerequisites

Ensure that there is a downtime during the data migration from Qubole-managed Hive metastore to the custom Hive metastore.

Migrating Data from the Qubole-managed Hive Metastore

Perform these steps to migrate data from Qubole-managed Hive metastore to the custom Hive metastore:

Note

If you face any intermittent lock or dead lock issues, see Intermittent Lock and Deadlock Issues in Hive Metastore Migration to resolve them.

  1. Ensure that you have a database instance that is appropriate for the workload.
  2. For security reasons, RDS is recommended to be in a private subnet and its bastion node can be in a public subnet.
  3. Create a ticket with Qubole Support requesting for the Qubole Hive metastore data dump.
  4. After receiving the DB dump from the Qubole Support, push the DB dump on the RDS instance that you have identified (refer to step 1).
  5. It is an optional step. For steps to create a metastore from the very beginning, see Creating a Custom Hive Metastore.