Connecting to a Custom Hive Metastore

Creating a Custom Hive Metastore describes how to create a custom Hive metastore from the very beginning.

Connecting to a Custom Hive Metastore

QDS provides a default Hive metastore, which may be satisfactory for your business purposes. If so, you don’t need to do any additional configuration. But you may want to use your own private metastore for reasons of security or performance; in this case, follow the instructions on this page.

Prerequisite

The metastore must reside on a MySQL database; this is the only type of database QDS supports for this purpose.

Connecting to the Metastore

You need to connect the metastore to:

  • Your QDS Hive cluster; and
  • The QDS service itself

Connecting the Metastore to the QDS Hive Cluster

  1. Configure the metastore in the same GCP network you are using for your QDS clusters.
  2. In the QDS UI (see step 3 below), check the Enable Cluster Access box.

Connecting the Metastore to QDS

You can configure QDS access to the metastore either through a Bastion node, or by whitelisting a private IP address.

  • To configure a Bastion node for QDS access, follow these instructions; or
  • To whitelist a private IP address, contact Qubole Support and provide the address to be whitelisted.

Specifying Your Configuration in the QDS UI

  1. From the QDS main menu, choose Explore.

  2. On the resulting page, pull down the menu to the right of Qubole Hive and choose Connect Custom Metastore.

  3. Fill out the fields as follows:

    Metastore Database: provide the name of the MySQL database hosting the metastore.

    Host Address:

    • If you are using a Bastion node, enter the Bastion node’s private IP address.
    • If you are whitelisting an address, enter the public IP address corresponding to the private IP address that you provided to Qubole support.

    Port: Accept the default (3306).

    User Name: Enter the name of the superuser or administrator user on the host you identified in the Host Address field.

    Password: Enter the password for the superuser or administrator user.

    Enable Cluster Access: Check this box to allow direct access between the QDS cluster and the metastore. Qubole recommends that you use direct access.

  4. If you are not using a Bastion node, leave Bastion Node unchecked and click Save to save your changes; otherwise continue with step 5.

  5. Check the box next to Bastion Node to enable access via your Bastion node.

  6. Enter the public IP address or hostname of the Bastion node.

  7. Enter the user name of the superuser or administrative user on the Bastion node.

  8. Enter the private key corresponding to the Bastion node’s public key.