Managing Sessions

How Sessions Work

Hive allows you to embed code (Python scripts, shell scripts, Java functions) in SQL queries. This is a way to add functionality that is not natively present in HiveQL. See this Hive page for more information and examples.

Qubole simulates this functionality by associating every command in QDS with a session. Sessions allow you to create temporary data sets, configure parameters which can be used to tune query behavior, and add your code as scripts to QDS to run transformations within HiveQL. These parameters, datasets, and user-defined transformations are active only for the session. A session’s duration is configurable; the default is two hours.

To ensure that scripts are accessible to Qubole’s Hive clusters, upload them to your Cloud storage. A script can be in any Cloud location that is readable using the keys associated with the account, but Qubole recommends that you place scripts in the default location’s scripts folder. Once you have uploaded a script, add it to your session by means of an add file … command.

Creating and Managing Sessions

Navigate to the Sessions page in the Control Panel.

To create a session, click the Add icon Add1Icon at the top right corner of the Sessions page.

A dialog appears:

../../_images/CreateSession.png

Select the cluster on which you want to create a session from the drop-down list and click Create Session. A session is created; for example:

../../_images/Sessions.png

The Sessions page contains:

  • Id: The session ID.

  • Cluster Id: The ID of the cluster on which session is created.

  • Start Time: The start time of the session

  • Duration: The default duration is two hours. You can change the number of hours to any value between 1 and 6.

  • Commands: The number of commands that have been run. To see the commands, click the number. This takes you to the Compose tab of the Workbench page.

  • Action: Click the down arrow in the Action column to see a list of actions:

    ../../_images/SessionAction.png

    You can perform the following operations on an existing session:

    • View Commands: Select this to see the commands that are running in the session; the Session Details dialog appears. Click the down arrow in Action column:

      ../../_images/ViewCommand.png

      You can choose to go to the Workbench page, or to Remove the command.

    • Change Duration: Select this to change the session duration. The Set Duration dialog appears:

      ../../_images/ChangeDuration.png

      The default is two hours. To change it, enter a new value between 1 and 6, and click OK. Click Cancel to restore the previous setting.

    • Deactivate: Select this to deactivate a session. You can re-activate a deactivated session within two hours. To re-activate a deactivated session, click the down arrow in the Action column and click Activate.

    • Delete: Select this to delete the session.