Can I use Python 2.7 for Hadoop tasks?

  • Python 2.7 is the default version configured on cluster nodes by default. This applies to older clusters as well if they are restarted after Python 2.7 has been set as the default version. Existing clusters that use Python 2.6 continue to use Python 2.6 until they are restarted.

  • Python 2.7 is the version installed by default on cluster nodes in the following QDS accounts:

    • AWS accounts created on or after September 29 2016.
    • All Azure accounts
    • All Oracle OCI accounts

Can I use Python 2.7 on an AWS account created before September 29 2016?

Python 2.7 is the default version configured on the cluster nodes by default. This applies to older clusters too only if they are restarted after Python 2.7 is set as the default version.

Can I enable Python 2.7 for an individual cluster?

By default, every cluster will be on Python 2.7 as Qubole is planning to remove the support for using Python 2.6.

In case if your Qubole account is using the older default Python version, 2.6, you can enable Python 2.7 for a specific cluster by adding the following lines to the node bootstrap file specified in the cluster configuration.

source /usr/lib/hustler/bin/qubole-bash-lib.sh
make-python2.7-system-default

The next invocation of the cluster will use Python 2.7.