All Commands Report

GET /api/v1.2/reports/all_commands

This API retrieves the All Commands report containing the query metrics in JSON format.

Note

The following points are related to a report API:

  • If the difference between start date and end date is more than 60 days, then the system defaults to 1 month window from the current day’s date.
  • If either start date or end date is not provided, then the system defaults to 1 month window from the current day’s date.
  • If you want to get data for a window more than 2 months, then create a ticket with Qubole Support.

Required Role

The following users can make this API call:

  • Users who belong to the system-admin or system-user group.
  • Users who belong to a group associated with a role that allows viewing commands reports. See Managing Groups and Managing Roles for more information.

Parameters

Parameter Description
start_date The date from which you want the report (inclusive). This parameter supports the timestamp in the UTC timezone (YYYY-MM-DDTHH:MM:SSZ) format. The date cannot be earlier than 180 days.
end_date The date until which you want the report (inclusive). The API default is today. This parameter also supports timestamp in the UTC timezone (YYYY-MM-DDTHH:MM:SSZ) format.
offset The starting point of the results. The API default is 0.
limit The number of results to fetch. The API default is 10.
sort_column The column used to sort the report. The valid choices are time, cpu, fs_bytes_read, fs_bytes_written. The API default is, time (chronological order).
by_user Report only those queries which are created by the current user. By default all queries by the current account are reported.
status
It enables you to filter queries based on the following status values: done, error, running, waiting, cancelled, and cancelling.

Response Parameters

Parameter Description
start_date The starting date of the report. This parameter filters on the created_at parameter value.
end_date The ending date of the report. This parameter filters on the created_at parameter value.
sort_column The sort column used.
queries Contains an array of query-related parameters and values as provided in the following table.

An array of parameters and values associated with the queries parameter.

id The query ID of the command.
created_at The time when the query was created.
submitted_by The Email address of the user who created the query.
command_type The type of the command (HiveCommand, PrestoCommand, and so on.)
command_summary The summary of the command (query/latin_statements/script_location, and so on.)
status The status of the command (whether it succeeded or failed, and so on.)
label It is the label of the cluster on which the command is run.
end_time Denotes the time at which the query execution completed.
cpu The total cumulative CPU, (in ms), consumed by this command. It signifies the cumulative CPU time spent by all cluster nodes in the cluster that processed the command.
fs_bytes_read The total bytes read by this command.
fs_bytes_written The total bytes written by this command.

Examples

Goal

To get the default report.

curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN"  -H "Accept: application/json"
"https://gcp.qubole.com/api/v1.2/reports/all_commands"

Goal

To get the report for commands executed only by the current user.

curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://gcp.qubole.com/api/v1.2/reports/all_commands?by_user"

Goal

To get the report for commands executed during a specific time period and sorted by total bytes read.

curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://gcp.qubole.com/api/v1.2/reports/all_commands?start_date=2017-06-10&end_date=2017-07-10&sort_column=fs_bytes_read"

Sample Response

{
"sort_column": "fs_bytes_read",
"start_date": "2017-06-10T00:00:00Z",
"end_date": "2017-07-10T00:00:00Z",
"queries": [
    {
        "id": 79520285,
        "created_at": "2017-06-19T08:56:06Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "select count(*) from default_qubole_memetracker where month=\"2008-08\";",
        "status": "done",
        "end_time": "2017-06-19 09:05:12 +0000",
        "tags": "",
        "fs_bytes_read": 4429520896,
        "fs_bytes_written": 8,
        "cpu": null
    },
    {
        "id": 78826792,
        "created_at": "2017-06-15T07:01:23Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT * FROM `tpcds_orc_1000`.`customer`",
        "status": "done",
        "end_time": "2017-06-15 07:02:53 +0000",
        "tags": "JDBC 1.0.7",
        "fs_bytes_read": 683353856,
        "fs_bytes_written": 1629612160,
        "cpu": 74910
    },
    {
        "id": 78828387,
        "created_at": "2017-06-15T07:11:24Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT COUNT(* ) FROM `tpcds_orc_1000`.`customer`",
        "status": "done",
        "end_time": "2017-06-15 07:12:03 +0000",
        "tags": "JDBC 1.0.7-SNAPSHOT",
        "fs_bytes_read": 683353856,
        "fs_bytes_written": 9,
        "cpu": 25680
    },
    {
        "id": 78830962,
        "created_at": "2017-06-15T07:23:20Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT * FROM `tpcds_orc_1000`.`customer`",
        "status": "done",
        "end_time": "2017-06-15 07:24:44 +0000",
        "tags": "JDBC 1.0.7",
        "fs_bytes_read": 683353856,
        "fs_bytes_written": 1629612160,
        "cpu": 76200
    },
    {
        "id": 78652381,
        "created_at": "2017-06-14T08:22:17Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "set hive.cli.print.header=false;\nset hive.resultset.use.unique.column.names=false; \nSELECT * FROM `table`.`calcs`",
        "status": "done",
        "end_time": "2017-06-14 08:24:14 +0000",
        "tags": "ODBC 1.0.0.1001",
        "fs_bytes_read": 23357610,
        "fs_bytes_written": 7470759,
        "cpu": 4330
    },
    {
        "id": 78825573,
        "created_at": "2017-06-15T06:54:44Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT * FROM `table`.`calcs`",
        "status": "done",
        "end_time": "2017-06-15 06:57:54 +0000",
        "tags": "JDBC 1.0.7",
        "fs_bytes_read": 23357610,
        "fs_bytes_written": 7470759,
        "cpu": 4320
    }
    ]
}