All Commands Report¶

GET /api/v1.2/reports/all_commands¶

This API retrieves the All Commands report containing the query metrics in JSON format.

Note

The following points are related to a report API:

If the difference between start date and end date is more than 60 days, then the system defaults to 1 month window from the current day’s date.
If either start date or end date is not provided, then the system defaults to 1 month window from the current day’s date.
If you want to get data for a window more than 2 months, then create a ticket with Qubole Support.

Required Role¶

The following users can make this API call:

Users who belong to the system-admin or system-user group.
Users who belong to a group associated with a role that allows viewing commands reports. See Managing Groups and Managing Roles for more information.

Parameters¶

Parameter	Description
start_date	The date from which you want the report (inclusive). This parameter supports the timestamp in the UTC timezone (YYYY-MM-DDTHH:MM:SSZ) format. The date cannot be earlier than 180 days.
end_date	The date until which you want the report (inclusive). The API default is today. This parameter also supports timestamp in the UTC timezone (YYYY-MM-DDTHH:MM:SSZ) format.
offset	The starting point of the results. The API default is 0.
limit	The number of results to fetch. The API default is 10.
sort_column	The column used to sort the report. The valid choices are `time`, `cpu`, `fs_bytes_read`, `fs_bytes_written`. The API default is, `time` (chronological order).
by_user	Report only those queries which are created by the current user. By default all queries by the current account are reported.
status	It enables you to filter queries based on the following status values: done, error, running, waiting, cancelled, and cancelling.

Response Parameters¶

Parameter	Description
start_date	The starting date of the report. This parameter filters on the `created_at` parameter value.
end_date	The ending date of the report. This parameter filters on the `created_at` parameter value.
sort_column	The sort column used.
queries	Contains an array of query-related parameters and values as provided in the following table.

An array of parameters and values associated with the queries parameter.

id	The query ID of the command.
created_at	The time when the query was created.
submitted_by	The Email address of the user who created the query.
command_type	The type of the command (HiveCommand, PrestoCommand, and so on.)
command_summary	The summary of the command (query/latin_statements/script_location, and so on.)
status	The status of the command (whether it succeeded or failed, and so on.)
label	It is the label of the cluster on which the command is run.
end_time	Denotes the time at which the query execution completed.
cpu	The total cumulative CPU, (in ms), consumed by this command. It signifies the cumulative CPU time spent by all cluster nodes in the cluster that processed the command.
fs_bytes_read	The total bytes read by this command.
fs_bytes_written	The total bytes written by this command.

Examples¶

Goal

To get the default report.

curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN"  -H "Accept: application/json"
"https://gcp.qubole.com/api/v1.2/reports/all_commands"

Goal

To get the report for commands executed only by the current user.

curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://gcp.qubole.com/api/v1.2/reports/all_commands?by_user"

Goal

To get the report for commands executed during a specific time period and sorted by total bytes read.

curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://gcp.qubole.com/api/v1.2/reports/all_commands?start_date=2017-06-10&end_date=2017-07-10&sort_column=fs_bytes_read"

Sample Response

{
"sort_column": "fs_bytes_read",
"start_date": "2017-06-10T00:00:00Z",
"end_date": "2017-07-10T00:00:00Z",
"queries": [
    {
        "id": 79520285,
        "created_at": "2017-06-19T08:56:06Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "select count(*) from default_qubole_memetracker where month=\"2008-08\";",
        "status": "done",
        "end_time": "2017-06-19 09:05:12 +0000",
        "tags": "",
        "fs_bytes_read": 4429520896,
        "fs_bytes_written": 8,
        "cpu": null
    },
    {
        "id": 78826792,
        "created_at": "2017-06-15T07:01:23Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT * FROM `tpcds_orc_1000`.`customer`",
        "status": "done",
        "end_time": "2017-06-15 07:02:53 +0000",
        "tags": "JDBC 1.0.7",
        "fs_bytes_read": 683353856,
        "fs_bytes_written": 1629612160,
        "cpu": 74910
    },
    {
        "id": 78828387,
        "created_at": "2017-06-15T07:11:24Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT COUNT(* ) FROM `tpcds_orc_1000`.`customer`",
        "status": "done",
        "end_time": "2017-06-15 07:12:03 +0000",
        "tags": "JDBC 1.0.7-SNAPSHOT",
        "fs_bytes_read": 683353856,
        "fs_bytes_written": 9,
        "cpu": 25680
    },
    {
        "id": 78830962,
        "created_at": "2017-06-15T07:23:20Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT * FROM `tpcds_orc_1000`.`customer`",
        "status": "done",
        "end_time": "2017-06-15 07:24:44 +0000",
        "tags": "JDBC 1.0.7",
        "fs_bytes_read": 683353856,
        "fs_bytes_written": 1629612160,
        "cpu": 76200
    },
    {
        "id": 78652381,
        "created_at": "2017-06-14T08:22:17Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "set hive.cli.print.header=false;\nset hive.resultset.use.unique.column.names=false; \nSELECT * FROM `table`.`calcs`",
        "status": "done",
        "end_time": "2017-06-14 08:24:14 +0000",
        "tags": "ODBC 1.0.0.1001",
        "fs_bytes_read": 23357610,
        "fs_bytes_written": 7470759,
        "cpu": 4330
    },
    {
        "id": 78825573,
        "created_at": "2017-06-15T06:54:44Z",
        "submitted_by": "[email protected]",
        "command_type": "HiveCommand",
        "command_summary": "SELECT * FROM `table`.`calcs`",
        "status": "done",
        "end_time": "2017-06-15 06:57:54 +0000",
        "tags": "JDBC 1.0.7",
        "fs_bytes_read": 23357610,
        "fs_bytes_written": 7470759,
        "cpu": 4320
    }
    ]
}