Using the CLI#

For many, the primary way of interacting with Globus automation services will be via the CLI.

Configuration#

It is recommended to install globus-automate command line completion to support use and discovery of the available commands and options. Currently only the bash, fish, and zsh shells are supported.

globus-automate --install-completion

Note

For users of zsh, make sure the zsh completion system is correctly initialized by running compinit near the end of your .zshrc file.

General Usage#

Each command support a --verbose option which can be used to see the underlying HTTP request information for what each command is doing. This can be useful for reusing tokens, debugging purposes, or for getting more familiar with the Globus Automate services’ APIs.

Each command also supports a --help option which provides concise information on the command and documents its expected inputs.

In almost all cases, the output from each command will be in JSON format. The CLI will format the output to try to improve readability, however, you may wish to filter the output using pipes and other command line tools such as grep or jq.

Many of the commands involve specifying JSON formatted input. Anywhere a command makes use of JSON, it can be specified in one of two ways:

  • Directly on the commmand line

    Typically by using single quote characters surrounding the content to protect it from interpretation by the shell:

    --option '{"property": "Value"}'
    

    where --option is the option requiring a JSON formatted parameter.

  • In a file

    Place the JSON object in a file and refer to the file instead:

    --option /path/to/your/input_file.json
    

    where --option is the option requiring a JSON formatted parameter.

Commands which can run or monitor an action or run support a --watch flag which can be used to stream updates on an activity’s execution state.

Many operations require a principal ID to delegate access. This value represents a single Globus user identity or a Globus Group. The format for a principal is the user ID or Group ID, prefixed with urn:globus:auth:identity: if it is a user, or urn:globus:groups:id: if it is a Group.

Using the CLI with Actions#

In Globus automation services, actions represent a single unit of work. Actions are created by invoking action providers. The CLI provides a means of interacting with action providers to create and manage actions, however in many cases, the fine grain nature of the actions means they are not often interacted with directly.

To interact with an action, you must know the URL for the action provider that created (or will create) it. This URL is specified using the --action-url option to the action subcommands.

Wherever an --action-url option is present, the --action-scope option may also be provided. The scope string is the Globus Auth scope registered for interacting with this action provider. This scope string is published as part of the action provider’s description (see Introspection) and the CLI will automatically retrieve this value when it is not specified by the user. In some cases, even retrieving the scope string may require authentication, and in these cases, introspecting the action provider is not possible without providing the --action-scope option.

All of the action providers operated by the Globus team are described in the hosted action providers documentation, which includes their URL and tips on using the CLI for interactions with these action providers directly. As these action providers are publicly viewable, there is no need to provide the --action-scope option when working with them from the CLI – the CLI will look up the scope string automatically.

As an example, we will work through the operations on the Hello World Action Provider at the URL https://actions.globus.org/hello_world.

Introspection#

The first step to learning more about an action provider is using the introspect operation to get a description of the action provider:

globus-automate action introspect --action-url https://actions.globus.org/hello_world
Command Output
{
    "admin_contact": "support@globus.org",
    "administered_by": [],
    "api_version": "1.0",
    "description": null,
    "event_types": null,
    "globus_auth_scope": "https://auth.globus.org/scopes/actions.globus.org/hello_world",
    "input_schema": {
        "additionalProperties": false,
        "properties": {
        "echo_string": {
            "type": "string"
        },
        "required_dependent_scope": {
            "type": "string"
        },
        "sleep_time": {
            "type": "integer"
        }
        },
        "type": "object"
    },
    "keywords": null,
    "log_supported": false,
    "maximum_deadline": "P30D",
    "runnable_by": [
        "all_authenticated_users"
    ],
    "subtitle": "An action responding Hello to an input value",
    "synchronous": false,
    "title": "Hello World",
    "types": [
        "ACTION"
    ],
    "visible_to": [
        "public"
    ]
}

From this introspection response we can see that the scope string for this action provider is the value of the globus_auth_scope field, https://auth.globus.org/scopes/actions.globus.org/hello_world. We can also see that the admin_contact is Globus.

For information on what this action provider does, it is useful to examine the title, subtitle, and description fields. We can also see that the action provider is visible_to public, meaning that anyone can make unauthenticated requests to the introspection endpoint. Similarly, it is runnable_by all_authenticated_users, meaning that any user with valid Globus Auth credentials may use this action provider to create actions.

The most important information for our next step is the input_schema element as it provides a description of the input we need to form for running an action on this action provider. The input_schema element is in JSON Schema format. This schema defines three properties: echo_string, sleep_time, and required_dependent_scope. We will use this information in the next section on running an action.

Running#

The first step to prepare for running an action is to create a file containing the input to the action. We’ll call the file hello_input.json and it contains the following:

{
  "echo_string": "Welcome to Globus Automate!",
  "sleep_time": 60
}

This input conforms to the input_schema from the Introspection call, and specifies that we will have the action echo a message back to us and that it will “sleep” for 60 seconds until the action is complete. We’ll use this sleep time to demonstrate monitoring the state of an action below.

With our input in place, run the action using the following command:

globus-automate action run --action-url https://actions.globus.org/hello_world --body hello_input.json

Note

If this is your first time running the Hello World Action Provider you will see text and a prompt appear on your terminal window. Follow the instructions to authenticate to Globus Auth to run this action. This will only appear on the first time you interact with an action provider.

The resulting output will look like:

{
    "action_id": "CBOXB3fUdKrO",
    "completion_time": null,
    "creator_id": "urn:globus:auth:identity:06a24bef-940e-418a-97bc-48229c64cc99",
    "details": {
        "Hello": "World",
        "hello": "Welcome to Globus Automate!"
    },
    "display_status": "ACTIVE",
    "label": null,
    "manage_by": [
        "urn:globus:auth:identity:6f8c1345-33c6-4235-86c6-90fbadbf4d35",
        "urn:globus:auth:identity:06a24bef-940e-418a-97bc-48229c64cc99"
    ],
    "monitor_by": [
        "urn:globus:auth:identity:6f8c1345-33c6-4235-86c6-90fbadbf4d35",
        "urn:globus:auth:identity:06a24bef-940e-418a-97bc-48229c64cc99"
    ],
    "release_after": null,
    "start_time": "2021-04-29 23:21:47.763653+00:00",
    "status": "ACTIVE"
}

This output is referred to as an Action Status document and all output from working with actions will follow this format.

The action_id is an identifier associated with this action provider invocation and is used to track this action’s lifecycle.

The status value of ACTIVE indicates that the action is in the process of executing. The possible values for status are:

  • ACTIVE

    The action is running and making progress towards completion.

  • INACTIVE

    The action has not yet completed and it is not making progress. Commonly, some intervention is necessary to help it continue to make progress.

  • SUCCEEDED

    The action is complete and the completion was considered to be normal.

  • FAILED

    The action has stopped running due to some error condition. It cannot make progress towards a successful completion.

Each action can be provided a label to help identity the purpose for which it was run.

The details field format is specific to every action provider and is the output or result of running the action. It will often contain information about why an action has reached the state it is in.

The release_after field is an ISO8601 format time duration value that indicates how long after completion the action provider will retain a record of the action’s execution. Until then, the record will persist and can be looked up.

monitor_by represents delegated read-only access to the action’s execution state, meaning that principals in an action’s monitor_by field will be able to retrieve the action’s execution state (see Retrieving Status). Principals may be either a Globus Auth user or a Globus Auth group. The format for a Globus Auth user is urn:globus:auth:identity:<UUID> and for a Globus Auth group is urn:globus:groups:id:<UUID>.

manage_by represents delegated write access to the action’s execution state, meaning that principals in an action’s manage_by field will have the ability to change the alter the state it is in (see Canceling and Releasing). Principals may be either a Globus Auth user or a Globus Auth group. The format for a Globus Auth user is urn:globus:auth:identity:<UUID> and for a Globus Auth group is urn:globus:groups:id:<UUID>.

Since the action has already been run, we cannot change any of these fields. If we wanted to run another action with updated values for any of the fields, we would pass those as command line options. For information on how to use the options, run the command with --help:

globus-automate action run --help

Tip

You can specify each of the --monitor-by and --manage-by flags multiple times to provide multiple principals with read or write access on the action.

Retrieving Status#

Once an action has been run, the user who initiated the action or anyone in the action’s monitor_by field can monitor or retrieve its status as follows:

globus-automate action status --action-url https://actions.globus.org/hello_world <action_id>

where the action_id is the value returned from the action run command from above. The output will be an Action Status document. When the action is completed, the completion_time field will be present indicating when the action reached its final state. You can continue requesting the action’s status as long as the action exists on the action provider.

In out example, we asked the action to “sleep” for 60 seconds. Therefore, the action will remain in an ACTIVE state until 60 seconds have passed, at which point the status should be SUCCEEDED.

Canceling and Releasing#

An action which is running, but which is no longer needed, may be canceled (or released) by the user who initiated the action execution or anyone in the action’s manage_by field using a command of the form:

globus-automate action cancel --action-url https://actions.globus.org/hello_world <action_id>

The cancel operation is considered to be an advisory request from the user. actions may not be cancelled immediately, or they may not be canceled at all. A request to cancel an action which has reached a final state of either SUCCEEDED or FAILED will result in an error return.

To remove an action’s state from the action provider, the user who initiated the action execution or anyone in the action’s manage_by field can use the release subcommand:

globus-automate action release --action-url https://actions.globus.org/hello_world <action_id>

Release may only be performed on actions which have reached a final state. If the action is in either the ACTIVE or INACTIVE state, the release will fail.

Once released, the action state is forever removed from the action provider and all attempts to access it will fail. Action providers use the maximum_deadline field to advertise how long they will keep a record of an action after it reaches a completed state. The time at which this will happen is equal to the completion_time plus the release_after values in the Action Status document.

Using the CLI with Flows#

As described in the Flows overview, a flow combines actions and other operations into a more complex operation. When a flow is invoked, it creates a run and the run’s interface is very much like an action’s; it has run, status, cancel and release operations defined. Because of this similarity, we sometimes refer to runs as actions in the documentation, CLI and SDK.

The CLI contains commands for creating, defining, and managing flow definitions and commands for running, monitoring, and managing flow runs (also known as actions).

Note

This section does not provide details on writing flows. That is covered in greater detail in the Authoring Flows documentation.

Finding and Displaying Flows#

When a flow is deployed to Automate, the creator can specify which identities the flow should be visible to and which identities the flow should be runnable by. As the names suggest, users in a flow’s visible_to field will be able to query the service to view a flow’s definition and metadata. Users in a flow’s runnable_by field will be able to run an instance of the flow.

The following command will list the flows you have created:

globus-automate flow list

To view flows which are visible or runnable by you as well, run the following command:

globus-automate flow list --role created_by --role visible_to --role runnable_by

This outputs a list of flows, where the description of each flow carries the same fields as the output from globus-automate action introspect described above. This emphasizes again the similarity between flows and actions. The title and description fields may be helpful in determining what a flow does and what its purpose is. Like actions, the input_schema may define what is required of the input when running the flow. However, not all flows are required to define an input_schema as a convenience to flow authors who may not be familiar with creating JSON Schema specifications. Importantly, each entry in the list of flows will also contain a value for id which we refer to as the “flow id” and denote as flow_id below. This value will be used for further interacting with a particular flow.

To display information about a single flow you may use:

globus-automate flow display <flow_id>

Or, to visualize the flow:

globus-automate flow display <flow_id> --format image

When focusing on one flow, it is also useful to notice the field definition. This is the actual encoding of the flow as it was created and deployed by the flow’s author. Looking at this value may give further information about how the flow works. This can be useful both to determine if a flow performs the function you desire, but also as a method to see how other flows have been defined if you are interested in creating new flows.

Executing and Monitoring Flows#

Execution and monitoring of flows follows the same pattern as actions: the run/status/cancel/release pattern is the same.

When initiating a flow run, you can delegate access to the flow instance to other Globus Auth identities. By providing the monitor-by option, you can delegate read-only access to other users or groups, allowing them to retrieve it execution state. By providing the manage-by option, you delegate write access to other users or groups, allowing them to alter its execution state. In the example below, we show how to run an instance of a flow and delegate monitor access to a Globus Group:

globus-automate flow run <flow_id> --flow-input input.json \
    --monitor-by urn:globus:groups:id:00000000-0000-0000-0000-000000000000

Note

If no manage_by or monitor_by values are specified, only the identity instantiating the flow run is allowed to monitor or manage a flow’s running state.

This acts like globus-automate action run with the flow id rather than the action_url specifying the “name” of the action to be run. The output, like for actions, will be an action status document including an action_id which is used in the following commands:

globus-automate flow action-status --flow-id <flow_id> <action_id>
globus-automate flow action-cancel --flow-id <flow_id> <action_id>
globus-automate flow action-release --flow-id <flow_id> <action_id>

For each of these, the details provides information about the most recent, potentially final, state executed by the flow. However, as the flow may execute many states, it is useful to be able to see what states have been executed and what their input and output have been. This can be seen via the “log” of the flow execution as follows:

globus-automate flow action-log --flow-id <flow_id> <action_id>

The log may have a large number of entries. You can request more entries be returned using the option -limit N where N is the number of log entries to return. The default value is 10.

Creating and managing Flows#

Many users will only ever use flows created by others, so they may not necessarily need to understand how to create flows including the commands listed in this section. For those that have created a flow, the first step is to deploy a flow as follows:

globus-automate flow deploy --title <title> \
    --definition <flow definition JSON> --input-schema <Input schema JSON> \
    --visible-to <urn of user or group which can see this flow> \
    --runnable-by <urn of user or group which can run this flow> \
    --administered-by <urn of user or group who can maintain this flow>

When deployed this way, only the identity that deployed the flow will be able to view the flow and only they will be able to run an instance of the flow. When deploying, it’s possible to specify who should be able to see and run the flow. Using the visible_to flag, you can indicate which Globus identities can view the deployed flow, or set it to public, which creates a flow viewable by anyone. Using the runnable_by flag, you can indicate which Globus identities can run an instance of the deployed flow, or set a value of all_authenticated_users which allows any authenticated user to run an instance of the flow.

Below, we demonstrate how to deploy a flow that is visible_to a single Globus group and runnable_by any authenticated user:

globus-automate flow deploy --title <title> \
    --definition <flow definition JSON> \
    --input-schema <Input schema JSON> \
    --visible-to urn:globus:groups:id:00000000-0000-0000-0000-000000000000 \
    --runnable-by all_authenticated_users

Once deployed, the output will be the flow description as displayed by the flow display command above. These command line options provide the values for the similarly named fields in the flow description. Of these, only title and definition are required. To aid users in using your flow, we highly recommend the use of input-schema as it provides them both a form of documentation and assurance at run-time that the input they provide is correct for executing the flow. By providing a value or values to administered-by you grant rights to others for updating or eventually removing the flow you have deployed. Commands for updating and removing flows are as follows.

globus-automate flow update --title <title> \
    --definition <flow definition JSON>  --input-schema <Input schema JSON> \
    --visible-to <urn of user or group which can see this flow> \
    --runnable-by <urn of user or group which can run this flow> \
    --administered-by <urn of user or group who can maintain this flow> \
    <flow_id>

This will update any of the fields or description of the flow, including the flow definition itself. Note the flow_id field is present at the end of the command line.

Deleting a flow is done via:

globus-automate flow delete <flow_id>

Care should be taken when issuing this command. There is no further prompting to ensure the flow should really be deleted. After deletion, no record of the flow definition or its execution history (i.e. the flow action-* commands) is maintained.

The bulk of the effort in creating flows is in authoring their definition which is covered in the Authoring Flows documentation.