Using the CLI
=============
For many, the primary way of interacting with Globus automation services will be
via the CLI.
Configuration
-------------
It is recommended to install ``globus-automate`` command line completion to
support use and discovery of the available commands and options. Currently only
the ``bash``, ``fish``, and ``zsh`` shells are supported.
.. code-block:: BASH
globus-automate --install-completion
.. note::
For users of ``zsh``, make sure the ``zsh`` completion system is correctly
initialized by running ``compinit`` near the end of your ``.zshrc`` file.
General Usage
-------------
Each command support a ``--verbose`` option which can be used to see the
underlying HTTP request information for what each command is doing. This can be
useful for reusing tokens, debugging purposes, or for getting more familiar with
the Globus Automate services' APIs.
Each command also supports a ``--help`` option which provides concise
information on the command and documents its expected inputs.
In almost all cases, the output from each command will be in JSON format. The
CLI will format the output to try to improve readability, however, you may
wish to filter the output using pipes and other command line tools such as
``grep`` or ``jq``.
Many of the commands involve specifying JSON formatted input. Anywhere a command
makes use of JSON, it can be specified in one of two ways:
- Directly on the commmand line
Typically by using single quote characters surrounding the content to protect it
from interpretation by the shell:
.. code-block:: BASH
--option '{"property": "Value"}'
where ``--option`` is the option requiring a JSON formatted parameter.
- In a file
Place the JSON object in a file and refer to the file instead:
.. code-block:: BASH
--option /path/to/your/input_file.json
where ``--option`` is the option requiring a JSON formatted parameter.
Commands which can run or monitor an action or run support a ``--watch``
flag which can be used to stream updates on an activity's execution state.
Many operations require a principal ID to delegate access. This value represents
a single Globus user identity or a Globus Group. The format for a principal is
the user ID or Group ID, prefixed with ``urn:globus:auth:identity:`` if it is a
user, or ``urn:globus:groups:id:`` if it is a Group.
Using the CLI with Actions
--------------------------
In Globus automation services, *actions* represent a single unit of work. Actions
are created by invoking *action providers*. The CLI provides a means of interacting
with action providers to create and manage actions, however in many cases, the
fine grain nature of the actions means they are not often interacted with
directly.
To interact with an action, you must know the URL for the action provider
that created (or will create) it. This URL is specified using the
``--action-url`` option to the ``action`` subcommands.
Wherever an ``--action-url`` option is present, the ``--action-scope`` option
may also be provided. The *scope string* is the Globus Auth scope registered for
interacting with this action provider. This *scope string* is published as part
of the action provider's description (see
:ref:`Introspection`) and the CLI will automatically retrieve
this value when it is not specified by the user. In some cases, even retrieving
the *scope string* may require authentication, and in these cases, introspecting
the action provider is not possible without providing the
``--action-scope`` option.
All of the action providers operated by the Globus team are described in the
`hosted action providers`_ documentation, which includes their URL and
tips on using the CLI for interactions with these action providers directly.
As these action providers are publicly viewable, there is no need to provide
the ``--action-scope`` option when working with them from the CLI -- the CLI
will look up the *scope string* automatically.
As an example, we will work through the operations on the ``Hello World Action
Provider`` at the URL ``https://actions.globus.org/hello_world``.
.. _Introspection:
Introspection
^^^^^^^^^^^^^
The first step to learning more about an action provider is using the
introspect operation to get a description of the action provider:
.. code-block:: BASH
globus-automate action introspect --action-url https://actions.globus.org/hello_world
.. raw:: html
Command Output
.. code-block:: JSON
{
"admin_contact": "support@globus.org",
"administered_by": [],
"api_version": "1.0",
"description": null,
"event_types": null,
"globus_auth_scope": "https://auth.globus.org/scopes/actions.globus.org/hello_world",
"input_schema": {
"additionalProperties": false,
"properties": {
"echo_string": {
"type": "string"
},
"required_dependent_scope": {
"type": "string"
},
"sleep_time": {
"type": "integer"
}
},
"type": "object"
},
"keywords": null,
"log_supported": false,
"maximum_deadline": "P30D",
"runnable_by": [
"all_authenticated_users"
],
"subtitle": "An action responding Hello to an input value",
"synchronous": false,
"title": "Hello World",
"types": [
"ACTION"
],
"visible_to": [
"public"
]
}
.. raw:: html
From this introspection response we can see that the *scope string* for
this action provider is the value of the ``globus_auth_scope`` field,
``https://auth.globus.org/scopes/actions.globus.org/hello_world``. We
can also see that the ``admin_contact`` is Globus.
For information on what this action provider does, it is useful to examine
the ``title``, ``subtitle``, and ``description`` fields. We can also see that
the action provider is ``visible_to`` *public*, meaning that anyone can make
unauthenticated requests to the introspection endpoint. Similarly, it is
``runnable_by`` *all_authenticated_users*, meaning that any user with valid
Globus Auth credentials may use this action provider to create actions.
The most important information for our next step is the ``input_schema`` element
as it provides a description of the input we need to form for running an action
on this action provider. The ``input_schema`` element is in `JSON Schema
`_ format. This schema defines three properties:
``echo_string``, ``sleep_time``, and ``required_dependent_scope``. We will use
this information in the next section on running an action.
Running
^^^^^^^
The first step to prepare for running an action is to create a file containing
the input to the action. We'll call the file ``hello_input.json`` and it
contains the following:
.. code-block:: JSON
{
"echo_string": "Welcome to Globus Automate!",
"sleep_time": 60
}
This input conforms to the ``input_schema`` from the :ref:`Introspection` call,
and specifies that we will have the action echo a message back to us and that it
will "sleep" for 60 seconds until the action is complete. We'll use this sleep
time to demonstrate monitoring the state of an action below.
With our input in place, run the action using the following command:
.. code-block:: BASH
globus-automate action run --action-url https://actions.globus.org/hello_world --body hello_input.json
.. note::
If this is your first time running the ``Hello World Action Provider`` you
will see text and a prompt appear on your terminal window. Follow the
instructions to authenticate to Globus Auth to run this action. This will
only appear on the first time you interact with an action provider.
The resulting output will look like:
.. code-block:: JSON
{
"action_id": "CBOXB3fUdKrO",
"completion_time": null,
"creator_id": "urn:globus:auth:identity:06a24bef-940e-418a-97bc-48229c64cc99",
"details": {
"Hello": "World",
"hello": "Welcome to Globus Automate!"
},
"display_status": "ACTIVE",
"label": null,
"manage_by": [
"urn:globus:auth:identity:6f8c1345-33c6-4235-86c6-90fbadbf4d35",
"urn:globus:auth:identity:06a24bef-940e-418a-97bc-48229c64cc99"
],
"monitor_by": [
"urn:globus:auth:identity:6f8c1345-33c6-4235-86c6-90fbadbf4d35",
"urn:globus:auth:identity:06a24bef-940e-418a-97bc-48229c64cc99"
],
"release_after": null,
"start_time": "2021-04-29 23:21:47.763653+00:00",
"status": "ACTIVE"
}
This output is referred to as an ``Action Status`` document and all output from
working with actions will follow this format.
The ``action_id`` is an identifier associated with this action provider
invocation and is used to track this action's lifecycle.
The ``status`` value of ``ACTIVE`` indicates that the action is in the process
of executing. The possible values for ``status`` are:
- ``ACTIVE``
The action is running and making progress towards completion.
- ``INACTIVE``
The action has not yet completed and it is not making
progress. Commonly, some intervention is necessary to help it continue to
make progress.
- ``SUCCEEDED``
The action is complete and the completion was considered to be normal.
- ``FAILED``
The action has stopped running due to some error condition. It cannot make
progress towards a successful completion.
Each action can be provided a ``label`` to help identity the purpose for which
it was run.
The ``details`` field format is specific to every action provider and is the
output or result of running the action. It will often contain information about
why an action has reached the state it is in.
The ``release_after`` field is an ISO8601 format time duration value that
indicates how long after completion the action provider will retain a record
of the action's execution. Until then, the record will persist and can be looked
up.
``monitor_by`` represents delegated read-only access to the action's execution
state, meaning that principals in an action's ``monitor_by`` field will be able
to retrieve the action's execution state (see :ref:`Retrieving Status`).
Principals may be either a Globus Auth user or a Globus Auth group. The format
for a Globus Auth user is ``urn:globus:auth:identity:`` and for a Globus
Auth group is ``urn:globus:groups:id:``.
``manage_by`` represents delegated write access to the action's execution state,
meaning that principals in an action's ``manage_by`` field will have the ability
to change the alter the state it is in (see :ref:`Canceling and Releasing`).
Principals may be either a Globus Auth user or a Globus Auth group. The format
for a Globus Auth user is ``urn:globus:auth:identity:`` and for a Globus
Auth group is ``urn:globus:groups:id:``.
Since the action has already been run, we cannot change any of these fields. If
we wanted to run another action with updated values for any of the fields, we
would pass those as command line options. For information on how to use the
options, run the command with ``--help``:
.. code-block:: BASH
globus-automate action run --help
.. admonition:: Tip
:class: tip
You can specify each of the ``--monitor-by`` and ``--manage-by`` flags
multiple times to provide multiple principals with read or write access on
the action.
.. _Retrieving Status:
Retrieving Status
^^^^^^^^^^^^^^^^^
Once an action has been run, the user who initiated the action or anyone in
the action's ``monitor_by`` field can monitor or retrieve its status as follows:
.. code-block:: BASH
globus-automate action status --action-url https://actions.globus.org/hello_world
where the ``action_id`` is the value returned from the ``action run`` command
from above. The output will be an Action Status document. When the action is
completed, the ``completion_time`` field will be present indicating when the
action reached its final state. You can continue requesting the action's status
as long as the action exists on the action provider.
In out example, we asked the action to "sleep" for 60 seconds. Therefore, the
action will remain in an ``ACTIVE`` state until 60 seconds have passed, at which
point the status should be ``SUCCEEDED``.
.. _Canceling and Releasing:
Canceling and Releasing
^^^^^^^^^^^^^^^^^^^^^^^
An action which is running, but which is no longer needed, may be canceled (or
released) by the user who initiated the action execution or anyone in the
action's ``manage_by`` field using a command of the form:
.. code-block:: BASH
globus-automate action cancel --action-url https://actions.globus.org/hello_world
The cancel operation is considered to be an advisory request from the user.
actions may not be cancelled immediately, or they may not be canceled at all. A
request to cancel an action which has reached a final state of either
``SUCCEEDED`` or ``FAILED`` will result in an error return.
To remove an action's state from the action provider, the user who initiated
the action execution or anyone in the action's ``manage_by`` field can use the
release subcommand:
.. code-block:: BASH
globus-automate action release --action-url https://actions.globus.org/hello_world
Release may only be performed on actions which have reached a final state. If
the action is in either the ``ACTIVE`` or ``INACTIVE`` state, the release will
fail.
Once released, the action state is forever removed from the action provider
and all attempts to access it will fail. Action providers use the
``maximum_deadline`` field to advertise how long they will keep a record of an
action after it reaches a completed state. The time at which this will happen is
equal to the ``completion_time`` plus the ``release_after`` values in the Action
Status document.
Using the CLI with Flows
------------------------
As described in the `Flows overview`_, a *flow* combines actions and
other operations into a more complex operation. When a flow is invoked, it
creates a *run* and the run's interface is very much like an action's; it
has ``run``, ``status``, ``cancel`` and ``release`` operations defined. Because
of this similarity, we sometimes refer to runs as actions in the
documentation, CLI and SDK.
The CLI contains commands for creating, defining, and managing flow definitions
and commands for running, monitoring, and managing flow runs (also known as
*actions*).
.. note::
This section does not provide details on writing flows. That is covered
in greater detail in the `Authoring Flows`_ documentation.
Finding and Displaying Flows
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When a flow is deployed to Automate, the creator can specify which identities
the flow should be visible to and which identities the flow should be runnable
by. As the names suggest, users in a flow's ``visible_to`` field will be able to
query the service to view a flow's definition and metadata. Users in a flow's
``runnable_by`` field will be able to run an instance of the flow.
The following command will list the flows you have created:
.. code-block:: BASH
globus-automate flow list
To view flows which are visible or runnable by you as well, run the following
command:
.. code-block:: BASH
globus-automate flow list --role created_by --role visible_to --role runnable_by
This outputs a list of flows, where the description of each flow carries the
same fields as the output from ``globus-automate action introspect`` described
above. This emphasizes again the similarity between flows and actions. The
``title`` and ``description`` fields may be helpful in determining what a flow
does and what its purpose is. Like actions, the ``input_schema`` may define what
is required of the input when running the flow. However, not all flows are
required to define an ``input_schema`` as a convenience to flow authors who may
not be familiar with creating JSON Schema specifications. Importantly, each
entry in the list of flows will also contain a value for ``id`` which we refer
to as the "flow id" and denote as ``flow_id`` below. This value will be used for
further interacting with a particular flow.
To display information about a single flow you may use:
.. code-block:: BASH
globus-automate flow display
Or, to visualize the flow:
.. code-block:: BASH
globus-automate flow display --format image
When focusing on one flow, it is also useful to notice the field ``definition``.
This is the actual encoding of the flow as it was created and deployed by the
flow's author. Looking at this value may give further information about how the
flow works. This can be useful both to determine if a flow performs the function
you desire, but also as a method to see how other flows have been defined if you
are interested in creating new flows.
Executing and Monitoring Flows
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Execution and monitoring of flows follows the same pattern as actions: the
run/status/cancel/release pattern is the same.
When initiating a flow run, you can delegate access to the flow instance to
other Globus Auth identities. By providing the ``monitor-by`` option, you can
delegate read-only access to other users or groups, allowing them to retrieve
it execution state. By providing the ``manage-by`` option, you delegate write
access to other users or groups, allowing them to alter its execution state. In
the example below, we show how to run an instance of a flow and delegate monitor
access to a Globus Group:
.. code-block:: BASH
globus-automate flow run --flow-input input.json \
--monitor-by urn:globus:groups:id:00000000-0000-0000-0000-000000000000
.. note::
If no ``manage_by`` or ``monitor_by`` values are specified, only the
identity instantiating the flow run is allowed to monitor or manage a flow's
running state.
This acts like ``globus-automate action run`` with the flow id rather than the
``action_url`` specifying the "name" of the action to be run. The output, like
for actions, will be an action status document including an ``action_id`` which
is used in the following commands:
.. code-block:: BASH
globus-automate flow action-status --flow-id
.. code-block:: BASH
globus-automate flow action-cancel --flow-id
.. code-block:: BASH
globus-automate flow action-release --flow-id
For each of these, the ``details`` provides information about the most recent,
potentially final, state executed by the flow. However, as the flow may execute
many states, it is useful to be able to see what states have been executed and
what their input and output have been. This can be seen via the "log" of the
flow execution as follows:
.. code-block:: BASH
globus-automate flow action-log --flow-id
The log may have a large number of entries. You can request more entries be
returned using the option ``-limit N`` where ``N`` is the number of log entries
to return. The default value is 10.
Creating and managing Flows
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Many users will only ever use flows created by others, so they may not
necessarily need to understand how to create flows including the commands
listed in this section. For those that have created a flow, the first step is
to deploy a flow as follows:
.. code-block:: BASH
globus-automate flow deploy --title \
--definition --input-schema \
--visible-to \
--runnable-by \
--administered-by
When deployed this way, only the identity that deployed the flow will be able to
view the flow and only they will be able to run an instance of the flow. When
deploying, it's possible to specify who should be able to see and run the flow.
Using the ``visible_to`` flag, you can indicate which Globus identities can view
the deployed flow, or set it to ``public``, which creates a flow viewable by
anyone. Using the ``runnable_by`` flag, you can indicate which Globus identities
can run an instance of the deployed flow, or set a value of
``all_authenticated_users`` which allows any authenticated user to run an
instance of the flow.
Below, we demonstrate how to deploy a flow that is ``visible_to`` a single
Globus group and ``runnable_by`` any authenticated user:
.. code-block:: BASH
globus-automate flow deploy --title \
--definition \
--input-schema \
--visible-to urn:globus:groups:id:00000000-0000-0000-0000-000000000000 \
--runnable-by all_authenticated_users
Once deployed, the output will be the flow description as displayed by the
``flow display`` command above. These command line options provide the values
for the similarly named fields in the flow description. Of these, only ``title``
and ``definition`` are required. To aid users in using your flow, we highly
recommend the use of ``input-schema`` as it provides them both a form of
documentation and assurance at run-time that the input they provide is correct
for executing the flow. By providing a value or values to ``administered-by``
you grant rights to others for updating or eventually removing the flow you have
deployed. Commands for updating and removing flows are as follows.
.. code-block:: BASH
globus-automate flow update --title \
--definition --input-schema \
--visible-to \
--runnable-by \
--administered-by \
This will update any of the fields or description of the flow, including the
flow definition itself. Note the ``flow_id`` field is present at the end of the
command line.
Deleting a flow is done via:
.. code-block:: BASH
globus-automate flow delete
Care should be taken when issuing this command. There is no further prompting to
ensure the flow should really be deleted. After deletion, no record of the flow
definition or its execution history (i.e. the ``flow action-*`` commands) is
maintained.
The bulk of the effort in creating flows is in authoring their definition which
is covered in the `Authoring Flows`_ documentation.
.. _Flows overview: https://docs.globus.org/api/flows/overview/#flows
.. _Authoring Flows: https://docs.globus.org/api/flows/authoring-flows/
.. _hosted action providers: https://docs.globus.org/api/flows/hosted-action-providers/