kfp CLI

All commands

kfp CLI
- kfp
  - component
    - build
  - diagnose-me
  - dsl
    - dsl-compile
  - experiment
    - archive
    - create
    - delete
    - get
    - list
    - unarchive
  - pipeline
    - create
    - create-version
    - delete
    - delete-version
    - get
    - get-version
    - list
    - list-versions
  - recurring-run
    - create
    - delete
    - disable
    - enable
    - get
    - list
  - run
    - archive
    - create
    - delete
    - get
    - list
    - unarchive

kfp

Kubeflow Pipelines CLI.

The KFP CLI is currently in Alpha feature stage (https://github.com/kubeflow/pipelines/blob/07328e5094ac2981d3059314cc848fbb71437a76/docs/release/feature-stages.md#alpha).

kfp [OPTIONS] COMMAND [ARGS]...

Options

--show-completion <show_completion>

Options: bash | zsh | fish

--install-completion <install_completion>

Options: bash | zsh | fish

--endpoint <endpoint>: Host name to use to talk to Kubeflow Pipelines. If not set, the in-cluster service DNS name will be used, which only works if the current environment is a pod in the same cluster (such as a Jupyter instance spawned by Kubeflow’s JupyterHub). (More information on connecting.)

--iap-client-id <iap_client_id>: Client ID used by Identity-Aware Proxy.

-n, --namespace <namespace>

Kubernetes namespace to use. Used for multi-user deployments. For single-user deployments, this should be left as None.

Default: kubeflow

--other-client-id <other_client_id>: Client ID used to obtain the auth codes and refresh tokens (reference).

--other-client-secret <other_client_secret>: Client secret used to obtain the auth codes and refresh tokens.

--output <output>

The formatting style for command output.

Default: table
Options: table | json

--version: Show the version and exit.

component

Builds shareable, containerized components.

kfp component [OPTIONS] COMMAND [ARGS]...

build

Builds containers for KFP v2 Python-based components.

kfp component build [OPTIONS] COMPONENTS_DIRECTORY

Options

--component-filepattern <component_filepattern>: Filepattern to use when searching for KFP components. The default searches all Python files in the specified directory.

--engine <engine>: Engine to use to build the component’s container.

--kfp-package-path <kfp_package_path>: A pip-installable path to the KFP package.

--overwrite-dockerfile: Set this to true to always generate a Dockerfile as part of the build process

--push-image, --no-push-image: Push the built image to its remote repository.

Arguments

COMPONENTS_DIRECTORY: Required argument

diagnose-me

Runs KFP environment diagnostic.

kfp diagnose-me [OPTIONS]

Options

-j, --json: Output in Json format, human readable format is set by default.

-p, --project-id <project_id>: Target project id. It will use environment default if not specified.

-n, --namespace <namespace>: Namespace to use for Kubernetes cluster.all-namespaces is used if not specified.

dsl

Command group for compiling DSL to IR.

kfp dsl [OPTIONS] COMMAND [ARGS]...

dsl-compile

Compiles a pipeline written in a .py file.

kfp dsl dsl-compile [OPTIONS]

Options

--py <py>: Required Local absolute path to a py file.

--output <output>: Required Path to write the compiled result.

--function <function_name>: The name of the pipeline or component to compile if there are multiple.

--pipeline-parameters <pipeline_parameters>: The pipeline parameters in JSON dict format.

--disable-type-check: Whether to disable type checking.

experiment

Manage experiment resources.

kfp experiment [OPTIONS] COMMAND [ARGS]...

archive

Archive an experiment.

kfp experiment archive [OPTIONS]

Options

--experiment-id <experiment_id>: ID of the experiment. Either –experiment-id or –experiment-name is required.

--experiment-name <experiment_name>: Name of the experiment. Either –experiment-id or –experiment-name is required.

create

Create an experiment.

kfp experiment create [OPTIONS] NAME

Options

-d, --description <description>: Description of the experiment.

Arguments

NAME: Required argument

delete

Delete an experiment.

kfp experiment delete [OPTIONS] EXPERIMENT_ID

Arguments

EXPERIMENT_ID: Required argument

get

Get information about an experiment.

kfp experiment get [OPTIONS] EXPERIMENT_ID

Arguments

EXPERIMENT_ID: Required argument

list

List experiments.

kfp experiment list [OPTIONS]

Options

--page-token <page_token>: Page token for obtaining page from paginated response.

-m, --max-size <max_size>: Size of the page.

--sort-by <sort_by>: Sort string of format '[field_name]', '[field_name] desc'. For example, 'name desc'.

--filter <filter>: A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto message). For a list of all filter operations 'op', see here. Example:

unarchive

Unarchive an experiment.

kfp experiment unarchive [OPTIONS]

Options

--experiment-id <experiment_id>: ID of the experiment. Either –experiment-id or –experiment-name is required.

--experiment-name <experiment_name>: Name of the experiment. Either –experiment-id or –experiment-name is required.

pipeline

Manage pipeline resources.

kfp pipeline [OPTIONS] COMMAND [ARGS]...

create

Upload a pipeline.

kfp pipeline create [OPTIONS] PACKAGE_FILE

Options

-p, --pipeline-name <pipeline_name>: Name of the pipeline to be shown in the UI.

-d, --description <description>: Description of the pipeline to be shown in the UI.

Arguments

PACKAGE_FILE: Required argument

create-version

Upload a version of a pipeline.

kfp pipeline create-version [OPTIONS] PACKAGE_FILE

Options

-v, --pipeline-version <pipeline_version>: Required Name of the pipeline version to be shown in the UI.

-p, --pipeline-id <pipeline_id>: ID of the pipeline. Either –pipeline-id or –pipeline-name is required.

-n, --pipeline-name <pipeline_name>: Name of the pipeline. Either –pipeline-id or –pipeline-name is required.

-d, --description <description>: Description of the pipeline version to show in the UI.

Arguments

PACKAGE_FILE: Required argument

delete

Delete a pipeline.

kfp pipeline delete [OPTIONS] PIPELINE_ID

Arguments

PIPELINE_ID: Required argument

delete-version

Delete a version of a pipeline.

kfp pipeline delete-version [OPTIONS] VERSION_ID

Arguments

VERSION_ID: Required argument

get

Get information about a pipeline.

kfp pipeline get [OPTIONS] PIPELINE_ID

Arguments

PIPELINE_ID: Required argument

get-version

Get information about a version of a pipeline.

kfp pipeline get-version [OPTIONS] VERSION_ID

Arguments

VERSION_ID: Required argument

list

List pipelines.

kfp pipeline list [OPTIONS]

Options

--page-token <page_token>: Page token for obtaining page from paginated response.

-m, --max-size <max_size>: Size of the page.

--sort-by <sort_by>: Sort string of format '[field_name]', '[field_name] desc'. For example, 'name desc'.

--filter <filter>: A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto message). For a list of all filter operations 'op', see here. Example:

list-versions

List versions of a pipeline.

kfp pipeline list-versions [OPTIONS] PIPELINE_ID

Options

--page-token <page_token>: Page token for obtaining page from paginated response.

-m, --max-size <max_size>: Size of the page.

--sort-by <sort_by>: Sort string of format '[field_name]', '[field_name] desc'. For example, 'name desc'.

--filter <filter>: A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto message). For a list of all filter operations 'op', see here. Example:

Arguments

PIPELINE_ID: Required argument

recurring-run

Manage recurring run resources.

kfp recurring-run [OPTIONS] COMMAND [ARGS]...

create

Create a recurring run.

kfp recurring-run create [OPTIONS] [ARGS]...

Options

--catchup, --no-catchup: Whether the recurring run should catch up if behind schedule. For example, if the recurring run is paused for a while and re-enabled afterwards. If no_catchup=False, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval if more than one interval is ready to be scheduled. Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate backfill.

--cron-expression <cron_expression>: Cron expression representing a set of times, using 6 space-separated fields (e.g., '0 0 9 ? * 2-6'). See cron format.

--description <description>: Description of the job.

--enable-caching, --disable-caching: Whether or not to enable caching for the run. If not set, defaults to the compile time settings, which is True for all tasks by default, while users may specify different caching options for individual tasks. If set, the setting applies to all tasks in the pipeline (overrides the compile time settings).

--enabled, --disabled: Whether to enable or disable the recurring run.

--end-time <end_time>: RFC3339 time string of the time when to end the job.

--experiment-id <experiment_id>: ID of the experiment. Either –experiment-id or –experiment-name is required.

--experiment-name <experiment_name>: The name of the experiment to create the recurring run under. Either –experiment-id or –experiment-name is required.

--job-name <job_name>: Name of the job.

--interval-second <interval_second>: Integer indicating the seconds between two recurring runs in for a periodic schedule.

--max-concurrency <max_concurrency>: Integer indicating how many jobs can be run in parallel.

--pipeline-id <pipeline_id>: ID of a pipeline.

--pipeline-package-path <pipeline_package_path>: Local path of the pipeline package (the filename should end with one of the following .tar.gz, .tgz, .zip, .json).

--start-time <start_time>: RFC3339 time string of the time when to start the job.

--version-id <version_id>: The id of a pipeline version.

Arguments

ARGS: Optional argument(s)

delete

Delete a recurring run.

kfp recurring-run delete [OPTIONS] JOB_ID

Arguments

JOB_ID: Required argument

disable

Disable a recurring run.

kfp recurring-run disable [OPTIONS] JOB_ID

Arguments

JOB_ID: Required argument

enable

Enable a recurring run.

kfp recurring-run enable [OPTIONS] JOB_ID

Arguments

JOB_ID: Required argument

get

Get information about a recurring run.

kfp recurring-run get [OPTIONS] JOB_ID

Arguments

JOB_ID: Required argument

list

List recurring runs.

kfp recurring-run list [OPTIONS]

Options

-e, --experiment-id <experiment_id>: Experiment ID to filter upon.

--page-token <page_token>: Page token for obtaining page from paginated response.

-m, --max-size <max_size>: Size of the page.

--sort-by <sort_by>: Sort string of format '[field_name]', '[field_name] desc'. For example, 'name desc'.

--filter <filter>: A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto message). For a list of all filter operations 'op', see here. Example:

run

Manage run resources.

kfp run [OPTIONS] COMMAND [ARGS]...

archive

Archive a pipeline run.

kfp run archive [OPTIONS] RUN_ID

Arguments

RUN_ID: Required argument

create

Submit a pipeline run.

kfp run create [OPTIONS] [ARGS]...

Options

-e, --experiment-name <experiment_name>: Required Experiment name of the run.

-r, --run-name <run_name>: Name of the job.

-f, --package-file <package_file>: Local path of the pipeline package (the filename should end with one of the following .tar.gz, .tgz, .zip, .json).

-p, --pipeline-id <pipeline_id>: ID of the pipeline.

-n, --pipeline-name <pipeline_name>: Name of the pipeline template.

-w, --watch: Watch the run status until it finishes.

-v, --version <version>: ID of the pipeline version to run. If both pipeline_id and version_id are specified, version_id will take precendence. If only pipeline_id is specified, the default version of this pipeline is used to create the run.

-t, --timeout <timeout>: Wait for a run to complete until timeout in seconds.

Arguments

ARGS: Optional argument(s)

delete

Delete a pipeline run.

kfp run delete [OPTIONS] RUN_ID

Arguments

RUN_ID: Required argument

get

Get information about a pipeline run.

kfp run get [OPTIONS] RUN_ID

Options

-w, --watch: Watch the run status until it finishes.

-d, --detail: Get detailed information of the run in json format.

Arguments

RUN_ID: Required argument

list

List pipeline runs.

kfp run list [OPTIONS]

Options

-e, --experiment-id <experiment_id>: Experiment ID to filter upon

--page-token <page_token>: Page token for obtaining page from paginated response.

-m, --max-size <max_size>: Size of the page.

--sort-by <sort_by>: Sort string of format '[field_name]', '[field_name] desc'. For example, 'name desc'.

--filter <filter>: A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto message). For a list of all filter operations 'op', see here. Example:

unarchive

Unarchive a pipeline run.

kfp run unarchive [OPTIONS] RUN_ID

Arguments

RUN_ID: Required argument