KFP extension modules

kfp.onprem module

kfp.onprem.mount_pvc(pvc_name='pipeline-claim', volume_name='pipeline', volume_mount_path='/mnt/pipeline')[source]

Modifier function to apply to a Container Op to simplify volume, volume mount addition and enable better reuse of volumes, volume claims across container ops. Usage:

train = train_op(…) train.apply(mount_pvc(‘claim-name’, ‘pipeline’, ‘/mnt/pipeline’))

kfp.gcp module

kfp.gcp.add_gpu_toleration(toleration: kubernetes.client.models.v1_toleration.V1Toleration = {'effect': 'NoSchedule', 'key': 'nvidia.com/gpu', 'operator': 'Equal', 'toleration_seconds': None, 'value': 'true'})[source]

An operator that configures the GKE GPU nodes in a container op.

Parameters:{V1Toleration} -- toleration to pods, default is the nvidia.com/gpu label. (toleration) –
kfp.gcp.use_gcp_secret(secret_name='user-gcp-sa', secret_file_path_in_volume=None, volume_name=None, secret_volume_mount_path='/secret/gcp-credentials')[source]

An operator that configures the container to use GCP service account by service account key stored in a Kubernetes secret.

For cluster setup and alternatives to using service account key, check https://www.kubeflow.org/docs/gke/authentication-pipelines/.

kfp.gcp.use_preemptible_nodepool(toleration: kubernetes.client.models.v1_toleration.V1Toleration = {'effect': 'NoSchedule', 'key': 'preemptible', 'operator': 'Equal', 'toleration_seconds': None, 'value': 'true'}, hard_constraint: bool = False)[source]

An operator that configures the GKE preemptible in a container op. :param toleration: toleration to pods, default is the preemptible label. :type toleration: V1Toleration :param hard_constraint: the constraint of scheduling the pods on preemptible

nodepools is hard. (Default: False)
kfp.gcp.use_tpu(tpu_cores: int, tpu_resource: str, tf_version: str)[source]

An operator that configures GCP TPU spec in a container op.


kfp.aws module

kfp.aws.use_aws_secret(secret_name='aws-secret', aws_access_key_id_name='AWS_ACCESS_KEY_ID', aws_secret_access_key_name='AWS_SECRET_ACCESS_KEY')[source]

An operator that configures the container to use AWS credentials.

AWS doesn’t create secret along with kubeflow deployment and it requires users to manually create credential secret with proper permissions. — apiVersion: v1 kind: Secret metadata:

name: aws-secret

type: Opaque data:


kfp.azure module


An operator that configures the container to use Azure user credentials.

The azcreds secret is created as part of the kubeflow deployment that stores the client ID and secrets for the kubeflow azure service principal.

With this service principal, the container has a range of Azure APIs to access to.