kfp.containers package

class kfp.containers.ContainerBuilder(gcs_staging=None, default_image_name=None, namespace=None, service_account='kubeflow-pipelines-container-builder', kaniko_executor_image='gcr.io/kaniko-project/executor@sha256:78d44ec4e9cb5545d7f85c1924695c89503ded86a59f92c7ae658afa3cff5400', k8s_client_configuration=None)[source]

Bases: object

ContainerBuilder helps build a container image.

build(local_dir, docker_filename: str = 'Dockerfile', target_image=None, timeout=1000)[source]
Parameters:
  • local_dir (str) – local directory that stores all the necessary build files
  • docker_filename (str) – the path of the Dockerfile relative to the local_dir
  • target_image (str) – The container image name where the data will be pushed. Can include tag. If not specified, the function will use the default_image_name specified when creating ContainerBuilder.
  • timeout (int) – time out in seconds. Default: 1000
kfp.containers.build_image_from_working_dir(image_name: str = None, working_dir: str = None, file_filter_re: str = '.*\\.py', timeout: int = 1000, base_image: str = None, builder: kfp.containers._container_builder.ContainerBuilder = None) → str[source]

Builds and pushes a new container image that captures the current python working directory.

This function recursively scans the working directory and captures the following files in the container image context:

  • requirements.txt files
  • All python files (can be overridden by passing a different file_filter_re argument)

The function generates Dockerfile that starts from a python container image, install packages from requirements.txt (if present) and copies all the captured python files to the container image. The Dockerfile can be overridden by placing a custom Dockerfile in the root of the working directory.

Parameters:
  • image_name – Optional. The image repo name where the new container image will be pushed. The name will be generated if not not set.
  • working_dir – Optional. The directory that will be captured. The current directory will be used if omitted.
  • file_filter_re – Optional. A regular expression that will be used to decide which files to include in the container building context.
  • timeout – Optional. The image building timeout in seconds.
  • base_image – Optional. The container image to use as the base for the new image. If not set, the Google Deep Learning Tensorflow CPU image will be used.
  • builder

    Optional. An instance of kfp.containers.ContainerBuilder or compatible class that will be used to build the image. The default builder uses “kubeflow-pipelines-container-builder” service account in “kubeflow” namespace. It works with Kubeflow Pipelines clusters installed in “kubeflow” namespace using Google Cloud Marketplace or Standalone with version > 0.4.0. If your Kubeflow Pipelines is installed in a different namespace, you should use ContainerBuilder(namespace='<your-kfp-namespace>', ...).

    Depending on how you installed Kubeflow Pipelines, you need to configure your ContainerBuilder instance’s namespace and service_account:

    • For clusters installed with Kubeflow >= 0.7, use ContainerBuilder(namespace='<your-user-namespace>', service_account='default-editor', ...). You can omit the namespace if you use kfp sdk from in-cluster notebook, it uses notebook namespace by default.
    • For clusters installed with Kubeflow < 0.7, use ContainerBuilder(service_account='default', ...).
    • For clusters installed using Google Cloud Marketplace or Standalone with version <= 0.4.0, use ContainerBuilder(namespace='<your-kfp-namespace>' service_account='default') You may refer to installation guide for more details about different installation options.
Returns:

The full name of the container image including the hash digest. E.g. gcr.io/my-org/my-image@sha256:86c1...793c.