kfp.containers package¶
-
class
kfp.containers.
ContainerBuilder
(gcs_staging=None, default_image_name=None, namespace=None, service_account='kubeflow-pipelines-container-builder', kaniko_executor_image='gcr.io/kaniko-project/executor@sha256:78d44ec4e9cb5545d7f85c1924695c89503ded86a59f92c7ae658afa3cff5400', k8s_client_configuration=None)[source]¶ Bases:
object
ContainerBuilder helps build a container image.
-
build
(local_dir, docker_filename: str = 'Dockerfile', target_image=None, timeout=1000)[source]¶ Parameters: - local_dir (str) – local directory that stores all the necessary build files
- docker_filename (str) – the path of the Dockerfile relative to the local_dir
- target_image (str) – The container image name where the data will be pushed. Can include tag. If not specified, the function will use the default_image_name specified when creating ContainerBuilder.
- timeout (int) – time out in seconds. Default: 1000
-
-
kfp.containers.
build_image_from_working_dir
(image_name: str = None, working_dir: str = None, file_filter_re: str = '.*\\.py', timeout: int = 1000, base_image: str = None, builder: kfp.containers._container_builder.ContainerBuilder = None) → str[source]¶ Builds and pushes a new container image that captures the current python working directory.
This function recursively scans the working directory and captures the following files in the container image context:
requirements.txt
files- All python files (can be overridden by passing a different file_filter_re argument)
The function generates Dockerfile that starts from a python container image, install packages from requirements.txt (if present) and copies all the captured python files to the container image. The Dockerfile can be overridden by placing a custom Dockerfile in the root of the working directory.
Parameters: - image_name – Optional. The image repo name where the new container image will be pushed. The name will be generated if not not set.
- working_dir – Optional. The directory that will be captured. The current directory will be used if omitted.
- file_filter_re – Optional. A regular expression that will be used to decide which files to include in the container building context.
- timeout – Optional. The image building timeout in seconds.
- base_image – Optional. The container image to use as the base for the new image. If not set, the Google Deep Learning Tensorflow CPU image will be used.
- builder –
Optional. An instance of
kfp.containers.ContainerBuilder
or compatible class that will be used to build the image. The default builder uses “kubeflow-pipelines-container-builder” service account in “kubeflow” namespace. It works with Kubeflow Pipelines clusters installed in “kubeflow” namespace using Google Cloud Marketplace or Standalone with version > 0.4.0. If your Kubeflow Pipelines is installed in a different namespace, you should useContainerBuilder(namespace='<your-kfp-namespace>', ...)
.Depending on how you installed Kubeflow Pipelines, you need to configure your
ContainerBuilder
instance’s namespace and service_account:- For clusters installed with Kubeflow >= 0.7, use
ContainerBuilder(namespace='<your-user-namespace>', service_account='default-editor', ...)
. You can omit the namespace if you use kfp sdk from in-cluster notebook, it uses notebook namespace by default. - For clusters installed with Kubeflow < 0.7, use
ContainerBuilder(service_account='default', ...)
. - For clusters installed using Google Cloud Marketplace or Standalone with version <= 0.4.0, use
ContainerBuilder(namespace='<your-kfp-namespace>' service_account='default')
You may refer to installation guide for more details about different installation options.
- For clusters installed with Kubeflow >= 0.7, use
Returns: The full name of the container image including the hash digest. E.g.
gcr.io/my-org/my-image@sha256:86c1...793c
.