kfp.components.structures package¶

Definitions for component spec.

class kfp.components.structures.ComponentSpec(name: str, implementation: kfp.components.structures.Implementation, description: Optional[str] = None, inputs: Optional[Dict[str, kfp.components.structures.InputSpec]] = None, outputs: Optional[Dict[str, kfp.components.structures.OutputSpec]] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

The definition of a component.

name¶: The name of the component.

description¶

the description of the component.

Type:	optional

inputs¶

the input definitions of the component.

Type:	optional

outputs¶

the output definitions of the component.

Type:	optional

implementation¶: The implementation of the component. Either an executor (container, importer) or a DAG consists of other components.

description = None

classmethod from_pipeline_spec_dict(pipeline_spec_dict: Dict[str, Any]) → kfp.components.structures.ComponentSpec[source]¶

classmethod from_pipeline_spec_yaml(pipeline_spec_yaml: str) → kfp.components.structures.ComponentSpec[source]¶

Creates a ComponentSpec from a pipeline spec in YAML format.

Parameters:	component_yaml (str) – Component spec in YAML format.
Returns:	The component spec object.
Return type:	ComponentSpec

classmethod from_v1_component_spec(v1_component_spec: kfp.components.v1_structures.ComponentSpec) → kfp.components.structures.ComponentSpec[source]¶

Converts V1 ComponentSpec to V2 ComponentSpec.

Parameters:	v1_component_spec – The V1 ComponentSpec.
Returns:	Component spec in the form of V2 ComponentSpec.
Raises:	`ValueError` – If implementation is not found. `TypeError` – if any argument is neither a str nor Dict.

inputs = None

classmethod load_from_component_yaml(component_yaml: str) → kfp.components.structures.ComponentSpec[source]¶

Loads V1 or V2 component yaml into ComponentSpec.

Parameters:	component_yaml – the component yaml in string format.
Returns:	Component spec in the form of V2 ComponentSpec.

outputs = None

save_to_component_yaml(output_file: str) → None[source]¶

Saves ComponentSpec into IR YAML file.

Parameters:	output_file – File path to store the component yaml.

to_pipeline_spec() → pipeline_spec_pb2.PipelineSpec[source]¶

Creates a pipeline instance and constructs the pipeline spec for a single component.

Parameters:	component_spec – The ComponentSpec to convert to PipelineSpec.
Returns:	A PipelineSpec proto representing the compiled component.

transform_command_input_placeholders() → None[source]¶: Converts command and args elements to a placholder type where applicable.

transform_inputs() → None[source]¶: Use None instead of empty list for inputs.

transform_name() → None[source]¶: Converts the name to a valid name.

transform_outputs() → None[source]¶: Use None instead of empty list for outputs.

validate_placeholders()[source]¶: Validates that input/output placeholders refer to an existing input/output.

class kfp.components.structures.ConcatPlaceholder(items: List[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]])[source]¶

Bases: kfp.components.base_model.BaseModel

Class that extends basePlaceholders for concatenation.

items¶: string or ValidCommandArgs for concatenation.

classmethod from_concat_string(concat_string: str) → kfp.components.structures.ConcatPlaceholder[source]¶

Creates a concat placeholder from an IR string indicating concatenation.

Parameters:	concat_string (str) – The IR string (e.g. {{$.inputs.parameters[‘input1’]}}+{{$.inputs.parameters[‘input2’]}})
Returns:	The ConcatPlaceholder instance.
Return type:	ConcatPlaceholder

class kfp.components.structures.ContainerSpec(image: str, command: Optional[List[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]]] = None, args: Optional[List[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]]] = None, env: Optional[Mapping[str, Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]]] = None, resources: Optional[kfp.components.structures.ResourceSpec] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

Container implementation definition.

image¶: The container image.

command¶

the container entrypoint.

Type:	optional

args¶

the arguments to the container entrypoint.

Type:	optional

env¶

the environment variables to be passed to the container.

Type:	optional

resources¶

the specification on the resource requirements.

Type:	optional

args = None

command = None

env = None

classmethod from_container_dict(container_dict: Dict[str, Any]) → kfp.components.structures.ContainerSpec[source]¶

Creates a ContainerSpec from a PipelineContainerSpec message in dict format (pipeline_spec.deploymentSpec.executors.<executor- key>.container).

Parameters:	container_dict (Dict[str, Any]) – PipelineContainerSpec message in dict format.
Returns:	The ContainerSpec instance.
Return type:	ContainerSpec

resources = None

transform_args() → None[source]¶: Use None instead of empty list for args.

transform_command() → None[source]¶: Use None instead of empty list for command.

transform_env() → None[source]¶: Use None instead of empty dict for env.

class kfp.components.structures.DagSpec(tasks: Mapping[str, kfp.components.structures.TaskSpec], outputs: Mapping[str, Any])[source]¶

Bases: kfp.components.base_model.BaseModel

DAG(graph) implementation definition.

tasks¶: The tasks inside the DAG.

outputs¶: Defines how the outputs of the dag are linked to the sub tasks.

class kfp.components.structures.IfPresentPlaceholder(if_structure: kfp.components.structures.IfPresentPlaceholderStructure)[source]¶

Bases: kfp.components.base_model.BaseModel

Class that extends basePlaceholders for conditional cases.

if_present¶

holds structure for conditional cases.

Type:	ifPresent

class kfp.components.structures.IfPresentPlaceholderStructure(input_name: str, then: List[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]], otherwise: Optional[List[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]]] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

Class that holds structure for conditional cases.

input_name¶: name of the input/output.

then¶: If the input/output specified in name is present, the command-line argument will be replaced at run-time by the expanded value of then.

otherwise¶: If the input/output specified in name is not present, the command-line argument will be replaced at run-time by the expanded value of otherwise.

otherwise = None

transform_otherwise() → None[source]¶: Use None instead of empty list for optional.

class kfp.components.structures.Implementation(container: Optional[kfp.components.structures.ContainerSpec] = None, graph: Optional[kfp.components.structures.DagSpec] = None, importer: Optional[kfp.components.structures.ImporterSpec] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

Implementation definition.

container¶: container implementation details.

graph¶: graph implementation details.

importer¶: importer implementation details.

container = None

classmethod from_deployment_spec_dict(deployment_spec_dict: Dict[str, Any], component_name: str) → kfp.components.structures.Implementation[source]¶

Creates an Implmentation object from a deployment spec message in dict format (pipeline_spec.deploymentSpec).

Parameters:	deployment_spec_dict (Dict[str, Any]) – PipelineDeploymentConfig message in dict format. component_name (str) – The name of the component.
Returns:	An implementation object.
Return type:	Implementation

graph = None

importer = None

class kfp.components.structures.ImporterSpec(artifact_uri: str, type_schema: str, reimport: bool, metadata: Optional[Mapping[str, Any]] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

ImporterSpec definition.

artifact_uri¶: The URI of the artifact.

type_schema¶: The type of the artifact.

reimport¶: Whether or not import an artifact regardless it has been imported before.

metadata¶

the properties of the artifact.

Type:	optional

metadata = None

class kfp.components.structures.InputPathPlaceholder(input_name: str)[source]¶

Bases: kfp.components.base_model.BaseModel, kfp.components.structures.PlaceholderSerializationMixin

Class that holds input path for conditional cases.

input_name¶: name of the input.

class kfp.components.structures.InputSpec(type: Union[str, dict], default: Optional[Any] = None)[source]¶

Bases: kfp.components.structures.InputSpec_, kfp.components.base_model.BaseModel

Component input definitions.

type¶: The type of the input.

default¶

the default value for the input.

Type:	optional

_optional¶: Wether the input is optional. An input is optional when it has an explicit default value.

classmethod from_ir_parameter_dict(ir_parameter_dict: Dict[str, Any]) → kfp.components.structures.InputSpec[source]¶

Creates an InputSpec from a ComponentInputsSpec message in dict format (pipeline_spec.components.<component- key>.inputDefinitions.parameters.<input-key>).

Parameters:	ir_parameter_dict (Dict[str, Any]) – The ComponentInputsSpec message in dict format.
Returns:	The InputSpec object.
Return type:	InputSpec

class kfp.components.structures.InputSpec_(type: Union[str, dict], default: Optional[Any] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

Component input definitions. (Inner class).

type¶: The type of the input.

default¶

the default value for the input.

Type:	optional

description¶: Optional: the user description of the input.

default = None

class kfp.components.structures.InputUriPlaceholder(input_name: str)[source]¶

Bases: kfp.components.base_model.BaseModel, kfp.components.structures.PlaceholderSerializationMixin

Class that holds input uri for conditional cases.

input_name¶: name of the input.

class kfp.components.structures.InputValuePlaceholder(input_name: str)[source]¶

Bases: kfp.components.base_model.BaseModel, kfp.components.structures.PlaceholderSerializationMixin

Class that holds input value for conditional cases.

input_name¶: name of the input.

class kfp.components.structures.OutputParameterPlaceholder(output_name: str)[source]¶

Bases: kfp.components.base_model.BaseModel, kfp.components.structures.PlaceholderSerializationMixin

Class that holds output path for conditional cases.

output_name¶: name of the output.

class kfp.components.structures.OutputPathPlaceholder(output_name: str)[source]¶

Bases: kfp.components.base_model.BaseModel, kfp.components.structures.PlaceholderSerializationMixin

Class that holds output path for conditional cases.

output_name¶: name of the output.

class kfp.components.structures.OutputSpec(type: Union[str, dict])[source]¶

Bases: kfp.components.base_model.BaseModel

Component output definitions.

type¶: The type of the output.

classmethod from_ir_parameter_dict(ir_parameter_dict: Dict[str, Any]) → kfp.components.structures.OutputSpec[source]¶

Creates an OutputSpec from a ComponentOutputsSpec message in dict format (pipeline_spec.components.<component- key>.outputDefinitions.parameters|artifacts.<output-key>).

Parameters:	ir_parameter_dict (Dict[str, Any]) – The ComponentOutputsSpec in dict format.
Returns:	The OutputSpec object.
Return type:	OutputSpec

class kfp.components.structures.OutputUriPlaceholder(output_name: str)[source]¶

Bases: kfp.components.base_model.BaseModel, kfp.components.structures.PlaceholderSerializationMixin

Class that holds output uri for conditional cases.

output_name¶: name of the output.

class kfp.components.structures.PlaceholderSerializationMixin[source]¶

Bases: object

Mixin for *Placeholder objects that handles the serialization/deserialization of the placeholder.

classmethod from_placeholder(placeholder_string: str) → P[source]¶

Converts a placeholder string into a placeholder object.

Parameters:	placeholder_string (str) – The placeholder.
Returns:	The placeholder object.
Return type:	PlaceholderSerializationMixin subclass

classmethod is_match(placeholder_string: str) → bool[source]¶

Determines if the placeholder_string matches the placeholder pattern.

Parameters:	placeholder_string (str) – The string (often “{{$.inputs/outputs…}}”) to check.
Returns:	Determines if the placeholder_string matches the placeholder pattern.
Return type:	bool

to_placeholder() → str[source]¶

Converts a placeholder object into a placeholder string.

Returns:	The placeholder string.
Return type:	str

class kfp.components.structures.ResourceSpec(cpu_limit: Optional[float] = None, memory_limit: Optional[float] = None, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

The resource requirements of a container execution.

cpu_limit¶

the limit of the number of vCPU cores.

Type:	optional

memory_limit¶

the memory limit in GB.

Type:	optional

accelerator_type¶

the type of accelerators attached to the container.

Type:	optional

accelerator_count¶

the number of accelerators attached.

Type:	optional

accelerator_count = None

accelerator_type = None

cpu_limit = None

memory_limit = None

class kfp.components.structures.TaskSpec(name: str, inputs: Mapping[str, Any], dependent_tasks: List[str], component_ref: str, trigger_condition: Optional[str] = None, trigger_strategy: Optional[str] = None, iterator_items: Optional[Any] = None, iterator_item_input: Optional[str] = None, enable_caching: bool = True, display_name: Optional[str] = None)[source]¶

Bases: kfp.components.base_model.BaseModel

The spec of a pipeline task.

name¶: The name of the task.

inputs¶: The sources of task inputs. Constant values or PipelineParams.

dependent_tasks¶: The list of upstream tasks.

component_ref¶: The name of a component spec this task is based on.

trigger_condition¶

an expression which will be evaluated into a boolean value. True to trigger the task to run.

Type:	optional

trigger_strategy¶

when the task will be ready to be triggered. Valid values include: “TRIGGER_STRATEGY_UNSPECIFIED”, “ALL_UPSTREAM_TASKS_SUCCEEDED”, and “ALL_UPSTREAM_TASKS_COMPLETED”.

Type:	optional

iterator_items¶

the items to iterate on. A constant value or a PipelineParam.

Type:	optional

iterator_item_input¶

the name of the input which has the item from the [items][] collection.

Type:	optional

enable_caching¶

whether or not to enable caching for the task. Default is True.

Type:	optional

display_name¶

the display name of the task. If not specified, the task name will be used as the display name.

Type:	optional

display_name = None

enable_caching = True

iterator_item_input = None

iterator_items = None

trigger_condition = None

trigger_strategy = None

class kfp.components.structures.TypeVar(name, *constraints, bound=None, covariant=False, contravariant=False)[source]¶

Bases: typing._Final, typing._Immutable

Type variable.

Usage:

T = TypeVar('T')  # Can be anything
A = TypeVar('A', str, bytes)  # Must be str or bytes

Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See class Generic for more information on generic types. Generic functions work as follows:

def repeat(x: T, n: int) -> List[T]:

‘’’Return a list containing n references to x.’’’ return [x]*n

def longest(x: A, y: A) -> A:

‘’’Return the longest of two strings.’’’ return x if len(x) >= len(y) else y

The latter example’s signature is essentially the overloading of (str, str) -> str and (bytes, bytes) -> bytes. Also note that if the arguments are instances of some subclass of str, the return type is still plain str.

At runtime, isinstance(x, T) and issubclass(C, T) will raise TypeError.

Type variables defined with covariant=True or contravariant=True can be used to declare covariant or contravariant generic types. See PEP 484 for more details. By default generic types are invariant in all type variables.

Type variables can be introspected. e.g.:

T.__name__ == ‘T’ T.__constraints__ == () T.__covariant__ == False T.__contravariant__ = False A.__constraints__ == (str, bytes)

Note that only type variables defined in global scope can be pickled.

kfp.components.structures.convert_str_or_dict_to_placeholder(element: Union[str, dict, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]) → Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder][source]¶

Converts command and args elements to a placholder type based on value of the key of the placeholder string, else returns the input.

Parameters:	element (Union[str, dict, ValidCommandArgs]) – A ContainerSpec.command or ContainerSpec.args element.
Raises:	`TypeError` – If element is invalid.
Returns:	Possibly converted placeholder or original input.
Return type:	Union[str, ValidCommandArgs]

kfp.components.structures.maybe_convert_command_arg_to_placeholder(arg: str) → Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.OutputParameterPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder][source]¶

Infers if a command is a placeholder and converts it to the correct Placeholder object.

Parameters:	arg (str) – The arg or command to possibly convert.
Returns:	The converted command or original string.
Return type:	ValidCommandArgs

kfp.components.structures.try_to_get_dict_from_string(string: str) → Union[dict, str][source]¶

Tries to parse a dictionary from a string if possible, else returns the string.

Parameters:	string (str) – The string to parse.
Returns:	The dictionary if one was successfully parsed, else the original string.
Return type:	Union[dict, str]

kfp.components.structures.kubernetes package
- kfp.components.structures.kubernetes.v1 module