agrifoodpy.pipeline
This module provides methods to build a pipeline for the AgriFoodPy package.
Submodules
Classes
Class for constructing and running pipelines of functions with |
Functions
|
Decorator to make a pipeline node available as a standalone function |
|
Extracts a list of items from a dataset using a coordinate-key tuple, |
|
Returns an element from a dictionary using a key or tuple of keys used |
|
Sets an element in a dictionary using a key or tuple of keys used to |
Package Contents
- class agrifoodpy.pipeline.Pipeline(datablock=None)
Class for constructing and running pipelines of functions with individual sets of parameters.
- nodes = []
- params = []
- names = []
- classmethod read(filename)
- Abstractmethod:
Read a pipeline from a configuration file
- Parameters:
filename (str) – The name of the configuration file.
- Returns:
pipeline – The pipeline object.
- Return type:
- datablock_write(path, value)
Writes a single value to the datablock at the specified path.
- Parameters:
path (list) – The datablock path to the value to be written.
value (any) – The value to be written.
- add_node(node, params={}, name=None)
Adds a node to the pipeline, including its function and execution parameters.
- Parameters:
node (function) – The function to be executed on this node.
params (dict, optional) – The parameters to be passed to the node function.
name (str, optional) – The name of the node. If not provided, a generic name will be assigned.
- run(from_node=0, to_node=None, timing=False)
Runs the pipeline
- Parameters:
from_node (int, optional) – The index of the first node to be executed. Defaults to 0.
to_node (int, optional) – The index of the last node to be executed. If not provided, all nodes will be executed
timing (bool, optional) – If True, the execution time of each node will be printed. Defaults to False.
- agrifoodpy.pipeline.standalone(input_keys, return_keys)
Decorator to make a pipeline node available as a standalone function
If datablock is not passed as a kwarg, and datasets are passed directly instead of datablock keys, a temporary datablock is created and the datasets associated with the arguments in input_keys are added to it. The function then returns the specified datasets in return_keys.
- Parameters:
input_keys (list of strings) – List of dataset keys to be added to the temporary datablock
return_keys (list of strings) – List of keys to datablock datasets to be returned by the decorated function.
- Returns:
wrapper – The decorated function
- Return type:
function
- agrifoodpy.pipeline.item_parser(fbs, items)
Extracts a list of items from a dataset using a coordinate-key tuple, or converts a scalar item to a list
- Parameters:
fbs (xarray.Dataset or xarray.DataArray) – The dataset containing the coordinate-key to extract items from.
items (tuple, scalar) – If a tuple, the first element is the name of the coordinate and the second element is a list of items to extract. If a scalar, the item is converted to a list.
- Returns:
A list of items matching the coordinate-key description, or containing the scalar item.
- Return type:
list
- agrifoodpy.pipeline.get_dict(datablock, keys)
Returns an element from a dictionary using a key or tuple of keys used to describe a path of keys
- Parameters:
datablock (dict) – The input dictionary
keys (str or tuple) – Dictionary key, or tuple of keys
- agrifoodpy.pipeline.set_dict(datablock, keys, object, create_missing=True)
Sets an element in a dictionary using a key or tuple of keys used to describe a path of keys
- Parameters:
datablock (dict) – The input dictionary
keys (str or tuple) – Dictionary key, or tuple of keys
object (any) – The object to set in the dictionary
create_missing (bool, optional) – If True, creates missing keys in the dictionary. Defaults to True.
- Raises:
KeyError – If a key in the path does not exist and create_missing is False.