leaf.models¶

Module Contents¶

class leaf.models.BaseModel(export_pattern_pred: str | Tuple[str, str], patch_sz: int | Tuple[int, int], input_scaling: float | Tuple[float, float], classes_dict: dict, debug: bool = False, model_name: str = 'latest', use_gpu: bool = True, cuda_device: str = 'cuda:0', search_pattern: str = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG'])¶

A base class to use for implementing new models to the pipeline. It defines most of the utilities required with respect to initialization, handling images, predicting and saving results.

This is a constructor placeholder, which needs to be overriden when used, otherwise it raises a NotImplementedError.

Parameters:

export_pattern_pred (Union[str, Tuple[str, str]]) – This argument controls how the results are saved. You can provide a single path string which defines where the images are saved. This then takes the filename and the provided path to create the path for the predictions. This means that the source folder structure is flattened. If you want to keep the source folder structure you can provide a tuple of two strings. This is then used with the str.replace() method and allows for specifying which part of the file path of the image is replaced with what.
patch_sz (Union[int, Tuple[int, int]]) – Size of patches cropped to use for inference. The patchsize needs to be chosen so that it recreates the whole image i.e., image is dividided without rests and overlaps. (e.g. 4096x4096 can use 2048x2048 patches but not 2000x2000)
input_scaling (Union[float, Tuple[float, float]]) – If one number is provide, the scaling is done symmetrically. It denotes a scaling factor applied before inference. The results are scaled back to the original resolution before saving.
classes_dict (dict) – This denotes a mapping between class names and their integer id.
debug (bool, optional) – When run in debug mode the models do not save predictions and some models provide additional insights. Defaults to False.
model_name (str) – Which model should be used. For this refer to model zoo. Defaults to ‘latest’.
use_gpu (bool, optional) – When true the inference is run on a GPU. Defaults to True.
cuda_device (str, optional) – which cuda device to use. Not recommended to use. Rather use the environment variable export CUDA_VISIBLE_DEVICES = … . Defaults to ‘cuda:0’.
search_pattern (str, optional) – List of image extensions used to search for images. Defaults to [’.jpg’, ‘.JPG’, ‘.jpeg’, ‘.png’, ‘*.PNG’].

Raises:

NotImplementedError – Error to notify that this method is meant to be implemented in the inherited class.

classes_dict = None¶

debug = None¶

export_pattern_pred = None¶

patch_sz = None¶

input_scaling = (None,)¶

use_gpu = None¶

cuda_device = None¶

model_path = None¶

patch_stride = None¶

search_pattern = None¶

current_input_name = None¶

predict(src: str) → None¶

Predicts on a folder or file. Uses pathlib Path internally

Parameters:: src (str) – Path to a specific file or folder

predict_folder(src: pathlib.Path) → None¶

This function predicts a folder. It uses recursive file search with extensions defined in the search_pattern: attribute.

Parameters:: src (pathlib.Path) – Pathlib Path pointing to a folder.

predict_image(src: pathlib.Path) → None¶

Tests if a file is read successfully. Then it subdivides an image to patches according to the specified patch size. Then it conducts the inference and merges the results from individual patches and saves the results, if not in debug mode.

Parameters:: src (pathlib.Path) – Pathlib Path to the image to be predicted.

subdivide_image(image: numpy.array, src: pathlib.Path) → numpy.array¶

This function subdivides an image according to the patch_sz attribute and returns a array of patches in form of n x patch_sz[0] x patch_sz[1] x 3 . Currently it only allows for subdivision that mathches the image size exactly.

Parameters:

image (np.array) – Image array
src (pathllib.Path) – Path to the corresponding image

Returns:

Batched image patches

Return type:

np.array

split_into_patches(image: numpy.array, patchsz: tuple | list) → numpy.array¶

Splits the image into patches according the patchsz and returns batched patches array.

Parameters:

image (np.array) – numpy array with the image
patchsz (Union[tuple, list]) – patchsize denoting the x and y size

Returns:

batched patches array

Return type:

np.array

merge_patches(patches: numpy.array) → numpy.array¶

Merge patches back to recreate the original image/pattern. It can only handle a stride which is equal to the patch size.

Parameters:: patches (np.array) – Batched array of patches
Raises:: NotImplementedError – Throw and exception regarding differing stride not implemented.
Returns:: merged output with the dimensions of the riginal image
Return type:: np.array

abstractmethod infer_image(model_input: torch.tensor) → numpy.array¶

A placeholder for downstream models inference. Needs to be implemented

Parameters:: model_input (torch.tensor) – model_input tensor
Raises:: NotImplementedError – Error denoting, it is not implemented.
Returns:: model output mask
Return type:: np.array

abstractmethod rgb_image2input(image: numpy.array) → torch.tensor¶

A placeholder for model specific preparation of the input for inference. Needs to be implemented in inherited classes.

Parameters:: image (np.array) – image array
Raises:: NotImplementedError – Error denoting, it is not implemented.
Returns:: torch tensor ready to be processed by the model
Return type:: torch.tensor

test() → None¶: A test method to see if inference can be done sucessfully. It downloads a sample image to test/images and predicts on it

download_file(url: str, root: str = 'models') → str¶

Download file at a given url to a specified directory if it does not already exists and keep the name of the file same as on the server.

Parameters:

url (str) – url to what to download
root (str, optional) – directory where to download to. Defaults to ‘models’.

Returns:

path to the downloaded file

Return type:

str

parse_name_from_header(header_value: str) → Tuple[str, dict[str, str]]¶

Parses the Content-Disposition header value and extracts the disposition type and parameters.

Parameters:: header_value (str) – The raw header string, e.g., ‘form-data; name=”file”; filename=”example.txt”’.
Returns:: A tuple where the first element is the disposition type (e.g., ‘form-data’), and the second is a dictionary of parameter key-value pairs (e.g., {‘name’: ‘file’, ‘filename’: ‘example.txt’}).
Return type:: Tuple[str, Dict[str, str]]

save_predictions(image_path: str, predictions: numpy.array) → None¶

This function determines the save path based on the source image name and export_pattern_pred attribute. Then it saves the predictions on a specified location.

Parameters:

image_path (str) – image path of the original image
predictions (np.array) – predictions array

Raises:

Exception – Error when the export path cannot be determined

abstractmethod get_model(model_name: str)¶

Download if not already existing and use a desired model.

Parameters:: model_name (str) – Model Name. Refer to Model Zoo
Raises:: NotImplementedError – Error that it is not implemented

class leaf.models.TorchscriptTransformer(export_pattern_pred: str | Tuple[str, str], patch_sz: int | Tuple[int, int], input_scaling: float | Tuple[float, float], classes_dict: dict, debug: bool = False, model_name: str = 'latest', use_gpu: bool = True, cuda_device: str = 'cuda:0', search_pattern: str = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG'])¶

Bases: BaseModel

Extension of the BaseModel class which implements further model specifics to torscript exported transformer model.

Constructor of this class

Parameters:

export_pattern_pred (Union[str, Tuple[str, str]]) – This argument controls how the results are saved. You can provide a single path string which defines where the images are saved. This then takes the filename and the provided path to create the path for the predictions. This means that the source folder structure is flattened. If you want to keep the source folder structure you can provide a tuple of two strings. This is then used with the str.replace() method and allows for specifying which part of the file path of the image is replaced with what.
patch_sz (Union[int, Tuple[int, int]]) – Size of patches cropped to use for inference. The patchsize needs to be chosen so that it recreates the whole image i.e., image is dividided without rests and overlaps. (e.g. 4096x4096 can use 2048x2048 patches but not 2000x2000)
input_scaling (Union[float, Tuple[float, float]]) – If one number is provide, the scaling is done symmetrically. It denotes a scaling factor applied before inference. The results are scaled back to the original resolution before saving.
classes_dict (dict) – This denotes a mapping between class names and their integer id.
debug (bool, optional) – When run in debug mode the models do not save predictions and some models provide additional insights. Defaults to False.
model_name (str) – Which model should be used. For this refer to model zoo. Defaults to ‘latest’.
use_gpu (bool, optional) – When true the inference is run on a GPU. Defaults to True.
cuda_device (str, optional) – which cuda device to use. Not recommended to use. Rather use the environment variable export CUDA_VISIBLE_DEVICES = … . Defaults to ‘cuda:0’.
search_pattern (str, optional) – List of image extensions used to search for images. Defaults to [’.jpg’, ‘.JPG’, ‘.jpeg’, ‘.png’, ‘*.PNG’].

Raises:

Exception – Error when GPU is requested but cannot be utilized.

classes_dict¶

debug = False¶

export_pattern_pred¶

patch_sz¶

input_scaling¶

use_gpu = True¶

cuda_device = 'cuda:0'¶

model_path = None¶

patch_stride¶

search_pattern = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG']¶

infer_image(model_input: torch.tensor) → numpy.array¶

Torchscript Transformer specific implementation of inference using a preprossed input.

Parameters:: model_input (torch.tensor) – prepared tensor for inference
Returns:: results encoded as a mask
Return type:: np.array

rgb_image2input(image: numpy.array) → torch.tensor¶

Prepare an image for inference with pytorch. This is specific for torchscript transformer.

Parameters:: image (np.array) – array with the image
Returns:: model input tensor ready for inference
Return type:: torch.tensor

class leaf.models.SymptomsSegmentation(classes_dict: dict = {1: 'leaf', 2: 'necrosis', 3: 'insect_damage', 4: 'powdery_mildew'}, export_pattern_pred: str | Tuple[str, str] = 'export/symptoms_seg/pred', input_scaling: float | Tuple[float, float] = 0.5, patch_sz: int | Tuple[int, int] = 2048, *args, **kwargs)¶

Bases: TorchscriptTransformer

Implementation of the symptoms segmentation bulding block.

Constructor of symptoms segmentation. It provides the correct default values for various parameters.

Parameters:

classes_dict (dict, optional) – This denotes a mapping between class names and their integer id. Defaults to {1: ‘leaf’, 2: ‘necrosis’, 3: ‘insect_damage’, 4: ‘powdery_mildew’, }.
export_pattern_pred (Union[str, Tuple[str, str]], optional) – This argument controls how the results are saved. You can provide a single path string which defines where the images are saved. This then takes the filename and the provided path to create the path for the predictions. This means that the source folder structure is flattened. If you want to keep the source folder structure you can provide a tuple of two strings. This is then used with the str.replace() method and allows for specifying which part of the file path of the image is replaced with what. Defaults to ‘export/symptoms_seg/pred’.
input_scaling (Union[float, Tuple[float, float]], optional) – If one number is provide, the scaling is done symmetrically. It denotes a scaling factor applied before inference. The results are scaled back to the original resolution before saving. Defaults to 0.5.
patch_sz (Union[int, Tuple[int, int]], optional) – Size of patches cropped to use for inference. The patchsize needs to be chosen so that it recreates the whole image i.e., image is dividided without rests and overlaps. (e.g. 4096x4096 can use 2048x2048 patches but not 2000x2000). Defaults to 2048.

get_model(model_name: str)¶

Download if not already existing and use a desired model. A specific implementation for symptoms segmentation.

Parameters:: model_name (str) – Model Name
Raises:: Exception – Error when model not found

class leaf.models.OrgansSegmentation(classes_dict: dict = {1: 'stem', 2: 'head'}, export_pattern_pred: str | Tuple[str, str] = 'export/organs/pred', input_scaling: float | Tuple[float, float] = 0.25, patch_sz: int | Tuple[int, int] = 4096, *args, **kwargs)¶

Bases: TorchscriptTransformer

Implementation of the organ segmentation bulding block.

Constructor of organ segmentation. It provides the correct default values for various parameters.

Parameters:

classes_dict (dict, optional) – This denotes a mapping between class names and their integer id. Defaults to {1: ‘stem’, 2: ‘head’}.
export_pattern_pred (Union[str, Tuple[str, str]], optional) – This argument controls how the results are saved. You can provide a single path string which defines where the images are saved. This then takes the filename and the provided path to create the path for the predictions. This means that the source folder structure is flattened. If you want to keep the source folder structure you can provide a tuple of two strings. This is then used with the str.replace() method and allows for specifying which part of the file path of the image is replaced with what. Defaults to ‘export/organs/pred’.
input_scaling (Union[float, Tuple[float, float]], optional) – If one number is provide, the scaling is done symmetrically. It denotes a scaling factor applied before inference. The results are scaled back to the original resolution before saving. Defaults to 0.25.
patch_sz (Union[int, Tuple[int, int]], optional) – Size of patches cropped to use for inference. The patchsize needs to be chosen so that it recreates the whole image i.e., image is dividided without rests and overlaps. (e.g. 4096x4096 can use 2048x2048 patches but not 2000x2000). Defaults to 4096.

get_model(model_name: str)¶

Download if not already existing and use a desired model. A specific implementation for organ segmentation.

Parameters:: model_name (str) – Model Name
Raises:: Exception – Error when model not found

class leaf.models.SymptomsDetection(export_pattern_pred: str | Tuple[str, str] = 'export/symptoms_det/pred', patch_sz: int | Tuple[int, int] = 1024, input_scaling: float | Tuple[float, float] = 1.0, classes_dict: dict = {1: 'pycnidia', 2: 'rust'}, debug: bool = False, model_name: str = 'latest', use_gpu: bool = True, cuda_device: str = 'cuda:0', keypoints_thresh: float = 0.212, max_det: int = 100000, search_pattern: str = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG'])¶

Bases: BaseModel

Implementation of the symptoms detection building block

Constructor of symptoms detection. It provides the correct default values for various parameters.

Parameters:

export_pattern_pred (Union[str, Tuple[str, str]], optional) – This argument controls how the results are saved. You can provide a single path string which defines where the images are saved. This then takes the filename and the provided path to create the path for the predictions. This means that the source folder structure is flattened. If you want to keep the source folder structure you can provide a tuple of two strings. This is then used with the str.replace() method and allows for specifying which part of the file path of the image is replaced with what. Defaults to ‘export/symptoms_det/pred’.
input_scaling (Union[float, Tuple[float, float]], optional) – If one number is provide, the scaling is done symmetrically. It denotes a scaling factor applied before inference. The results are scaled back to the original resolution before saving. Defaults to 1.0.
classes_dict (dict, optional) – This denotes a mapping between class names and their integer id. Defaults to {1: ‘pycnidia’, 2: ‘rust’}.
debug (bool, optional) – When run in debug mode the models do not save predictions and some models provide additional insights. Defaults to False.
model_name (str, optional) – Which model should be used. For this refer to model zoo. Defaults to ‘latest’.
use_gpu (bool, optional) – When true the inference is run on a GPU. Defaults to True.
cuda_device (str, optional) – which cuda device to use. Not recommended to use. Rather use the environment variable export CUDA_VISIBLE_DEVICES = … . For the current version of ultralytics only ‘cuda:0’ works as pointed out in https://github.com/ultralytics/ultralytics/issues/5801 . Defaults to ‘cuda:0’.
keypoints_thresh (float, optional) – Confidence threshold for acceptance of predictions. Typically try to use a value that optimizes the f1 score during training. Defaults to 0.212.
search_pattern (str, optional) – List of image extensions used to search for images. Defaults to [’.jpg’, ‘.JPG’, ‘.jpeg’, ‘.png’, ‘*.PNG’].

Raises:

Exception – Error when GPU is requested but cannot be utilized.

classes_dict¶

debug = False¶

export_pattern_pred = 'export/symptoms_det/pred'¶

patch_sz = 1024¶

input_scaling = 1.0¶

use_gpu = True¶

cuda_device = 'cuda:0'¶

model_path = None¶

keypoints_thresh = 0.212¶

max_det = 100000¶

patch_stride = 1024¶

search_pattern = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG']¶

get_model(model_name: str)¶

Download if not already existing and use a desired model. A specific implementation for symptoms detection.

Parameters:: model_name (str) – Model Name
Raises:: Exception – Error when model not found

infer_image(model_input: torch.tensor) → numpy.array¶

Symptoms detection specific implementation of inference using a preprossed input.

Parameters:: model_input (torch.tensor) – prepared tensor for inference
Returns:: results encoded as a mask
Return type:: np.array

rgb_image2input(image: numpy.array) → torch.tensor¶

Prepare an image for inference with pytorch. This is specific for symptoms detection.

Parameters:: image (np.array) – array with the image
Returns:: model input tensor ready for inference
Return type:: torch.tensor

class leaf.models.Resize(width, height, resize_target=True, keep_aspect_ratio=False, ensure_multiple_of=1, resize_method='lower_bound', image_interpolation_method=cv2.INTER_AREA)¶

Bases: object

Resize sample to given size (width, height).

Init.

Parameters:

width (int) – desired output width
height (int) – desired output height
resize_target (bool, optional) – True: Resize the full sample (image, mask, target). False: Resize image only. Defaults to True.
keep_aspect_ratio (bool, optional) – True: Keep the aspect ratio of the input sample. Output sample might not have the given width and height, and resize behaviour depends on the parameter ‘resize_method’. Defaults to False.
ensure_multiple_of (int, optional) – Output width and height is constrained to be multiple of this parameter. Defaults to 1.
resize_method (str, optional) – “lower_bound”: Output will be at least as large as the given size. “upper_bound”: Output will be at max as large as the given size. (Output size might be smaller than given size.) “minimal”: Scale as least as possible. (Output size might be smaller than given size.) Defaults to “lower_bound”.

constrain_to_multiple_of(x, min_val=0, max_val=None)¶

get_size(width, height)¶

__call__(sample)¶

class leaf.models.NormalizeImage(mean, std)¶

Bases: object

Normlize image by given mean and std.

__call__(sample)¶

class leaf.models.PrepareForNet¶

Bases: object

Prepare sample for usage as network input.

__call__(sample)¶

class leaf.models.FocusSegmentation(export_pattern_pred: str | Tuple[str, str] = 'export/focus/pred', patch_sz: int | Tuple[int, int] = 4096, classes_dict: dict = {1: 'out_of_focus'}, debug: bool = False, model_name: str = 'latest', use_gpu: bool = True, cuda_device: str = 'cuda:0', input_scaling: float | Tuple[float, float] = 0.25, search_pattern: str = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG'], buffer_scaling: float = 1.0)¶

Bases: BaseModel

Implementation of the focus segmentation building block

Constructor of focus segmentation. It provides the correct default values for various parameters.

Parameters:

export_pattern_pred (Union[str, Tuple[str, str]], optional) – This argument controls how the results are saved. You can provide a single path string which defines where the images are saved. This then takes the filename and the provided path to create the path for the predictions. This means that the source folder structure is flattened. If you want to keep the source folder structure you can provide a tuple of two strings. This is then used with the str.replace() method and allows for specifying which part of the file path of the image is replaced with what.. Defaults to ‘export/focus/pred’.
classes_dict (dict, optional) – This denotes a mapping between class names and their integer id.. Defaults to {1: ‘out_of_focus’}.
debug (bool, optional) – When run in debug mode the models do not save predictions and some models provide additional insights.. Defaults to False.
model_name (str, optional) – Which model should be used. For this refer to model zoo. Defaults to ‘latest’.
use_gpu (bool, optional) – When true the inference is run on a GPU. Defaults to True.
cuda_device (str, optional) – which cuda device to use. Not recommended to use. Rather use the environment variable export CUDA_VISIBLE_DEVICES = … . Defaults to ‘cuda:0’.
input_scaling (Union[float, Tuple[float, float]], optional) – If one number is provide, the scaling is done symmetrically. It denotes a scaling factor applied before inference. The results are scaled back to the original resolution before saving. Defaults to 0.25.
search_pattern (str, optional) – List of image extensions used to search for images. Defaults to [’.jpg’, ‘.JPG’, ‘.jpeg’, ‘.png’, ‘*.PNG’].
buffer_scaling (float, optional) – Paramer of the focus acceptance buffer. Lower values make the focus estimation more aggressive. Defaults to 1.0.

Raises:

Exception – _description_

classes_dict¶

debug = False¶

export_pattern_pred = 'export/focus/pred'¶

patch_sz = 4096¶

input_scaling = 0.25¶

use_gpu = True¶

cuda_device = 'cuda:0'¶

model_path = None¶

buffer_scaling = 1.0¶

patch_stride = 4096¶

search_pattern = ['*.jpg', '*.JPG', '*.jpeg', '*.png', '*.PNG']¶

get_model(model_name: str)¶

Download if not already existing and use a desired model. A specific implementation for focus segmentation.

Parameters:: model_name (str) – Model Name
Raises:: Exception – Error when model not found

infer_image(model_input: torch.tensor) → numpy.array¶

Focus segmentation specific implementation of inference using a preprossed input.

Parameters:: model_input (torch.tensor) – prepared tensor for inference
Returns:: results encoded as a mask
Return type:: np.array

rgb_image2input(image: numpy.array) → torch.tensor¶

Prepare an image for inference with pytorch. This is specific for focus segmentation.

Parameters:: image (np.array) – array with the image
Returns:: model input tensor ready for inference
Return type:: torch.tensor

image2tensor(raw_image: numpy.array, input_size_w: int, input_size_h: int) → torch.tensor¶

Focus segmentation specific implementation of converting the image to a tensor for inference.

Parameters:

raw_image (np.array) – image to be processed
input_size_w (int) – width shape of the image
input_size_h (int) – heigth shape of the image

Returns:

tensor ready for inference with DepthAnythingv2

Return type:

torch.tensor

determine_focus(depth: numpy.array, image: numpy.array) → numpy.array¶

Using the predicted depth map and original image, determine regions that are in focus.

Parameters:

depth (np.array) – depth map
image (np.array) – original image

Returns:

_description_

Return type:

np.array

leaf.models.test()¶: This function runs a dry run of the complete pipeline to validate your installation. It produces predictions in newly created test folder.