leaf.metrics¶

Module Contents¶

class leaf.metrics.NamingConstants¶

A collection of constant names used for organizing different types of image predictions and ground truths. These names refer to the folder structure of the results, respectively where to find what. If new block are introduced to the processing pipeline, extend it here.

SYMPTOMS_DET¶

Identifier for symptom detection predictions.

Type:: str

SYMPTOMS_SEG¶

Identifier for symptom segmentation predictions.

Type:: str

ORGANS¶

Identifier for organ segmentation predictions.

Type:: str

FOCUS¶

Identifier for focus classification.

Type:: str

GT_SYMPTOMS_DET¶

Identifier for ground truth symptom detection.

Type:: str

GT_SYMPTOMS_SEG¶

Identifier for ground truth symptom segmentation.

Type:: str

GT_ORGANS¶

Identifier for ground truth organ segmentation.

Type:: str

SYMPTOMS_DET = 'symptoms_det'¶

SYMPTOMS_SEG = 'symptoms_seg'¶

ORGANS = 'organs'¶

FOCUS = 'focus'¶

GT_SYMPTOMS_DET = 'gt_symptoms_det'¶

GT_SYMPTOMS_SEG = 'gt_symptoms_seg'¶

GT_ORGANS = 'gt_organs'¶

class leaf.metrics.PredictionIds¶

Integer ID mappings for different prediction classes used in segmentation and detection tasks for plant imagery. When new models can handle additional classes, their new respective ids should be added here

LEAF_BACKGROUND¶

Background class in leaf segmentation (ID: 0).

Type:: int

LEAF¶

Leaf class in leaf segmentation (ID: 1).

Type:: int

LESION¶

Lesion symptom in segmentation (ID: 2).

Type:: int

INSECT_DAMAGE¶

Insect damage symptom in segmentation (ID: 3).

Type:: int

POWDERY_MILDEW¶

Powdery mildew symptom in segmentation (ID: 4).

Type:: int

PYCNIDIA¶

Pycnidia symptom in detection (ID: 1).

Type:: int

RUST¶

Rust symptom in detection (ID: 2).

Type:: int

OUT_OF_FOCUS¶

Class for out-of-focus regions (ID: 0).

Type:: int

IN_FOCUS¶

Class for in-focus regions (ID: 1).

Type:: int

ORGAN_BACKGROUND¶

Background class in organ segmentation (ID: 0).

Type:: int

HEAD¶

Head organ class in organ segmentation (ID: 1).

Type:: int

STEM¶

Stem organ class in organ segmentation (ID: 2).

Type:: int

LEAF_BACKGROUND = 0¶

LEAF = 1¶

LESION = 2¶

INSECT_DAMAGE = 3¶

POWDERY_MILDEW = 4¶

PYCNIDIA = 1¶

RUST = 2¶

OUT_OF_FOCUS = 0¶

IN_FOCUS = 1¶

ORGAN_BACKGROUND = 0¶

HEAD = 1¶

STEM = 2¶

class leaf.metrics.BasePredictionsMerger(root_folder: str, file_extension: str)¶

Bases: torch.utils.data.Dataset

Base class for merging prediction masks from multiple subfolders in a structured directory.

This class is used to iterate through synchronized sets of prediction files (e.g., segmentation masks) located in multiple subfolders. It ensures that each set of files is present in all expected subfolders and prepares them for further processing.

root_folder¶

Root directory containing prediction subfolders.

Type:: str

file_extension¶

File extension used to identify prediction files.

Type:: str

subfolder_filepaths¶

Mapping of subfolder names to their respective file paths.

Type:: dict

max_count¶

Number of synchronized prediction sets.

Type:: int

current_id¶

Current index used for iteration.

Type:: int

Initializes the BasePredictionsMerger.

Parameters:

root_folder (str) – Root directory containing prediction subfolders.
file_extension (str) – File extension to look for (e.g., ‘*.png’).

root_folder¶

file_extension¶

subfolder_filepaths¶

max_count¶

current_id = 0¶

__len__()¶

Returns the total number of prediction sets.

Returns:: Total number of file sets across subfolders.
Return type:: int

__iter__()¶

Resets the iterator.

Returns:: Iterator object.
Return type:: BasePredictionsMerger

__getitem__(idx)¶

Retrieves the prediction stack for a given index.

Parameters:: idx (int) – Index of the prediction set.
Returns:: Filepath and dictionary of masks for each subfolder.
Return type:: tuple

__next__()¶

Retrieves the next prediction stack in the iteration.

Returns:: Filepath and dictionary of masks for each subfolder.
Return type:: tuple
Raises:: StopIteration – When all prediction sets have been iterated through.

scan_predictions_root(root_folder: str)¶

Scans the root directory for prediction files and validates file consistency across subfolders.

Parameters:: root_folder (str) – The root folder containing prediction subfolders.
Returns:: Number of prediction sets found.
Return type:: int
Raises:: Exception – If subfolders have mismatched file counts or missing files.

get_reference_subfolder()¶

Returns the reference subfolder used for consistency checks.

Returns:: Name of the first subfolder to use as reference.
Return type:: str

check_if_missing(file_path: str)¶

Checks whether a given reference file is present in all other subfolders.

Parameters:: file_path (str) – Path to the file in the reference subfolder.
Raises:: Exception – If the corresponding file is missing in any subfolder.

processing_step(id: int) → tuple[str, dict[str:np.array]]¶

Loads prediction masks for a specific index from all subfolders.

Parameters:: id (int) – Index of the prediction set.
Returns:: str: Filepath of the reference mask. dict: Dictionary mapping subfolder keys to their corresponding prediction masks (as tensors).
Return type:: tuple
Raises:: Exception – If the mask encoding is not supported.

class leaf.metrics.CanopyPredictionsMerger(root_folder: str, file_extension: str = '*.png', prediction_subfolders: dict = {NamingConstants.SYMPTOMS_DET: 'symptoms_det/pred', NamingConstants.SYMPTOMS_SEG: 'symptoms_seg/pred', NamingConstants.ORGANS: 'organs/pred', NamingConstants.FOCUS: 'focus/pred'})¶

Bases: BasePredictionsMerger

Merges canopy prediction masks from predefined subfolders.

Inherits from BasePredictionsMerger and specifies subfolders for canopy-related predictions: symptoms detection, segmentation, organs, and focus. This class handles merging of prediction masks for specific tasks within the canopy.

Initializes the CanopyPredictionsMerger object with the specified prediction subfolders.

Parameters:

root_folder (str) – Root directory containing prediction subfolders.
file_extension (str) – File extension to look for (default: ‘*.png’).
prediction_subfolders (dict) – Mapping of prediction types to subfolder paths.

prediction_subfolders¶

class leaf.metrics.CanopyBenchmarkMerger(root_folder: str, file_extension: str = '*.png', prediction_subfolders: dict = {NamingConstants.SYMPTOMS_DET: 'symptoms_det/pred', NamingConstants.SYMPTOMS_SEG: 'symptoms_seg/pred', NamingConstants.ORGANS: 'organs/pred', NamingConstants.FOCUS: 'focus/pred', NamingConstants.GT_SYMPTOMS_DET: 'symptoms_det/gt', NamingConstants.GT_SYMPTOMS_SEG: 'symptoms_seg/gt', NamingConstants.GT_ORGANS: 'organs/gt'}, focus_override: str = None)¶

Bases: BasePredictionsMerger

Merges benchmark prediction masks and ground truth from predefined subfolders.

This class handles both predictions and corresponding ground truth masks for evaluation purposes. It inherits from BasePredictionsMerger and includes additional ground truth folders.

prediction_subfolders¶

Mapping of prediction and ground truth types to subfolder paths.

Type:: dict

focus_override¶

Custom path to override the default focus path.

Type:: str, optional

Initializes the CanopyBenchmarkMerger object with the specified prediction and ground truth subfolders.

Parameters:

root_folder (str) – Root directory containing prediction subfolders.
file_extension (str) – File extension to look for (default: ‘*.png’).
prediction_subfolders (dict) – Mapping of prediction and ground truth types to subfolder paths.
focus_override (str, optional) – Custom path to override the default focus path (default: None).

prediction_subfolders¶

class leaf.metrics.FlatLeavesPredictionsMerger(root_folder: str, file_extension: str = '*.png', prediction_subfolders: dict = {NamingConstants.SYMPTOMS_DET: 'symptoms_det/pred', NamingConstants.SYMPTOMS_SEG: 'symptoms_seg/pred'})¶

Bases: BasePredictionsMerger

Merges flat leaf prediction masks from predefined subfolders.

This class handles merging of prediction masks specific to flat leaf prediction tasks, including symptoms detection and segmentation.

prediction_subfolders¶

Mapping of prediction types to subfolder paths.

Type:: dict

Initializes the FlatLeavesPredictionsMerger object with the specified prediction subfolders.

Parameters:

root_folder (str) – Root directory containing prediction subfolders.
file_extension (str) – File extension to look for (default: ‘*.png’).
prediction_subfolders (dict) – Mapping of prediction types to subfolder paths.

prediction_subfolders¶

class leaf.metrics.BaseEvaluator(results_path: str)¶

Base class for evaluating prediction results and logging them to a CSV file.

This class provides methods for evaluating prediction results, logging them to a CSV file, and computing evaluation metrics. It is intended to be subclassed, with the compute_metrics method implemented in subclasses to provide specific evaluation logic.

results_path¶

Path to the CSV file for storing evaluation results.

Type:: str

filename_key¶

Column name for filenames in the results file.

Type:: str

Initializes the evaluator and prepares the output CSV file.

Parameters:: results_path (str) – Path to a .csv file or directory where the results will be saved.
Raises:: ValueError – If the provided path does not end with ‘.csv’.

results_path = ''¶

filename_key = 'filename'¶

predict(filepath: str, prediction_stack: dict[str:np.array])¶

Method to predict and log results.

Parameters:

filepath (str) – Path of the image or file.
(dict[str (prediction_stack) – np.array]): Prediction results in a dictionary form.

log_results(filepath: str, results: Dict)¶

Method to log the results into a CSV file.

Parameters:

filepath (str) – Path to the file.
results (Dict) – The results dictionary to be logged.

abstractmethod compute_metrics(prediction_stack: dict[str:np.array]) → dict[str:float]¶

Abstract method to compute evaluation metrics. This method should be implemented by subclasses.

Parameters:: (dict[str (prediction_stack) – np.array]): Dictionary of predicted data.
Returns:: float]: A dictionary of computed metrics.
Return type:: dict[str

class leaf.metrics.CanopyEvaluator(results_path: str)¶

Bases: BaseEvaluator

A subclass of BaseEvaluator for evaluating canopy-related metrics.

This evaluator focuses on metrics specific to canopy evaluation, such as leaf area, pycnidia density, rust density, and the fraction of damaged areas. It computes these metrics based on the prediction results for canopy symptoms and structures.

resulting_keys¶

The list of keys for metrics related to canopy evaluation.

Type:: list

Initializes the CanopyEvaluator and sets up the result keys for canopy evaluation.

Parameters:: results_path (str) – Path to a .csv file or directory where the results will be saved.

resulting_keys = ['reference_leaf_1e6', 'placl', 'n_pycnidia', 'pycnidia_density_1e-6', 'n_rust',...¶

compute_metrics(prediction_stack: dict[str:np.array]) → dict[str:float]¶

Method to compute specific evaluation metrics for canopy evaluation.

Parameters:: (dict[str (prediction_stack) – np.array]): Dictionary of predicted data.
Returns:: float]: A dictionary containing computed metrics.
Return type:: dict[str

class leaf.metrics.CanopyBenchmarkEvaluator(results_path: str)¶

Bases: BaseEvaluator

A subclass of BaseEvaluator for evaluating benchmark metrics for canopy predictions.

This evaluator extends the base evaluator to include benchmark metrics, such as intersection-over-union (IoU), mean squared error (MSE), and mean absolute percentage error (MAPE) for various symptoms and lesions on the canopy. It compares predicted values against ground truth.

resulting_keys¶

The list of keys for benchmark metrics for canopy evaluation.

Type:: list

Initializes the CanopyBenchmarkEvaluator and sets up the result keys for benchmark evaluation.

Parameters:: results_path (str) – Path to a .csv file or directory where the results will be saved.

resulting_keys = ['necrosis_iou', 'pred_placl', 'gt_placl', 'placl_mse', 'pred_n_pycndia', 'gt_n_pycnidia',...¶

compute_metrics(prediction_stack: dict[str:np.array]) → dict[str:float]¶

Method to compute benchmark evaluation metrics.

Parameters:: (dict[str (prediction_stack) – np.array]): Dictionary of predicted data.
Returns:: float]: A dictionary containing computed metrics.
Return type:: dict[str

class leaf.metrics.FlatLeavesEvaluator(results_path: str)¶

Bases: BaseEvaluator

A subclass of BaseEvaluator for evaluating leaf-related metrics for flat leaves.

This evaluator focuses on metrics related to leaf symptoms, such as lesions and pycnidia, within flat leaves. It computes relevant metrics such as leaf area, lesion area, and pycnidia density.

resulting_keys¶

The list of keys for metrics related to flat leaf evaluation.

Type:: list

Initializes the FlatLeavesEvaluator and sets up the result keys for flat leaf evaluation.

Parameters:: results_path (str) – Path to a .csv file or directory where the results will be saved.

resulting_keys = ['reference_leaf_1e6', 'placl', 'n_pycnidia', 'pycnidia_density_1e-6', 'n_rust',...¶

compute_metrics(prediction_stack: dict[str:np.array]) → dict[str:float]¶

Method to compute specific evaluation metrics for flat leaves evaluation.

Parameters:: (dict[str (prediction_stack) – np.array]): Dictionary of predicted data.
Returns:: float]: A dictionary containing computed metrics.
Return type:: dict[str

leaf.metrics.flat_leaves_predictions_iterator(root_folder: str, num_workers: int = 16)¶

Creates a DataLoader for iterating over flat leaves prediction data.

This function prepares a DataLoader to iterate over flat leaves prediction data from the specified root folder. It utilizes a custom data-merging class (FlatLeavesPredictionsMerger) to load and process the data in parallel using multiple workers.

Parameters:

root_folder (str) – The path to the folder containing the flat leaves predictions data.
num_workers (int, optional) – The number of workers to use for data loading. Default is 16.

Returns:

A DataLoader object that loads the flat leaves prediction data in batches.

Return type:

DataLoader

leaf.metrics.canopy_predictions_iterator(root_folder: str, num_workers: int = 16)¶

Creates a DataLoader for iterating over canopy prediction data.

This function prepares a DataLoader to iterate over canopy prediction data from the specified root folder, utilizing the CanopyPredictionsMerger class to handle data merging and loading.

Parameters:

root_folder (str) – The path to the folder containing the canopy predictions data.
num_workers (int, optional) – The number of workers to use for data loading. Default is 16.

Returns:

A DataLoader object that loads the canopy prediction data in batches.

Return type:

DataLoader

leaf.metrics.canopy_benchmark_iterator(root_folder: str, num_workers: int = 16, focus_src_override: str = None)¶

Creates a DataLoader for iterating over canopy benchmark data.

This function prepares a DataLoader to iterate over canopy benchmark data, leveraging the CanopyBenchmarkMerger class. It is used for tasks that involve benchmarking predictions against ground truth data. The optional focus_src_override argument allows overriding the focus data source.

Parameters:

root_folder (str) – The path to the folder containing the canopy benchmark data.
num_workers (int, optional) – The number of workers to use for data loading. Default is 16.
focus_src_override (str, optional) – An optional path to override the focus data source.

Returns:

A DataLoader object that loads the canopy benchmark data in batches.

Return type:

DataLoader

leaf.metrics.canopy_evaluation_wrapper(root_folder: str = 'export', results_path: str = 'canopy_results.csv', num_workers: int = 16)¶

Wrapper for evaluating canopy predictions and computing metrics.

This function wraps the entire process of evaluating canopy predictions, including loading the prediction data, computing evaluation metrics using a CanopyEvaluator, and saving the results in a CSV file. The computation is parallelized using a thread pool to improve performance.

Parameters:

root_folder (str, optional) – The path to the folder containing the canopy prediction data. Default is ‘export’.
results_path (str, optional) – The path to the CSV file where results will be saved. Default is ‘canopy_results.csv’.
num_workers (int, optional) – The number of workers to use for data loading. Default is 16.

leaf.metrics.flat_leaves_evaluation_wrapper(root_folder: str = 'export', results_path: str = 'flat_leaves_results.csv', num_workers: int = 16)¶

Wrapper for evaluating flat leaves predictions and computing metrics.

This function wraps the entire process of evaluating flat leaves predictions, including loading the prediction data, computing evaluation metrics using a FlatLeavesEvaluator, and saving the results in a CSV file. It also uses parallel computation for efficient processing.

Parameters:

root_folder (str, optional) – The path to the folder containing the flat leaves prediction data. Default is ‘export’.
results_path (str, optional) – The path to the CSV file where results will be saved. Default is ‘flat_leaves_results.csv’.
num_workers (int, optional) – The number of workers to use for data loading. Default is 16.

leaf.metrics.focus_evaluation_wrapper(root_folder: str = 'benchmark', results_path: str = 'benchmark.csv', focus_src_override: str = None)¶

Wrapper for evaluating focus data in canopy benchmark tasks.

This function wraps the process of evaluating focus-specific data by loading the data and computing metrics with the CanopyBenchmarkEvaluator. It then logs the results in the specified CSV file.

Parameters:

root_folder (str, optional) – The path to the folder containing the benchmark focus data. Default is ‘benchmark’.
results_path (str, optional) – The path to the CSV file where results will be saved. Default is ‘benchmark.csv’.
focus_src_override (str, optional) – An optional path to override the focus data source.

leaf.metrics.test() → None¶

Test function for running the canopy evaluation.

This function configures logging and then triggers the canopy evaluation process on a specified dataset by calling the canopy_evaluation_wrapper function.

Returns:: None