solaris.nets
API reference¶
Contents
solaris.nets.callbacks
Keras-like callbacks¶
-
class
solaris.nets.callbacks.
KerasTerminateOnMetricNaN
(*args: Any, **kwargs: Any)[source]¶ Callback to stop training if a metric has value NaN or infinity.
Notes
Instantiate as you would any other keras callback. For example, to end training if a validation metric called f1_score reaches value NaN:
m = Model(inputs, outputs) m.compile() m.fit(X, y, callbacks=[TerminateOnMetricNaN('val_f1_score')])
-
checkpoint
¶ One of
['epoch', 'batch']
: Should the metric be checked at the end of every epoch (default) or every batch?- Type
str, optional
-
on_epoch_end : operations to complete at the end of each epoch.
-
on_batch_end : operations to complete at the end of each batch.
-
-
solaris.nets.callbacks.
get_callbacks
(framework, config)[source]¶ Load callbacks based on a config file for a specific framework.
Note that this function is primarily intended for use with Keras. PyTorch does not use the same object-oriented training approach as Keras, and therefore doesn’t generally have the same checkpointing objects to pass to model compilers - instead these are defined in model training code. See solaris.nets.train for examples of this. The only torch callback instantiated here is a learning rate scheduler.
- Parameters
- Returns
callbacks – A list of callbacks to pass to the compiler (Keras) or to wrap the optimizer (torch learning rate scheduling) for model training.
- Return type
-
solaris.nets.callbacks.
get_lr_schedule
(framework, config)[source]¶ Get a LR scheduling function for model training.
- Parameters
- Returns
lr_scheduler (
tensorflow.keras.callbacks.LearningRateScheduler
or)torch.optim.lr_schedule
scheduler class – A scheduler to provide during training. For Keras, this takes the form of a callback passed to the optimizer; for PyTorch, it’s a class object that wraps the optimizer. Because the torch version must wrap the optimizer, it’s not instantiated here - the class is returned instead.
-
solaris.nets.callbacks.
keras_lr_schedule
(schedule_type, initial_lr=0.001, update_frequency=1, factor=0, schedule_dict=None)[source]¶ Create a learning rate schedule for Keras from a schedule dict.
- Parameters
schedule_type (str) – Type of learning rate schedule to use. Options are:
['arbitrary', 'exponential', 'linear']
.initial_lr (float, optional) – The initial learning rate to use. Defaults to
0.001
.update_frequency (int, optional) – How frequently should learning rate be reduced (or increased)? Defaults to
1
(every epoch). Has no effect ifschedule_type='arbitrary'
.factor (float, optional) – The magnitude by which learning rate should be changed at each update. Use a positive number to increase learning rate and a negative number to decrease learning rate. See Usage for more details.
schedule_dict (dict, optional) – A dictionary with
{epoch: learning rate}
pairs. The learning rate defined in each pair will be used beginning at the specified epoch and continuing until the next highest epoch number is reached during training.
- Returns
lr_schedule (func) – a function that takes epoch number integers as an argument and returns a learning rate.
Usage
—–
schedule_type='arbitrary'
usage is documented in the arguments above.For
schedule_type='exponential'
, the following equation is applied todetermine the learning rate at each epoch
.. math:: – lr = initial_lr*e^{factor imes(floor(epoch/update_frequency))}
For
schedule_type='linear'
, the following equation is applied.. math:: – lr = initial_lr imes(1+factor imes(floor(epoch/update_frequency)))
PyTorch Callbacks.
-
class
solaris.nets.torch_callbacks.
TorchEarlyStopping
(patience=5, threshold=0.0, verbose=False)[source]¶ Tracks if model training should stop based on rate of improvement.
- Parameters
patience (int, optional) – The number of epochs to wait before stopping the model if the metric didn’t improve. Defaults to 5.
threshold (float, optional) – The minimum metric improvement required to count as “improvement”. Defaults to
0.0
(any improvement satisfies the requirement).verbose (bool, optional) – Verbose text output. Defaults to off (
False
). _NOTE_ : This currently does nothing.
-
class
solaris.nets.torch_callbacks.
TorchModelCheckpoint
(filepath='', monitor='loss', verbose=False, save_best_only=False, mode='auto', period=1, weights_only=True)[source]¶ Save the model at specific points using Keras checkpointing args.
- Parameters
filepath (str, optional) – Path to save the model file to. The end of the path (before the file extension) will have
'_[epoch]'
added to it to ID specific checkpoints.monitor (str, optional) – The loss value to monitor. Options are
['loss', 'val_loss', 'periodic']
or a metric from the keys insolaris.nets.metrics.metric_dict
. Defaults to'loss'
. If'periodic'
, it saves every n epochs (see period below).verbose (bool, optional) – Verbose text output. Defaults to
False
.save_best_only (bool, optional) – Save only the model with the best value? Defaults to no (
False
).mode (str, optional) – One of
['auto', 'min', 'max']
. Is a better value higher or lower? Defaults to'auto'
in which case it tries to infer it (ifmonitor='loss'
ormonitor='val_loss'
, it assumes'min'
, if it’s a metric it assumes'max'
.) If'min'
, it assumes lower values are better; if'max'
, it assumes higher values are better.period (int, optional) – If using
monitor='periodic'
, this saves models every period epochs. Otherwise, it sets the minimum number of epochs between checkpoints.
-
save
(model, weights_only)[source]¶ Save the model.
- Parameters
model (
torch.nn.Module
) – A PyTorch model instance to save.weights_only (bool, optional) – Should the entire model be saved, or only its weights (also known as the state_dict)? Defaults to
False
(saves entire model). The entire model must be saved to resume training without re-defining the model architecture, optimizer, and loss function.
-
class
solaris.nets.torch_callbacks.
TorchTerminateOnMetricNaN
(stopping_metric, patience=1, verbose=False)[source]¶ Sets a stop condition if a training metric achieves an NaN or inf value.
- Parameters
stopping_metric (str) – The name of the metric to stop on. The name must match a key in
solaris.nets.metrics.metric_dict
.patience (int, optional) – The number of epochs that must display an NaN loss value before stopping. Defaults to
1
.verbose (bool, optional) – Verbose text output. Defaults to off (
False
). _NOTE_ : This currently does nothing.
solaris.nets.losses
Loss functions for Geo CV model training¶
-
class
solaris.nets.losses.
TorchCompositeLoss
(*args: Any, **kwargs: Any)[source]¶ Composite loss function.
-
solaris.nets.losses.
get_loss
(framework, loss, loss_weights=None, custom_losses=None)[source]¶ Load a loss function based on a config file for the specified framework.
- Parameters
framework (string) – Which neural network framework to use.
loss (dict) – Dictionary of loss functions to use. Each key is a loss function name, and each entry is a (possibly-empty) dictionary of hyperparameter-value pairs.
loss_weights (dict, optional) – Optional dictionary of weights for loss functions. Each key is a loss function name (same as in the
loss
argument), and the corresponding entry is its weight.custom_losses (dict, optional) – Optional dictionary of Pytorch classes or Keras functions of any user-defined loss functions. Each key is a loss function name, and the corresponding entry is the Python object implementing that loss.
-
solaris.nets.losses.
keras_composite_loss
(loss_dict, weight_dict, custom_losses=None)[source]¶ Wrapper to other loss functions to create keras-compatible composite.
-
solaris.nets._keras_losses.
k_jaccard_loss
(y_true, y_pred)[source]¶ Jaccard distance for semantic segmentation.
Modified from the keras-contrib package.
-
solaris.nets._keras_losses.
k_layered_weighted_bce
(y_true, y_pred, weights)[source]¶ Binary cross-entropy function with different weights for mask channels.
y_true (tensor): passed silently by Keras during model training. y_pred (tensor): passed silently by Keras during model training. weights (list-like): Weights to assign to mask foreground pixels for each
channel in the 3rd axis of the mask.
The binary cross-entropy loss function output multiplied by a weighting mask.
See implementation instructions for weighted_bce.
This loss function is intended to allow different weighting of different segmentation outputs - for example, if a model outputs a 3D image mask, where the first channel corresponds to foreground objects and the second channel corresponds to object edges. weights must be a list of length equal to the depth of the output mask. The output mask’s “z-axis” corresponding to the mask channel must be the third axis in the output array.
-
solaris.nets._keras_losses.
k_lovasz_hinge
(per_image=False)[source]¶ Wrapper for the Lovasz Hinge Loss Function, for use in Keras.
This is a mess. I’m sorry.
-
solaris.nets._keras_losses.
k_weighted_bce
(y_true, y_pred, weight)[source]¶ Weighted binary cross-entropy for Keras.
- y_true
tf.Tensor
passed silently by Keras during model training.
- y_pred
tf.Tensor
passed silently by Keras during model training.
- weight
float
orint
Weight to assign to mask foreground pixels. Use values >1 to over-weight foreground or 0<value<1 to under-weight foreground. weight=1 is identical to vanilla binary cross-entropy.
The binary cross-entropy loss function output multiplied by a weighting mask.
Because Keras doesn’t make it easy to implement loss functions that take arguments beyond y_true and y_pred `, this function’s arguments must be partially defined before passing it into your `model.compile command. See example below, modified from ternausnet.py:
model = Model(input=inputs, output=output_layer) # defined in ternausnet.py loss_func = partial(weighted_bce, weight=loss_weight) loss_func = update_wrapper(loss_func, weighted_bce) model.compile(optimizer=Adam(), loss=loss_func)
If you wish to save and re-load a model which used this loss function, you must pass the loss function as a custom object:
model.save('path_to_your_model.hdf5') wbce_loss = partial(weighted_bce, weight=loss_weight) wbce_loss = update_wrapper(wbce_loss, weighted_bce) reloaded_model = keras.models.load_model( 'path_to_your_model.hdf5', custom_objects={'weighted_bce': wbce_loss} )
- y_true
-
solaris.nets._keras_losses.
tf_lovasz_grad
(gt_sorted)[source]¶ Code from Maxim Berman’s GitHub repo for Lovasz.
Computes gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper
-
class
solaris.nets._torch_losses.
TorchFocalLoss
(*args: Any, **kwargs: Any)[source]¶ Implementation of Focal Loss[1]_ modified from Catalyst 2 .
- Parameters
References
-
forward
(outputs, targets)[source]¶ Calculate the loss function between outputs and targets.
- Parameters
outputs (
torch.Tensor
) – The output tensor from a model.targets (
torch.Tensor
) – The training target.
- Returns
loss – The loss value.
- Return type
torch.Variable
-
solaris.nets._torch_losses.
binary_xloss
(logits, labels, ignore=None)[source]¶ - Binary Cross entropy loss
logits: [B, H, W] Variable, logits at each pixel (between -inf and +inf) labels: [B, H, W] Tensor, binary ground truth masks (0 or 1) ignore: void class id
-
solaris.nets._torch_losses.
flatten_binary_scores
(scores, labels, ignore=None)[source]¶ Flattens predictions in the batch (binary case) Remove labels equal to ‘ignore’
-
solaris.nets._torch_losses.
iou
(preds, labels, C, EMPTY=1.0, ignore=None, per_image=False)[source]¶ Array of IoU for each (non ignored) class
-
solaris.nets._torch_losses.
iou_binary
(preds, labels, EMPTY=1.0, ignore=None, per_image=True)[source]¶ IoU for foreground class binary: 1 foreground, 0 background
-
solaris.nets._torch_losses.
lovasz_grad
(gt_sorted)[source]¶ Computes gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper
-
solaris.nets._torch_losses.
lovasz_hinge_flat
(logits, labels)[source]¶ Binary Lovasz hinge loss.
- Parameters
logits (
torch.Variable
) – Logits at each prediction (between -inf and +inf)labels (
torch.Tensor
) – binary ground truth labels (0 or 1)
- Returns
loss – Lovasz loss value for the input logits and labels.
- Return type
torch.Variable
-
solaris.nets._torch_losses.
mean
(l, ignore_nan=False, empty=0)[source]¶ nanmean compatible with generators.
-
solaris.nets._torch_losses.
torch_lovasz_hinge
(logits, labels, per_image=False, ignore=None)[source]¶ Lovasz Hinge Loss. Implementation edited from Maxim Berman’s GitHub.
References
https://github.com/bermanmaxim/LovaszSoftmax/ https://arxiv.org/abs/1705.08790
- Parameters
logits (
torch.Variable
) – logits at each pixel (between -inf and +inf)labels (
torch.Tensor
) – binary ground truth masks (0 or 1)per_image (bool, optional) – compute the loss per image instead of per batch. Defaults to
False
.ignore (optional void class id.) –
- Returns
loss – Lovasz loss value for the input logits and labels. Compatible with
loss.backward()
as its atorch.Variable
.- Return type
torch.Variable
solaris.nets.transform
Augmentation pipeline prep for Geo imagery¶
Image transformation, augmentation, etc. for use in models.¶
Where possible, the codebase uses albumentations implementations for transforms because they checked various different implementations and use the fastest one. However, in some cases albumentations uses a cv2 backend, which is incompatible with unusual channel counts in imagery, and therefore other implementations are used for those functions here.
Note: Some augmentations are unavailable in this library.
Functionality used directly from albumentations: - Crop - VerticalFlip - HorizontalFlip - Flip - Transpose - Resize - CenterCrop - RandomCrop - RandomSizedCrop - OpticalDistortion - GridDistortion - ElasticTransform - Normalize - HueSaturationValue # NOTE: CAN ONLY HANDLE RGB 3-CHANNEL! - RGBShift # NOTE: CAN ONLY HANDLE RGB 3-CHANNEL! - RandomRotate90 - RandomBrightnessContrast - Blur - MotionBlur - MedianBlur - GaussNoise - CLAHE - RandomGamma - ToFloat - NoOp - PadIfNeeded
Implemented here: - Rotate - RandomScale - Cutout
-
class
solaris.nets.transform.
RandomScale
(*args: Any, **kwargs: Any)[source]¶ Randomly resize the input array in X and Y.
- Parameters
scale_limit (
(float, float)
tuple or float) – Limit to the amount of scaling to perform on the image. If provided as a tuple, the limits are[shape*scale_limit[0], shape*scale_limit[1]]
. If only a single vaue is passed, this is converted to a tuple by converting to(1-scale_limit, 1+scale_limit)
, i.e.scale_limit=0.2
is equivalent toscale_limit=(0.8, 1.2)
.axis (str, optional) – Which axis should be rescaled? Options are
['width', 'height', 'both'].
interpolation (str, optional) – Interpolation method to use for resizing. One of
['bilinear', 'bicubic', 'lanczos', 'nearest', or 'hamming']
. Defaults to'bicubic'
. See the Pillow_ documentation for more information.always_apply (bool, optional) – Apply this transformation to every image? Defaults to no (
False
).p (float [0, 1], optional) – Probability that the augmentation is performed to each image. Defaults to
0.5
._ (.) –
-
class
solaris.nets.transform.
Rotate
(*args: Any, **kwargs: Any)[source]¶ Array rotation using scipy.ndimage’s implementation.
- Parameters
limit (
[int, int]
orint
) – Range from which a random angle is picked. If only a single int is provided, an angle is picked from range(-angle, angle)border_mode (str, optional) – One of
['reflect', 'nearest', 'constant', 'wrap']
. Defaults to'reflect'
. Seescipy.ndimage.interpolation.rotate()
mode
argument.cval (int or float, optional) – constant value to fill borders with if
border_mode=='constant'
. Defaults to 0.always_apply (bool, optional) – Apply this transformation to every image? Defaults to no (
False
).p (float [0, 1], optional) – Probability that the augmentation is performed to each image. Defaults to
0.5
.
-
class
solaris.nets.transform.
SwapChannels
(*args: Any, **kwargs: Any)[source]¶ Swap channels in an input image.
- Parameters
first_idx (int) – The first channel in the pair to swap.
second_idx (int) – The second channel in the pair to swap.
axis (int, optional (default: 1)) – The axis to drop the channel from. Defaults to
0
(torch channel axis). Set to2
for TF models where the channel is the last axis of an image.always_apply (bool, optional (default: False)) – Apply this transformation to every image? Defaults to no (
False
).p (float [0, 1], optional (default: 1.0)) – Probability that the augmentation is performed to each image. Defaults to
1.0
.
-
solaris.nets.transform.
build_pipeline
(config)[source]¶ Create train and val augmentation pipelines from a config object.
- Parameters
config (dict) – A configuration dictionary created by parsing a .yaml config file. See documentation to the project.
- Returns
Two
albumentations.core.composition.Compose
instances with the entireaugmentation pipeline assembled (one for training and one for validation/)
inference.
-
solaris.nets.transform.
get_augs
(aug_dict, meta_augs_list=['oneof', 'oneorother'])[source]¶ Get the set of augmentations contained in a dict.
- aug_dictdict
The
'augmentations'
sub-dict of a'training_augmentation'
or'validation_augmentation'
item in the'config'
object. sub-dict from theconfig
object.- meta_augs_listdict, optional
The list of augmentation names that correspond to “meta-augmentations” in all lowercase (e.g.
oneof
,oneorother
). This will be used to find augmentation dictionary items that need further parsing.
- Returns
list of augmentations to pass to a
Compose
object.- Return type
-
solaris.nets.transform.
process_aug_dict
(pipeline_dict, meta_augs_list=['oneof', 'oneorother'])[source]¶ Create a Compose object from an augmentation config dict.
Notes
See the documentation for instructions on formatting the config .yaml to enable utilization by get_augs.
- Parameters
aug_dict (dict) – The
'training_augmentation'
or'validation_augmentation'
sub-dict from theconfig
object.meta_augs_list (dict, optional) – The list of augmentation names that correspond to “meta-augmentations” in all lowercase (e.g.
oneof
,oneorother
). This will be used to find augmentation dictionary items that need further parsing.
- Returns
The composed augmentation pipeline.
- Return type
Compose
instance
solaris.nets.optimizers
Model training optimizer management¶
Wrappers for training optimizers.
-
class
solaris.nets.optimizers.
TorchAdamW
(*args: Any, **kwargs: Any)[source]¶ AdamW algorithm as implemented in Torch_AdamW.
The original Adam algorithm was proposed in Adam: A Method for Stochastic Optimization. The AdamW variant was proposed in Decoupled Weight Decay Regularization. :param params: iterable of parameters to optimize or dicts defining
parameter groups
- Parameters
lr (float, optional) – learning rate (default: 1e-3)
betas (Tuple[float, float], optional) – coefficients used for computing running averages of gradient and its square (default: (0.9, 0.999))
eps (float, optional) – term added to the denominator to improve numerical stability (default: 1e-8)
weight_decay (float, optional) – weight decay coefficient (default: 1e-2)
amsgrad (boolean, optional) – whether to use the AMSGrad variant of this algorithm from the paper On the Convergence of Adam and Beyond (default: False)
solaris.nets.model_io
Model I/O and model weight management¶
-
solaris.nets.model_io.
get_model
(model_name, framework, model_path=None, pretrained=False, custom_model_dict=None, num_classes=1)[source]¶ Load a model from a file based on its name.
-
solaris.nets.model_io.
reset_weights
(model, framework)[source]¶ Re-initialize model weights for training.
- Parameters
model (
tensorflow.keras.Model
ortorch.nn.Module
) – A pre-trained, compiled model with weights saved.framework (str) – The deep learning framework used. Currently valid options are
['torch', 'keras']
.
- Returns
reinit_model – The model with weights re-initialized. Note this model object will also lack an optimizer, loss function, etc., which will need to be added.
- Return type
model object
solaris.nets.datagen
Data generators for model training¶
-
class
solaris.nets.datagen.
InferenceTiler
(framework, width, height, x_step=None, y_step=None, augmentations=None)[source]¶ An object to tile fragments of images for inference.
This object allows you to pass images of arbitrary size into Solaris for inference, similar to the pre-existing CosmiQ Works tool, BASISS. The object will step across an input image creating tiles of size
[height, width]
, taking steps of size[y_step, x_step]
as it goes. When it reaches an edge, it will take tiles from-height
or-width
to the edge. Clearly, these can overlap with one another; the intention is that overlaps will be resolved usingsolaris.raster.image.stitch_images()
when re-creating the output.-
aug
¶ Augmentations to apply before passing to a neural net. Generally used for pre-processing.
- Type
albumentations.core.composition.Compose
-
-
class
solaris.nets.datagen.
KerasSegmentationSequence
(*args: Any, **kwargs: Any)[source]¶ An object to stream images from files into a Keras model in solaris.
-
df
¶ The
pandas.DataFrame
specifying where inputs are stored.- Type
-
aug
¶ An albumentations Compose object to pass imagery through before passing it into the neural net. If an augmentation config subdict was provided during initialization, this is created by parsing the dict with
solaris.nets.transform.process_aug_dict()
.- Type
albumentations.core.composition.Compose
-
n_batches
¶ The number of batches per epoch. Inferred based on the number of input files in df and batch_size.
- Type
-
-
class
solaris.nets.datagen.
TorchDataset
(*args: Any, **kwargs: Any)[source]¶ A PyTorch dataset object for solaris.
Note that this object is wrapped in a
torch.utils.data.DataLoader
before being passed to the :class:solaris.nets.train.Trainer` instance.-
df
¶ The
pandas.DataFrame
specifying where inputs are stored.- Type
-
aug
¶ An albumentations Compose object to pass imagery through before passing it into the neural net. If an augmentation config subdict was provided during initialization, this is created by parsing the dict with
solaris.nets.transform.process_aug_dict()
.- Type
albumentations.core.composition.Compose
-
n_batches
¶ The number of batches per epoch. Inferred based on the number of input files in df and batch_size.
- Type
-
dtype
¶ The numpy dtype that image inputs should be when passed to the model.
- Type
-
dtype
¶ The data type images should be converted to before being passed to neural nets.
- Type
class:numpy.dtype
-
-
solaris.nets.datagen.
make_data_generator
(framework, config, df, stage='train')[source]¶ Create an appropriate data generator based on the framework used.
A wrapper for the high-end
solaris
API to create data generators. Using theconfig
dictionary, this function creates an instance of eitherKerasSegmentationSequence
orTorchDataset
(depending on the framework used for the pipeline). If using Torch, this instance is then wrapped in atorch.utils.data.DataLoader
and returned; if Keras, the sequence object is directly returned.- Parameters
framework (str) – One of [‘keras’, ‘pytorch’, ‘simrdwn’, ‘tf’, ‘tf_obj_api’], the deep learning framework used for the model to be used.
config (dict) – The config dictionary for the entire pipeline.
df (
pandas.DataFrame
orstr
) – Apandas.DataFrame
containing two columns:'image'
, with the path to images for training, and'label'
, with the path to the label file corresponding to each image.stage (str, optional) – Either
'train'
or'validate'
, indicates whether the object created is being used for training or validation. This determines which augmentations from the config file are applied within the returned object.
- Returns
data_gen – An object to pass data into the
solaris.nets.train.Trainer
instance during model training.- Return type
See also
solaris.nets.metrics
Metrics for evaluating model performance¶
-
solaris.nets.metrics.
dice_coef_binary
(y_true, y_pred, smooth=1e-07)[source]¶ Dice coefficient for 2 categories. Ignores background pixel label 0 Pass to model as metric during compile statement
-
solaris.nets.metrics.
f1_score
(y_true, y_pred)[source]¶ F1 score for foreground pixels ONLY.
Calculates pixelwise F1 score for the foreground pixels (mask value == 1). Returns NaN if the model does not identify any foreground pixels in the image.
-
solaris.nets.metrics.
get_metrics
(framework, config)[source]¶ Load model training metrics from a config file for a specific framework.
solaris.nets.train
Model training functionality¶
Training code for solaris models.
-
class
solaris.nets.train.
Trainer
(config, custom_model_dict=None, custom_losses=None)[source]¶ Object for training solaris models using PyTorch or Keras.
-
solaris.nets.train.
get_train_val_dfs
(config)[source]¶ Get the training and validation dfs based on the contents of
config
.This function uses the logic described in the documentation for the config files to determine where to find training and validation dataset files. See the docs and the comments in solaris/data/config_skeleton.yml for details.
solaris.nets.infer
Prediction with Geo CV models¶
-
class
solaris.nets.infer.
Inferer
(config, custom_model_dict=None)[source]¶ Object for training solaris models using PyTorch or Keras.
-
solaris.nets.infer.
get_infer_df
(config)[source]¶ Get the inference df based on the contents of
config
. This function uses the logic described in the documentation for the config file to determine where to find images to be used for inference. See the docs and the comments in solaris/data/config_skeleton.yml for details. :param config: The loaded configuration dict for model training and/or inference. :type config: dict