`solaris.nets` API reference ¶

Contents

solaris.nets API reference

`solaris.nets.callbacks` Keras-like callbacks ¶

class solaris.nets.callbacks.KerasTerminateOnMetricNaN(*args: Any, **kwargs: Any)[source]¶

Callback to stop training if a metric has value NaN or infinity.

Notes

Instantiate as you would any other keras callback. For example, to end training if a validation metric called f1_score reaches value NaN:

m = Model(inputs, outputs)
m.compile()
m.fit(X, y, callbacks=[TerminateOnMetricNaN('val_f1_score')])

metric¶

Name of the metric being monitored.

Type: str, optional

checkpoint¶

One of ['epoch', 'batch']: Should the metric be checked at the end of every epoch (default) or every batch?

Type: str, optional

on_epoch_end : operations to complete at the end of each epoch.

on_batch_end : operations to complete at the end of each batch.

solaris.nets.callbacks.get_callbacks(framework, config)[source]¶

Load callbacks based on a config file for a specific framework.

Note that this function is primarily intended for use with Keras. PyTorch does not use the same object-oriented training approach as Keras, and therefore doesn’t generally have the same checkpointing objects to pass to model compilers - instead these are defined in model training code. See solaris.nets.train for examples of this. The only torch callback instantiated here is a learning rate scheduler.

Parameters

framework (str) – Deep learning framework used for the model. Options are ['keras', 'torch'] .
config (dict) – Configuration dict generated from the YAML config file.

Returns

callbacks – A list of callbacks to pass to the compiler (Keras) or to wrap the optimizer (torch learning rate scheduling) for model training.

Return type

list

solaris.nets.callbacks.get_lr_schedule(framework, config)[source]¶

Get a LR scheduling function for model training.

Parameters

framework (str) – Deep learning framework used for the model. Options are ['keras', 'torch'] .
config (dict) – Configuration dict generated from the YAML config file.

Returns

lr_scheduler (tensorflow.keras.callbacks.LearningRateScheduler or)
torch.optim.lr_schedule scheduler class – A scheduler to provide during training. For Keras, this takes the form of a callback passed to the optimizer; for PyTorch, it’s a class object that wraps the optimizer. Because the torch version must wrap the optimizer, it’s not instantiated here - the class is returned instead.

solaris.nets.callbacks.keras_lr_schedule(schedule_type, initial_lr=0.001, update_frequency=1, factor=0, schedule_dict=None)[source]¶

Create a learning rate schedule for Keras from a schedule dict.

Parameters

schedule_type (str) – Type of learning rate schedule to use. Options are: ['arbitrary', 'exponential', 'linear'] .
initial_lr (float, optional) – The initial learning rate to use. Defaults to 0.001 .
update_frequency (int, optional) – How frequently should learning rate be reduced (or increased)? Defaults to 1 (every epoch). Has no effect if schedule_type='arbitrary'.
factor (float, optional) – The magnitude by which learning rate should be changed at each update. Use a positive number to increase learning rate and a negative number to decrease learning rate. See Usage for more details.
schedule_dict (dict, optional) – A dictionary with {epoch: learning rate} pairs. The learning rate defined in each pair will be used beginning at the specified epoch and continuing until the next highest epoch number is reached during training.

Returns

lr_schedule (func) – a function that takes epoch number integers as an argument and returns a learning rate.
Usage
—–
schedule_type='arbitrary' usage is documented in the arguments above.
For schedule_type='exponential', the following equation is applied to
determine the learning rate at each epoch
.. math:: – lr = initial_lr*e^{factor imes(floor(epoch/update_frequency))}
For schedule_type='linear', the following equation is applied
.. math:: – lr = initial_lr imes(1+factor imes(floor(epoch/update_frequency)))

PyTorch Callbacks.

class solaris.nets.torch_callbacks.TorchEarlyStopping(patience=5, threshold=0.0, verbose=False)[source]¶

Tracks if model training should stop based on rate of improvement.

Parameters

patience (int, optional) – The number of epochs to wait before stopping the model if the metric didn’t improve. Defaults to 5.
threshold (float, optional) – The minimum metric improvement required to count as “improvement”. Defaults to 0.0 (any improvement satisfies the requirement).
verbose (bool, optional) – Verbose text output. Defaults to off (False). _NOTE_ : This currently does nothing.

class solaris.nets.torch_callbacks.TorchModelCheckpoint(filepath='', monitor='loss', verbose=False, save_best_only=False, mode='auto', period=1, weights_only=True)[source]¶

Save the model at specific points using Keras checkpointing args.

Parameters

filepath (str, optional) – Path to save the model file to. The end of the path (before the file extension) will have '_[epoch]' added to it to ID specific checkpoints.
monitor (str, optional) – The loss value to monitor. Options are ['loss', 'val_loss', 'periodic'] or a metric from the keys in solaris.nets.metrics.metric_dict . Defaults to 'loss' . If 'periodic', it saves every n epochs (see period below).
verbose (bool, optional) – Verbose text output. Defaults to False .
save_best_only (bool, optional) – Save only the model with the best value? Defaults to no (False ).
mode (str, optional) – One of ['auto', 'min', 'max']. Is a better value higher or lower? Defaults to 'auto' in which case it tries to infer it (if monitor='loss' or monitor='val_loss' , it assumes 'min' , if it’s a metric it assumes 'max' .) If 'min', it assumes lower values are better; if 'max' , it assumes higher values are better.
period (int, optional) – If using monitor='periodic' , this saves models every period epochs. Otherwise, it sets the minimum number of epochs between checkpoints.

check_is_best_value(value)[source]¶: Check if value is better than the best stored value.

save(model, weights_only)[source]¶

Save the model.

Parameters

model (torch.nn.Module) – A PyTorch model instance to save.
weights_only (bool, optional) – Should the entire model be saved, or only its weights (also known as the state_dict)? Defaults to False (saves entire model). The entire model must be saved to resume training without re-defining the model architecture, optimizer, and loss function.

class solaris.nets.torch_callbacks.TorchTerminateOnMetricNaN(stopping_metric, patience=1, verbose=False)[source]¶

Sets a stop condition if a training metric achieves an NaN or inf value.

Parameters

stopping_metric (str) – The name of the metric to stop on. The name must match a key in solaris.nets.metrics.metric_dict .
patience (int, optional) – The number of epochs that must display an NaN loss value before stopping. Defaults to 1.
verbose (bool, optional) – Verbose text output. Defaults to off (False). _NOTE_ : This currently does nothing.

class solaris.nets.torch_callbacks.TorchTerminateOnNaN(patience=1, verbose=False)[source]¶

Sets a stop condition if the model loss achieves an NaN or inf value.

Parameters

patience (int, optional) – The number of epochs that must display an NaN loss value before stopping. Defaults to 1.
verbose (bool, optional) – Verbose text output. Defaults to off (False). _NOTE_ : This currently does nothing.

`solaris.nets.losses` Loss functions for Geo CV model training ¶

class solaris.nets.losses.TorchCompositeLoss(*args: Any, **kwargs: Any)[source]¶: Composite loss function.

solaris.nets.losses.get_loss(framework, loss, loss_weights=None, custom_losses=None)[source]¶

Load a loss function based on a config file for the specified framework.

Parameters

framework (string) – Which neural network framework to use.
loss (dict) – Dictionary of loss functions to use. Each key is a loss function name, and each entry is a (possibly-empty) dictionary of hyperparameter-value pairs.
loss_weights (dict, optional) – Optional dictionary of weights for loss functions. Each key is a loss function name (same as in the loss argument), and the corresponding entry is its weight.
custom_losses (dict, optional) – Optional dictionary of Pytorch classes or Keras functions of any user-defined loss functions. Each key is a loss function name, and the corresponding entry is the Python object implementing that loss.

solaris.nets.losses.keras_composite_loss(loss_dict, weight_dict, custom_losses=None)[source]¶: Wrapper to other loss functions to create keras-compatible composite.

solaris.nets._keras_losses.k_jaccard_loss(y_true, y_pred)[source]¶

Jaccard distance for semantic segmentation.

Modified from the keras-contrib package.

solaris.nets._keras_losses.k_layered_weighted_bce(y_true, y_pred, weights)[source]¶

Binary cross-entropy function with different weights for mask channels.

y_true (tensor): passed silently by Keras during model training. y_pred (tensor): passed silently by Keras during model training. weights (list-like): Weights to assign to mask foreground pixels for each

channel in the 3rd axis of the mask.

The binary cross-entropy loss function output multiplied by a weighting mask.

See implementation instructions for weighted_bce.

This loss function is intended to allow different weighting of different segmentation outputs - for example, if a model outputs a 3D image mask, where the first channel corresponds to foreground objects and the second channel corresponds to object edges. weights must be a list of length equal to the depth of the output mask. The output mask’s “z-axis” corresponding to the mask channel must be the third axis in the output array.

solaris.nets._keras_losses.k_lovasz_hinge(per_image=False)[source]¶

Wrapper for the Lovasz Hinge Loss Function, for use in Keras.

This is a mess. I’m sorry.

solaris.nets._keras_losses.k_weighted_bce(y_true, y_pred, weight)[source]¶

Weighted binary cross-entropy for Keras.

y_truetf.Tensor: passed silently by Keras during model training.
y_predtf.Tensor: passed silently by Keras during model training.
weightfloat or int: Weight to assign to mask foreground pixels. Use values >1 to over-weight foreground or 0<value<1 to under-weight foreground. weight=1 is identical to vanilla binary cross-entropy.

The binary cross-entropy loss function output multiplied by a weighting mask.

Because Keras doesn’t make it easy to implement loss functions that take arguments beyond y_true and y_pred `, this function’s arguments must be partially defined before passing it into your `model.compile command. See example below, modified from ternausnet.py:

model = Model(input=inputs, output=output_layer) # defined in ternausnet.py

loss_func = partial(weighted_bce, weight=loss_weight)
loss_func = update_wrapper(loss_func, weighted_bce)

model.compile(optimizer=Adam(), loss=loss_func)

If you wish to save and re-load a model which used this loss function, you must pass the loss function as a custom object:

model.save('path_to_your_model.hdf5')
wbce_loss = partial(weighted_bce, weight=loss_weight)
wbce_loss = update_wrapper(wbce_loss, weighted_bce)
reloaded_model = keras.models.load_model(
    'path_to_your_model.hdf5', custom_objects={'weighted_bce': wbce_loss}
    )

solaris.nets._keras_losses.tf_lovasz_grad(gt_sorted)[source]¶

Code from Maxim Berman’s GitHub repo for Lovasz.

Computes gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper

class solaris.nets._torch_losses.TorchDiceLoss(*args: Any, **kwargs: Any)[source]¶

class solaris.nets._torch_losses.TorchFocalLoss(*args: Any, **kwargs: Any)[source]¶

Implementation of Focal Loss[1]_ modified from Catalyst 2 .

Parameters

gamma (int or float) – Focusing parameter. See 1 .
alpha (int or float) – Normalization factor. See 1 .

References

1(1,2): https://arxiv.org/pdf/1708.02002.pdf
2: https://catalyst-team.github.io/catalyst/

forward(outputs, targets)[source]¶

Calculate the loss function between outputs and targets.

Parameters

outputs (torch.Tensor) – The output tensor from a model.
targets (torch.Tensor) – The training target.

Returns

loss – The loss value.

Return type

torch.Variable

class solaris.nets._torch_losses.TorchJaccardLoss(*args: Any, **kwargs: Any)[source]¶

class solaris.nets._torch_losses.TorchStableBCELoss(*args: Any, **kwargs: Any)[source]¶

solaris.nets._torch_losses.binary_xloss(logits, labels, ignore=None)[source]¶

Binary Cross entropy loss: logits: [B, H, W] Variable, logits at each pixel (between -inf and +inf) labels: [B, H, W] Tensor, binary ground truth masks (0 or 1) ignore: void class id

solaris.nets._torch_losses.flatten_binary_scores(scores, labels, ignore=None)[source]¶: Flattens predictions in the batch (binary case) Remove labels equal to ‘ignore’

solaris.nets._torch_losses.iou(preds, labels, C, EMPTY=1.0, ignore=None, per_image=False)[source]¶: Array of IoU for each (non ignored) class

solaris.nets._torch_losses.iou_binary(preds, labels, EMPTY=1.0, ignore=None, per_image=True)[source]¶: IoU for foreground class binary: 1 foreground, 0 background

solaris.nets._torch_losses.lovasz_grad(gt_sorted)[source]¶: Computes gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper

solaris.nets._torch_losses.lovasz_hinge_flat(logits, labels)[source]¶

Binary Lovasz hinge loss.

Parameters

logits (torch.Variable) – Logits at each prediction (between -inf and +inf)
labels (torch.Tensor) – binary ground truth labels (0 or 1)

Returns

loss – Lovasz loss value for the input logits and labels.

Return type

torch.Variable

solaris.nets._torch_losses.mean(l, ignore_nan=False, empty=0)[source]¶: nanmean compatible with generators.

solaris.nets._torch_losses.torch_lovasz_hinge(logits, labels, per_image=False, ignore=None)[source]¶

Lovasz Hinge Loss. Implementation edited from Maxim Berman’s GitHub.

References

https://github.com/bermanmaxim/LovaszSoftmax/ https://arxiv.org/abs/1705.08790

Parameters

logits (torch.Variable) – logits at each pixel (between -inf and +inf)
labels (torch.Tensor) – binary ground truth masks (0 or 1)
per_image (bool, optional) – compute the loss per image instead of per batch. Defaults to False.
ignore (optional void class id.) –

Returns

loss – Lovasz loss value for the input logits and labels. Compatible with loss.backward() as its a torch.Variable .

Return type

torch.Variable

`solaris.nets.transform` Augmentation pipeline prep for Geo imagery ¶

Image transformation, augmentation, etc. for use in models.¶

Where possible, the codebase uses albumentations implementations for transforms because they checked various different implementations and use the fastest one. However, in some cases albumentations uses a cv2 backend, which is incompatible with unusual channel counts in imagery, and therefore other implementations are used for those functions here.

Note: Some augmentations are unavailable in this library.

Functionality used directly from albumentations: - Crop - VerticalFlip - HorizontalFlip - Flip - Transpose - Resize - CenterCrop - RandomCrop - RandomSizedCrop - OpticalDistortion - GridDistortion - ElasticTransform - Normalize - HueSaturationValue # NOTE: CAN ONLY HANDLE RGB 3-CHANNEL! - RGBShift # NOTE: CAN ONLY HANDLE RGB 3-CHANNEL! - RandomRotate90 - RandomBrightnessContrast - Blur - MotionBlur - MedianBlur - GaussNoise - CLAHE - RandomGamma - ToFloat - NoOp - PadIfNeeded

Implemented here: - Rotate - RandomScale - Cutout

class solaris.nets.transform.RandomScale(*args: Any, **kwargs: Any)[source]¶

Randomly resize the input array in X and Y.

Parameters

scale_limit ((float, float) tuple or float) – Limit to the amount of scaling to perform on the image. If provided as a tuple, the limits are [shape*scale_limit[0], shape*scale_limit[1]]. If only a single vaue is passed, this is converted to a tuple by converting to (1-scale_limit, 1+scale_limit), i.e. scale_limit=0.2 is equivalent to scale_limit=(0.8, 1.2).
axis (str, optional) – Which axis should be rescaled? Options are ['width', 'height', 'both'].
interpolation (str, optional) – Interpolation method to use for resizing. One of ['bilinear', 'bicubic', 'lanczos', 'nearest', or 'hamming']. Defaults to 'bicubic'. See the Pillow_ documentation for more information.
always_apply (bool, optional) – Apply this transformation to every image? Defaults to no (False).
p (float [0, 1], optional) – Probability that the augmentation is performed to each image. Defaults to 0.5.
_ (.) –

class solaris.nets.transform.Rotate(*args: Any, **kwargs: Any)[source]¶

Array rotation using scipy.ndimage’s implementation.

Parameters

limit ([int, int] or int) – Range from which a random angle is picked. If only a single int is provided, an angle is picked from range(-angle, angle)
border_mode (str, optional) – One of ['reflect', 'nearest', 'constant', 'wrap']. Defaults to 'reflect'. See scipy.ndimage.interpolation.rotate() mode argument.
cval (int or float, optional) – constant value to fill borders with if border_mode=='constant'. Defaults to 0.
always_apply (bool, optional) – Apply this transformation to every image? Defaults to no (False).
p (float [0, 1], optional) – Probability that the augmentation is performed to each image. Defaults to 0.5.

class solaris.nets.transform.SwapChannels(*args: Any, **kwargs: Any)[source]¶

Swap channels in an input image.

Parameters

first_idx (int) – The first channel in the pair to swap.
second_idx (int) – The second channel in the pair to swap.
axis (int, optional (default: 1)) – The axis to drop the channel from. Defaults to 0 (torch channel axis). Set to 2 for TF models where the channel is the last axis of an image.
always_apply (bool, optional (default: False)) – Apply this transformation to every image? Defaults to no (False).
p (float [0, 1], optional (default: 1.0)) – Probability that the augmentation is performed to each image. Defaults to 1.0.

solaris.nets.transform.build_pipeline(config)[source]¶

Create train and val augmentation pipelines from a config object.

Parameters

config (dict) – A configuration dictionary created by parsing a .yaml config file. See documentation to the project.

Returns

Two albumentations.core.composition.Compose instances with the entire
augmentation pipeline assembled (one for training and one for validation/)
inference.

solaris.nets.transform.get_augs(aug_dict, meta_augs_list=['oneof', 'oneorother'])[source]¶

Get the set of augmentations contained in a dict.

aug_dictdict: The 'augmentations' sub-dict of a 'training_augmentation' or 'validation_augmentation' item in the 'config' object. sub-dict from the config object.
meta_augs_listdict, optional: The list of augmentation names that correspond to “meta-augmentations” in all lowercase (e.g. oneof, oneorother). This will be used to find augmentation dictionary items that need further parsing.

Returns: list of augmentations to pass to a Compose object.
Return type: list

solaris.nets.transform.process_aug_dict(pipeline_dict, meta_augs_list=['oneof', 'oneorother'])[source]¶

Create a Compose object from an augmentation config dict.

Notes

See the documentation for instructions on formatting the config .yaml to enable utilization by get_augs.

Parameters

aug_dict (dict) – The 'training_augmentation' or 'validation_augmentation' sub-dict from the config object.
meta_augs_list (dict, optional) – The list of augmentation names that correspond to “meta-augmentations” in all lowercase (e.g. oneof, oneorother). This will be used to find augmentation dictionary items that need further parsing.

Returns

The composed augmentation pipeline.

Return type

Compose instance

`solaris.nets.optimizers` Model training optimizer management ¶

Wrappers for training optimizers.

class solaris.nets.optimizers.TorchAdamW(*args: Any, **kwargs: Any)[source]¶

AdamW algorithm as implemented in Torch_AdamW.

The original Adam algorithm was proposed in Adam: A Method for Stochastic Optimization. The AdamW variant was proposed in Decoupled Weight Decay Regularization. :param params: iterable of parameters to optimize or dicts defining

parameter groups

Parameters

lr (float, optional) – learning rate (default: 1e-3)
betas (Tuple[float, float], optional) – coefficients used for computing running averages of gradient and its square (default: (0.9, 0.999))
eps (float, optional) – term added to the denominator to improve numerical stability (default: 1e-8)
weight_decay (float, optional) – weight decay coefficient (default: 1e-2)
amsgrad (boolean, optional) – whether to use the AMSGrad variant of this algorithm from the paper On the Convergence of Adam and Beyond (default: False)

step(closure=None)[source]¶

Performs a single optimization step. :param closure: A closure that reevaluates the model

and returns the loss.

solaris.nets.optimizers.get_optimizer(framework, config)[source]¶

Get the optimizer specified in config for model training.

Parameters

framework (str) – Name of the deep learning framework used. Current options are ['torch', 'keras'].
config (dict) – The config dict generated from the YAML config file.

Returns

Return type

An optimizer object for the specified deep learning framework.

`solaris.nets.model_io` Model I/O and model weight management ¶

solaris.nets.model_io.get_model(model_name, framework, model_path=None, pretrained=False, custom_model_dict=None, num_classes=1)[source]¶: Load a model from a file based on its name.

solaris.nets.model_io.reset_weights(model, framework)[source]¶

Re-initialize model weights for training.

Parameters

model (tensorflow.keras.Model or torch.nn.Module) – A pre-trained, compiled model with weights saved.
framework (str) – The deep learning framework used. Currently valid options are ['torch', 'keras'] .

Returns

reinit_model – The model with weights re-initialized. Note this model object will also lack an optimizer, loss function, etc., which will need to be added.

Return type

model object

`solaris.nets.datagen` Data generators for model training ¶

class solaris.nets.datagen.InferenceTiler(framework, width, height, x_step=None, y_step=None, augmentations=None)[source]¶

An object to tile fragments of images for inference.

This object allows you to pass images of arbitrary size into Solaris for inference, similar to the pre-existing CosmiQ Works tool, BASISS. The object will step across an input image creating tiles of size [height, width], taking steps of size [y_step, x_step] as it goes. When it reaches an edge, it will take tiles from -height or -width to the edge. Clearly, these can overlap with one another; the intention is that overlaps will be resolved using solaris.raster.image.stitch_images() when re-creating the output.

framework¶

The deep learning framework used. Can be one of "torch", "pytorch", or "keras".

Type: str

width¶

The width of images to load into the neural net.

Type: int

height¶

The height of images to load into the neural net.

Type: int

x_step¶

The step size taken in the x direction when sampling for new images.

Type: int, optional

y_step¶

The step size taken in the y direction when sampling for new images.

Type: int, optional

aug¶

Augmentations to apply before passing to a neural net. Generally used for pre-processing.

Type: albumentations.core.composition.Compose

`solaris.nets.metrics` Metrics for evaluating model performance ¶

solaris.nets.metrics.dice_coef_binary(y_true, y_pred, smooth=1e-07)[source]¶: Dice coefficient for 2 categories. Ignores background pixel label 0 Pass to model as metric during compile statement

solaris.nets.metrics.f1_score(y_true, y_pred)[source]¶

F1 score for foreground pixels ONLY.

Calculates pixelwise F1 score for the foreground pixels (mask value == 1). Returns NaN if the model does not identify any foreground pixels in the image.

solaris.nets.metrics.get_metrics(framework, config)[source]¶: Load model training metrics from a config file for a specific framework.

solaris.nets.metrics.precision(y_true, y_pred)[source]¶

Precision for foreground pixels.

Calculates pixelwise precision TP/(TP + FP).

solaris.nets.metrics.recall(y_true, y_pred)[source]¶

Precision for foreground pixels.

Calculates pixelwise recall TP/(TP + FN).

`solaris.nets.zoo` Model definitions for geospatial image analysis ¶

`solaris.nets.train` Model training functionality ¶

Training code for solaris models.

class solaris.nets.train.Trainer(config, custom_model_dict=None, custom_losses=None)[source]¶

Object for training solaris models using PyTorch or Keras.

initialize_model()[source]¶: Load in and create all model training elements.

save_model()[source]¶: Save the final model output.

train()[source]¶: Run training on the model.

solaris.nets.train.get_train_val_dfs(config)[source]¶

Get the training and validation dfs based on the contents of config.

This function uses the logic described in the documentation for the config files to determine where to find training and validation dataset files. See the docs and the comments in solaris/data/config_skeleton.yml for details.

Parameters: config (dict) – The loaded configuration dict for model training and/or inference.
Returns: train_df, val_df – dict s containing two columns: 'image' and 'label'. Each column corresponds to paths to find matching image and label files for training.
Return type: tuple of dict s

`solaris.nets.infer` Prediction with Geo CV models ¶

class solaris.nets.infer.Inferer(config, custom_model_dict=None)[source]¶: Object for training solaris models using PyTorch or Keras.

solaris.nets.infer.get_infer_df(config)[source]¶

Get the inference df based on the contents of config . This function uses the logic described in the documentation for the config file to determine where to find images to be used for inference. See the docs and the comments in solaris/data/config_skeleton.yml for details. :param config: The loaded configuration dict for model training and/or inference. :type config: dict

Returns: infer_df – dict containing at least one column: 'image' . The values in this column correspond to the path to filenames to perform inference on.
Return type: dict