`solaris.utils` API reference ¶

Contents

solaris.utils API reference

`solaris.utils.core` Core utilities ¶

solaris.utils.core.get_data_paths(path, infer=False)[source]¶

Get a pandas dataframe of images and labels from a csv.

This file is designed to parse image:label reference CSVs (or just image) for inferencde) as defined in the documentation. Briefly, these should be CSVs containing two columns:

'image': the path to images. 'label': the path to the label file that corresponds to the image.

Parameters

path (str) – Path to a .CSV-formatted reference file defining the location of training, validation, or inference data. See docs for details.
infer (bool, optional) – If infer=True , the 'label' column will not be returned (as it is unnecessary for inference), even if it is present.

Returns

df – A pandas.DataFrame containing the relevant image and label information from the CSV at path (unless infer=True , in which case only the image column is returned.)

Return type

pandas.DataFrame

solaris.utils.core.get_files_recursively(path, traverse_subdirs=False, extension='.tif')[source]¶: Get files from subdirs of path, joining them to the dir.

`solaris.utils.config` Configuration file utilities ¶

solaris.utils.config.parse(path)[source]¶

Parse a config file for running a model.

Parameters: path (str) – Path to the YAML-formatted config file to parse.
Returns: config – A dict containing the information from the config file at path.
Return type: dict

`solaris.utils.io` Imagery and vector I/O utilities ¶

Utility functions for data io.

solaris.utils.io.imread(path, make_8bit=False, rescale=False, rescale_min='auto', rescale_max='auto')[source]¶

Read in an image file and rescale pixel values (if applicable).

Note

Because overhead imagery is often either 16-bit or multispectral (i.e. >3 channels or bands that don’t directly translate into the RGB scheme of photographs), this package using scikit-image io algorithms. Though slightly slower, these algorithms are compatible with any bit depth or channel count.

Parameters

path (str) – Path to the image file to load.
make_8bit (bool, optional) – Should the image be converted to an 8-bit format? Defaults to False.
rescale (bool, optional) – Should pixel intensities be rescaled? Defaults to no (False).
rescale_min ('auto' or int or float or list) – The minimum pixel value(s) for rescaling. If rescale=True but no value is provided for rescale_min, the minimum pixel intensity in each channel of the image will be subtracted such that the minimum value becomes zero. If a single number is provided, that number will be subtracted from each channel. If a list of values is provided that is the same length as the number of channels, then those values will be subtracted from the corresponding channels.
rescale_max ('auto' or int or float or list) – The max pixel value(s) for rescaling. If rescale=True but no value is provided for rescale_max, each channel will be rescaled such that the maximum value in the channel is set to the bit range’s max. If a single number is provided, that number will be set as the upper limit for all channels. If a list of values is provided that is the same length as the number of channels, then those values will be set to the maximum value in the corresponding channels.

Returns

im – A NumPy array of shape [Y, X, C] containing the imagery, with dtype uint8.

Return type

numpy.array()

solaris.utils.io.preprocess_im_arr(im_arr, im_format, rescale=False, rescale_min='auto', rescale_max='auto')[source]¶

Convert image to standard shape and dtype for use in the pipeline.

Notes

This repo will require the following of images:

Their shape is of form [X, Y, C]

Input images are dtype uint8

This function will take an image array im_arr and reshape it accordingly.

Parameters

im_arr (numpy.array()) – A numpy array representation of an image. im_arr should have either two or three dimensions.
im_format (str) – One of 'uint8', 'uint16', 'z-scored', 'zero-one normalized', '255 float', or '65535 float'. String indicating the dtype of the input, which will dictate the preprocessing applied.
rescale (bool, optional) – Should pixel intensities be rescaled? Defaults to no (False).
rescale_min ('auto' or int or float or list) – The minimum pixel value(s) for rescaling. If rescale=True but no value is provided for rescale_min, the minimum pixel intensity in each channel of the image will be subtracted such that the minimum value becomes zero. If a single number is provided, that number will be subtracted from each channel. If a list of values is provided that is the same length as the number of channels, then those values will be subtracted from the corresponding channels.
rescale_max ('auto' or int or float or list) – The max pixel value(s) for rescaling. If rescale=True but no value is provided for rescale_max, each channel will be rescaled such that the maximum value in the channel is set to the bit range’s max. If a single number is provided, that number will be set as the upper limit for all channels. If a list of values is provided that is the same length as the number of channels, then those values will be set to the maximum value in the corresponding channels.

Returns

Return type

A numpy.array() with shape [X, Y, C] and dtype uint8.

solaris.utils.io.rescale_arr(im_arr, im_format, rescale_min='auto', rescale_max='auto')[source]¶

Rescale array values in a 3D image array with channel order [Y, X, C].

Parameters

im_arr (numpy.array) – A numpy array representation of an image. im_arr should have either two or three dimensions.
im_format (str) – One of 'uint8', 'uint16', 'z-scored', 'zero-one normalized', '255 float', or '65535 float'. String indicating the dtype of the input, which will dictate the preprocessing applied.
rescale_min ('auto' or int or float or list) – The minimum pixel value(s) for rescaling. If rescale=True but no value is provided for rescale_min, the minimum pixel intensity in each channel of the image will be subtracted such that the minimum value becomes zero. If a single number is provided, that number will be subtracted from each channel. If a list of values is provided that is the same length as the number of channels, then those values will be subtracted from the corresponding channels.
rescale_max ('auto' or int or float or list) – The max pixel value(s) for rescaling. If rescale=True but no value is provided for rescale_max, each channel will be rescaled such that the maximum value in the channel is set to the bit range’s max. If a single number is provided, that number will be set as the upper limit for all channels. If a list of values is provided that is the same length as the number of channels, then those values will be set to the maximum value in the corresponding channels.

Returns

normalized_arr

Return type

numpy.array

solaris.utils.io.scale_for_model(image, output_type=None)[source]¶

Scale an image to a model’s required parameters.

Parameters

image (np.array) – The image array to be transformed to a desired output format.
output_type (str, optional) –
The data format of the output to pass into the model. There are five possible values:
- 'normalized' : values rescaled to 0-1.
- 'zscored' : image converted to zero mean and unit stdev.
- '8bit' : image converted to 8-bit format.
- '16bit' : image converted to 16-bit format.
If no value is provided, no re-scaling is performed (input array is returned directly).

`solaris.utils.geo` Geographic coordinate system management utilities ¶

solaris.utils.geo.affine_to_list(affine_obj)[source]¶: Convert a affine.Affine instance to a list for Shapely.

solaris.utils.geo.bbox_corners_to_coco(bbox)[source]¶

Convert bbox from [minx, miny, maxx, maxy] to coco format.

COCO formats bounding boxes as [minx, miny, width, height].

Parameters: bbox (list-like of numerics) – A 4-element list of the form [minx, miny, maxx, maxy].
Returns: coco_bbox – [minx, miny, width, height] shape.
Return type: list

solaris.utils.geo.gdf_get_projection_unit(vector_file)[source]¶

Get the projection unit for a vector_file or gdf.

Parameters: vector_file (geopandas.GeoDataFrame or geojson/shapefile) – A vector file or gdf with georeferencing

Notes

If vector file is already in UTM coords, the projection WKT is complex:: https://www.spatialreference.org/ref/epsg/wgs-84-utm-zone-11n/html/

In this case, return the second instance of ‘UNIT’.

Returns: unit – The unit i.e. meter, metre, or degree, of the projection
Return type: String

solaris.utils.geo.geometries_internal_intersection(polygons)[source]¶

Get the intersection geometries between all geometries in a set.

Parameters: polygons (list-like) – A list-like containing geometries. These will be placed in a geopandas.GeoSeries object to take advantage of rtree spatial indexing.
Returns: A list of geometric intersections between polygons in polygons, in the same CRS as the input.
Return type: intersect_list

solaris.utils.geo.get_bounds(geo_obj, crs=None)[source]¶

Get the [left, bottom, right, top] bounds in any CRS.

Parameters

geo_obj (a georeferenced raster or vector dataset.) –
crs (int, optional) – The EPSG code (or other CRS format supported by rasterio.warp) for the CRS the bounds should be returned in. If not provided, the bounds will be returned in the same crs as geo_obj.

Returns

bounds – [left, bottom, right, top] bounds in the input crs (if crs is None) or in crs if it was provided.

Return type

list

solaris.utils.geo.get_crs(obj)[source]¶: Get a coordinate reference system from any georegistered object.

solaris.utils.geo.get_projection_unit(crs)[source]¶

Get the units of a specific SRS.

Parameters: crs (pyproj.crs.CRS, rasterio.crs.CRS, str, or int) – The coordinate reference system to retrieve a unit for.
Returns: unit – The string-formatted unit.
Return type: str

solaris.utils.geo.get_subgraph(G, node_subset)[source]¶

Create a subgraph from G. Code almost directly copied from osmnx.

Parameters

G (networkx.MultiDiGraph) – A graph to be subsetted
node_subset (list-like) – The subset of nodes to induce a subgraph of G

Returns

G2 – The subgraph of G that includes node_subset

Return type

networkx.MultiDiGraph

solaris.utils.geo.latlon_to_utm_epsg(latitude, longitude, return_proj4=False)[source]¶

Get the WGS84 UTM EPSG code based on a latitude and longitude value.

Parameters

latitude (numeric) – The latitude value for the coordinate.
longitude (numeric) – The longitude value for the coordinate.
return_proj4 (bool, optional) – Should the proj4 string be returned as well as the EPSG code? Defaults to no (False)`

Returns

epsg (int) – The integer corresponding to the EPSG code for the relevant UTM zone in WGS 84.
proj4 (str) – The proj4 string for the CRS. Only returned if return_proj4=True.

solaris.utils.geo.list_to_affine(xform_mat)[source]¶

Create an Affine from a list or array-formatted [a, b, d, e, xoff, yoff]

Parameters: xform_mat (list or numpy.array) – A list of values to convert to an affine object.
Returns: aff – An affine transformation object.
Return type: affine.Affine

solaris.utils.geo.polygon_to_coco(polygon)[source]¶: Convert a geometry to COCO polygon format.

solaris.utils.geo.raster_get_projection_unit(image)[source]¶

Get the projection unit for an image.

Parameters: image (raster image, GeoTIFF or other format) – A raster file with georeferencing

Notes

If raster is already in UTM coords, the projection WKT is complex:: https://www.spatialreference.org/ref/epsg/wgs-84-utm-zone-11n/html/

In this case, return the second instance of ‘UNIT’.

Returns: unit – The unit i.e. meters or degrees, of the projection
Return type: String

solaris.utils.geo.reproject(input_object, input_crs=None, target_crs=None, target_object=None, dest_path=None, resampling_method='cubic')[source]¶

Reproject a dataset (df, gdf, or image) to a new coordinate system.

This function takes a georegistered image or a dataset of vector geometries and converts them to a new coordinate reference system. If no target CRS is provided, the data will be converted to the appropriate UTM zone by default. To convert a pixel-coordinate dataset to geographic coordinates or vice versa, use solaris.vector.polygon.georegister_px_df() or solaris.vector.polygon.geojson_to_px_gdf() instead.

Parameters

input_object (str or rasterio.DatasetReader or gdal.Dataset or geopandas.GeoDataFrame) – An object to transform to a new CRS. If a string, it must be a path to a georegistered image or vector dataset (e.g. a .GeoJSON). If the object itself does not contain georeferencing information, the coordinate reference system can be provided with input_crs.
input_crs (int, optional) – The EPSG code integer for the input data’s CRS. If provided and a CRS is also associated with input_object, this argument’s value has precedence.
target_crs (int, optional) – The EPSG code for the output projection. If values are not provided for this argument or target_object, the input data will be re-projected into the appropriate UTM zone. If both target_crs and target_object are provided, target_crs takes precedence (and a warning is raised).
target_object (str, optional) – An object in the desired destination CRS. If neither this argument nor target_crs is provided, the input will be projected into the appropriate UTM zone. target_crs takes precedence if both it and target_object are provided.
dest_path (str, optional) – The path to save the output to (if desired). This argument is only required if the input is a gdal.Dataset; otherwise, it is optional.
resampling_method (str, optional) – The resampling method to use during reprojection of raster data. Only has an effect if the input is a :class:`rasterio.DatasetReader` ! Possible values are ['cubic' (default), 'bilinear', 'nearest', 'average'].

Returns

output – An output in the same format as input_object, but reprojected into the destination CRS.

Return type

rasterio.DatasetReader or gdal.Dataset or geopandas.GeoDataFrame

solaris.utils.geo.reproject_geometry(input_geom, input_crs=None, target_crs=None, affine_obj=None)[source]¶

Reproject a geometry or coordinate into a new CRS.

Parameters

input_geom (str, list, or Shapely geometry) – A geometry object to re-project. This can be a 2-member list, in which case input_geom is assumed to coorespond to [x, y] coordinates in input_crs. It can also be a Shapely geometry object or a wkt string.
input_crs (int, optional) – The coordinate reference system for input_geom’s coordinates, as an EPSG int. Required unless affine_transform is provided.
target_crs (int, optional) – The target coordinate reference system to re-project the geometry into. If not provided, the appropriate UTM zone will be selected by default, unless affine_transform is provided (and therefore CRSs are ignored.)
affine_transform (affine.Affine, optional) – An affine.Affine object (or a [a, b, c, d, e, f] list to convert to that format) to use for transformation. Has no effect unless input_crs and target_crs are not provided.

Returns

output_geom – A shapely geometry object: - in target_crs, if one was provided; - in the appropriate UTM zone, if input_crs was provided and

target_crs was not;

with affine_transform applied to it if neither input_crs nor target_crs were provided.

Return type

Shapely geometry

solaris.utils.geo.reproject_to_utm(input_data, input_type, input_crs=None, dest_path=None, resampling_method='bicubic')[source]¶: Convert an input to a UTM CRS (after determining the correct UTM zone).

solaris.utils.geo.split_geom(geometry, tile_size, resolution=None, use_projection_units=False, src_img=None)[source]¶

Splits a vector into approximately equal sized tiles.

Adapted from @lossyrob’s Gist__

The more complex the geometry, the slower this will run, but geometrys with around 10000 coordinates run in a few seconds time. You can simplify geometries with shapely.geometry.Polygon.simplify if necessary.

Parameters

geometry (str, optional) – A shapely.geometry.Polygon, path to a single feature geojson, or list-like bounding box shaped like [left, bottom, right, top]. The geometry must be in the projection coordinates corresponding to the resolution units.
tile_size (tuple of `int`s) – The size of the input tiles in (y, x) coordinates. By default, this is in pixel units; this can be changed to metric units using the use_metric_size argument.
use_projection_units (bool, optional) – Is tile_size in pixel units (default) or distance units? To set to distance units use use_projection_units=True. If False, resolution must be supplied.
resolution (tuple of `float`s, optional) – (x resolution, y resolution). Used by default if use_metric_size is False. Can be acquired from rasterio dataset object’s metadata.
src_img (str or raster, optional) – A rasterio raster object or path to a geotiff. The bounds of this raster and the geometry will be intersected and the result of the intersection will be tiled. Useful in cases where the extent of collected labels and source imagery partially overlap. The src_img must have the same projection units as the geometry.

Returns

tile_bounds

Return type

list (containing sublists like [left, bottom, right, top])

solaris.utils.geo.split_multi_geometries(gdf, obj_id_col=None, group_col=None, geom_col='geometry')[source]¶

Split apart MultiPolygon or MultiLineString geometries.

Parameters

gdf (geopandas.GeoDataFrame or str) – A geopandas.GeoDataFrame or path to a geojson containing geometries.
obj_id_col (str, optional) – If one exists, the name of the column that uniquely identifies each geometry (e.g. the "BuildingId" column in many SpaceNet datasets). This will be tracked so multiple objects don’t get produced with the same ID. Note that object ID column will be renumbered on output. If passed, group_col must also be provided.
group_col (str, optional) – A column to identify groups for sequential numbering (for example, 'ImageId' for sequential number of 'BuildingId'). Must be provided if obj_id_col is passed.
geom_col (str, optional) – The name of the column in gdf that corresponds to geometry. Defaults to 'geometry'.

Returns

A geopandas.GeoDataFrame that’s identical to the input, except with the multipolygons split into separate rows, and the object ID column renumbered (if one exists).

Return type

geopandas.GeoDataFrame

`solaris.utils.tile` Tiling utilities ¶

`solaris.utils.raster` Raster image and array management utilities ¶

solaris.utils.raster.reorder_axes(arr, target='tensorflow')[source]¶

Check order of axes in an array or tensor and convert to desired format.

Parameters

arr (numpy.array or torch.Tensor or tensorflow.Tensor) –
target (str, optional) – Desired axis order type. Possible values: - 'tensorflow' (default): [N, Y, X, C] or [Y, X, C] - 'torch' : [N, C, Y, X] or [C, Y, X]

Returns

out_arr (an object of the same class as arr with axes in the desired)
order.

`solaris.utils.data` Dataset CSV utilities ¶

solaris.utils.data.make_dataset_csv(im_dir, im_ext='tif', label_dir=None, label_ext='json', output_path='dataset.csv', stage='train', match_re=None, recursive=False, ignore_mismatch=None, verbose=0)[source]¶

Automatically generate dataset CSVs for training.

This function creates basic CSVs for training and inference automatically. See the documentation tutorials for details on the specification. A regular expression string can be provided to extract substrings for matching images to labels; if not provided, it’s assumed that the filename for the image and label files is identical once extensions are stripped. By default, this function will raise an exception if there are multiple label files that match to a given image file, or if no label file matches an image file; see the ignore_mismatch argument for alternatives.

Parameters

im_dir (str) – The path to the directory containing images to be used by your model. Images in sub-directories can be included by setting recursive=True.
im_ext (str, optional) – The file extension used by your images. Defaults to "tif". Not case sensitive.
label_dir (str, optional) – The path to the directory containing images to be used by your model. Images in sub-directories can be included by setting recursive=True. This argument is required if stage is "train" (default) or "val", but has no effect if stage is "infer".
output_path (str, optional) – The path to save the generated CSV to. Defaults to "dataset.csv".
stage (str, optional) – The stage that the csv is generated for. Can be "train" (default), "val", or "infer". If set to "train" or "val", label_dir must be provided or an error will occur.
match_re (str, optional) – A regular expression pattern to extract substrings from image and label filenames for matching. If not provided and labels must be matched to images, it’s assumed that image and label filenames are identical after stripping directory and extension. Has no effect if stage="infer". The pattern must contain at least one capture group for compatibility with pandas.Series.str.extract().
recursive (bool, optional) – Should sub-directories in im_dir and label_dir be traversed to find images and label files? Defaults to no (False).
ignore_mismatch (str, optional) – Dictates how mismatches between image files and label files should be handled. By default, having != 1 label file per image file will raise a ValueError. If ignore_mismatch="skip", any image files with != 1 matching label will be skipped.
verbose (int, optional) – Verbose text output. By default, none is provided; if True or 1, information-level outputs are provided; if 2, extremely verbose text is output.

Returns

output_df – A pandas.DataFrame with one column titled "image" and a second titled "label" (if stage != "infer"). The function also saves a CSV at output_path.

Return type

pandas.DataFrame