ConvNeXt Segmentation Models

Semantic segmentation models that use a ConvNeXt backbone with either an FCN (Fully Convolutional Network) head or a UPerNet (Unified Perceptual Parsing Network) head.

These models follow the architecture patterns from mmsegmentation and can be used for semantic segmentation tasks.

model_convnext_tiny_fcn(
  num_classes = 21,
  aux_loss = FALSE,
  pretrained_backbone = FALSE,
  ...
)

model_convnext_small_fcn(
  num_classes = 21,
  aux_loss = FALSE,
  pretrained_backbone = FALSE,
  ...
)

model_convnext_base_fcn(
  num_classes = 21,
  aux_loss = FALSE,
  pretrained_backbone = FALSE,
  ...
)

model_convnext_tiny_upernet(
  num_classes = 21,
  aux_loss = FALSE,
  pretrained = FALSE,
  pretrained_backbone = FALSE,
  pool_scales = c(1, 2, 3, 6),
  ...
)

model_convnext_small_upernet(
  num_classes = 21,
  aux_loss = FALSE,
  pretrained = FALSE,
  pretrained_backbone = FALSE,
  pool_scales = c(1, 2, 3, 6),
  ...
)

model_convnext_base_upernet(
  num_classes = 21,
  aux_loss = FALSE,
  pretrained = FALSE,
  pretrained_backbone = FALSE,
  pool_scales = c(1, 2, 3, 6),
  ...
)

Arguments

num_classes

Number of output segmentation classes. Default: 21 (PASCAL VOC).

aux_loss

If TRUE, includes an auxiliary classifier branch. Default: FALSE.

pretrained_backbone

If TRUE, loads ImageNet pretrained weights for the ConvNeXt backbone. Default: FALSE.

...

Additional arguments passed to the backbone.

pretrained

If TRUE, loads convnext pretrained weights of backbone and segmentation heads.

pool_scales

Numeric vector. Pooling scales used in the Pyramid Pooling Module for UPerNet models. Default: c(1, 2, 3, 6).

Value

An nn_module representing the segmentation model.

Functions

• model_convnext_tiny_fcn(): ConvNeXt Tiny with FCN head

• model_convnext_small_fcn(): ConvNeXt Small with FCN head

• model_convnext_base_fcn(): ConvNeXt Base with FCN head

• model_convnext_tiny_upernet(): ConvNeXt Tiny with UPerNet head

• model_convnext_small_upernet(): ConvNeXt Small with UPerNet head

• model_convnext_base_upernet(): ConvNeXt Base with UPerNet head

Available FCN Models

• model_convnext_tiny_fcn()

• model_convnext_small_fcn()

• model_convnext_base_fcn()

Available UPerNet Models

• model_convnext_tiny_upernet()

• model_convnext_small_upernet()

• model_convnext_base_upernet()

See also

Other semantic_segmentation_model: model_deeplabv3, model_fcn_resnet

Examples

if (FALSE) { # \dontrun{
library(magrittr)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants
norm_std  <- c(0.229, 0.224, 0.225)

# Use a publicly available image
wmc <- "https://upload.wikimedia.org/wikipedia/commons/thumb/"
url <- "e/ea/Morsan_Normande_vache.jpg/120px-Morsan_Normande_vache.jpg"
img <- base_loader(paste0(wmc, url))

input <- img %>%
  transform_to_tensor() %>%
  transform_resize(c(520, 520)) %>%
  transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)

# ConvNeXt Tiny FCN segmentation
model <- model_convnext_tiny_fcn(num_classes = 21, pretrained_backbone = TRUE)
model$eval()
output <- model(batch)

# Visualize result
segmented <- draw_segmentation_masks(input, output$out$squeeze(1))
tensor_image_display(segmented)

# ConvNeXt Tiny UPerNet segmentation
model <- model_convnext_tiny_upernet(num_classes = 21, pretrained_backbone = TRUE)
model$eval()
output <- model(batch)

# Visualize result
segmented <- draw_segmentation_masks(input, output$out$squeeze(1))
tensor_image_display(segmented)
} # }