ConvNeXt Detection Models (Faster R-CNN style)

Source: R/models-convnext_detection.R

Object detection models that use a ConvNeXt backbone with a Feature Pyramid Network (FPN) and the same detection head as the Faster R-CNN models implemented in model_fasterrcnn_*.

These helpers mirror the architecture used in model_fasterrcnn_resnet50_fpn(), but swap the ResNet backbone for ConvNeXt variants.

model_convnext_tiny_detection(
  num_classes = 91,
  pretrained_backbone = FALSE,
  ...
)

model_convnext_small_detection(
  num_classes = 91,
  pretrained_backbone = FALSE,
  ...
)

model_convnext_base_detection(
  num_classes = 91,
  pretrained_backbone = FALSE,
  ...
)

Arguments

num_classes

Number of output classes (default: 91 for COCO).

pretrained_backbone

Logical, if TRUE the ConvNeXt backbone weights are loaded from ImageNet pretraining.

...

Other arguments (unused).

Functions

  • model_convnext_tiny_detection(): ConvNeXt Tiny with FPN detection head

  • model_convnext_small_detection(): ConvNeXt Small with FPN detection head

  • model_convnext_base_detection(): ConvNeXt Base with FPN detection head

Note

Currently, detection head weights are randomly initialized, so predicted bounding-boxes are random. For meaningful results, you need to train the model detection head on your data.

Available Models

  • model_convnext_tiny_detection()

  • model_convnext_small_detection()

  • model_convnext_base_detection()

See also

Other object_detection_model: model_facenet, model_fasterrcnn, model_maskrcnn

Examples

if (FALSE) { # \dontrun{
library(magrittr)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants
norm_std  <- c(0.229, 0.224, 0.225)

# Use a publicly available image
url <- paste0("https://upload.wikimedia.org/wikipedia/commons/thumb/",
       "e/ea/Morsan_Normande_vache.jpg/120px-Morsan_Normande_vache.jpg")
img <- magick_loader(url) %>%
  transform_to_tensor() %>%
  transform_resize(c(520, 520))

input <- img %>%
  transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)    # Add batch dimension (1, 3, H, W)

# ConvNeXt Tiny detection
model <- model_convnext_tiny_detection(pretrained_backbone = TRUE)
model$eval()
# Please wait 2 mins + on CPU
pred <- model(batch)$detections[[1]]
num_boxes <- as.integer(pred$boxes$size()[1])
topk <- pred$scores$topk(k = 5)[[2]]
boxes <- pred$boxes[topk, ]
labels <- imagenet_classes(as.integer(pred$labels[topk]))

# `draw_bounding_box()` may fail if bbox values are not consistent.
if (num_boxes > 0) {
  boxed <- draw_bounding_boxes(img, boxes, labels = labels)
  tensor_image_browse(boxed)
}
} # }