RF-DETR Implementation

Source: R/models-rfdetr_detection.R

RF-DETR: Neural Architecture Search for Real-Time Detection Transformers (https://arxiv.org/abs/2511.09554)

model_rfdetr_nano(pretrained = FALSE, progress = TRUE, ...)

model_rfdetr_small(pretrained = FALSE, progress = TRUE, ...)

model_rfdetr_medium(pretrained = FALSE, progress = TRUE, ...)

model_rfdetr_base(pretrained = FALSE, progress = TRUE, ...)

model_rfdetr_base_2(pretrained = FALSE, progress = TRUE, ...)

model_rfdetr_base_o365(pretrained = FALSE, progress = TRUE, ...)

model_rfdetr_large(pretrained = FALSE, progress = TRUE, ...)

Arguments

pretrained

(bool): If TRUE, returns a model pre-trained on ImageNet.

progress

(bool): If TRUE, displays a progress bar of the download to stderr.

...

Other parameters passed to the model implementation.

Details

Object detection transformer models combining a DINOv2 backbone (windowed attention, register tokens) with a deformable-attention decoder and two-stage query proposal. Supports Nano, Small, Medium, Base, Base with O365 pretraining, and Large variants.

Model Variants

| Variant   | Backbone           | Decoder Layers | Resolution | # Queries | Group DETR | Weights                     |
|-----------|--------------------|----------------|------------|-----------|------------|-----------------------------|
| nano      | DINOv2 Small (win) | 2              | 384        | 300       | 13         | COCO (91 classes)           |
| small     | DINOv2 Small (win) | 3              | 512        | 300       | 13         | COCO (91 classes)           |
| medium    | DINOv2 Small (win) | 4              | 576        | 300       | 13         | COCO (91 classes)           |
| base      | DINOv2 Small (win) | 3              | 560        | 300       | 13         | COCO (91 classes)           |
| base_2    | DINOv2 Small (win) | 3              | 560        | 300       | 13         | COCO (91 classes, alt run)  |
| base_o365 | DINOv2 Small (win) | 3              | 560        | 300       | 13         | Objects365 (366 classes)    |
| large     | DINOv2 Base (win)  | 3              | 560        | 300       | 13         | COCO (91 classes)           |

  • All models use group DETR (group_detr=13) with two-stage query proposal, Lite Refpoint Refine, and BBox reparameterisation.

  • The large variant corresponds to the deprecated RF-DETR-Large config (DINOv2 Base encoder, hidden_dim=384).

  • model(x, mask = NULL) expects ImageNet-normalised images (B, 3, H, W), which are resized to the variant resolution in eval mode. The optional mask (B, H, W) is TRUE on padded pixels; boxes are then returned in the coordinates of the unpadded image.

Functions

  • model_rfdetr_nano(): RF-DETR Nano (fastest, COCO, 384px)

  • model_rfdetr_small(): RF-DETR Small (lightweight, COCO, 512px)

  • model_rfdetr_medium(): RF-DETR Medium (balanced speed/accuracy, COCO, 576px)

  • model_rfdetr_base(): RF-DETR Base (COCO pretrained, 560px)

  • model_rfdetr_base_2(): RF-DETR Base variant 2 (alternative COCO training run)

  • model_rfdetr_base_o365(): RF-DETR Base O365 (Objects365, 366 classes)

  • model_rfdetr_large(): RF-DETR Large (DINOv2 Base backbone, COCO, 560px)

Examples

if (FALSE) { # \dontrun{
url <- "https://upload.wikimedia.org/wikipedia/commons/6/6f/Toy_Poodle_wearing_clothes_in_Tokyo.jpg"
img <- base_loader(url) %>%
  transform_to_tensor()
# the model expects ImageNet-normalised input, as in the reference predict()
input <- img %>%
  transform_normalize(c(0.485, 0.456, 0.406), c(0.229, 0.224, 0.225))

model <- model_rfdetr_large(pretrained = TRUE)
model$eval()

pred <- torch::with_no_grad(
  model(input$unsqueeze(1))
)$detections[[1]]

topk <- pred$scores$topk(k = 5L)[[2]]
boxes <- pred$boxes[topk, ]
labels <- coco_classes(as.integer(pred$labels[topk]))
canvas <- (img * 255)$to(dtype = torch_uint8())
boxed <- draw_bounding_boxes(canvas, boxes, labels = labels,
 colors = "black", width = 6)
tensor_image_browse(boxed)
} # }