Cityscapes Dataset for Instance and Semantic Segmentation

The Cityscapes dataset contains street scene images from 50 European cities with pixel-level annotations for semantic and instance segmentation tasks. This dataset is widely used for urban scene understanding research.

cityscapes_dataset(
  root = tempdir(),
  split = "train",
  mode = "fine",
  target_type = "instance",
  transform = NULL,
  target_transform = NULL,
  item_transform = NULL
)

Arguments

root

Character. Root directory where dataset is stored (default: tempdir()).

split

Character. One of "train", "val", or "test" (default: "train").

mode

Character. Annotation quality: "fine" for high-quality dense annotations or "coarse" for weaker annotations (default: "fine").

target_type

Character vector. Types of annotations to load. Can include:

• "instance": Instance segmentation masks (each object has unique ID)

• "semantic": Semantic segmentation masks (class IDs only)

• "polygon": Polygon annotations in JSON format

• "color": Color-coded visualization of annotations

Multiple types can be specified (default: "instance").

transform

Function to transform the input image (default: NULL).

target_transform

Function to transform the target annotations (default: NULL).

item_transform

Optional transform function applied to the whole item, such as item_transform_rotate(), once transform and target_transform have been applied.

Value

A torch dataset object. Each item is a named list:

• x: a RGB magick image.

• y: Named list containing requested target types:

  • instance: Instance segmentation mask (1, H, W) with unique IDs per object

  • semantic: Semantic segmentation mask (1, H, W) with class IDs

  • color: Color-coded visualization (3, H, W)

  • polygon: Polygon annotations (nested list structure)

The dataset inherits segmentation_dataset and its y inherits segmentation_target, which target transforms dispatch on.

Dataset Structure

The Cityscapes dataset includes:

• 5,000 finely annotated images (2,975 train, 500 val, 1,525 test)

• 20,000 coarsely annotated images

• 19 semantic classes for urban scenes

• Dense pixel-level annotations at 1024x2048 resolution

• Images from 50 different European cities

Semantic Classes

The dataset includes 19 evaluation classes: road, sidewalk, building, wall, fence, pole, traffic light, traffic sign, vegetation, terrain, sky, person, rider, car, truck, bus, train, motorcycle, bicycle.

Download

Cityscapes requires manual download and registration:

1. Register at https://www.cityscapes-dataset.com/

2. Download packages:

   • leftImg8bit_trainvaltest.zip (11GB) - RGB images

   • gtFine_trainvaltest.zip (241MB) - Fine annotations

   • gtCoarse.zip (1.3GB) - Coarse annotations (optional)

3. Extract to root directory

Expected directory structure:

root/
+-- leftImg8bit/
|   +-- train/
|   +-- val/
|   +-- test/
+-- gtFine/
|   +-- train/
|   +-- val/
|   +-- test/
+-- gtCoarse/
    +-- train/
    +-- train_extra/
    +-- val/

Integration with draw_segmentation_masks

The output is directly compatible with draw_segmentation_masks:

item <- dataset[1]
# For instance segmentation
overlay <- draw_segmentation_masks(item$x, item$y$instance > 0)
# For semantic segmentation
overlay <- draw_segmentation_masks(item$x, item$y$semantic == class_id)

See also

Other segmentation_dataset: coco_segmentation_dataset(), oxfordiiitpet_segmentation_dataset(), pascal_voc_datasets, rf100_peixos_segmentation_dataset()

Examples

if (FALSE) { # \dontrun{
# Load Cityscapes with instance segmentation
cityscapes_train <- cityscapes_dataset(
  root = "~/datasets/cityscapes",
  split = "train",
  mode = "fine",
  target_type = "instance",
  transform = transform_to_tensor
)

# Get first item
first <- cityscapes_train[1]
first$x  # Image tensor (3, H, W)
first$y$instance  # Instance mask (H, W)

# Visualize with draw_segmentation_masks
# Create boolean mask for all instances
mask <- first$y$instance > 0
overlay <- draw_segmentation_masks(first$x, mask$unsqueeze(1), alpha = 0.5)
tensor_image_browse(overlay)

# Load with multiple target types
cityscapes_multi <- cityscapes_dataset(
  root = "~/datasets/cityscapes",
  split = "val",
  mode = "fine",
  target_type = c("instance", "semantic"),
  transform = transform_to_tensor
)

item <- cityscapes_multi[1]
# Semantic mask for specific class (e.g., cars = class 13)
car_mask <- item$y$semantic == 13
overlay <- draw_segmentation_masks(item$x, car_mask$unsqueeze(1))
tensor_image_browse(overlay)

# Use with dataloader
dl <- torch::dataloader(cityscapes_train, batch_size = 4, shuffle = TRUE)
batch <- dl$.iter()$.next()
} # }