Object-aware domain generalization for object detection

AAAI'24 (Oral)
1Urban Robotics Lab, Korea Advanced Institute of Science and Technology*These authors contributed equally; †Corresponding authors

Abstract

OA-DG is an effective method for single-domain object detection generalization (S-DGOD). It consists of two components: OA-Mix for data augmentation and OA-Loss for reducing domain gaps.

Video

Features

  • OA-DG consists of OA-Mix for data augmentation and OA-Loss for reducing the domain gap.
  • OA-Mix is a data augmentation methods that increases image diversity while preserving important semantic feature with multi-level transformations and object-aware mixing.
  • OA-Loss is a novel contrastive loss functions that reduces the domain gap by training semantic relations of foreground and background instances from multi-domain.
  • Extensive experiments on standard benchmarks (Cityscapes-C and Diverse Weather Dataset) show that OA-DG outperforms SOTA methods on unseen target domains.
  • OA-DG can be generally applied to improve robustness regardless of the augmentation set and object detector architectures.

OA-Mix

Multi-Level Transformations

Multi-level transformation enhances the domain diversity of the augmented image without damaging the object locations.

Foreground-level transformation
Background-level transformation
Random-level transformation
  • An image is randomly divided into several regions such as foreground, background, and random box regions.
  • Different transformations, such as color and spatial transformations, are randomly applied to each region.
  • Spatial transformations are applied at the foreground level to preserve the location of the object.

Object-Aware Mixing

In object detection, each object in an image has different characteristics, such as size, location, and color distribution. Depending on these object-specific characteristics, some transformations can damage the semantic features of an object.
Object-aware mixing enhances the affinity of the augmented image, mitigating the negative effects of transformations.

(Left) Multi-level transformed image. (Right) Object-aware mixed image.

Examples

The left and right images are the original and OA-Mixed images, respectively:

Results

We evaluated the robustness of our method for common corruptions and various weather conditions in urban scenes. mPC is an evaluation metric of robustness against out-of-distribution (OOD).

Cityscapes-C

DWD

☀️ Daytime-Sunny
ClassGTsDetsRecallAP
aeroplane173897110.7990.561
bicycle104661650.7160.491
bird953393259820.8800.763
boat53731510.7020.462
bottle12309763180.7640.557
bus78734100.6540.489
car5029282290.8350.582
mAP0.558
🌃 Night-Sunny
ClassGTsDetsRecallAP
aeroplane2012153070.6880.395
bicycle141091510.6160.371
bird24161614095870.8460.639
boat665131910.4980.178
bottle175661854150.7100.439
bus84149070.4470.271
car4853416330.7140.412
mAP0.386
🌧️ Dusk-Rainy
ClassGTsDetsRecallAP
aeroplane82039530.6040.382
bicycle32224690.4810.285
bird342401802930.8350.681
boat11015080.3360.132
bottle5144270220.5250.325
bus16911860.3310.214
car2235131580.7030.449
mAP0.353
🌙 Night-Rainy
ClassGTsDetsRecallAP
aeroplane24811580.4680.289
bicycle12110880.2230.123
bird216551748570.6680.356
boat4916350.1430.010
bottle1532209630.3780.139
bus715600.1690.120
car49943830.4630.220
mAP0.180
🌫️ Daytime-Foggy
ClassGTsDetsRecallAP
aeroplane55418820.4930.324
bicycle4920174700.5000.324
bird33392814600.7140.626
boat91143010.4970.319
bottle21530627590.5270.420
bus236366090.5300.426
car73660680.4970.267
mAP0.387

Citation

@inproceedings{lee2024object, title={Object-Aware Domain Generalization for Object Detection}, author={Lee, Wooju and Hong, Dasol and Lim, Hyungtae and Myung, Hyun}, booktitle={Proceedings of the AAAI Conference on Artificial Intelligence}, volume={38}, number={4}, pages={2947--2955}, year={2024} }