Object detection

The task#

Given an image I, produce a set of predictions of unknown size:

D = { (b_i, c_i, s_i) }  for i = 1..N

• b_i — a box, conventionally (x1, y1, x2, y2) in image coordinates, or (cx, cy, w, h) in centre form.
• c_i — a class label from {1, ..., K}.
• s_i — a confidence score in [0, 1].
• N — not fixed, and not known in advance.

That last point is the structural difficulty. Classification maps an image to a fixed-size output. Detection maps an image to a set of varying cardinality, which does not fit the standard supervised template. Every detector architecture is, at bottom, a different answer to "how do I turn set prediction into fixed-size prediction".

Three answers, historically#

Two-stage. Propose regions, then classify each. R-CNN (Girshick et al., 2014) used selective search plus a CNN per crop, at roughly 47 seconds per image. Fast R-CNN (2015) shared the convolutional computation and introduced RoI pooling. Faster R-CNN (Ren et al., 2015) replaced selective search with a learned Region Proposal Network, making the whole system end-to-end and roughly 200 times faster than R-CNN.

One-stage. Skip proposals; predict boxes and classes directly from a dense grid of anchors. SSD (Liu et al., 2016) and YOLO (Redmon et al., 2016) established the family. The accuracy gap against two-stage methods was diagnosed by Lin et al. (2017) as a foreground-background imbalance problem — roughly 10^4 to 10^5 easy negatives per positive — and addressed with focal loss:

FL(p_t) = -alpha_t * (1 - p_t)^gamma * log(p_t)

• p_t — predicted probability of the true class.
• gamma — focusing parameter, 2 in the paper. It down-weights already-easy examples.
• alpha_t — per-class balancing weight, 0.25 for the foreground in RetinaNet.

Set prediction. DETR (Carion et al., 2020) removed anchors and NMS entirely. A transformer decoder emits a fixed set of N queries, and training uses bipartite matching between predictions and ground truth, solved by the Hungarian algorithm. The matching makes the loss permutation-invariant, so duplicate suppression becomes part of the objective rather than a post-process.

DETR's original weakness was convergence — 500 epochs against Faster R-CNN's 12 — and poor small-object performance. Deformable DETR (Zhu et al., 2021) fixed both with sparse attention over multi-scale features. DINO (Zhang et al., 2022) added denoising queries and now anchors the top of the COCO leaderboard.

Anchors and box parameterisation#

Anchor-based detectors regress an offset from a prior box rather than absolute coordinates. The R-CNN parameterisation is still the standard:

t_x = (x - x_a) / w_a          t_w = log(w / w_a)
t_y = (y - y_a) / h_a          t_h = log(h / h_a)

• (x, y, w, h) — the target box, centre form.
• (x_a, y_a, w_a, h_a) — the anchor.
• t_* — the regressed targets.

Two design choices are doing real work here. Dividing the centre offsets by anchor size makes the target scale-invariant. Taking the log of the size ratio makes the target symmetric — halving and doubling are equal-magnitude errors — and keeps predicted widths positive without a constraint.

Anchor-free detectors (FCOS, Tian et al., 2019; CenterNet, Zhou et al., 2019) predict distances to the four box edges from each location instead. That removes the anchor hyperparameters which dominated tuning effort in the anchor era: scales, aspect ratios and assignment IoU thresholds.

Box regression losses#

Smooth L1 on the t_* targets was the original choice. It optimises four numbers independently, which does not correspond to the IoU that evaluation actually uses.

The gradient problem GIoU solves is worth stating. When two boxes do not overlap, IoU is 0 and its gradient is 0 everywhere. Plain IoU loss therefore gives no learning signal in exactly the case that needs one.

NMS and its variants#

Greedy NMS sorts by score and removes any box whose IoU with a kept box exceeds N_t:

s_i = 0   if   IoU(M, b_i) >= N_t

• M — the currently highest-scoring box.
• N_t — the suppression threshold, typically 0.5 to 0.7.

The hard cut is the flaw. A genuinely distinct object overlapping a detected one is deleted outright. Soft-NMS (Bodla et al., ICCV 2017) decays the score instead of zeroing it:

s_i = s_i * exp( -IoU(M, b_i)^2 / sigma )

• sigma — decay width, 0.5 in the paper.

This gives a consistent gain of roughly 1 point of AP on COCO with no retraining, and it particularly helps crowded scenes. Crowd-specific alternatives exist — CrowdDet (Chu et al., 2020) predicts a set per proposal — and the general problem of dense overlapping instances remains open.

Cost: greedy NMS is O(n^2) in the worst case over n surviving boxes, though n is small after score thresholding. It is also sequential, which is why it shows up as a latency floor in optimised inference graphs and why NMS-free detectors are attractive for deployment.

Evaluation: what mAP is, and is not#

Average precision for one class is the area under the precision-recall curve, computed by ranking all detections by score and sweeping the threshold. A detection counts as a true positive if its IoU with an unmatched ground-truth box of the same class exceeds a threshold. Each ground-truth box can be matched at most once; extra matches are false positives.

COCO mAP — reported as AP or mAP@[.5:.95] — averages AP over ten IoU thresholds from 0.50 to 0.95 in steps of 0.05, and over all 80 categories. The precision-recall curve is interpolated at 101 recall points.

Three things follow, and each is routinely misread:

1. It is not accuracy. A COCO AP of 21.3, as in the model above, is a respectable score for a 3.4M-parameter mobile detector. State-of-the-art large models sit around 60 to 65. Nothing about these numbers maps onto "percent correct".
2. Averaging over IoU thresholds means localisation quality dominates. A model that finds everything but boxes it loosely scores far below its AP@0.5. The gap between AP@0.5 and AP@[.5:.95] is a direct readout of box tightness.
3. The size breakdown is where the information is. COCO reports AP_small, AP_medium, AP_large for objects under 32^2, between 32^2 and 96^2, and above 96^2 pixels. AP_small is typically less than half of AP_large for the same model. If your application is small objects, the headline number is actively misleading.

Cost#

For a single-stage detector at input H x W, the dense head evaluates A anchors per location over the feature pyramid:

total anchors  ≈  A * sum over levels l of (H / s_l) * (W / s_l)

• s_l — the stride of pyramid level l, typically 8, 16, 32, 64, 128.
• A — anchors per location, 9 in RetinaNet.

At 640 x 640 with RetinaNet's configuration this is roughly 76,000 anchors, essentially all negative. That imbalance is the reason focal loss exists.

Backbone FLOPs dominate compute; NMS dominates the non-parallelisable tail. On mobile targets the practical lever is input resolution, which scales cost quadratically and small-object AP roughly linearly.

Papers#

• Girshick, R. et al. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv:1311.2524
• Ren, S. et al. (2015). Faster R-CNN. arXiv:1506.01497
• Liu, W. et al. (2016). SSD: Single Shot MultiBox Detector. arXiv:1512.02325
• Lin, T.-Y. et al. (2017). Feature Pyramid Networks for Object Detection. arXiv:1612.03144
• Lin, T.-Y. et al. (2017). Focal Loss for Dense Object Detection. arXiv:1708.02002
• Bodla, N. et al. (2017). Soft-NMS. arXiv:1704.04503
• Tian, Z. et al. (2019). FCOS: Fully Convolutional One-Stage Object Detection. arXiv:1904.01355
• Rezatofighi, H. et al. (2019). Generalized Intersection over Union. arXiv:1902.09630
• Carion, N. et al. (2020). End-to-End Object Detection with Transformers. arXiv:2005.12872
• Zhu, X. et al. (2021). Deformable DETR. arXiv:2010.04159
• Zhang, H. et al. (2022). DINO: DETR with Improved DeNoising Anchor Boxes. arXiv:2203.03605
• Liu, S. et al. (2023). Grounding DINO. arXiv:2303.05499
• Howard, A. et al. (2019). Searching for MobileNetV3. arXiv:1905.02244 — the source of the SSDLite model used above.

Open problems#

Small objects. The gap between AP_small and AP_large has narrowed but not closed, and it is the binding constraint in aerial imagery, retail shelf analysis and long-range driving perception. Higher input resolution is the reliable fix and it is expensive.

Crowds and heavy occlusion. Greedy NMS cannot distinguish duplicate detections from genuinely overlapping instances, and set-prediction methods still degrade sharply in dense scenes.

Long-tail categories. LVIS (Gupta et al., 2019) exposed how badly COCO-tuned methods handle rare classes with a handful of examples. Copy-paste augmentation and decoupled classifier training help; the problem is not solved.

Open vocabulary. Grounding DINO, GLIP and OWL-ViT detect classes named in text at inference time, removing the fixed-K constraint. Accuracy trails a fine-tuned closed-set detector on any specific category, and the flexibility frequently wins in practice.

Calibration. Detection confidences are poorly calibrated, and unlike classification the literature on fixing it is thin. Any system that gates an action on a score threshold needs its own reliability measurement on its own data.

What to learn next#

• YOLO — the one-stage family that made detection fast enough for video.
• Image segmentation — from rectangles to exact object outlines.
• Model evaluation — precision, recall and the curves mAP is built from.