Dice and other segmentation losses

Definitions#

For a predicted probability map $p \in [0,1]^{N}$ and binary target $g \in {0,1}^{N}$:

Soft Dice, from Milletari et al. (2016), V-Net:

$$ \mathcal{L}_{\text{Dice}} = 1 - \frac{2\sum_i p_i g_i + \epsilon}{\sum_i p_i + \sum_i g_i + \epsilon} $$

Soft Jaccard (IoU):

$$ \mathcal{L}_{\text{IoU}} = 1 - \frac{\sum_i p_i g_i + \epsilon}{\sum_i p_i + \sum_i g_i - \sum_i p_i g_i + \epsilon} $$

The two are related by $J = \frac{D}{2 - D}$, a strictly increasing map, so they induce the same ranking over predictions. They do not induce the same gradients, and they are not interchangeable during optimisation.

Tversky, from Salehi et al. (2017):

$$ \mathcal{L}_{\text{T}} = 1 - \frac{TP + \epsilon}{TP + \alpha FP + \beta FN + \epsilon}, \quad TP = \sum_i p_i g_i,\; FP = \sum_i p_i (1-g_i),\; FN = \sum_i (1-p_i) g_i $$

$\alpha = \beta = 0.5$ recovers Dice; $\alpha = \beta = 1$ recovers Jaccard. Setting $\beta > \alpha$ penalises misses more heavily and raises recall.

Focal, from Lin et al. (2017), with $p_t = p$ if $g = 1$ else $1 - p$:

$$ \mathcal{L}_{\text{focal}} = -\alpha_t (1 - p_t)^{\gamma} \log p_t $$

The modulating factor $(1-p_t)^\gamma$ shrinks the contribution of well-classified pixels. It rebalances by difficulty, not by region overlap, which is why it complements Dice rather than replacing it.

Gradient behaviour, and why it matters#

The Dice gradient with respect to one prediction $p_k$ is

$$ \frac{\partial \mathcal{L}_{\text{Dice}}}{\partial p_k} = -\frac{2 g_k (\sum_i p_i + \sum_i g_i) - 2\sum_i p_i g_i}{(\sum_i p_i + \sum_i g_i + \epsilon)^2} $$

Three consequences follow directly.

• The gradient is non-local. Every pixel's gradient depends on the sums over all pixels, so one pixel's update depends on the rest of the image. Cross-entropy has no such coupling.
• The denominator scales with object size. Small objects produce large per-pixel gradients. That is the desired imbalance correction, and the source of instability when an object is a handful of pixels.
• Near-empty predictions produce vanishing signal. With $\sum p_i \approx 0$ and $\sum g_i \approx 0$, the loss is dominated by $\epsilon$, and gradients are tiny. This is the empty-mask pathology in the developer section, stated analytically.

Direct surrogates for IoU#

Dice and soft-Jaccard are relaxations of the discrete metric, not tight surrogates. Two lines of work address that.

Lovász-Softmax (Berman, Rannen Triki, Blaschko, CVPR 2018) uses the Lovász extension of the submodular Jaccard set function. The result is a convex, piecewise-linear surrogate that is tight on the discrete metric. It requires sorting the per-pixel errors, so it costs $O(N \log N)$ per class. It reliably improves mIoU when used as a fine-tuning loss after cross-entropy pre-training, and is less reliable trained from scratch.

Eelbode et al. (2020) analyse the relationship between the soft relaxations and the metrics. They give conditions under which optimising one optimises the other. The paper is Optimization for Medical Image Segmentation: Theory and Practice when Evaluating with Dice Score or Jaccard Index.

Boundary-aware terms#

Region losses are blind to how far a wrong pixel is from the truth. The one-pixel-offset wire in the developer section scores like a total miss.

Boundary loss (Kervadec et al., MIDL 2019) integrates the predicted region against a pre-computed distance map of the true boundary. Its gradient is then proportional to distance from that boundary. It is used with a scheduled weight, starting near zero and rising, because alone it is unstable.

Hausdorff-distance surrogates and surface Dice (also called normalised surface distance) target the same failure. Surface Dice with a tolerance in millimetres is now standard in medical challenges, because clinicians care about boundary error in physical units, not pixel overlap.

What the comparisons find#

Ma et al. (2021), Loss Odyssey in Medical Image Segmentation, benchmark around twenty losses across four segmentation tasks. Their headline conclusions are stable and worth internalising:

• No single loss wins across tasks.
• Compound losses, particularly Dice combined with cross-entropy, are the most robust default.
• Combining a region term with a boundary term is the most effective choice for small structures.

That is close to what the training run above shows on a toy problem: the compound loss is fastest, and the final gaps are small.

Practical recipe#

1. Start with cross-entropy plus Dice, equally weighted. Deviating needs a reason and a measurement.
2. If misses matter more than false alarms, move to Tversky with $\beta > \alpha$ and tune on validation data, not intuition.
3. If your evaluation is boundary-based, add a boundary or surface term with a warm-up schedule.
4. Report the metric you optimised and one you did not. A model tuned on Dice alone can have poor boundary accuracy and poor calibration at once.
5. For multi-class, decide explicitly how absent classes are handled. That single decision moves reported mean Dice by several points.

Papers#

• Milletari, Navab, Ahmadi, V-Net, 3DV 2016 — arxiv.org/abs/1606.04797
• Salehi, Erdogmus, Gholipour, Tversky loss function for image segmentation using 3D fully convolutional deep networks, MLMI 2017 — arxiv.org/abs/1706.05721
• Lin, Goyal, Girshick, He, Dollár, Focal Loss for Dense Object Detection, ICCV 2017 — arxiv.org/abs/1708.02002
• Berman, Rannen Triki, Blaschko, The Lovász-Softmax loss, CVPR 2018 — arxiv.org/abs/1705.08790
• Kervadec et al., Boundary loss for highly unbalanced segmentation, MIDL 2019 — arxiv.org/abs/1812.07032
• Ma et al., Loss Odyssey in Medical Image Segmentation, Medical Image Analysis 2021 — arxiv.org/abs/2005.13449

What to learn next#

• Loss functions — the general picture these are special cases of.
• Imbalanced data — the same problem outside segmentation.
• Cleaning up predicted masks — what to do once the loss has done all it can.