Relative velocity
A bounded linear multiplier captures relative impact speed.
CVPR 2026 · Simulation for Autonomous Driving Workshop
Beyond Binary Metrics
Unveiling the Safety Illusion in
Autonomous Driving Simulation
The University of Hong Kong
Binary metrics count collisions.CCM measures consequences.
01 / The safety illusion
A graze and a severe crash
should not count the same.
Binary collision labels collapse very different events into the same outcome. Bounding-box jitter, a brief contact, and a high-speed collision can all be recorded as a collision. A similar collision rate can therefore conceal very different severity distributions.
We study the gap between benchmark-facing realism and physical risk in autonomous driving simulation. Composite Severity Analysis represents collisions through impact velocity, penetration depth, and contact duration. The Composite Collision Metric (CCM) then summarizes severe outcomes in the tail of the distribution.
02 / Composite Collision Metric
From an event to its consequences.
Penetration depth
A quadratic penalty emphasizes structural penetration.
Contact duration
A temporal gate filters brief contact noise and saturates for sustained contact.
S(event) = m(vrel) × δ(d) × g(Δt)
For each collision event, vrel is relative impact velocity, d is penetration depth, and Δt is contact duration. The functions m, δ, and g convert these quantities into severity factors.
Event severity
Relative velocity
\[v_{\mathrm{rel}}=\|\vec v_1-\vec v_2\|\]
\[m(v_{\mathrm{rel}})=\frac{\min(\max(v_{\mathrm{rel}},v_{\min}),v_{\max})}{v_{\mathrm{ref}}}\]
Penetration depth
\[\delta(d)=\left(\frac{\max(d-\epsilon,0)}{d_{\mathrm{ref}}}\right)^2\]
Contact duration
\[g(\Delta t)=\begin{cases}0,&\Delta t\le t_{\mathrm{res}},\\[4pt]\left(\frac{\Delta t-t_{\mathrm{res}}}{t_{\mathrm{noise}}-t_{\mathrm{res}}}\right)^2,&t_{\mathrm{res}}<\Delta t\le t_{\mathrm{noise}},\\[4pt]1,&\Delta t>t_{\mathrm{noise}}.\end{cases}\]
Default parameters: vref = 5.0 m/s, dref = 0.5 m, vmin = 1.0 m/s, vmax = 40.0 m/s, tres = 0.1 s, tnoise = 0.2 s, and ε = 10−4.
Events are filtered before severity scoring: pedestrian–pedestrian contacts and pedestrian–vehicle contacts where the pedestrian's impact speed exceeds the vehicle's are excluded.
Unconditional tail risk
CCM = CVaR95(S)
Collision-free instances receive S = 0. CCM evaluates the worst 5% of the global severity distribution, capturing both collision frequency and consequence severity.
Global severity, including collision-free instances
95% of instances Worst 5%
CCM · Equations (12–13)
\[\operatorname{VaR}_{\alpha}(S)=\inf\{s\in\mathbb R:\Pr(S\le s)\ge\alpha\}\]
\[\mathrm{CCM}=\operatorname{CVaR}_{0.95}(S)=\mathbb E[S\mid S\ge\operatorname{VaR}_{0.95}(S)]\]
The paper separately reports conditional CVaR95 over meaningful collisions (C = 1), which describes severity within the collision subset:
\[\operatorname{CVaR}_{\alpha}(S\mid C=1)=\mathbb E[S\mid S\ge\operatorname{VaR}_{\alpha}(S\mid C=1),\ C=1]\]
03 / Results on WOSAC
Similar benchmark scores.
Different tail risks.
Safety-oriented rejection sampling in Safe CAT-K reduces CCM from 2.792 to 1.611, while collision likelihood changes only slightly.
| Model | Collision likelihood ↑ | Original collision rate (%) ↓ | Inter-vehicle collision rate (%) ↓ | CCM ↓ |
|---|---|---|---|---|
| Log Replay | — | 4.69 | 0.72 | 0.028 |
| SMART | 0.9643 | 6.75 | 2.36 | 5.557 |
| CAT-K | 0.9672 | 6.16 | 1.62 | 2.792 |
| Safe CAT-K | 0.9637 | 6.23 | 1.51 | 1.611 |
04 / Qualitative examples
Inspect collision behavior.
Selected collision scenarios illustrating vehicle interactions and contact dynamics.
05 / Poster
The full picture.
High realism scores do not
guarantee physical safety.
Severity-aware evaluation reveals rare, consequential failures that binary collision rates can hide.
Read the paper ↗