Closed-loop videos
Every candidate the planner scores, drawn as the path it traces, coloured by the reward that the world model predicts for each imagined step, with the executed plan and the risk signal over the last 30 s. In CARLA the planner sees only the bird's-eye view on the right; the camera view is for display.
CARLA · Leaderboard routes · full routes, stops of 3 s or more at 10×
Town02, route 1. The full route, completed without a collision or a red-light violation.
Town01, route 7. The full route, completed without a collision or a red-light violation.
Town06, route 49. The full route, completed without a collision or a red-light violation.
MetaDrive · CAT test scenarios · logged traffic, then again against CAT's opponent
CAT scenario 449.
CAT scenario 405.
CAT scenario 416.
Offline experience, online refinement
SAFER learns from failures that others make, judged by an expert, and then keeps refining itself on its own driving.
Learn from others' failures
Partially trained agents and scripted, recoverable perturbations fail in simulation, and an expert labels every state they visit with its action and value. The world model learns what mistakes lead to, and the planner learns what should have been done instead, so that much of what it explores is covered before it drives.
Keep refining on its own driving
Once the planner drives, an online loop continuously refines the world model's dynamics, reward scorers and risk head on the planner's own driving, wherever that driving leaves the coverage of the offline data. The planner scores with slowly moving copies of the refined heads, so that consecutive plans stay consistent.
The dilemma
A world model lets a planner compare imagined futures before committing to one, but it can only foresee what its training data contain. A planner that keeps to this coverage can rely on the predictions but can hardly do better than the driving in the data. A planner that explores beyond it meets actions that the model misjudges, and because it picks the candidate with the highest predicted value, it is drawn to exactly those.
Widening the coverage takes experience of failure. Logs of competent driving rarely contain it, and agents that learn from interaction gather it only by failing themselves, many times, learning the outcome of each failure but not what should have been done instead.
How SAFER learns
A child is not left to discover the consequences of every choice alone: parents praise what works and point to negative examples, and school and work later keep reshaping these beliefs in situations the parents never showed. SAFER follows the same course, in five stages.
Partially trained agents and scripted, recoverable perturbations drive in simulation until their failures run to their consequences.
The expert labels every visited state with its action and value: what should have been done instead.
A chunked latent world model imagines a whole 16-step action chunk in one pass and predicts reward factors, expert values and collision probabilities.
A risk-aware MPPI planner scores candidate manoeuvres by their predicted return minus a bounded collision penalty.
An online loop continuously refines the dynamics, the reward scorers and the risk head on the planner's own driving.
Inside the planner
At every decision the planner scores 270 candidate action chunks of 16 steps in two MPPI iterations. The world model imagines each chunk in one pass and predicts the reward of every imagined step, the terminal value and, through the risk head, the probability that a collision has happened by then. A candidate loses score the longer, and the deeper, it runs through the area where the risk head expects a collision, so the plan follows the part of the reachable set where that risk is lowest. The planner decides at 43–45 Hz on a single GPU, with 15.5 M parameters.
CARLA · Town02 · a pedestrian stands at the inner kerb of a left bend · 6.3 m/s
MetaDrive · CAT test scenario 449 · CAT's opponent cuts in from the right lane · 14.4 m/s
The imagined reachable set and its risk area in two recorded closed-loop decisions of the full system: the front camera (for display; the planner does not see it) with the candidates and the risk area projected onto the road, the top-down view (in CARLA the bird's-eye view that the planner observes), and the risk-head probability against imagined time for every candidate, the prior and the plan, with their risk terms Wρ̄. In CARLA the prior cuts the bend into the risk area and the plan keeps its lane outside it. In MetaDrive CAT's opponent cuts into the ego's lane, and the plan keeps its lane at the edge of the reachable set, where the risk is lowest.
Results
SAFER is evaluated in closed loop on the 76 CARLA Leaderboard routes and on the safety-critical benchmark of CAT in MetaDrive, each with three seeds.
The highest among the compared methods on the Leaderboard routes, from a privileged bird's-eye view.
The highest among the compared methods, in regular and in safety-critical traffic, with the highest road completion in both.
270 imagined chunks per decision on a single GPU, more than four times the 10 Hz at which the agent acts.
CARLA Leaderboard
| Method | Input | Driving score ↑ | Route compl. ↑ | Infraction penalty ↑ |
|---|---|---|---|---|
| CILRS | RGB | 0.05 | 0.14 | 0.55 |
| LBC | RGB | 0.09 | 0.18 | 0.73 |
| TransFuser (2021) | RGB + LiDAR | 0.17 | 0.52 | 0.42 |
| NEAT | RGB | 0.22 | 0.42 | 0.65 |
| MMFN | RGB + LiDAR + HD map | 0.23 | 0.47 | 0.63 |
| IARL | RGB | 0.25 | 0.47 | 0.52 |
| Rails | RGB | 0.31 | 0.58 | 0.56 |
| GRIAD | RGB | 0.37 | 0.62 | 0.60 |
| Latent TransFuser | RGB | 0.45 | 0.66 | 0.72 |
| TransFuser (2023) | RGB + LiDAR | 0.61 | 0.87 | 0.71 |
| LAV | RGB + LiDAR | 0.62 | 0.94 | 0.64 |
| TF++ | RGB + LiDAR | 0.66 | 0.79 | 0.84 |
| TCP | RGB | 0.75 | 0.86 | 0.87 |
| InterFuser | RGB + LiDAR | 0.76 | 0.88 | 0.84 |
| TD-MPC2 | BEV + ego state | 0.43 | 0.70 | 0.59 |
| TD3+BC | BEV + ego state | 0.63 | 0.84 | 0.76 |
| IQL | BEV + ego state | 0.73 | 0.88 | 0.82 |
| DreamerV3 | BEV + ego state | 0.82 | 0.94 | 0.88 |
| SAFER (ours) | BEV + ego state | 0.80 | 0.95 | 0.84 |
The end-to-end methods (top) are given with their published CARLA Leaderboard 1.0 results. TD-MPC2, TD3+BC, IQL and DreamerV3 (middle) are trained by us on SAFER's corpus, with the frozen encoder of the expert and without its labels, and evaluated as SAFER. SAFER observes the privileged bird's-eye view of Roach and is scored with the Leaderboard's metrics on the 76 Leaderboard routes; the comparison places its driving decisions among those of the baselines, not complete pipelines from sensors to control.
CARLA · methods with SAFER's input
| Method | Reward per step ↑ | Speed (m/s) ↑ | Completed (%) ↑ | Blocked (%) ↓ | Per 100 km ↓ | |||
|---|---|---|---|---|---|---|---|---|
| Pedestrians | Vehicles | Red lights | Stop signs | |||||
| TD-MPC2 | −0.51 | 2.61 | 43.0 | 52.6 | 5.6 | 27.2 | 3.8 | 0.7 |
| TD3+BC | 0.53 | 3.08 | 68.0 | 30.3 | 8.4 | 27.9 | 4.7 | 2.6 |
| IQL | 0.47 | 3.20 | 75.4 | 21.5 | 4.2 | 19.3 | 3.9 | 3.9 |
| DreamerV3 | 0.71 | 3.26 | 88.2 | 11.8 | 1.1 | 17.1 | 2.1 | 0.0 |
| SAFER (ours) | 0.77 | 3.31 | 89.0 | 11.0 | 5.5 | 16.0 | 1.6 | 1.8 |
The methods of the table above whose input is exactly that of SAFER, the bird's-eye view and the ego state, on the same routes under the same seeds; mean over three seeds. Reward per step: SAFER's reward function; speed: route distance completed over the duration of the episode; collisions and traffic-rule infractions per 100 km driven.
What each stage adds
Driving score on the 76 routes, mean over three seeds. Planning with the world model alone is misled beyond the data and falls below the behaviour-cloned prior; pricing the collision risk learned from the staged failures makes planning pay off, and online refinement adds more on top.
MetaDrive with CAT's safety-critical opponents
| Method | Regular traffic | Safety-critical traffic | ||||
|---|---|---|---|---|---|---|
| Reward ↑ | Road compl. ↑ | Crash ↓ | Reward ↑ | Road compl. ↑ | Crash ↓ | |
100 test scenarios of CAT's benchmark, mean over three seeds; road completion and crash rate in %. Baseline results are from CurricuVLM (Sheng et al., 2026), except DreamerV3, which we trained. In safety-critical traffic, CAT's opponent is generated against the regular drive.
The risk head foresees recorded collisions
Replayed on 450 recorded collisions of the corpus and 150 collision-free control segments, the predicted probability of a collision within the 1.6 s horizon rises before the impact while the controls stay near zero: the separation reaches an AUC of 0.97 two seconds before the impact and at least 0.998 from 1.1 s on.
Citation
@article{qu2026safer,
title = {{SAFER}: Navigating the Exploration--Exploitation Dilemma in
World-Model-Based Planning for Autonomous Driving},
author = {Qu, Yansong and Huang, Zilin and Sheng, Zihao and Chen, Sikai and Labi, Samuel},
journal = {arXiv preprint},
year = {2026}
}