SAFER

Navigating the Exploration–Exploitation Dilemma in World-Model-Based Planning for Autonomous Driving

Yansong Qu1, Zilin Huang2, Zihao Sheng2, Sikai Chen2,*, Samuel Labi1

1Purdue University2University of Wisconsin–Madison*Corresponding author

A recorded decision of the planner (CARLA, Town02, 6.3 m/s), unfolded over the 1.6 s that it imagines, on the bird's-eye view that it observed. Its 270 candidate action chunks, each the path it traces at the world model's predicted speeds and coloured by the predicted reward of each imagined step (low to high); risk area, the risk head's probability that a collision has happened by then (drawn more opaque than in the paper's figure); dotted behaviour-cloned prior, which cuts the bend towards the pedestrian; executed plan, which keeps its lane.

Closed-loop videos

Every candidate the planner scores, drawn as the path it traces, coloured by the reward that the world model predicts for each imagined step, with the executed plan and the risk signal over the last 30 s. In CARLA the planner sees only the bird's-eye view on the right; the camera view is for display.

CARLA · Leaderboard routes · full routes, stops of 3 s or more at 10×

Town02, route 1. The full route, completed without a collision or a red-light violation.

Town01, route 7. The full route, completed without a collision or a red-light violation.

Town06, route 49. The full route, completed without a collision or a red-light violation.

MetaDrive · CAT test scenarios · logged traffic, then again against CAT's opponent

CAT scenario 449.

CAT scenario 405.

CAT scenario 416.

Offline experience, online refinement

SAFER learns from failures that others make, judged by an expert, and then keeps refining itself on its own driving.

offline · before it drives

Learn from others' failures

Partially trained agents and scripted, recoverable perturbations fail in simulation, and an expert labels every state they visit with its action and value. The world model learns what mistakes lead to, and the planner learns what should have been done instead, so that much of what it explores is covered before it drives.

online · while it drives

Keep refining on its own driving

Once the planner drives, an online loop continuously refines the world model's dynamics, reward scorers and risk head on the planner's own driving, wherever that driving leaves the coverage of the offline data. The planner scores with slowly moving copies of the refined heads, so that consecutive plans stay consistent.

The dilemma

A world model lets a planner compare imagined futures before committing to one, but it can only foresee what its training data contain. A planner that keeps to this coverage can rely on the predictions but can hardly do better than the driving in the data. A planner that explores beyond it meets actions that the model misjudges, and because it picks the candidate with the highest predicted value, it is drawn to exactly those.

Widening the coverage takes experience of failure. Logs of competent driving rarely contain it, and agents that learn from interaction gather it only by failing themselves, many times, learning the outcome of each failure but not what should have been done instead.

Learning paradigms for autonomous driving, each with a glyph of how it treats the coverage of its data: imitation learning, reinforcement learning, world models learned online or from logs, and SAFER.
Where the experience beyond competent driving comes from. The glyph under each paradigm shows the coverage of its data (gray) and how the paradigm treats it: exploiting (blue) takes the best candidate up to the edge of the coverage but no further; exploring (orange) grows the coverage at the price of failures (red crosses), the paradigm's own or, for SAFER, failures that others make, labeled by the expert (teal); a step out of the coverage that does not grow it is misjudged (?).

How SAFER learns

A child is not left to discover the consequences of every choice alone: parents praise what works and point to negative examples, and school and work later keep reshaping these beliefs in situations the parents never showed. SAFER follows the same course, in five stages.

offline
SStage

Partially trained agents and scripted, recoverable perturbations drive in simulation until their failures run to their consequences.

AAnnotate

The expert labels every visited state with its action and value: what should have been done instead.

planning
FForesee

A chunked latent world model imagines a whole 16-step action chunk in one pass and predicts reward factors, expert values and collision probabilities.

EEvaluate

A risk-aware MPPI planner scores candidate manoeuvres by their predicted return minus a bounded collision penalty.

online
RRefine

An online loop continuously refines the dynamics, the reward scorers and the risk head on the planner's own driving.

An everyday analogy for SAFER: parents, the child learning what leads where, and school and work, matched to the offline data, the world model and planner, and online refinement.
An everyday analogy for SAFER (top) and the corresponding stages of the method (bottom); the numbered badges link each stage to its scene. Parents praise what works and point to negative examples (1), as weaker drivers and scripted perturbations stage failures that the expert labels with its action and value (a). The child learns what each choice leads to (2), as the world model foresees the reward, value and risk of each candidate chunk (b) and the planner scores the candidates by their return minus a price on risk (c). Away from home, school and work keep reshaping the beliefs while the core ones stay (3), as the planner's own driving refines the dynamics, reward and risk while the prior and the value stay (d).
Technical overview of SAFER in its CARLA instance.
Technical overview of SAFER in its CARLA instance; the numbered badges link each part to its scene of the analogy above. (a) Stage and Annotate: the expert, weaker checkpoints and scripted events drive until the first violation, and the expert labels every visited state. (b) Foresee: the chunked latent world model imagines the 16 latents of a candidate chunk in one pass and reads them as prior, reward, survival, value and collision risk. (c) Evaluate: every 0.1 s the risk-aware MPPI planner scores 135 candidate chunks in each of two iterations and executes the first action. (d) Refine: the planner's own driving adapts the dynamics, the reward scorers and the risk head online.

Inside the planner

At every decision the planner scores 270 candidate action chunks of 16 steps in two MPPI iterations. The world model imagines each chunk in one pass and predicts the reward of every imagined step, the terminal value and, through the risk head, the probability that a collision has happened by then. A candidate loses score the longer, and the deeper, it runs through the area where the risk head expects a collision, so the plan follows the part of the reachable set where that risk is lowest. The planner decides at 43–45 Hz on a single GPU, with 15.5 M parameters.

CARLA · Town02 · a pedestrian stands at the inner kerb of a left bend · 6.3 m/s

Front camera and bird's-eye view of a recorded CARLA decision: the candidates fan out along a left bend, the risk area lies at a pedestrian on the inner kerb, the prior cuts the bend into it and the plan keeps its lane.

MetaDrive · CAT test scenario 449 · CAT's opponent cuts in from the right lane · 14.4 m/s

Front camera and top-down view of a recorded MetaDrive decision: CAT's opponent cuts in from the right lane and the plan keeps its lane at the edge of the reachable set where the risk is lowest.
predicted reward CARLA −1 … 1, MetaDrive −1 … 2 risk area risk-head probability 0 … 1 executed plan behaviour-cloned prior CAT's opponent, generated path candidates

The imagined reachable set and its risk area in two recorded closed-loop decisions of the full system: the front camera (for display; the planner does not see it) with the candidates and the risk area projected onto the road, the top-down view (in CARLA the bird's-eye view that the planner observes), and the risk-head probability against imagined time for every candidate, the prior and the plan, with their risk terms Wρ̄. In CARLA the prior cuts the bend into the risk area and the plan keeps its lane outside it. In MetaDrive CAT's opponent cuts into the ego's lane, and the plan keeps its lane at the edge of the reachable set, where the risk is lowest.

Results

SAFER is evaluated in closed loop on the 76 CARLA Leaderboard routes and on the safety-critical benchmark of CAT in MetaDrive, each with three seeds.

0.95 / 0.77CARLA route completion / reward per step

The highest among the compared methods on the Leaderboard routes, from a privileged bird's-eye view.

53.4 / 49.0MetaDrive episode reward

The highest among the compared methods, in regular and in safety-critical traffic, with the highest road completion in both.

43–45 HzReal-time planning

270 imagined chunks per decision on a single GPU, more than four times the 10 Hz at which the agent acts.

CARLA Leaderboard

MethodInputDriving score ↑Route compl. ↑Infraction penalty ↑
CILRSRGB0.050.140.55
LBCRGB0.090.180.73
TransFuser (2021)RGB + LiDAR0.170.520.42
NEATRGB0.220.420.65
MMFNRGB + LiDAR + HD map0.230.470.63
IARLRGB0.250.470.52
RailsRGB0.310.580.56
GRIADRGB0.370.620.60
Latent TransFuserRGB0.450.660.72
TransFuser (2023)RGB + LiDAR0.610.870.71
LAVRGB + LiDAR0.620.940.64
TF++RGB + LiDAR0.660.790.84
TCPRGB0.750.860.87
InterFuserRGB + LiDAR0.760.880.84
TD-MPC2BEV + ego state0.430.700.59
TD3+BCBEV + ego state0.630.840.76
IQLBEV + ego state0.730.880.82
DreamerV3BEV + ego state0.820.940.88
SAFER (ours)BEV + ego state0.800.950.84

The end-to-end methods (top) are given with their published CARLA Leaderboard 1.0 results. TD-MPC2, TD3+BC, IQL and DreamerV3 (middle) are trained by us on SAFER's corpus, with the frozen encoder of the expert and without its labels, and evaluated as SAFER. SAFER observes the privileged bird's-eye view of Roach and is scored with the Leaderboard's metrics on the 76 Leaderboard routes; the comparison places its driving decisions among those of the baselines, not complete pipelines from sensors to control.

CARLA · methods with SAFER's input

MethodReward per step ↑Speed (m/s) ↑Completed (%) ↑Blocked (%) ↓Per 100 km ↓
PedestriansVehiclesRed lightsStop signs
TD-MPC2−0.512.6143.052.65.627.23.80.7
TD3+BC0.533.0868.030.38.427.94.72.6
IQL0.473.2075.421.54.219.33.93.9
DreamerV30.713.2688.211.81.117.12.10.0
SAFER (ours)0.773.3189.011.05.516.01.61.8

The methods of the table above whose input is exactly that of SAFER, the bird's-eye view and the ego state, on the same routes under the same seeds; mean over three seeds. Reward per step: SAFER's reward function; speed: route distance completed over the duration of the episode; collisions and traffic-rule infractions per 100 km driven.

What each stage adds

Driving score on the 76 routes, mean over three seeds. Planning with the world model alone is misled beyond the data and falls below the behaviour-cloned prior; pricing the collision risk learned from the staged failures makes planning pay off, and online refinement adds more on top.

MetaDrive with CAT's safety-critical opponents

MethodRegular trafficSafety-critical traffic
Reward ↑Road compl. ↑Crash ↓Reward ↑Road compl. ↑Crash ↓

100 test scenarios of CAT's benchmark, mean over three seeds; road completion and crash rate in %. Baseline results are from CurricuVLM (Sheng et al., 2026), except DreamerV3, which we trained. In safety-critical traffic, CAT's opponent is generated against the regular drive.

The risk head foresees recorded collisions

Replayed on 450 recorded collisions of the corpus and 150 collision-free control segments, the predicted probability of a collision within the 1.6 s horizon rises before the impact while the controls stay near zero: the separation reaches an AUC of 0.97 two seconds before the impact and at least 0.998 from 1.1 s on.

Predicted collision probability against time to impact for recorded collisions and control segments, and the AUC of the separation.

Citation

@article{qu2026safer,
  title   = {{SAFER}: Navigating the Exploration--Exploitation Dilemma in
             World-Model-Based Planning for Autonomous Driving},
  author  = {Qu, Yansong and Huang, Zilin and Sheng, Zihao and Chen, Sikai and Labi, Samuel},
  journal = {arXiv preprint},
  year    = {2026}
}