Machine learning researchers have developed a technique that allows world models to make better predictions about how objects will behave. World models are used to train physical AI systems like robots how to navigate the real world. They predict the consequences of actions. But they do so based on observed data. These models could provide better direction to robots if they were able to make inferences about possible actions without being trained for all the possible outcomes. Randall Balestriero (Brown University, AMI Labs), Kerda Varaku, and Turing Award winner Yann LeCun (NYU, AMI Labs) describe an approach that might allow that to happen in a pre-print paper, "Counterfactual World Models: Natura Non Facit Saltus." The Latin refers to Gottfried Leibniz’s axiom " nature makes no leaps ," which had a good run until it ran headlong into quantum mechanics. Yet for the authors, the saying explains how they have been able to devise a method for zero-shot physics inference. Counterfactual world models, said Balestriero in an email to The Register use "the mismatch between your prediction and the real world to figure out what did you get wrong, and from that you can better understand what happened for better future predictions, debugging, reasoning, all at test time." Joint Embedding Predictive Architecture ( JEPA ) models, developed by LeCunn and colleagues back in 2023, have been proposed for robots because they can take an action as input and ask what the world would be like with that action. But a JEPA world model produces an average of outcomes and would not be able to anticipate counterfactual scenarios – events that didn’t happen. The technique that the researchers developed is not specific to JEPAs – it can be applied to any world model that predicts the average next state under any action. But JEPA models are particularly well suited to being adapted. Balestriero said that the counterfactual world model experiments were done using off-the-shelf encoders (DINOv2 and MAE) without any fine-tuning and only a small predictor trained on top. Creating a counterfactual world model from that starting point would only require minor additions. Counterfactual world models predict alternative scenarios more effectively by inferring possibilities from what did occur. "In causal inference, answering ‘What would have happened had I acted differently?’ takes three steps," explained Balestriero. "You use what you observed to learn something about those hidden factors, you change the action, and you predict again with the hidden factors kept exactly as they were. "Our counterfactual world model does all three steps with one subtraction." Balestriero cited an example from the paper: a boat crossing a river with a hidden current and how the JEPA world model and counterfactual world model predicted the boat’s movements. "The world model predicted where the boat would end up after steering left, and it actually ended up two meters further downstream," he explained. "Those two meters are the signature of the current that occurred. To answer ‘where would the boat be had I steered right?’ , we take the world model’s prediction for steering right and add the same two meters. In other words, the counterfactual is what actually happened, plus the difference between the model’s predictions for the alternative action and for the action taken. "Physics enters to justify this rule, not as an equation the model fits…