Generalization by Construction

In solution inference, learning provides an evaluator that scores proposed solutions. Given the current problem, a sampler or optimizer uses this evaluator to compare candidates and revise them toward a better solution. The evaluator remains fixed during this search, while the candidate changes.

A major advantage of solution inference is that verification is often easier than generation. In the coffee task, a direct predictor must produce a suitable placement from the kitchen scene. An evaluator can judge familiar properties of a proposed placement, such as stability and reachability, without learning how to produce the placement itself. A model that learns these simpler judgments can generalize better to unfamiliar scenes than one trained to predict a complete solution. Search uses them to construct a placement for the current scene.

An energy-based model is one way to implement solution inference. A neural network learns an energy landscape over the space of possible solutions, assigning low energy to correct candidates. Langevin dynamics performs inference by iteratively refining a sample along the negative energy gradient while adding random noise. By varying the number of sampling steps, we can trade off sample quality against computation time. We can initialize inference either from random noise or from a generated proposal. We can also adapt inference to satisfy specific constraints and obtain desired solutions. For instance, we can implement image inpainting by fixing the values of observed pixels and letting inference generate the rest of the image.

This formulation also lets a model spend more computation on harder reasoning problems. In iterative reasoning through energy minimization, each optimization step revises a candidate answer under a learned energy function. A model trained to predict shortest-path distances on graphs with up to ten nodes achieves lower error on fifteen-node graphs than the feedforward and iterative baselines tested. The verifier–generator analysis shows how recognizing a solution can require far less computation than constructing one.

A language model can also be used in this way. The negative log probability it assigns to a candidate sequence acts as an energy. We can then perform solution inference through iterative sampling that favors lower-energy responses. This procedure substantially improves reasoning performance over ordinary sampling from the base language model. On several reasoning tasks, it performs comparably to versions of the same models finetuned with reinforcement learning, without any additional training.

The effectiveness of this search depends on how training shapes the geometry of the energy landscape. A sequence of landscapes can guide inference from broad, smooth basins toward sharper constraints, helping the solver approach a solution before resolving its finer details. Equilibrium Matching instead learns a single smoothed energy landscape for optimization.

A new task may require a combination of goals and constraints that no single learned model represents. Model composition lets the system construct a task model from reusable factors after the task is known. Each factor evaluates a particular aspect of a candidate solution. Inference selects and combines the relevant factors to guide the search for a solution.

For the coffee task, the robot can combine factors that favor an upright mug, clearance from nearby dishes, stable support on the table, and reachability for the person. A support factor can evaluate the mug and its supporting surface, while a clearance factor evaluates the mug and nearby objects. Neither factor needs to model the entire kitchen arrangement. By composing these local judgments, the robot can evaluate an arrangement it has never seen as a whole.

Composition provides a way to program a new task using factors learned from earlier data. Diffusion-CCSP learns factors for constraint types such as collision avoidance and spatial relations. A task specifies a graph connecting these factors to the object poses they constrain. Adding their energies defines a joint criterion, allowing inference to search for poses that satisfy unfamiliar combinations of familiar constraints. Set It Up! builds on this idea by using a language model to construct relation graphs from instructions, examples, and a task sketch. The graph specifies which learned factors guide the arrangement.

Visual factors can support both generating a scene and understanding one. In compositional visual generation, learned factors for individual concepts are combined to specify what an image should contain. For compositional scene understanding, inference searches for the object locations or attributes under which the combined factors best explain an observed image. The learned factors remain reusable, while the inferred locations and attributes describe the particular scene.

Composing additional instances of a learned factor can also extend inference to larger problems than those used in training. In Compositional Generative Inverse Design, factors trained on trajectories of two interacting bodies are composed to design initial conditions for systems with four or eight bodies. Each factor evaluates the motion of a pair within the same global trajectory. Inference jointly revises the initial conditions and trajectory under the combined dynamics factors and the design objective. The larger system does not have to be learned as a whole.

The same principle applies to reasoning. For Boolean satisfiability, clause-level factors can be learned and their composition refined on small problems, then composed into larger instances at test time. With training instances of at most twenty variables, this approach solves more fifty-variable problems than the neural baselines tested. Familiar local constraints can therefore support search over a larger, unfamiliar problem.

Factors can also draw on knowledge from models pretrained for different purposes. Iterative Consensus combines a pretrained generator with factors supplied by separately learned evaluators. For image generation, these factors judge whether candidate images match a requested description. Product of Experts for Visual Generation combines factors derived from visual models with constraints from graphics or physics. An appearance factor can favor realistic video, while a motion factor favors agreement with a simulated trajectory. The full task is expressed by selecting and combining the relevant factors at inference time.

The vocabulary of factors does not need to be manually designed. Learning can discover factors in an unsupervised manner. It can also infer a new task factor from demonstrations while leaving a pretrained generator unchanged. A newly learned factor can then be combined with earlier ones to express further tasks.

The combined factors must be paired with an inference procedure suited to them. For example, summing the scores of diffusion factors and running standard reverse diffusion does not in general sample the intended product distribution. Reduce, Reuse, Recycle develops sampling procedures to address this mismatch. Effective composition depends on both the reusable factors and the procedure used to reconcile them.

For decision making, factors can express both what should happen and what actions can cause. Objective factors represent goals, rewards, and constraints, while dynamics factors capture the behavior of objects, agents, and their interactions. Action planning uses their combination to search for actions whose predicted consequences satisfy the objective.

Composing a desired outcome with a model of what can happen defines a decision problem, but it does not determine how the agent should act. In the coffee task, the robot must still choose a grasp, lift the mug clear of nearby dishes, carry it without spilling, and place it at the selected location. Planning leaves this action sequence open. The planner predicts the consequences of possible actions and searches for a sequence expected to reach the desired state.

A direct policy resolves the action choice through a learned mapping from observations and goals. When a new situation falls outside the regularities captured by this mapping, the policy may fail even if the relevant dynamics and skills are familiar. Planning keeps the action sequence available for search under the current task model. Behavior initially constructed through search can later be absorbed into a policy or skill when it recurs.

Planning does not always operate directly on individual motor commands. It can search over subgoals, intermediate states, or complete trajectories, depending on what must be decided for the current task. Searching over intermediate states under learned dynamics allowed a planner to route around obstacles not encountered during training. A generative model of complete state-action trajectories provides a space of possible behaviors in which new rewards, endpoints, or constraints can guide search. In both cases, learning captures what recurs across tasks while search resolves the route required by the task at hand.

Predictive control repeats this search as the world changes. For locomotion, a learned trajectory prior can be combined with new rewards and constraints to adapt behavior to the current task. A task planner can also be composed with action-conditioned dynamics to improve the feasibility of trajectories that pursue the current goal.

For longer-horizon tasks, video provides an intermediate representation between a high-level goal and low-level control. It can describe how the scene should change while an embodiment-specific model determines how a particular robot makes that change happen. A learned video model can propose a task-level visual trajectory from which actions are extracted. Search can compare and connect these proposals into longer-horizon plans.

Different models can also divide the problem across levels of abstraction. We can compose language, video, and action models so that language proposes high-level steps, video grounds them in the scene, and an action model tests their physical executability. The components provide complementary views of the same evolving future. Planning searches for a course of action supported at every level, using the structure supplied by model composition.

Action proposal, consequence prediction, and search can also be separated within a planner. A vision-language model specifies a sequence of action primitives, which a low-level controller translates into robot actions. An action-conditioned world model predicts their consequences, and global correction and local search revise the proposals for the current task. The same verification–generation separation that motivates solution inference therefore reappears in sequential decision making. Judging whether an imagined consequence makes progress can be easier than generating the entire successful behavior at once.

Generalization by construction also applies to the computation used to solve a problem. An unfamiliar task may require a new sequence of familiar reasoning operations or a different combination of specialists. By leaving these choices open to inference, a system can reuse learned capabilities to construct a reasoning process suited to the current problem.

Results from previous rounds of inference can guide what computation the system performs next. Procedure control combines this feedback with the current setting. A failed verification can identify which parts of a candidate to preserve and which to revise, while a successful check can let the system stop reasoning and begin execution. The system can also estimate task difficulty from these results and adjust the sequence and amount of computation accordingly.

For the robot, an uncertain estimate of the mug's pose may call for another perception step before planning. If verification identifies a problem with the grasp, the next round of planning can focus on revising that part of the trajectory. If the trajectory passes verification, the controller can proceed to execution. During execution, the model's estimated likelihood of the current plan can provide further feedback for deciding whether to continue or replan from the existing trajectory.

Verification can also determine where the next round of generation begins. If a verifier identifies where a generated trajectory first goes wrong, the system can preserve a valid prefix and resume generation from it. The next round can then focus on extending a partial solution that has already passed verification. The generate–verify–restart loop stays fixed, while feedback determines which prefix to expand and when to stop. On Keys-and-Doors puzzles with solution paths longer than those in training, this procedure improves success over sampling complete rollouts at the same generation budget. Symbolic verification identifies the prefixes that can be retained, and every solved episode in this held-out setting comes from a restarted branch.

Repeated failures or slow progress can indicate that the current task is harder than expected. The controller can then devote more computation to unresolved decisions or try a different reasoning operation. The choice of model size and numerical precision also affects both answer quality and latency. For an embodied system, the value of another reasoning step depends on whether it can improve the action before that action is needed.

Procedure control chooses the next operation within an already assembled system. Reasoner organization leaves the assembly itself open. Inference may determine which specialists and tools participate, what roles they take on, what state is shared, and how information moves among them. Model composition constructs the task model that defines what counts as a solution. Reasoner organization assembles the specialists that supply or evaluate the relevant information.

One benefit of using multiple reasoners is diversity in how they approach a problem. A single reasoning process can become stuck repeatedly revising an unproductive candidate. Different reasoners can explore alternative candidates, assumptions, or strategies. By exchanging and debating their proposals, they can use one another's progress to redirect their own search. This gives the system ways to move beyond an unproductive line of reasoning on a hard problem. The benefit depends on preserving useful differences among reasoners and evaluating the alternatives they propose.

A second benefit is the ability to divide a complex task into simpler subproblems. In the coffee task, separate specialists can estimate the mug's pose, choose a stable grasp, and plan a collision-free route. Each specialist reasons about a narrower problem using knowledge relevant to that part of the task. Their decisions must then be coordinated so that the grasp and route are compatible. This decomposition can reduce the difficulty faced by each reasoner and let the system construct a solution to an unfamiliar task from familiar local capabilities. Verification can also be decomposed by assigning reasoners to check different aspects of a candidate and combining their judgments.

These benefits are possible even when the participants and coordination rules are fixed before deployment. Training can produce complementary generator and critic behaviors that preserve diverse reasoning chains and are reused across problems. In such systems, inference uses the interaction among reasoners to solve the current problem. Leaving the organization open adds the ability to choose which reasoners participate, how the task is divided, and how their contributions are combined for the problem at hand.

In the coffee task, suppose a single planner repeatedly fails to find a reliable grasp. The system can then assign several reasoners to explore different grasp strategies and ask another to compare their proposals. If the proposals still fail verification, it can recruit a grasp specialist and route that specialist's feedback to the proposing reasoners. The goal and task model remain fixed while inference changes the participants, their responsibilities, and the information they exchange.

Reusable incentives and local rules can govern this adaptation by determining when reasoners participate and how their contributions are valued, allowing a task-specific organization to emerge without a central scheduler. For example, auctions can select a problem-dependent sequence of specialists from a fixed population. The population stays fixed during evaluation, while economic selection reshapes it across optimization episodes. Selecting who acts also determines what computation runs next, so this mechanism spans procedure control and reasoner organization.