About Yilun Du

I study how intelligent systems can learn reusable models, then compose and reason with them at inference time.

Energy-based and diffusion models are common tools I use for capturing and composing this structure. I study these ideas across generation, reasoning, and decision making, including systems that coordinate multiple models or agents. A central goal of my research is to build robots that can combine knowledge of objects, dynamics, and skills to plan and act in unfamiliar environments.

I lead the Embodied Minds Lab at Harvard and am a part-time research scientist at NVIDIA Research. I received my PhD from MIT EECS, advised by Leslie Kaelbling, Tomas Lozano-Pérez, and Joshua B. Tenenbaum. Before Harvard, I held research roles at OpenAI and Google DeepMind.

Research

Generative Inference

KnowledgeSolution spacesReasoningSample and refine

I study how to construct and refine solutions using a learned generative model. Sampling and optimization use the model to propose candidates and improve them. I develop representations and inference procedures to make this search reliable and adapt computation to the problem.

Examples: Reasoning with Sampling improves language-model reasoning by changing the sampling procedure while keeping the model fixed. Deep EBMs learn energy landscapes for generation and conditional inference. Reduce, Reuse, Recycle develops samplers for composing pretrained diffusion models.

Compositional Inference

KnowledgeConcepts, constraints, and skillsReasoningCompose and solve

I study how to combine learned models and task requirements to define new inference problems. Models of concepts, constraints, and skills provide reusable components, while inference searches for solutions that satisfy the combined requirements. This lets us tackle new combinations without learning a separate model for each complete task.

Examples: Composable diffusion models combine learned concepts to generate unfamiliar combinations. Diffuser reuses a trajectory model under different goals and constraints. Compositional energy minimization assembles subproblem energies to solve larger reasoning problems.

World Models & Robotics

KnowledgeWorld dynamicsReasoningPredict and plan

I study how robots can use learned world models to plan and act in unfamiliar environments. These models predict the consequences of possible actions, while planning searches for actions that achieve the current goal. This lets robots reuse the same knowledge across tasks and adjust their behavior as goals and environments change.

Examples: UniPi generates video plans that can be translated into actions. Video Language Planning searches language and video predictions for long-horizon plans. Large Video Planner uses a pretrained video model to guide robot control.

Multi-Agent Intelligence

KnowledgeDistributed across agentsReasoningDebate and verify

I study multiagent computation as a way to scale language-model intelligence. Rather than relying only on stronger individual models or longer reasoning traces, we train complementary agents and organize how they explore, verify, and combine solutions. The central question is how additional collective computation can produce capabilities beyond those of individual models.

Examples: Multi-agent debate lets models critique and revise candidate answers. Multi-agent verification combines multiple verifiers to evaluate candidate solutions. Economy of Minds organizes agents through task-dependent economic interactions.

I develop the broader argument connecting these directions in Generalization by Construction, a research perspective on what learning should leave open for inference. My PhD thesis and thesis defense provide a detailed account of this view through energy-based modeling, compositional inference, and planning.

News & updates

  • [Perspective] I wrote Generalization by Construction, a research perspective on what learning should leave open for inference.
  • [Website] We launched an overview of our work on Energy-Based Models, showing how learned energy landscapes can be optimized and composed at inference time.
  • [2026] I am recruiting PhD students for the December 2026 application cycle through the Embodied Minds Lab.
  • [2026] We are organizing the ICML 2026 workshop on compositional learning.
  • [Talk] Presented compositional world models for embodied intelligence at the Kempner Frontiers of NeuroAI Symposium. Watch the talk.

Publications

Topics:
Showing 13 of 29 selected publications

* Equal contribution; † or + equal advising.