Predictive Physical Intelligence: Learning to Understand and Simulate the 3D World

Speaker: Chanho Kim, Oregon State University

Date: Oct. 30

Time: 2-3 p.m.

Q&A: 3-3:30 p.m.

Location: KEC 1001

Zoom

Title: Predictive Physical Intelligence: Learning to Understand and Simulate the 3D World

Abstract: Modern AI systems have achieved remarkable progress in visual recognition and 3D reconstruction, yet they still have limited ability to predict how the physical world will evolve or respond to actions. Building such predictive capabilities is important for both autonomous systems, which must reason about the consequences of their actions, and scientific and engineering applications, where learned models could complement conventional physical simulation. In this talk, I will present our recent work toward predictive physical intelligence: AI systems that construct structured representations of the 3D world, learn its dynamics from observations, and predict future physical states. I will discuss several complementary directions, including hierarchical point cloud representations for modeling physical interactions, learning particle-based dynamics directly from real-world videos, action-conditioned 3D point world models for robot planning, and semantic SLAM for constructing persistent 3D representations of geometry, appearance, and semantics from real-world observations. Together, these efforts point toward a broader goal: AI systems that can observe the physical world, build useful 3D and 4D representations, and learn and predict how the world changes. I will conclude with ongoing challenges and opportunities in scaling these models toward more general physical environments.

Bio: Chanho Kim is an Assistant Professor (Senior Research) in the School of Electrical Engineering and Computer Science at Oregon State University. His research lies at the intersection of computer vision and robotics, with a focus on building AI systems that can understand and predict the physical world. His recent work spans learned physical simulation, SLAM, 3D point tracking, and predictive world models for robotic systems. More broadly, his research aims to develop unified frameworks for learning structured 3D representations and physical dynamics directly from real-world observations. He received his Ph.D. from Georgia Tech, where his research focused on multi-object tracking.