Latent World Models for Vision-Based Surgical Navigation: Pretraining and Planning for Robotic Endoluminal Intervention

  • Forschungsbereich:Medizinische Bildgebung für Modellierung und Simulation
  • Typ:Forschungsarbeit
  • Betreuung:

    M.Sc. Edoardo De Rose

  • Bearbeitung:

    M.Sc. Francesca Filice

  • Accurate intraoperative navigation is essential for robot-assisted endoluminal intervention, but remains difficult because of limited endoscopic field of view and dynamic artifacts. Existing navigation platforms often rely on external localization technologies, such as electromagnetic tracking or shape sensing, which increase hardware complexity and remain vulnerable to intraoperative anatomical mismatch. Despite recent advances in deep learning for surgical vision, most current approaches rely on supervised learning for specific tasks (e.g., segmentation, phase recognition), and do not explicitly model temporal dynamics of the environment; lack planning capabilities and remain reactive. Recent progress in world models and self-supervised learning, particularly Joint Embedding Predictive Architectures (JEPA), offers a new paradigm: learning latent representations of visual dynamics from large-scale unlabeled video enabling prediction in latent space rather than pixel space; supporting goal-directed planning through.