Language-Guided Closed-Loop Manipulation
We develop research prototypes that connect object perception, language instructions, grasp planning and feedback control for tabletop manipulation.
Project investigator: Haonan Zhou
Ongoing research
We investigate how robots can track task progress, maintain useful context and select subsequent actions during extended manipulation tasks.
During an extended manipulation task, how can a robot judge where it is in the task, decide which context to keep, and choose what to do next — including after a step has not gone to plan?
Many useful tasks take more than a single action. Each step changes the scene, and the robot needs to know whether the step worked before it moves on. We study the loop between observing, judging progress and acting.
Concept loop: observation and memory, task context, action selection and outcome verification, then re-observation or recovery.
↺ Re-observe, then continue or recover
Concept diagram of the approach under investigation, not an experimental result.
We develop research prototypes that connect object perception, language instructions, grasp planning and feedback control for tabletop manipulation.
Project investigator: Haonan Zhou