Ongoing research
Task-Aware Communication for Embodied Intelligence
How much information does a robot need to act? We investigate compact representations and communication strategies that preserve task-relevant information across constrained and unreliable links.
Research question
Which parts of a robot's observations need to cross a wireless link for a task to succeed, and how should they be represented when bandwidth is limited and the link is unreliable?
Approach under investigation
We study the path from what a robot observes to what a model or controller at the other end of a link needs in order to act. Rather than treating every pixel or measurement as equally important, we ask which information the task depends on.
- Visual and semantic representations that keep task-relevant content and leave out detail the task does not use.
- Compression and coding of these representations for links with limited bandwidth.
- Link impairments — delay, loss and fluctuating capacity — and how they carry through to the receiving end.
- Receiver-side inference and its relationship to task performance, so that task feedback can inform what is encoded and sent.
Diagrams and media
Concept: observation, representation and encoding, a constrained wireless link, and receiver-side inference, with task feedback informing what is encoded.
- Observation Camera views and robot state
- Representation & encoding Keep what the task needs
- Wireless link Constrained and unreliable
- Receiver-side inference Model or controller acts on what arrives
- Task performance
↺ Task feedback informs what is encoded and sent
Task-aware communication pipeline
Concept diagram of the approach under investigation, not an experimental result.