Research talk by Ruidong Zhang and Ke Li (Cornell University)

Title: Wireless Sensing Around the Body: Towards Tracking and Interpreting Body Postures in the Wild on Wearables

Abstract:

Despite the fast growth of AI, computers have limited ability to interpret human physical behaviors in everyday life. For example, computers still cannot yet accurately distinguish basic daily activities such as eating and drinking. One key challenge is the lack of high-quality behavior data in the wild, such as body postures on the hands, face, and limbs. Existing body posture sensing technologies encounter at least one of the following issues when used continuously in daily settings: 1) limited sensing area, 2) cumbersome form factor, 3) high energy consumption, and 4) low resolution. The inability to continuously track high-quality fine-grained body postures in daily settings has hindered computers’ ability to comprehensively and continuously understand complex human behaviors and interpret their intentions.

The research in the SciFi lab aims to develop a low-power, privacy-sensitive, and intelligent sensing platform with minimally obtrusive wearables that can be deployed in everyday life to track and interpret a key set of fine-grained body postures. The key sensing principle behind these projects is that different body postures lead to unique patterns and formations of nearby skin or contours of the body, which can be captured by wearable wireless sensors to infer fine-grained body postures and activities. In this talk, we will present a series of novel sensing technologies from the SciFi lab, including an earable and a pair of glasses that can track facial expressions (IMWUT’22 & CHI’24, Ke Li), a pair of glasses that can continuously track gaze movements (MobiCom’24, Ke Li), a pair of glasses and a headphone that can continuously interpret speech from subtle skin deformations (CHI’23 & ISWC’23, Ruidong Zhang), and a wristband that can reconstruct hand postures and recognize hand-object interaction (CHI’24, Ruidong Zhang).

Web:

Ruidong’s web: https://ruidongzhang.com/

Ke’s web: https://keli97.github.io/

Time and Location:

Monday, May 20th, 2024

10:00am – 11:00am – Eng. Bldg. 2, 12F Room 121B1（電気系会議室4）

Zoom: Join the lecture