We believe "earables" is the next significant milestone in wearable computing. With sensing, processing, and communications converging into these devices, we envision a host of new possibilities within the next 5 years. The leap from today’s ear-phones to "earables" could mimic the transformation from basic-phones to smartphones. Today’s smartphones are hardly a calling device, much like how tomorrow’s earables will hardly be a wireless speaker or microphone. See our vision slides

Projects

Algorithms & Learning Problems

Under-determined blind source separation (UBSS)
Learning multi-modal translators for speech de-noising
Near-Far mapping of head related transfer functions (HRTF)
Blind channel interpolation for personal sound zones (PSZ)
Derivative free optimization for human feedback functions
Sensor fusion for motion tracking
Geometric sub-spaces for direction of arrival (DoA) estimation
Interference alignment via angular aliasing
Self-supervised region-wise source extraction

Systems & Applications

Immersive acoustic augmented reality (AAR)
Indoor localization and navigation
Head, face, and mouth activity sensing
Vibratory communication
Inaudible acoustics
Health and vital sign sensing from glasses
Reading in the acoustic domain
Continuous multi-modal security
Audio-visual systems

Platforms & Experiments

Videos

Invited Seminars & Talks

Invited seminar at 2020 NUS Computer Science Week, Singapore

Keynote at IASA, 2022

Workshop talk on Earable Computing, ACM HotMobile, 2021

Conference talk on AoA Factorization, ICASSP 2021

Conference talk on Ear-AR, ACM MobiCom 2020

Conference talk on EarSense, ACM MobiCom 2020

Conference talk on UNIQ, ACM SIGCOMM 2020

Conference talk on VoLoc, ACM MobiCom 2020

Conference talk on MUTE, ACM SIGCOMM 2018

Invited seminar at Charles Babbage Seminar (Cambridge Univ.)

Keynote at EarComp workshop 2019

Keynote at MobiUK workshop (Oxford Univ.)

Publications

ICRA, 2023	RoSS: Rotation-induced Aliasing for Audio Source Separation RoSS exploits the intrinsic angular/delay aliasing phenomenon in microphone arrays to solve under-determined source separation and AoA estimation problems via rotational motion. Hyungjoo Seo , Sahil Bhandary Karnoor , Romit Roy Choudhury
ICML, 2022	Learning to Separate Voices by Spatial Regions This paper explores a self-supervised technique for binaural source separation which relaxes the maximum number of sources constraint. Zhongweiyang Xu , Romit Roy Choudhury
ACM HotMobile, 2021	Earable Computing: A New Area to Think About This position paper argues that earphones hold the potential to disrupt mobile, wearable computing. Romit Roy Choudhury
arXiv, 2021	Estimating multiple Angles of Arrival in a Steering Vector Space This paper estimates the AoA of multiple uncorrelated and correlated signals (echoes) by analyzing them in a steering-vector sub-space. Yu-lin Wei , Romit Roy Choudhury
ACM SIGCOMM, 2021	Personalizing Head Related Transfer Functions for Earables UNIQ enables better spatial acoustics on earables by estimating personalized HRTF using off-the-shelf mobile devices. Zhijian Yang , Romit Roy Choudhury
IEEE ICASSP, 2021	Angle-of-arrival (AOA) Factorization in Multipath Channels TThis paper aims to estimate K angle of arrivals (AoA) using an array of M > K microphones, for unknown and correlated source signals. Yu-Lin Wei , Romit Roy Choudhury
ACM MOBICOM, 2020	Ear-AR: Indoor Acoustic Augmented Reality on Earphones Ear-AR enables indoor localization and acoustic AR via sensor fusion between ear IMU, phone IMU, and acoustics. Zhijian Yang , Yu-Lin Wei , Sheng Shen , Romit Roy Choudhury
ACM MOBICOM, 2020	EarSense: Earphones as a Teeth Activity Sensor EarSense uses today’s earphone speaker and microphone to sense and localize teeth gestures (applications in health monitoring and HCI) Jay Prakash , Zhijian Yang , Yu-Lin Wei , Haitham Hassanieh , Romit Roy Choudhury
ACM MOBICOM, 2020	Voice Localization Using Nearby Wall Reflections VoLoc shows the feasibility of inferring user location from voice commands, useful for voice assistants like Amazon Alexa and Google Home. Sheng Shen , Daguan Chen , Yu-Lin Wei , Zhijian Yang , Romit Roy Choudhury
ACM Earcomp(Workshop with Ubicomp'19), 2019	STEAR: Robust Step Counting from Earables STEAR discusses why earphone IMU serves as a much better sensor than phone or watch IMU, for motion tracking applications. Jay Prakash , Zhijian Yang , Yu-Lin Wei , Romit Roy Choudhury
ACM SIGCOMM, 2018	MUTE: Bringing IoT to Noise Cancellation MUTE exploits the velocity gap between RF and sound to improve active noise cancellation. Sheng Shen , Nirupam Roy , Junfeng Guan , Haitham Hassanieh , Romit Roy Choudhury