Office: 5519 Gates & Hillman Centers
Language Technologies Institute
Principal Systems Scientist
My interests center on all aspects of multimedia -- incorporating research in video and image analysis, speech, and language processing in effective interfaces for a wide range of applications. I am guiding many successful, on-going projects. Following are a few examples:
Fixed Camera Applications
Surveillance event detection inside an airport: This project seeks to identify actions such as running, pointing, answering a cell phone, meeting, embracing, etc. from airport surveillance cameras. In annual evaluations conducted by NIST our system has consistently been a top performer over the last 5 years.
Guiding cars to the best available parking spaces in large outdoor parking areas: This project, nearing its completion, detects empty parking spots using cameras mounted on light posts and communicates the open spaces to a navigation app that guides arriving cars to the optimal spot.
Multimedia Analysis of Social and Personal Media
Event reconstruction: This research is developing a modular platform to analyze and visually organize streams of event-based videos and texts. This includes analysis modules for activity, person and object recognition, video synchronization and geolocation without GPS metadata, text parsing and real-time visual interfaces for exploring the combined multimedia data information.
Processing and searching video archives of millions of videos: This flagship project has been developing a modular system that can process a large collection of YouTube-style videos and allow users to query for content items or find videos similar in content to provided video clip examples. A personal app for users to query their own video/image collections is also available.
Safety and well-being of seniors in the home and healthcare settings: ‘Marauders Map’ of the Caremedia project uses cameras and sensors to observe elderly individuals in their living environments and tracks activities and trends in their daily lives.
Medical dvice coaching: These applications use customized knowledge to monitor patients as they using medical devices and prevents or corrects incorrect actions. This was successful prototyped for home infusion pumps; an asthma inhaler coach as is currently in a patient evaluation phase with the promise to significantly reduce adverse outcomes.