Cross-Modal Matching

Dr. Soma Biswas, Dept. of EE, IISC

Short Bio:
Dr. Soma Biswas is an Associate Professor in the Electrical Engineering department in IISc. She received her PhD degree in Electrical and Computer Engineering from the University of Maryland, College Park, in 2009. Then she worked as a Research Faculty at the University of Notre Dame and as a Research Scientist at GE Research before joining IISc. Her research interests include computer vision, pattern recognition and machine learning. She is a senior member of IEEE. She has received the prestigious “IEEE Late Shri Pralhad Chhabria Award for Best Women in Engineering” award in 2018 and the “Google India AI/ML Research Award” in 2020.

Date:July 9, 2020
Time: 4:00 PM IST


Due to the increase in the number of sources of data, research in cross-modal matching is becoming an increasingly important area of research. It has several applications like matching text with image, matching near infra-red images with visible images for night-time or low-light surveillance, matching sketch images with pictures for forensic applications, etc. Given a query from one modality, the goal of cross-modal retrieval is to retrieve semantically meaningful data from another modality. This is an extremely challenging task due to significant differences between data from different modalities. In this talk, first, I will start with a brief overview of the area of computer vision and then discuss the applications and different challenges in the area of cross-modal matching. Then we will look at some of the work done in this area in our IACV (Image Analysis and Computer Vision) Lab. We will also discuss approaches to handle interesting challenges like “How can we incorporate more annotated data into the approach without training the model from scratch?”, “how do we handle data we have never seen before”, “how do we personalise our searches”, etc.