[Abstract]

Humans perceive the world through multiple modalities, including the basic senses of sight, hearing, smell, taste, and touch. For example, a person in a coffee shop can see nearby people, hear the ambient noise in that setting, smell the coffee in their cup, and feel its warmth while holding it. These modalities work together to provide them with a rich and reliable sense of their surroundings. A multimodal human-computer interface is a user interface that offers different types of sensory stimuli at the same time (e.g., visual, auditory, haptic, olfactory, etc.). Multimodal interfaces are necessary to support multi-sensory experiences, which can benefit usability and user experience. They can lead to more “natural” interaction with digital environments, providing multiple types of sensory information similar to what we perceive when interacting with physical environments. It is challenging to create high-quality multimodal interfaces: This requires an in-depth understanding of human perception and device output capabilities and how these can be combined to make interactions with a convincing multi-sensory experience. In this talk, I will discuss how I have designed and engineered interactive multimodal interfaces and how they could impact the user experience by delivering multi-sensory experiences in an immersive virtual reality.

[Biography]

Professor Jin Ryong Kim is an Assistant Professor of Computer Science at the University of Texas at Dallas. Dr. Kim directs the Multimodal Interaction Lab, where they pursue fundamental and applied research on the future of interactive technologies, with emphasis on haptics, virtual reality, and human-computer interaction. Their lab is committed to contributing to the sense of touch’s fundamental science and how it can benefit the user experience in immersive environments. He is actively publishing in the field of haptics, HCI, and VR, appearing in IEEE Transactions on Haptics, IEEE Haptics Symposium, ACM CHI, ACM VRST, and Advanced Functional Materials. He is currently serving as an Associate Editor of IEEE Robotics and Automation Letters, a Guest Editor of IEEE Transactions on Haptics, and a Guest Editor of Frontiers in Virtual Reality. Dr. Kim received his Ph.D. in Electrical and Computer Engineering and M.S. in Computer Science, both from Purdue University.