Deep Learning for Localized-Haptic Feedback in Tactile Surfaces

Touchscreens are nowadays the preferred choice for user interfaces in consumer electronics. Significant technological advances have been made in terms of touch sensing and visual quality. However, the haptic feedback offered by commercial products is still primitive, primarily because it affects the whole touch surface and can only render vibration buzzes.

Several researchers are currently concerned with enriching haptic feedback on touch surfaces. One approach is to use an array of transducers, bonded to the surface, and exploit the wave propagation phenomenon to operate in the far field of the actuators. For instance, the time-reversal method has been used to create elastic wave-fronts and obtain localized vibrations. One of the limitations of a wave-focusing strategy is the appearance of secondary displacement peaks in undesired locations, which lowers the contrast of the focalization.

This thesis explores the potential of state-of-the-art Deep Learning strategies to create alternative signals that allow to reduce the number of actuators required, improve the contrast ratio, and evoke novel feelings to the users of the device with vibrotactile feedback.

In this work, machine learning models are used to extract relevant features from an impact signal and predict the location where the impact has occurred. A transformation of the time-domain signal permits to obtain a representation that improves the precision on the prediction of the impact source location. 

Furthermore, a new approach to optimize the localized peaks obtained with time-reversed impulse response signals is developed using Deep Neural Networks (DNNs) and Reinforcement Learning (RL). The optimization increases the peak amplitude and contrast ratio while ensuring that the peaks appear at the desired location.  

Moreover, a novel approach to storing and generating time-reversed signals is developed using Generative Adversarial Networks (GANs). The effect of the diversity provided by deep generative models on the generated signals is evaluated. The effect on the properties of the localized peaks created with the GAN-generated signals is studied in an experimental setup. This effect inspires the development of a novel stimulation pattern for vibrotactile feedback. 

A transparent haptic demonstrator surface is developed. The demonstrator can create localized vibrations within the human perception range. This haptic surface is used to perform a human perception experiment that compares the perceived alertness provided by the novel pattern with the traditional stimulation pattern. The novel pattern is perceived as more alerting and evokes different perceptual sensations that can lead to novel vibrotactile cues.

This exploratory work introduces state-of-the-art machine learning techniques into the time-reversal haptics field of study. It demonstrates the potential of these approaches to bring new knowledge into the haptics research