2/5/2025 | 11:00 AM - 12:20 PM | Grand Peninsula B

Multisensory aesthetic perception: A quantitative-qualitative study on visuo-tactile interactions with material textures

Author(s)

Marella Campagna | Bamberg University
Alexander Pastukhov | Bamberg University
Claus-Christian Carbon | Bamberg University

Abstract

In real-world interactions, material textures elicit a complex interplay of sensory modalities, primarily vision and touch. Recent research advances have sought to unravel material perception, focusing on how visual and haptic systems interact. Despite these efforts, a clear understanding of bottom-up and top-down processes in visuo-tactile aesthetics remains limited. This study aimed to explore visuo-tactile sensory perception and material texture interactions using a multimethodological approach.Thirty right-handed participants with normal or corrected-to-normal vision, engaged with various textures through a Touch Perception Task (TPT). A think-out-loud protocol captured participants’ qualitative feedback during visuo-tactile exploration, while self-reports detailed their sensory and affective experiences, personality traits, and need for touch. Eye-tracking technology measured fixation and pupillary responses, complemented by video analysis of hand movements during material engagement. We classified diverse sensory and affective responses toward the material textures by analyzing both pupillometry data and prototypical hand movements. The findings demonstrated that materials such as sandpapers (K40, K1200) and silks showed contrasting sensory/affective profiles. Sandpaper K40 was unexpectedly pleasant, despite perceived roughness, due to its larger, smoother particles and possible low-frequency vibrations. In contrast, K1200, although visually softer, was experienced as irritating due to its fine grain and unengaging visual properties. Faux fur emerged as the most favored material, evoking comfort and safety. Hand movement analysis revealed deviations from traditional lateral motion used in texture assessments, possibly driven by cognitive, affective, and aesthetic factors. Our integrated approach, combining physiological and behavioral data, provides deeper insights into the cognitive-affective interplay of visuo-tactile experiences. These findings suggest that visual complexity, (in)congruence between visual anticipation and actual haptic experience, and material properties like texture and vibration influence aesthetic judgments, expanding our understanding of sensory integration processes beyond singular assessments of independent dimensions.