Date of Award

5-2026

Document Type

Thesis (Master's)

Department or Program

Computer Science

First Advisor

Nikhil Singh

Second Advisor

Lorie Loeb

Third Advisor

John P. Bell

Abstract

Vibrotactile feedback can communicate system state, interaction progress, and event salience through short physical sensations. For temporal visual events, however, effective feedback must align with both what an event means and how it unfolds over time, including rhythm, duration, intensity, and repetition. This thesis examines this problem through UI flows, a controlled and common HCI setting. The proposed semantic VAE pipeline represents each visual transition as semantic haptic events, described by behavior class, communicative role, style, strength, timing, and duration. These events guide a curated behavior atlas and a convolutional variational autoencoder. The atlas identifies interpretable regions of the learned haptic latent space; feature-guided search generates waveform chunks; an event timeline mixer composes those chunks into a two-second response; and the export stage converts the result into an ERM-ready playback sequence. This structure links visual interpretation to physical playback through an intermediate haptic representation. The system was evaluated in a 27-participant user study comparing Semantic VAE with Random Noise, Random-Plan VAE, HapticGen, and Direct LLM Pattern baselines across four UI flows: Payment Confirmation, Failed Submission, Notification Received, and Pull to Refresh. The results show that Semantic VAE produced competitive haptic feedback, improved perceived meaningfulness over weaker baselines, and was selected as the worst match least often. At the same time, Random-Plan VAE achieved the highest overall ratings, indicating that the VAE-atlas backend can generate strong candidates when it explores a broader structured design space. Qualitative explanations and follow-up survey responses further showed that participants judged haptics through timing, rhythm, semantic fit, salience, and appropriateness, not intensity alone. Together, these results show that visual-to-haptic generation is a structured, preference-shaped design problem: semantic event planning reduces failure risk, while atlas-guided generative exploration expands the space of preferred haptic candidates. The thesis therefore contributes a framework for treating haptic generation not as a single visual-to-vibration translation, but an interpretable search over generated haptic possibilities.

Download

Share

COinS