Date of Award
Spring 6-4-2025
Document Type
Thesis (Undergraduate)
Department
Computer Science
First Advisor
SouYoung Jin
Abstract
Automatic audio description (AD) systems support visually impaired audiences by narrating visual content, but they often fail to capture the interpersonal dynamics that underpin narrative understanding. In this work, we introduce a novel framework for character relationship prediction as a means of enriching audio descriptions with socially grounded context. Our contributions are threefold: (1) we propose the Character Relationship Module (CRM), which extends identity-aware video captioning with directed sentiment inference between character pairs; (2) we develop a scalable weak supervision pipeline that uses large language models to generate 669,520 relationship annotations across 202 films; and (3) we construct a complementary synthetic benchmark of 2,215 generated video clips with controlled emotional dynamics for evaluation. CRM achieves 68.29\% accuracy on the LLM-annotated dataset, demonstrating that character relationships can be inferred from visual signals alone. We analyze the challenges of visual relationship prediction and establish baselines for this new task. Our work opens new directions for emotionally-aware audio descriptions and provides datasets and benchmarks for future research.
Recommended Citation
Hahm, Seung Hyun, "Character Relationship Prediction in Movies: Toward Emotionally-Aware Automatic Audio Descriptions" (2025). Computer Science Senior Theses. 86.
https://digitalcommons.dartmouth.edu/cs_senior_theses/86
