Author ORCID Identifier
https://orcid.org/0000-0002-7110-6502
Date of Award
2026
Document Type
Thesis (Ph.D.)
Department or Program
Cognitive Neuroscience
Abstract
Understanding observed actions requires more than recognizing bodies, objects, and movements; it requires representing how agents, goals, objects, and social context are organized into meaningful events. This dissertation examines how action meaning is organized in human cortical representational geometry across naturalistic viewing conditions. Behavioral arrangement tasks yielded models of perceived action similarity that were compared with cortical response representational geometries measured using functional magnetic resonance imaging (fMRI) and evaluated against alternative models of visual content, semantic and categorical properties, visual motion, and auditory structure.
Chapter 1 investigated naturalistic human action videos presented as repeated, isolated clips, without audio. Transitivity (goal similarity) and sociality (social-interaction similarity) explained neural geometry across lateral occipitotemporal and ventral temporal cortex. These effects remained after accounting for person, object, and scene similarity; verb and nonverb semantics; gaze; motion energy; body parts; number of people; and action category. Action-related geometry was therefore not reducible to the measured content, semantic, and sensory properties of the clips.
Chapter 2 tested whether this organization generalized to continuous naturalistic viewing. Neural responses to the same 90 monkey action events were compared across two fMRI experiments: repeated isolated-clip viewing and continuous movie viewing of the documentary from which the clips were extracted. Transitivity explained unique neural geometry beyond visual and auditory controls in both contexts. Sociality explained focal unique variance during isolated-clip viewing but not during continuous movie viewing after transitivity, and the remaining models were considered. Neural representational geometries were directly correlated across viewing contexts, peaking in right ventral temporal cortex and providing model-independent evidence that relationships among action events were partly conserved despite differences in repetition, temporal context, and participant samples.
Across the dissertation, transitivity emerged as the most robust organizing dimension, whereas the uniquely estimable contribution of sociality depended more strongly on its covariance with other properties. Ventral temporal representational geometry reflected the goal-directed organization of events beyond measured similarities among their constituent action agents, objects, scenes, and sensory features. Together, these findings support an account in which observed actions are represented through distributed, multidimensional cortical geometries that remain partly conserved as actions are embedded within an unfolding narrative.
Recommended Citation
Han, Jane, "Action Understanding in Naturalistic Vision: Neural and behavioral representational geometry across controlled and continuous viewing" (2026). Dartmouth College Ph.D Dissertations. 546.
https://digitalcommons.dartmouth.edu/dissertations/546
