Date of Award
Winter 2023
Document Type
Thesis (Master's)
Department or Program
Computer Science
First Advisor
Michael Casey
Second Advisor
Lorie Loeb
Third Advisor
James Mahoney
Abstract
Moving images contain a wealth of information pertaining to motion. Motivated by the interconnectedness of music and movement, we present a framework for transforming the kinetic qualities of moving images into music. We developed an interactive software system that takes video as input and maps its motion attributes into the musical dimension based on perceptually grounded principles. The system combines existing sonification frameworks with theories and techniques of generative music. To evaluate the system, we conducted a two-part experiment. First, we asked participants to make judgements on video-audio correspondence from clips generated by the system. Second, we asked participants to give ratings for audiovisual works created using the system. These experiments revealed that 1) the system is able to generate music with a significant level of perceptual correspondence to the source video’s motion and 2) the system can effectively be used as an artistic tool for generative composition.
Recommended Citation
Wang, Hanlin, "An Interactive System for Generating Music from Moving Images" (2023). Dartmouth College Master’s Theses. 63.
https://digitalcommons.dartmouth.edu/masters_theses/63
