Date of Award
2024
Document Type
Thesis (Master's)
Department or Program
Computer Science
First Advisor
Lorie Loeb
Second Advisor
SouYoung Jin
Third Advisor
John Bell
Abstract
With the advancement of computer vision and machine learning, techniques have emerged to capture movement data from humans in video footage. The Deep Screens Project utilizes these techniques to extract 3D human body pose data from film using machine learning and computer vision algorithms. Once data is extracted, it will undergo statistical analysis to quantitatively characterize the acting styles specific to various eras, genres, and actors. My thesis contributes to the Deep Screens Project by developing a pipeline that integrates machine learning and computer vision libraries to import data into a 3D visualization environment such as Unity. This pipeline includes steps such as 2D pose estimation, initial smoothing, 2D to 3D pose lifting, secondary smoothing, tracking ID matching, data loading in Unity, and updating humanoid character animations. Additionally, my thesis involves creating a small art demo to demonstrate the potential for artistic expression and educational use of the data obtained. After evaluating various tools and methods, we constructed a custom pipeline and demo that not only fulfills the project’s requirements but also enhances user engagement through an artistic showcase. Feedback from 30 participants con- firmed the effectiveness of the pipeline and the artistic representations in capturing the essence of the original film scenes. While the current pipeline meets its objectives within the constraints of contemporary technology, future enhancements will focus on improving tracking accuracy and expanding capabilities to accommodate larger groups and more complex motion capture.
Recommended Citation
Xu, Yaming, "DEEP SCREENS: A PIPELINE FOR TRANSFORMING POSE ESTIMATION IN FILM INTO 3D CHARACTER ANIMATIONS" (2024). Dartmouth College Master’s Theses. 158.
https://digitalcommons.dartmouth.edu/masters_theses/158