Date of Award

Spring 5-15-2024

Document Type

Thesis (Master's)

Department or Program

Computer Science

First Advisor

Adithya Pediredia

Second Advisor

Suren Jayasuriya

Third Advisor

Alberto Quattrini Li

Abstract

The world’s information can be conceptualized as a holistic scene, often represented as a four-dimensional vector encapsulating spatial coordinates (x, y, z) and temporal data (t). Advancements in human technology have facilitated the measurement of this feature vector through various sensors. For instance, a video captures spatial dimensions (x, y) with temporal resolution (t), while sonar pulses gauge depth (z). Leveraging these captures, we can easily perform post-modification of a scene and provide computationally enhanced perception and visual results. However, capturing with only one type of sensor poses challenges, especially when capturing surround view (360◦ viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, interior rooms, and autonomous navigation, a problem also known as the missing cone problem. This thesis delves into two main areas. Firstly, it explores a post-processing method for dynamic holistic scenes. The proposed method creates controllable long exposure images from video or light-field camera, effectively integrating temporal information into the mathematical representation. Secondly, it investigates a method to reconstruct holistic scenes within a small baseline by fusing camera and acoustic sensor data to combine 2D spatial (x, y) and depth (z) information. This fusion is achieved through Gaussian Splatting, a widely used 3D representation method. Referred to as Z-Splat, this novel method enables the reconstruction of holistic scenes with a small baseline.

Download

Share

COinS