Author ORCID Identifier
https://orcid.org/0009-0005-2310-2581
Date of Award
Spring 6-3-2026
Document Type
Thesis (Undergraduate)
Department
Computer Science
First Advisor
Yaoqing Yang
Abstract
With the rapid development of open-sourced models on Huggingface, there is a strong need for a way to systematically determine the similarity between models. More strongly, for intellectual property and organization, we need a way to determine the "lineage" of models. We borrow principles from Heavy-Tailed Self-Regularization and Random Matrix Theory to provide an inference-free method to accomplish this. We cluster a corpus of several model families by their spectral fingerprints and demonstrate that each model family occupies a distinct region in weight space. This confirms prior ideas of training setups leaving artifacts on model weights and allows us to formally capture intrinsic similarities embedded into model structures.
Recommended Citation
Prasad, Ishan Verma, "Capturing Large Language Model Similarity Through Spectral Analysis" (2026). Computer Science Senior Theses. 61.
https://digitalcommons.dartmouth.edu/cs_senior_theses/61
