Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging.
Authors
Affiliations (8)
Affiliations (8)
- Institute of Ophthalmology, University College London London EC1V 9EL U.K.
- Wake Forest University School of Medicine Winston-Salem NC 27101 USA.
- Virginia Tech-Wake Forest University School of Biomedical Engineering and Sciences Blacksburg VA 24061 USA.
- Faculty of Electrical Engineering and ComputingUniversity of Zagreb Zagreb 10000 Croatia.
- Department of Ophthalmology and Visual SciencesThe University of British Columbia Vancouver BC V5Z 3N9 Canada.
- School of Biomedical EngineeringThe University of British Columbia Vancouver BC V6T 1Z3 Canada.
- NIHR Biomedical Research CentreMoorfields Eye Hospital NHS Foundation Trust London EC1V 2PD U.K.
- Department of Medical Physics and Biomedical EngineeringUniversity College London London WC1E 6BT U.K.
Abstract
Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a patch-based multiple instance learning (MIL) framework for disease classification on UWF images. We compare Vision Transformer encoders pretrained with supervised, Masked Autoencoder (MAE), and self-distillation objectives, while keeping the downstream aggregation architecture unchanged. Within a controlled comparison of ViT-B encoders pretrained on ImageNet-1k, the choice of pretraining objective substantially influenced frozen representation transfer, with supervised and self-distillation-based models outperforming MAE. A contemporary DINOv3 model pretrained at a larger scale achieved the strongest overall performance, with a quadratic weighted kappa of 0.863 for five-class diabetic retinopathy grading, comparable with DINOv1. Attention analysis further revealed distinct patch-aggregation behaviours associated with the different pretrained representations, while partial fine-tuning reduced the performance gap for MAE. These findings suggest that pretraining strategy influences both representation transferability and the subsequent aggregation of patch-level evidence within MIL, resulting in differences in downstream classification performance.