Back to all papers

Capturing reader-specific annotation biases in ordinal classification: a controlled simulation study.

November 18, 2026pubmed logopapers

Authors

Ripaud E,Jailin C,Milioni de Carvalho P,Vancamberg L,Bloch I

Affiliations (3)

  • GE Healthcare France, 283 Rue de la Minière, Buc, 78530, France.
  • CentraleSupélec, ENS Paris-Saclay, CNRS, LMPS, Université Paris-Saclay, 4 Av. des Sciences, Gif-sur-Yvette, 91190, France.
  • CNRS, LIP6, Sorbonne Universite, 4 place Jussieu, Paris, Île-de-France, 75005, France.

Abstract

<i>Objective.</i>Inter-reader variability remains a major challenge in medical image classification, particularly for tasks involving ordinal labels, due to subjective interpretation and institutional biases. While deep learning is widely used to automate and standardize such classification tasks, including breast density assessment, handling reader-dependent annotation behavior remains an open problem. This study introduces a controlled setting to assess the extent to which such variability can be captured by a reader-adaptive model.<i>Approach.</i>This work builds on ReaderAdaptNet, a recently proposed reader-adaptive framework that models individual annotation styles through jointly learned reader-specific embeddings. Through a simulation study, it aims to deepen the understanding of this framework and to address the evaluation limitations imposed by small expert-annotated datasets, for which the underlying bias mechanisms are neither known nor controllable. Synthetic reader profiles are generated to reproduce realistic annotation behaviors, enabling a controlled analysis of the model's ability to adapt to diverse annotation biases. The effects of the embedding dimension and of the fusion strategy of image-reader features on classification accuracy are assessed. The structure of the embedding space is also analyzed.<i>Main results.</i>Results show that reader embeddings effectively capture identifiable, systematic biases, with moderate embedding dimensionality being sufficient. Systematic threshold shifts are recovered, whereas stochastic variability is not learnable. Conditioning image-reader fusion through feature-wise linear modulation or cross-attention mechanisms enables the correction of view-related discrepancies. The learned embedding space further reveals meaningful clustering of annotation profiles, reflecting similarity in reader behavior.<i>Significance.</i>Overall, the proposed synthetic framework identifies which aspects of inter-reader variability are learnable by ReaderAdaptNet and provides a controlled proxy for anticipating model behavior in clinical settings.

Topics

Journal Article

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.