An Interpretable Vision-Language Framework for Discriminating Clinically Defined Groups from Segmented Metacarpal X-ray Regions.
Authors
Affiliations (7)
Affiliations (7)
- Department of Electronic Engineering, Feng Chia University, Taichung, 40724, Taiwan R.O.C.
- School of Medicine, College of Medicine, China Medical University, Taichung, 404, Taiwan R.O.C.. [email protected].
- Department of Orthopedic Surgery, China Medical University Hospital, Taichung, 404, Taiwan R.O.C.. [email protected].
- School of Medicine, College of Medicine, China Medical University, Taichung, 404, Taiwan R.O.C.
- Department of Orthopedic Surgery, China Medical University Hospital, Taichung, 404, Taiwan R.O.C.
- School of Architecture, Feng Chia University, Taichung, 40724, Taiwan R.O.C.
- Department of Information Management, National United University, Miaoli, 36003, Taiwan, R.O.C.. [email protected].
Abstract
Current osteoporosis screening approaches are still difficult to apply broadly because they often require specialized imaging systems, clinical resources, and additional operational procedures. To overcome these practical limitations, this study establishes a hand X-ray-based framework for discriminating between two clinically defined groups using quantitative analysis of the metacarpal regions, comprising 2100 healthy radiographs and 518 osteoporosis radiographs, collected from China Medical University Hospital. Specifically, this work proposes a vision-language model (VLM)-driven framework that integrates zero-shot prompt engineering and chain-of-thought (CoT) prompting to generate interpretable and computable diagnostic formulas based on the quantified anatomical information extracted from segmented metacarpal regions. Using three VLMs, including Gemini 3, GPT-5, and Qwen3-VL-8B, the proposed method analyzes segmented X-ray regions, anatomical definitions, and quantitative metacarpal measurements to suggest several logical diagnostic formulas. With the guidance of the CoT prompt, the model first interprets the segmented images, then identifies the anatomical meaning of each region, and finally generates candidate formulas. This process links the visual characteristics of metacarpal structures with quantitative indices that can be used for classification. The experimental evaluation shows that the best VLM-generated formula, <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>(</mo> <mi>o</mi> <mo>-</mo> <mi>i</mi> <mo>)</mo> <mo>/</mo> <mi>i</mi></mrow> </math> , achieved an AUC of 0.9344 under threshold-based classification and an accuracy of 87.5% with an AUC of 0.9120 under ANN-based classification. Compared with the manually defined baseline ratio <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>(</mo> <mi>o</mi> <mo>-</mo> <mi>i</mi> <mo>)</mo> <mo>/</mo> <mi>m</mi></mrow> </math> , this formula demonstrated better classification performance under the same experimental setting. In short, the proposed framework provides a practical blueprint for integrating regional segmentation, anatomical measurement, VLM-based reasoning, and formula-driven classification for discriminating between the two predefined clinical groups.