A Multimodal Approach to Breast-Lesion Classification Using Ultrasound and Patient Metadata
Keywords:
Breast cancer, lesion prognosis, machine learning, deep learning , multimodal fusionAbstract
The diagnosis and prognosis of breast cancer have been greatly improved by incorporating machine learning methods, especially through medical imaging analysis as well as clinical information. In this study, the potential of deep learning models for breast lesion prognosis was explored by integrating imaging features with clinical data to enhance predictive accuracy. Clinical data were analyzed using multilayer perceptron (MLP) classifiers, XGBoost, and Random Forest, while several convolutional neural network (CNN) architectures, such as ResNet optimized with Adam, DenseNet with stochastic gradient descent (SGD), and EfficientNet with RMSprop, were evaluated. The integration of imaging-based features with clinical data was found to significantly improve model performance, enabling more accurate risk stratification and the development of individualized treatment strategies. The highest validation accuracy and area under the curve (AUC) were achieved by the most effective models, highlighting the advantages of a multimodal approach. Although the study was conducted on a relatively small dataset and faced challenges such as missing data, the results suggest that these methods hold considerable promise for implementation in clinical practice.
Downloads
References
World Health Organization. “Breast cancer.” WHO Fact Sheets, 2021. Available: https://www.who.int/news-room/fact-sheets/detail/breast-cancer
R. L. Siegel, K. D. Miller, H. E. Fuchs, and A. Jemal. “Cancer statistics, 2023.” CA: A Cancer Journal for Clinicians, vol. 73, no. 1, pp. 17–48, 2023.
W. A. Berg et al. “Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer.” JAMA, vol. 299, no. 18, pp. 2151–2163, 2008.
E. B. Mendelson. “Evaluation of the breast with ultrasound.” Radiologic Clinics of North America, vol. 40, no. 3, pp. 485–506, 2002.
J. G. Elmore et al. “Variability in interpretation of mammograms.” New England Journal of Medicine, vol. 331, no. 22, pp. 1493–1499, 1994.
M. L. Giger et al. “Computer-aided diagnosis in breast imaging: CADx and CAD.” Academic Radiology, vol. 15, no. 4, pp. 452–462, 2008.
C. J. D’Orsi, E. A. Sickles, E. B. Mendelson, E. A. Morris, et al. ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System, 5th ed., Reston, VA: American College of Radiology, 2013.
M. H. Yap et al. “Automated breast ultrasound lesions detection using convolutional neural networks.” IEEE Journal of Biomedical and Health Informatics, vol. 22, no. 4, pp. 1218–1226, 2018.
Q. Cao et al. “Deep learning-based computer-aided diagnosis systems for breast cancer: a survey.” Artificial Intelligence in Medicine, vol. 132, p. 102360, 2022.
G. Litjens et al. “A survey on deep learning in medical image analysis.” Medical Image Analysis, vol. 42, pp. 60–88, 2017.
A. Yala et al. “Deep learning mammography-based model for improved breast cancer risk prediction.” Radiology, vol. 292, no. 1, pp. 60–66, 2019.
J. Wang et al. “Multi-modal deep learning for breast cancer diagnosis by integrating ultrasound and mammographic images.” IEEE Transactions on Medical Imaging, vol. 39, no. 10, pp. 2735–2744, 2020.
Q. Dou et al. “Multimodal breast cancer diagnosis by fusing qualitative and quantitative features from ultrasound and mammograms.” Medical Image Analysis, vol. 70, p. 102006, 2021.
Y. Zhang et al. “Multi-modal deep learning model for breast cancer prognosis prediction using both imaging and non-imaging data.” Frontiers in Oncology, vol. 12, p. 876543, 2022.
Y. Liu et al. “Integrating clinical and imaging data for improved breast cancer diagnosis using deep learning.” Journal of Biomedical Informatics, vol. 108, p. 103518, 2020.
B. E. Bejnordi, M. Veta, P. J. van Diest, et al. “Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.” JAMA, 2017.
C. Aumente‐Maestro, J. Díez, and B. Remeseiro. “A multi‐task framework for breast cancer segmentation and classification in ultrasound imaging.” Medical Image Analysis, Mar. 2025.
L. Shen, L. R. Margolies, J. H. Rothstein, E. Fluder, R. McBride, and W. Sieh. “Deep learning to improve breast cancer detection on screening mammography.” Radiology, Aug. 2019.
D. Ribli, A. Horváth, Z. Unger, P. Pollner, and I. Csabai. “Detecting and classifying lesions in mammograms with deep learning.” Scientific Reports, Mar. 2018.
Y. Zhang et al. “Deep learning-based multi-modal data integration enhancing breast cancer prognosis prediction.” Frontiers in Oncology, vol. 13, p. 11190375, 2023.
S. Banerjee and M. K. H. Monir. “CEIMVEN: An approach of cutting-edge implementation of modified versions of EfficientNet (v1‐v2) architecture for breast cancer detection and classification from ultrasound images.” arXiv preprint, 2023.
A. Pawłowska, A. Ćwierz Pieńkowska, A. Domalik, D. Jaguś, P. Kasprzak, R. Matkowski, A. Fura, A. Nowicki, and N. Zolek. “A curated benchmark dataset for ultrasound based breast lesion analysis (breast-lesions-usg).” Dataset, Version 1. DOI: 10.7937/9WKK-Q141, 2024.
F. Jiang, H. Liu, S. Yu, and Y. Xie. “Breast mass lesion classification in mammograms by transfer learning.” Journal of Healthcare Engineering, Jan. 2017.
Downloads
Published
How to Cite
Issue
Section
ARK
License
Copyright (c) 2025 AMINA ABOULMIRA, Mohamed Ouhami, Hamid Hrimech, Mohamed Lachgar
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright on any article published in the International Journal of Computer Engineering and Data Science (IJCEDS) is retained by the author(s). All articles are published under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0), which permits any non-commercial use, distribution, and reproduction in any medium, provided that the original work is properly cited.
License Agreement
By submitting and publishing their work in IJCEDS, the authors:
-
Grant IJCEDS the non-exclusive right to publish the article and to identify IJCEDS as the original publisher.
-
Authorize any third party to use, share, and reproduce the article for non-commercial purposes, provided that appropriate credit is given to the original authors and source, and a link to the license is included.
International Journal of Computer Engineering and Data Science (IJCEDS) is an Open Access Journal ISSN : 2737-8543
