Volume 15, Issue 9 · IJARST · ISSN 2319-1783 / 2320-1126

A Hybrid CNN–Transformer Framework for Enhanced Medical Image Classification

A Hybrid CNN–Transformer Framework for Enhanced Medical Image Classification is an open-access, peer-reviewed research paper by Shubham Vashishtha , Shiwangi Choudhary, published in Volume 15, Issue 9 of the International Journal of Advanced Research in Science and Technology (IJARST), a UGC-approved journal (Print ISSN 2319-1783, Online ISSN 2320-1126).

Author

Shubham Vashishtha , Shiwangi Choudhary

Abstract

Medical image classification is important for computer-aided disease diagnosis due to its potential in identifying diseases from clinical images. Convolutional Neural Networks have been proved capable of modelling local spatial and textural features but are believed to be weak in capturing long-range dependencies. On the other hand, Transformer architectures are good at capturing global contextual relationships through self-attention but consume lots of computation power and training data. This paper seeks to integrate these two approaches through developing an architecture for medical image classification. Specifically, this paper develops a CNN-Transformer hybrid architecture for medical image classification that fuses convolutional feature extraction and Transformer-based global context modelling. The developed architecture is tested using two publicly available medical imaging datasets covering different medical diagnosis problems. These datasets include HAM10000 dermatoscopic skin lesion images and chest X-ray pneumonia. Proper preprocessing and augmentation techniques are applied to increase the quality of images and generalization of models and strategies to handle class imbalance are used. The performance of the developed architecture is compared with that of CNN architectures (ResNet50, DenseNet121, and EfficientNet-B0) and Transformer architectures (ViT and Swin Transformer). Evaluation metrics include accuracy, precision, recall, F1-score, specificity and Area Under the Receiver Operating Characteristic Curve (AUC).

Keywords: Medical Image Classification, Convolutional Neural Network, Transformer, Hybrid Deep Learning, Vision Transformer, Feature Fusion, HAM10000, Chest X-Ray

DOI:https://doi.org/10.62226/ijarst20262815

References

• Hu, Q., Zhang, Y., & He, Y. (2022). Hybrid cnn-transformer architecture for medical image classification. Computers in Biology and Medicine, 143, 105307.
• Manzari, O. N., Kaleybar, J. M., Saadat, H., & Maleki, S. (2024). BEFUnet: A hybrid CNN-transformer
• architecture for precise medical image segmentation. arXiv preprint arXiv:2402.08793.
• Elkholy, D. A., Shehata, M. S., & Braun, J. W. (2026). Comparative Study of CNN, Hybrid, and Transformer Architectures in Medical Image Classification. Journal of Imaging Informatics in Medicine, 1-22.
• Hu, C., Cao, N., Zhou, H., & Guo, B. (2024). Medical image classification with a hybrid SSM model based on CNN and transformer. Electronics, 13(15), 3094.
• Tang, H., Chen, Y., Wang, T., Zhou, Y., Zhao, L., Gao, Q., ... & Tong, T. (2024). HTC-Net: A hybrid CNN-transformer framework for medical image segmentation. Biomedical Signal Processing and Control, 88, 105605.
• Balakrishnan, M., Ananthi, K., Sindhuja, A., Kalpana, G., & CR, R. (2026). Hybrid CNN-Transformer Framework for Multi-Disease Detection from Medical Imaging Data. International Journal of Computer Information Systems and Industrial Management Applications, 18(21s), 259-280.
• Hasan, M. R. (2024). Transformer and convolutional neural network based hybrid approaches in medical image classification, caption generation, and retrieval processes (Master's thesis, Morgan State University).
• Yousafzai, S. N., Nasir, I. M., Mansour, S., Negm, N., Alhashmi, A. A., Alharbi, M. A., & Kim, E. (2026). A hybrid deep learning approach integrating CNN and transformer for lung cancer classification using CT scans. Scientific Reports, 16(1), 15420.
• Ali, R. S., Hassoun, R. K., Al-Adhami, A., Jabber, S. A., & Hashem, S. H. (2025). Proposal medical image protection system based on hybrid CNN-transformer
• model and chaotic maps. IEEE Access, 13, 123793-123807.
• Thanekar, S. A. (2026). Hybrid CNN–Transformer Architecture for Multi-Organ Disease Classification. Natural Resources for Human Health (NRFHH), 6(3), 391-401.
• Gupta, A., Kumar, S., Kumar, A., & Raj, U. (2025, September). CNNs and Transformers: Unraveling Their Impact on Modern Medical Image Classification. In 2025 12th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO) (pp. 1-6). IEEE.
• Li, B., Zhou, W., & Li, H. (2026). A hybrid CNN-Transformer network integrating multiscale spatially detailed features for medical image segmentation. Plos one, 21(4), e0345549.
• Mahadik, R. V., Asudani, D. S., Singh, S., Jawahire, S. S., Nimgade, J., & Kawale, R. M. (2026, March). Hybrid CNN–Transformer Architecture for High-Precision Image Classification. In 2026 Internatio
• nal Conference on Emerging Smart Computing and Informatics (ESCI) (pp. 1-6). IEEE.

DOI

10.62226/ijarst20262815

PAGES : 2641-2646 | 3 VIEWS | 2 DOWNLOADS

How do you cite this paper?

Shubham Vashishtha , Shiwangi Choudhary — “A Hybrid CNN–Transformer Framework for Enhanced Medical Image Classification.” International Journal of Advanced Research in Science and Technology (IJARST), Volume 15, Issue 9. DOI: 10.62226/ijarst20262815.

Read / Download Full Article