Improving Healthcare Sustainability using Advanced Brain Simulations using a Multi-Modal Deep Learning Strategy with VGG19 and Bidirectional LSTM

Saravanan Chandrasekaran ¹ Aaarthi S. ² Abdulmajeed Alqhatni ³ Surbhi Bhatia Khan ^4* Mohammad Tabrez Quasim ^4,5 Shakila Basheer ⁶

• ¹ SRM Institute of Science and Technology, Chennai, Tamil Nadu, India
• ² Dayananda Sagar College of Engineering, Bangalore, Karnataka, India
• ³ Najran University, Najran, Saudi Arabia
• ⁴ University of Salford, Salford, United Kingdom
• ⁵ University of Bisha, BISHA, Saudi Arabia
• ⁶ Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia

Background: Brain tumor categorization on MRI is a challenging but crucial task in medical imaging, requiring high resilience and accuracy for effective diagnostic applications. This study describe a unique multimodal scheme combining the capabilities of deep learning with ensemble learning approaches to overcome these issues.The system integrates three new modalities, spatial feature extraction using a pre-trained VGG19 network, sequential dependency learning using a Bidirectional LSTM, and classification efficiency through a LightGBM classifier.The combination of both methods leverages the complementary strengths of convolutional neural networks and recurrent neural networks, thus enabling the model to achieve state-of-the-art performance scores. The outcomes confirm the efficacy of this multimodal approach, which achieves a total accuracy of 97%, an F1-score of 0.97, and a ROC AUC score of 0.997.With synergistic harnessing of spatial and sequential features, the model enhances classification rates and effectively deals with high-dimensional data, compared to traditional single-modal methods. The scalable methodology has the possibility of greatly augmenting brain tumor diagnosis and planning of treatment in medical imaging studies.