About this Research Topic
Advances in storage systems are vital to meet the growing data demands of high-performance computing (HPC). Modern HPC applications now encompass traditional modelling and simulation workloads and scale-out tasks like AI, big data analytics, deep learning, and complex workflows, all requiring enhanced data exchange and in-situ processing capabilities. The emergence of Exascale systems such as Frontier and Aurora integrates heterogeneous components from both scale-up and scale-out communities, driving scientific discovery and innovation. Flexible storage systems are essential to support hybrid workloads with varying demands, especially given the conflicting needs of write- and read-intensive processes. Furthermore, the complexity and heterogeneity of parallel file and storage systems in large-scale environments are increasing. Innovations are moving beyond the traditional two-tier model, introducing new, fast, composable storage tiers closer to computing resources. The rise of varied HPC deployment models and virtualization further complicates storage management, necessitating new approaches for a seamless computing continuum.
The evolving landscape of hybrid HPC workloads and the widening gap between compute and storage performance underscore the need for smarter data management techniques that can provide the persistence of traditional storage systems while providing performance from state-of-the-art accelerators in HPC systems. Innovative I/O optimization techniques, incorporating machine learning and AI algorithms such as intelligent load balancing, I/O pattern prediction, and pipelining asynchronous I/O operations with computations, are essential for efficiently managing the exponential growth of data and the complexities of storage hierarchies. User-friendly, transparent, and adaptable approaches that enable API flexibility and performance are crucial for meeting the requirements of diverse HPC workloads, streamlining scientific and commercial code development, and efficiently utilizing extreme-scale parallel storage resources. This Research Topic aims to collect novel work in data-related areas such as storage, I/O, processing, and analytics on extreme-scale infrastructures, including HPC systems. Submissions should focus on proposing solutions and demonstrating their effectiveness for improving I/O performance for the exascale era and beyond using novel techniques, analyzing I/O behavior on large-scale HPC systems, providing efficient strategies for mitigating I/O bottlenecks in large-scale data analytics platforms, and identify new challenges in data and storage management for emerging HPC workloads.
This Research Topic aims to provide a comprehensive overview of advancements in I/O and data-related research for HPC systems. The topics of interest include but are not limited to:
- Innovative approaches for extreme-scale storage systems and multi-tier architectures with alternative data storage models and synergy between different storage models.
- Performance modelling, benchmarking, and I/O characterization studies of emerging HPC workloads.
- Efficient monitoring tools and data collection techniques for understanding data movement.
- High-performance I/O middleware libraries, I/O services, metadata management, and complex data management.
- Case studies of I/O services and data processing architectures to support various scientific applications.
We expect this Research Topic will stimulate further research and innovation and foster collaboration among researchers, practitioners, and stakeholders from academia, industry, government, and society.
The evolving landscape of hybrid HPC workloads and the widening gap between compute and storage performance underscore the need for smarter data management techniques that can provide the persistence of traditional storage systems while providing performance from state-of-the-art accelerators in HPC systems. Innovative I/O optimization techniques, incorporating machine learning and AI algorithms such as intelligent load balancing, I/O pattern prediction, and pipelining asynchronous I/O operations with computations, are essential for efficiently managing the exponential growth of data and the complexities of storage hierarchies. User-friendly, transparent, and adaptable approaches that enable API flexibility and performance are crucial for meeting the requirements of diverse HPC workloads, streamlining scientific and commercial code development, and efficiently utilizing extreme-scale parallel storage resources. This Research Topic aims to collect novel work in data-related areas such as storage, I/O, processing, and analytics on extreme-scale infrastructures, including HPC systems. Submissions should focus on proposing solutions and demonstrating their effectiveness for improving I/O performance for the exascale era and beyond using novel techniques, analyzing I/O behavior on large-scale HPC systems, providing efficient strategies for mitigating I/O bottlenecks in large-scale data analytics platforms, and identify new challenges in data and storage management for emerging HPC workloads.
This Research Topic aims to provide a comprehensive overview of advancements in I/O and data-related research for HPC systems. The topics of interest include but are not limited to:
- Innovative approaches for extreme-scale storage systems and multi-tier architectures with alternative data storage models and synergy between different storage models.
- Performance modelling, benchmarking, and I/O characterization studies of emerging HPC workloads.
- Efficient monitoring tools and data collection techniques for understanding data movement.
- High-performance I/O middleware libraries, I/O services, metadata management, and complex data management.
- Case studies of I/O services and data processing architectures to support various scientific applications.
We expect this Research Topic will stimulate further research and innovation and foster collaboration among researchers, practitioners, and stakeholders from academia, industry, government, and society.
Keywords: Parallel file and storage systems, Emerging HPC workloads, I/O optimization and benchmarking, Storage and data processing architectures and systems, AI/ML-centric I/O management, Storage runtime architectures
Important Note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.
