Project abstract for group dudh

Parallel File System and Hierarchical Storage System Workload Characterization

This project aims to create a framework of modeling and generation for parallel I/O workload, including parallel file system I/O workloads. Parallel I/O workload in this project is defined mainly as the I/O workload generated by high performance MPI programs such as HPC applications that utilize MPI-IO to achieve high data parallelism. These parallel I/O workloads generally contain rich information about application behavior, I/O patterns, and system performance, which can be used in multiple purposes such as system bottleneck identification and performance optimization.

These researchers have identified several unique and related challenges in this project, including inter-process correlations, and are providing solutions to them. The synthetic parallel I/O workload generator is designated to generate artificial workloads at file system input level so that it can be used to test parallel file system performance as well as other file systems. The framework of workload modeling and generation is also designated to be able to adjust for general file systems. The researchers run specific HPC applications on an MSI cluster, capture their I/O workload, characterize the I/O traces, and replay the traces back in the system. Benchmark applications like IOR2, MPI-IO Test, and Flash-IO may be used. 

A bibliography of this group’s publications acknowledging MSI is attached.

Attachment: