DICE: Data Intensive Computing Environment

Real Testing * Real Data * Real Results

BOF "Parallel File Systems Benchmarking Update"

Birds of Feather (BOF) Session
Tuesday, November 16
5:30-7:00 pm

Tracey Wilson opened the session with a report on the survey of the HPC community concerning parallel file systems, benchmarks, and trace collection.  The full report on the survey is now available on the DICE Program website.

One result from the survey is that very few respondents were willing and able to provide traces of the I/O activity of applications.  This result informed much of the resulting discussion.  Some comments were:

  • we need a better description of what is desired for traces
  • we need a description of how to obtain the desired traces
  • strace is the obvious mechanism for tracing but is not usable on big clusters
  • some see a need for tracing MPI calls as well as I/O calls; perhaps MPI profiling could be used
  • NFS traces may be useful because the protocol is relatively small and well understood
  • traces on parallel file system such as Lustre are not reproducible since the file system rearranges I/O based on what else is going on in the file system
  • can we develop benchmarks based on the traces and will such benchmarks be better than IOR
  • the current website is not clearly identified as belonging to the DICE Program