DICE: Data Intensive Computing Environment

Real Testing * Real Data * Real Results

Parallel File System Comparison Project

Project Description

File system performance is a key component of user application efficiency in high performance computing (HPC) data centers. Storage administrators and data center directors must choose wisely to select the correct file system to meet their users’ requirements and properly adjust settings for optimal performance. But which file system is the best for each situation? Today, there is no direct correlation between parallel file systems and many of the decisions are based upon vendor marketing. The DICE Program, in partnership with the Department of Energy’s Sandia National Laboratories, is conducting a project to develop a comprehensive benchmarking framework for evaluating file systems. This project also seeks to develop a normalization factor between different file system architectures. HPC community involvement is key to providing a viable tool and understanding all the factors for an accurate comparison, especially for file systems at large scale.

Project Core Technical Team

Tracey Wilson, Project Manager/Team Lead
Dr. Paul Buerger, Avetec
Ed Wahl, Avetec
Lee Ward, Sandia National Lab
Matt Leininger, Lawrence Livermore National Lab
Dr. Daniel Duffy, NASA Goddard

Project Goals

  1. Survey the HPC technical community concerning parallel file system, benchmarks and I/O measurement, and trace collection
  2. Collect trace and benchmark data for analysis
  3. Develop comprehensive benchmark framework for evaluating file systems
  4. Develop normalization factor for accurate comparison of file system
  5. Release final report concerning all goals and publish results

Project Schedule and Status

Click the image below for a larger view.

Parallel File System Project Schedule

Figure 1: Parallel File System Project Schedule

How You Can Participate

Initial results to the surveys and a status of the project will be presented at the DICE Alliance conference in Dearborn, Michigan on April 15. More information can be found at the following: http://www.diceprogram.org/news/alliance/

If you are interested in participating in this project or would like additional information, please contact Tracey Wilson, DICE Program Manager at twilson@avetec.org or by phone 937.322.5000 x2037.

If you wish to provide trace or benchmark data, please register at our collection site: https://webproj.usa40.net