Previous | 
Home
Section: New Results
Scalable I/Os: visualization and processing
Modeling and predicting I/O patterns of large-scale simulations
Participants : Matthieu Dorier, Shadi Ibrahim, Gabriel Antoniu.
The increasing gap between the computation performance of
post-petascale machines and the performance of their I/O subsystem
has motivated many I/O optimizations including prefetching, caching,
and scheduling. In order to further improve these techniques,
modeling and predicting spatial and temporal I/O patterns of HPC
applications as they run has become crucial. Our work in this
context focuses on Omnisc'IO, an approach that builds a
grammar-based model of the I/O behavior of HPC applications and uses
it to predict when future I/O operations will occur, and where and
how much data will be accessed. To infer grammars, Omnisc'IO is
based on StarSequitur, a novel algorithm extending Nevill-Manning's
Sequitur algorithm [11] . Omnisc'IO is
transparently integrated into the POSIX and MPI I/O stacks and does
not require any modification in applications or higher-level I/O
libraries. It works without any prior knowledge of the application
and converges to accurate predictions of any
In situ analysis and visualization workflows
Participants : Matthieu Dorier, Lokman Rahmani, Gabriel Antoniu.
In situ visualization has been proposed in the past few years to
couple running simulations with parallel visualization and analysis
tools. While many parallel visualization tools now provide in situ
visualization capabilities, the trend has been to feed such tools
with what previously was large amounts of unprocessed output data
and let them render everything at the highest possible
resolution. This leads to an increased run time of simulations that
still have to complete within a fixed-length job allocation. In this
work, we tackle the challenge of enabling in situ visualization
under performance constraints. Our approach shuffles data across
processes according to its content and filters out part of it in
order to feed a visualization pipeline with only a reorganized
subset of the data produced by the simulation. Our framework
monitors its own performance and reconfigures itself dynamically to
achieve the best possible visual fidelity within predefined
performance constraints. Experiments on the Blue Waters
supercomputer with the CM1 simulation show that our approach enables
a