Interleaving Simulation and Data Analysis for the Edge to Exascale Data Path
Presenter
DescriptionLarge-scale data science workflows and long-running and dynamically changing scientific campaigns require cross-facility data science peering, but also need intra-facility peering of analytics and forward simulations. The model for execution of a scientific campaign increasingly calls for an interleaving of a forward simulation and data analysis. In the case of experimental data analysis taking place at the edge (for reduction and staging), analysis begins early in the data gathering process. The simulations that have helped design the experiment must now evaluate whether the collected data is aligned with the experimental plan. If not, careful reruns of the model simulation must take place rapidly to more effectively use the experiment apparatus. Similarly, in the case of large-scale forward HPC jobs (for modeling and simulation), generated data needs to be analyzed to exploit modern analytics (Machine Learning and Deep Learning) in order to speed up, short circuit, or steer various phases of the computation. The data analysis interleaved with forward simulation in this case is analogous to the first case of experimental facility data being analyzed in an interleaved feedback loop along with simulation data. We present a life-cycle view to data analysis that requires simulation and data analysis in a workflow that connects the Oak Ridge Leadership Computing Facility (OLCF) - the largest supercomputing platform in the world - via an interface through ORNL’s Compute and Data Environment for Science (CADES) to observational facilities at ORNL including the Spallation Neutron Source and Center for Nanophase Materials Science.
TimeWednesday, 7 July 202112:00 - 12:30 CEST
LocationMère Royaume
Session Chair
Event Type
Minisymposium
CS and Math
Emerging Applications
Chemistry and Materials
Physics
Life Sciences
Engineering
