Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Dynamic Workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Integrating Ultra-Coarse-Grained Protein Models into Accessible Workflows for Multiscale Molecular Dynamics

To capture protein conformational transitions using molecular dynamics (MD), several simulation resolutions covering different spatial and temporal scales are typically needed. All-atom (AA) simulations provide fine resolution, but are computationally infeasible for large systems over longer durations. Coarse-grained (CG) and ultra-coarse-grained (UCG) models have a lower resolution and computational cost while still being able to conserve essential protein features. Prior work on a Multiscale Machinelearned Modeling Infrastructure (MuMMI) combined both AA and CG simulations to study RAS-RAF protein interactions, leveraging CG models for longer time scales and using AA to investigate unusual conformations in greater detail. However, MuMMI is still resource-intensive, and this study aims to maximize exploration of the protein conformational space while reducing computational cost. In this paper, we build on prior work that integrates UCG models based on heterogeneous elastic network modeling (hENM) into the MuMMI workflow. We demonstrate that UCG models enable accurate sampling of protein conformations, focusing on simulating RAS-RAF protein interactions. Using higher-resolution CG Martini simulation data, we can automatically refine intramolecular interactions in UCG models. We present a scalable Python package that uses fluctuations observed in higher-resolution CG Martini simulations to estimate bond coefficients of the UCG model. We built novel machine learning-based backmapping methods to recover more detailed CG Martini structures from UCG structures, using diffusion models to learn the mapping between scales. Finally, we present UCG-mini-MuMMI, an accessible and less compute-intensive version of MuMMI as a resource for the scientific community. Incorporating UCG models into MD studies is applicable to a broad range of systems and proteins, and our study offers insights into the advantages and limitations of these methods.

Chemical structure↗

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)↗

DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics

The combination of ever-growing scientific datasets and distributed workflow complexity creates I/O performance bottlenecks due to data volume, velocity, and variety. Although the increasing use of descriptive data formats (e.g., HDF5, netCDF) helps organize these datasets, it also creates obscure bottlenecks due to the need to translate high level operations into file addresses and then into low-level I/O operations. To address this challenge, we introduce DaYu, a method and toolset for analyzing (a) semantic relationships between logical datasets and file addresses, (b) how dataset operations translate into I/O, and (c) the combination across entire workflows. DaYu's analysis and visualization enables identification of critical bottlenecks and reasoning about remediation. We describe our methodology and propose optimization guidelines. Evaluation on scientific workflows demonstrates up to 3.7x performance improvements in I/O time for obscure bottlenecks. The time and storage overhead for DaYu's time-ordered data is typically under 0.2% of runtime and 0.25% of data volume, respectively.

Tang, Meng↗

From 2D to 4D: a containerized workflow and browser to explore dynamic chromatin architecture

Background Characterizing the physical organization of the genome is essential for understanding long-range gene regulation, chromatin compartmentalization, and epigenetic accessibility. Hi-C experiments generate two-dimensional (2D) genome-wide contact maps of chromatin interactions by capturing the spatial proximity between genomic loci, which reveal interaction frequencies but lack the spatial resolution needed to interpret the three-dimensional (3D) genome structure(s). Emerging evidence suggests that epigenetic regulation is closely linked to 3D genome architecture, and that structural changes over time (4D) drive key biological processes in development, disease, and environmental response. Thus, integrating 3D structure with functional data is critical for a more complete understanding of genome regulation. Previous work, most notably the 4DHiC chromosome modeling framework, has shown that physical multi-dimensional modeling approaches rooted in polymer physics and molecular dynamics can resolve these structures at biologically meaningful resolutions by integrating temporal Hi-C data with physical constraints to uncover dynamic chromosome reorganization. Thus, molecular dynamics simulations, constrained by Hi-C contact matrices, can resolve fine-scale structural changes and reveal functionally significant transitions in chromatin conformation. Results Herein, we present the 4D Genome Browser Workflow (4DGBWorkflow) and the 4D Genome Browser (4DGB). The algorithm is based on the 4DHiC method, and the containerized tool is an end-to-end workflow that can transform, filter, and view 4D epigenomics and chromatin datasets, allowing non-specialists to apply three-dimensional modeling principles to diverse datasets and experimental conditions. The software executes on a laptop running macOS, Linux or Windows. From input Hi-C files (.hic), the 4DGBWorkflow produces 3D reconstructions of chromosomes, integrates the reconstruction with track data (e.g., epigenetic marks, transcriptome profiles), and provides comparative visualization of the results in a single workflow. Conclusions The 4DGBWorkflow and 4D Genome Browser are open-source tools for comparative analysis and visualization of 4D chromosome datasets, including chromatin architecture and epigenomic signals. Automatic integration of Hi-C data with molecular dynamics democratizes the construction of time resolved 3D genome structures, simplifying complex simulations and data integration schemes.

3D Genome Browser↗

MARS: Malleable Actor-Critic Reinforcement Learning Scheduler

In this paper, we introduce MARS, a new scheduling system for HPC-cloud infrastructures based on a cost-aware, flexible reinforcement learning approach, which serves as an intermediate layer for next generation HPC-cloud resource manager. MARS ensembles the pre-trained models from heuristic workloads and decides on the most cost-effective strategy for optimization. A whole workflow application would be split into several optimizable dependent sub-tasks, then based on the pre- defined resource management plan, a reward will be generated after executing a scheduled task. Lastly, MARS updates the Deep Neural Network (DNN) model based on the reward. MARS is designed to optimize the existing models through reinforcement mechanisms. MARS adapts to the dynamics of workflow applications, selects the most cost-effective scheduling solution among pre-built scheduling strategies (backfilling, SJF, etc.) and self- learning deep neural network model at run-time. We evaluate MARS with different real-world workflow traces. MARS can achieve 5%-60% increased performance compare to state-of-the- art approaches.

Baheri, Betis↗

Unstructured Grid Development for the Space Launch System Liftoff and Transition Lineloads Computational Analysis

Production of aerodynamic lineloads databases for the Space Launch System (SLS) vehicle at Liftoff and Transition (LOT) conditions has required the development of a Computational Fluid Dynamics (CFD) workflow capable of producing high-quality solutions for this unique phase of flight. Aerodynamic considerations included a wide range of flow angles (from 0°up to 90°total angle of attack), resulting leeside separation, and interaction effects between the three bodies of the integrated SLS vehicle, as well as the nearby launch tower. Computational mesh development for similar problems at the NASA Langley Research Center (such as for the Constellation/Ares launch vehicles) has primarily relied on in-house tools such as VGRID/POSTGRID, with grids designed for NASA-developed and maintained flow solvers such as USM3D and FUN3D. The workflow for such problems has evolved over the development of the various SLS configurations to incorporate new tools such as the Heldenpatch/Heldenmesh grid generator (Helden Aerospace) and CREATE-AV Kestrel (US Department of Defense) flow solver. This paper describes efforts to benchmark a grid generation approach for LOT problems using Heldenpatch/Heldenmesh and Kestrel, verified against prior best practices from VGRID/POSTGRID. Parameters studied include surface grid density, first-layer viscous cell height, and volume grid growth rate parameters. Resulting solutions are compared based on total force and moment values, sectional line loads, and surface pressures, all validated against existing wind tunnel aerodynamic databases where available for the SLS Block 1B Cargo configuration.

Space launch systems↗

Accelerated Assessment of Critical Infrastructure in Aiding Recovery Efforts During Natural and Human-made Disaster

Relief and recovery from disasters (both natural and human-made) require a coordinated approach across several federal and state government agencies. In order to achieve optimal resource allocation and deployment of first responders, accurate and timely assessment of the impact and extent of destruction are the cornerstones to any recovery effort. Ideally, this knowledge should be gathered and shared within the first 0-24 hours (termed as "Acute Phase" by the U.S. CDC guideline) for informed decision-making. But achieving this poses significant challenges for the data collection and data harmonization processes, particularly when voluminous data are being generated from diverse and distributed sources during the disaster responses. To this end, this work developed a scalable and efficient workflow to dynamically collect and harmonize crowd-sourced geographic multi-modal data, and then assess critical infrastructure (CI) damaged during disaster events. We demonstrate the application of our framework with two real-world experiences in addressing post-disaster recovery efforts - for the Bahamas (Natural - due to Hurricane Dorian, 2019) and Beirut (Human-made - due to explosion caused by the ammonium nitrate stored in a warehouse, 2020). We have illustrated that a coordinated effort is needed for planning as well as for execution to achieve informed decision making.

Thakur, Gautam Malviya↗

A high-throughput framework for lattice dynamics

We develop an automated high-throughput workflow for calculating lattice dynamical properties from first principles including those dictated by anharmonicity. The pipeline automatically computes interatomic force constants (IFCs) up to 4th order from perturbed training supercells, and uses the IFCs to calculate lattice thermal conductivity, coefficient of thermal expansion, and vibrational free energy and entropy. It performs phonon renormalization for dynamically unstable compounds to obtain real effective phonon spectra at finite temperatures and calculates the associated free energy corrections. The methods and parameters are chosen to balance computational efficiency and result accuracy, assessed through convergence testing and comparisons with experimental measurements. Deployment of this workflow at a large scale would facilitate materials discovery efforts toward functionalities including thermoelectrics, contact materials, ferroelectrics, aerospace components, as well as general phase diagram construction.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Visual HPC Workflows for the Analysis of System Dynamics Models

Visual analytics supported by high performance computing (HPC) accelerates and enhances the discovery, exploration, and analysis of causal patterns in complex system dynamics (SD) models. We present a suite of visualization-assisted ensemble-based techniques for hypothesis generation and testing, and for sensitivity analysis. By employing HPC to provide parallel, on-demand simulation of SD models, one can “steer” an ensemble of simulated scenarios in real time as one first formulates and then informally tests those hypotheses: this provides rapid feedback for analysts to refine their understanding of the causal relationships emergent from a model. Such understandings can be followed and augmented by rigorous application of statistical methods, namely global variance-based sensitivity analysis, Monte-Carlo filtering, adaptive regional sensitivity analysis, and self-organized maps: here timely computation relies on HPC, while effective presentation emphasizes high-dimensional multivariate data visualization. Immersive visualization in virtual 3D environments provides an excellent adjunct to the traditional 2D graphics typically used for SD models, as it generates an embodied understanding of model behavior and facilitates an active, collaborative critique of model structure and output. Finally, we summarize prospects for HPC-enabled visual analytics applied to SD modeling.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Intelligent Process Visualization through Nuclear Operation Process Modeling, Reasoning, and Object Detection from Field Videos (Final Report)

This report is a deliverable for the “Final Report” task of DOE NEET Project 19-16790, "Context-Aware Safety Information Display for Nuclear Field Workers." This project's overall goal is to test the hypothesis that integrating computer vision and process reasoning methods will enable proactive visualization of the safe operation and maintenance processes of Nuclear Power Plants (NPP) for field workers. Augmented Reality (AR) glasses adopting such proactive safety information visualization techniques can significantly increase personnel safety and reduce the NPP’s operating costs. The current practice of monitoring NPPs requires workers to switch between digital models, data, and physical workspaces in identifying relevant but potentially occluded objects and in assessing the risks of operation and maintenance processes. On the other hand, frequently changed field conditions require field workers to report to supervisors for real-time guidance. Such guidance is essential to ensure that changing conditions will not invalidate or endanger the work order and other ongoing processes that may jeopardize NPP operations. Additionally, incorrect recognition of equipment objects can result in communication errors and safety problems. AR techniques can assist engineers in viewing the physical workspaces with objects labeled with detailed operation procedures and safety reminders during field operations. The project team developed an “Intelligent Context-Aware Safety Information Display” (ICAD) for supporting Nuclear Power Plant (NPP) field workers in achieving safe and efficient execution of a series of operational tasks in uncertain and changing workspaces of an NPP. Before designing the ICAD-AR prototype, the project team synthesized NPP operational knowledge models through literature review studies, surveys, interviews with domain experts, and knowledge modeling. The project team conducted an extensive study of the operational procedures of various NPPs, and digital technologies that can support the safe and efficient execution of those procedures in different NPP operational contexts. This literature review helped the project team conduct surveys and interviews with nuclear engineers and field workers to identify three categories of information. The NPP knowledge modeling efforts reveal that the three categories of information identified have different levels of importance in a typical procedure of carrying out a series of tasks to achieve a specific NPP operation goal (e.g., shutdown, mode changes). These three categories of information include 1) Workspace dynamics – the changing spatial arrangements of workspaces, tools, protection equipment, and supporting materials, 2) Workflow prognostics – the dynamic dependencies between different parts of an NPP that functionally support and influence each other in terms of safety and efficiency, and 3) Hazards – objects and spaces that contain hazardous materials or physical conditions that can pose risks to workers or mechanical systems. The project team has profiled the importance levels of these categories of information into a knowledge model. This knowledge model specifies what types of information are more critical for a given task in a given workspace so that computers can automatically identify critical objects and sensors in a scene for delivering context-ware safety information to field workers through AR devices. Significant research development of this project results in technical research outcomes and a prototyping system that illustrates the technical feasibility of establishing an ICAD-AR system supporting the proactive safety information display for nuclear field workers. This final report summarizes the project team’s technological achievements in the past three years. Overall, the project team completed the development and integration of five techniques into a prototype ICAD Augmented Reality (ICAD-AR) system and demonstrated the developed system’s real-time execution in a mechanical room. The project team completed the analysis of using this prototype in other types of workspaces based on 3D image data and digital design models collected from two additional workspaces (a water treatment plant and a flow loop training facility). The integrated techniques include 1) Natural Language Processing (NLP) algorithms supporting the generation and updates of nuclear fieldwork process models based on text analysis of work packages and operation manuals; 2) sensor log analysis for predicting control actions in given sensor reading contexts; 3) computer vision algorithms for automatic localization and navigation of workers; 4) object detection algorithms for identifying task-related objects and correlated sensors for safety checking; 5) AR technique as a platform for supporting the integration. The testing results of these five techniques have shown that 1) the sensor log analysis model can predict the next control action with an accuracy of 0.883; 2) the trained natural language processing model can extract more than 80% of the critical information from paper-based procedures (PBPs); 3) the navigation algorithm with the integration of Visual Inertial Odometry (VIO) and Non-Recursive Bayesian Filter methods make operator’s trajectory estimation resilient to drift error; 4) the computer vision algorithm can detect task-specific and safety-critical objects with an average accuracy of 95.3%. The project team used work procedures collected from a flow loop training facility and two datasets collected from two mechanical rooms simulating the workspaces of NPPs to demonstrate the technical capabilities of the developed ICAD-AR prototype. The demonstration validated the technical feasibility of establishing the ICAD-AR system for nuclear field workers and identified the challenges in 1) automatic text analysis of work packages; 2) use of limited samples of sensor logs for predicting the proper timings of control actions; 3) reliably tracking workers and their task progress in mechanical rooms with many similar objects.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Simulation process and data flow for a large system dynamics model

This paper documents the workflow and supporting technologies that a large system dynamics model, the biomass scenario model, employs to streamline the data preparation, simulation, quality control, and analysis process at the National Renewable Energy Laboratory. The workflow centers on automation of routine aspects of the flow of data between data stores, simulations, and visualizations. It enforces quality checks on data, reproducibility of computations, and traceability of results, while maintaining complete archives of modeling and analysis artifacts. The resulting frictionless simulation/analysis environment supports large-scale sensitivity analysis, interactive creation of ensembles of simulations, and rapid visualization-based exploration of simulation results.

09 BIOMASS FUELS↗

On the integration of molecular dynamics, data science, and experiments for studying solvent effects on catalysis

Computational workflows that combine molecular dynamics (MD) simulations and emerging data-centric (DC) methods can accelerate the screening and analysis of solvent systems experimentally and computationally. Here, MD simulations provide atomic positions and velocities of reactant, solvent, and catalyst materials that can be manipulated into data representations that in turn can be used by DC techniques to conduct predictive modeling, feature extraction, and experimental design. For liquid-phase catalytic applications, emerging DC techniques such as Convolutional and Graph Neural Networks (CNN/GNN), Topological Data Analysis (TDA), and Active Learning (AL) can leverage MD and experimental data to quickly predict solvent effects on reaction outcomes. For instance, in recent studies, 3D solvent environments obtained with MD have been exploited by CNNs to predict experimental reaction rates for homogeneous acid-catalyzed lignocellulosic processes. In this perspective, we discuss basic principles of DC methods and how these can be combined with MD to enable high-throughput screening of solvent selection for diverse catalysis applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Catching the Killer: Dynamic Disorder Design Rules for Small–Molecule Organic Semiconductors

Small-molecule organic semiconductors are promising materials for applications ranging from solar cells to medical sensors. Their nearly infinite design space means that, in theory, it is possible to tailor the material to the exact specifications of a particular application. In reality; however, design rules to improve mobility (μ) remain elusive because of the complex calculations required to understand its limiters. Transient localization theory posits that charge carriers are slowed down by the collective phonon motions, called dynamic disorder, which localize charge carriers temporarily. Traditionally, a mode-by-mode analysis is performed to try and identify “killer” modes. Herein, the flaws are demonstrated with this analysis and present a streamlined simulation workflow that simplifies simulation of dynamic disorder and enables engineering of new molecular structures based on phonon dynamics. This workflow is combined with a novel visualization technique that enables per-atom insights into how μ is limited. Furthermore, this workflow is applied and analyzed to a series of -acenes and a series of BTBT-based molecules. Then design rules are identified in each series that identify possible ways to improve the μ beyond current experimental limits using phonon engineering.

36 MATERIALS SCIENCE↗

Performance assessment of ensembles of in situ workflows under resource constraints

Summary Scientific breakthroughs in biomolecular methods and improvements in hardware technology have shifted from a long‐running simulation to a large set of shorter simulations running simultaneously, called an ensemble. In an ensemble, simulations are usually coupled with analyses of data produced by the simulations. In situ methods can be used to analyze large volumes of data generated by scientific simulations at runtime (i.e., simulations and analyses are performed concurrently). In this work, we study the execution of ensemble‐based simulations paired with in situ analyses using in‐memory staging methods. Using an ensemble of molecular dynamics in situ workflows with multiple simulations and analyses, we first show that collecting traditional metrics such as makespan, instructions per cycle, memory usage, or cache miss ratio is not sufficient to characterize complex behaviors of ensembles. We propose a method to evaluate the performance of ensembles of workflows that captures multiple resource usage aspects: resource efficiency, resource allocation, and resource provisioning. Experimental results demonstrate that the proposed method can effectively distinguish the performance of different component placements in an ensemble with up to 32 ensemble members. By evaluating different co‐location scenarios, our proposed performance indicators demonstrate benefits of co‐locating simulation and coupled analyses within a compute node.

Do, Tu Mai Anh↗

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

This report details recent progress for the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows”. We refer to the project as IPPD/2, reflecting the 2017 renewal under expanded scope and partners In IPPD/2, we increased our research scope to include data motion. We are focusing on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. This new work on data motion will augment and complement IPPD/2’s research that focused on the computational aspects of tasks. We leverage and extend our existing tools and demonstrate our work on the Belle II workflow suite as well as on workflows from NSLS-II. The highlights of our work are as follows: Provenance for Workflows: Provenance is used to provide information enabling quality control, re-run computational workflows, and reproduce results. IPPD/2 has been building a scalable provenance management system that enables the capture of provenance from the high-level workflow through all relevant system levels in one integrated environment. Leveraging this work, our recent efforts have included using provenance as an enabling technique. Workload characterization: Leveraging provenance and analysis, we characterize data movement within network, storage, and memory over a variety of workloads. This characterization enables an understanding by performance analysts and application developers of the range of behaviors that could be expected. Performance Prediction for Workflows: The goal of modeling distributed workflows is to understand performance bottlenecks and enable more intelligent task scheduling to optimize selected metrics of interest (e.g., task throughput or output data rate). IPPD/2 has utilized both analytical and AI/ML modeling methodologies for performance modeling. Advanced Scheduling and Fault Modeling for Workflows: Scheduling of large-scale scientific workflows on geographically distributed resources is a challenging problem. To improve workflow throughput, we combined novel scheduling algorithms with task predictions from performance modeling and fault modeling. Dynamically Alleviating Bottlenecks in Workflows: Exploiting our provenance, analysis, and modeling efforts, we have explored and developed several techniques for dynamically detecting and alleviating bottlenecks in data movement. In particular, we have spent considerable effort demonstrating our techniques on production-like workflow configurations.

97 MATHEMATICS AND COMPUTING↗

Hybrid Simulation Framework

HYBRID is a modeling toolset to assess the economic viability of Nuclear-Renewable Integrated Energy Systems (N-R IES). The frameworks enabling this toolset are INL’s RAVEN, its CashFlow plugin and the Modelica language. The toolset includes sample RAVEN workflows performing economic assessments. These workflows consist of: generation of stochastic time series and application of probabilistic analysis and optimization algorithms (RAVEN); a library of Modelica models representing the physical behavior of N-R IES; and the CashFlow plugin mapping physical performance to economic performance. The toolset allows assembling existing and new models such as nuclear reactors, renewable energy sources, energy storage, gas turbines, industrial processes, etc. into an N-R IES. The toolset workflows evaluate the dynamics of the N-R IES responding to stochastic conditions (electricity demand, price, etc.) and optimize the dispatch economics as well as N-R IES capacity planning.

Epiney, Aaron↗