Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Preparing an on-Demand Cloud Processing Workflow for NISAR Ecosystems Science Products

In preparation for the NISAR launch and data collection in 2024, the NISAR Project Science Team is building workflows for each Science Team discipline (Ecosystems, Cryosphere, and Solid Earth). This abstract focuses on the Ecosystem disciplines and the development of on-demand cloud-processing workflows for wetlands inundation, forest biomass, agricultural active crop area, and forest disturbance. The workflow simulates NISAR data using UAVSAR or ALOS-2 Single Look Complex data, which are processed to Level 2 geocoded polarimetric covariance matrix products using InSAR Scientific Computing Environment 3.0 software and to Level 3 science products using the Algorithm Theoretical Basis Documents. In this presentation, we describe these workflows and efforts to improve efficiency and data accessibility by using a cloud processing system. We present preliminary sample products from each Ecosystem discipline: inundation, forest biomass, crop area, and forest disturbance.

Christensen, Alexandra↗
DOE OSTI2024

A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization

Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. These optimizations can improve the compression ratio of SOTA approaches by up to 3.3× under the same data quality loss. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.

Wang, Daoce↗
DOE OSTI2025

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research

Earth observation satellites and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose significant challenges for processing and analysis. In this paper we discuss our experiences in developing and using a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our workflow can be seamlessly integrated and orchestrated across research facilities managed by different federal agencies, thus allowing users to extract new scientific insights from climate datasets. The experimental results indicate that the multi-facility workflow significantly reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Notably, our workflow processes 12,000 high-resolution satellite images in just 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems. Such high throughput is essential for dynamic tokenization and sharding of petascale satellite data for distributed AI model training and inferencing at scale across thousands of GPUs.

Kurihana, Takuya [ORNL] (ORCID:0000000156698565)↗
DOE OSTI2021

Emerging Frameworks for Advancing Scientific Workflows Research, Development, and Education

Lightning talks of the Workflows in Support of Large-Scale Science (WORKS) workshop are a venue where the workflow community (researchers, developers, and users) can discuss work in progress, emerging technologies and frameworks, and training and education materials. This paper summarizes the WORKS 2021 lightning talks, which cover four broad topics: (i) libEnsemble, a Python library to coordinate the concurrent evaluation of dynamic ensembles of calculations; (ii) Edu WRENCH, a set of online pedagogic modules that provides simulation-driven hands-on activity in the browser; (iii) VisDict, an envisioned visual dictionary framework that will translate terms, jargon, and concepts between research domains and workflow providers; and (iv) Pegasus Kickstart, a lightweight tool for capturing workflow tasks' performance, including performance metrics from Nvidia GPUs.

Casanova, Henri↗
DOE OSTI2022

Novel Proposals for FAIR, Automated, Recommendable, and Robust Workflows

Lightning talks of the Workflows in Support of Large-Scale Science (WORKS) workshop are a venue where the workflow community (researchers, developers, and users) can discuss work in progress, emerging technologies and frameworks, and training and education materials. This paper summarizes the WORKS 2022 lightning talks, which cover five broad topics: data integrity of scientific workflows; a machine learning-based recommendation system; a Python toolkit for running dynamic ensembles of simulations; a cross-platform, high-performance computing utility for processing shell commands; and a meta(data) framework for reproducing hybrid workflows.

Abhinit, Ishan↗
DOE OSTI2022

An automated workflow that generates atom mappings for large‐scale metabolic models and its application to Arabidopsis thaliana

SUMMARY Quantification of reaction fluxes of metabolic networks can help us understand how the integration of different metabolic pathways determines cellular functions. Yet, intracellular fluxes cannot be measured directly but are estimated with metabolic flux analysis (MFA), which relies on the patterns of isotope labeling of metabolites in the network. The application of MFA also requires a stoichiometric model with atom mappings that are currently not available for the majority of large‐scale metabolic network models, particularly of plants. While automated approaches such as the Reaction Decoder Toolkit (RDT) can produce atom mappings for individual reactions, tracing the flow of individual atoms of the entire reactions across a metabolic model remains challenging. Here we establish an automated workflow to obtain reliable atom mappings for large‐scale metabolic models by refining the outcome of RDT, and apply the workflow to metabolic models of Arabidopsis thaliana . We demonstrate the accuracy of RDT through a comparative analysis with atom mappings from a large database of biochemical reactions, MetaCyc. We further show the utility of our automated workflow by simulating 15 N isotope enrichment and identifying nitrogen (N)‐containing metabolites which show enrichment patterns that are informative for flux estimation in future 15 N‐MFA studies of A. thaliana . The automated workflow established in this study can be readily expanded to other species for which metabolic models have been established and the resulting atom mappings will facilitate MFA and graph‐theoretic structural analyses with large‐scale metabolic networks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Streaming Data in HPC Workflows Using ADIOS

The “IO Wall” problem, in which the gap between computation rate and data access rate grows continuously, poses significant problems to scientific workflows which have traditionally relied upon using the filesystem for intermediate storage between workflow stages. One way to avoid this problem in scientific workflows is to stream data directly from producers to consumers and avoiding storage entirely. However, the manner in which this is accomplished is key to both performance and usability. This paper presents the Sustainable Staging Transport, an approach which allows direct streaming between traditional file writers and readers with few application changes. SST is an ADIOS “engine”, accessible via standard ADIOS APIs, and because ADIOS allows engines to be chosen at run-time, many existing file-oriented ADIOS workflows can utilize SST for direct application-to-application communication without any source code changes. This paper describes the design of SST and presents performance results from various applications that use SST, for feeding model training with simulation data with substantially higher bandwidth than the theoretical limits of Frontier’s file system, for strong coupling of separately developed applications for multiphysics multiscale simulation, or for in situ analysis and visualization of data to complete all data processing shortly after the simulation finishes.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
DOE OSTI2025

The (R)evolution of Scientific Workflows in the Agentic AI Era: Towards Autonomous Science

Modern scientific discovery increasingly requires coordinating distributed facilities and heterogeneous resources, forcing researchers to act as manual workflow coordinators rather than scientists. Advances in AI leading to AI agents show exciting new opportunities that can accelerate scientific discovery by providing intelligence as a component in the ecosystem. However, it is unclear how this new capability would materialize and integrate in the real world. To address this, we propose a conceptual framework where workflows evolve along two dimensions which are intelligence (from static to intelligent) and composition (from single to swarm) to chart an evolutionary path from current workflow management systems to fully autonomous scientific laboratories. With these trajectories in mind, we present an architectural blueprint that can help the community take the next steps towards harnessing the opportunities in autonomous science with the potential for 100x discovery acceleration and transformational scientific workflows.

Shin, Woong [ORNL] (ORCID:0000000172077814)↗
DOE OSTI2023

Improv Dynamic Workflows

A workflow is a series of dependent computations which are executed to yield an experimental result, much like sheet music describes a musical performance. Dynamic workflows are like improvisational jazz, in which the musicians create unique, situation-driven, collaborative variations on themes. Dynamic workflows can potentially yield better results, save computation, and/or save human decision-making effort versus pre-specified or manual workflows, as they need not run the exhaustive set of studies which might be required in a pre-specified set, and they make their own experimental design decisions at run time. If you have an existing Maestro study where you would like to save computation and/or human time, Improv is for you. Improv provides core capabilities for creating a simple, hierarchical structure over Maestro studies, allowing you to connect ``experimental'' Maestro strudies with decision-making code to select parameters and run corresponding studies.

Goforth, John↗
DOE OSTI2022

Sim2Ls: FAIR simulation workflows and data

Just like the scientific data they generate, simulation workflows for research should be findable, accessible, interoperable, and reusable (FAIR). However, while significant progress has been made towards FAIR data, the majority of science and engineering workflows used in research remain poorly documented and often unavailable, involving ad hoc scripts and manual steps, hindering reproducibility and stifling progress. We introduce Sim2Ls (pronounced simtools) and the Sim2L Python library that allow developers to create and share end-to-end computational workflows with well-defined and verified inputs and outputs. The Sim2L library makes Sim2Ls , their requirements, and their services discoverable, verifies inputs and outputs, and automatically stores results in a globally-accessible simulation cache and results database. This simulation ecosystem is available in nanoHUB, an open platform that also provides publication services for Sim2Ls , a computational environment for developers and users, and the hardware to execute runs and store results at no cost. We exemplify the use of Sim2Ls using two applications and discuss best practices towards FAIR simulation workflows and associated data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Development of Time Lapse VSP Integration Workflow: A Case Study at Farnsworth CO2-EOR Project

Abstract This study aims to develop a 4D Vertical Seismic Profile (VSP) integration workflow to improve the prediction of subsurface stress changes. The selected study site is a 5-spot pattern within the ongoing CO2-EOR operations at the Farnsworth Field Unit FWU in Ochiltree County, Texas. The specific pattern has undergone extensive geological and geomechanical characterization through the acquisition of 3D seismic data, geophysical well logs, and core. This workflow constrains a numerical hydromechanical model by applying a penalty function formed between "modeled" versus "observed" time-lapse compressional and shear seismic velocity changes. Analyses of geophysical logs and ultra-sonic measurements on core exhibit measurable sensitivities to changes in both fluid saturation and mean effective stress. These data are used to develop a site-specific rock physics model and stress-velocity relationship, which inform the numerical models used to generate the "modeled" portion of the penalty function. The "observed" portion of the penalty function is provided by a novel elastic full-waveform inversion of the available 3D baseline and three monitor surveys to produce high-quality estimates of time-lapse compressional and shear seismic velocity changes. The modeling workflow accounts sequentially for fluid substitution and stress impacts. Hydrodynamic and geomechanical properties of the 3D coupled numerical model are estimated through geostatistical integration of well log and core data with 3D seismic inversion products. Changes in seismic velocities due to fluid substitution are computed using the Biot-Gassmann workflow and site-specific rock physics. Stress impacts on time-lapse seismic velocity changes are modeled from the effective stress output of the hydromechanical model and are initially based on the velocity versus effective stress relationship extracted from core mechanical testing. Based on the principle of superposition of seismic wavefields, seismic velocity changes attributed to fluid substitution and that due to changes in mean effective stress are treated as linearly additive. The modeled results are upscaled using Backus averaging to reconcile scale discrepancies between the modeled and measured datasets to formulate the penalty function. This manuscript presents the forward modeling process and concludes that for the base case, the seismic velocity changes due to mean effective stress dominates over the seismic velocity changes attributed to fluid substitution because of the extensive range of the pressure perturbations. Successful minimization of this penalty function calibrates the coupled hydrodynamic geomechanical numerical model and affirms the suitability of acoustic time-lapse measurements such as 4D-VSP for geomechanical calibration.

02 PETROLEUM↗
DOE OSTI2021

Execute BEE workflows on private cloud infrastructure (STNS01-22 BEE - FY21 P6-2)

Scope and objectives: BEE provides a portable, modular, HPC-focused workflow engine capable of managing containerized applications at scale. In FY21 BEE will expand its capabilities to provide more sophisticated handling of workflows. The ability to archive, clone, and re-run workflows will be added to BEE. The kinds of resources that BEE can use to execute workflow tasks will be expanded to include public and private clouds, such as Google Cloud Platform and OpenStack.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Applying 3D Geologic Modeling Workflows to the Argillite Reference Case (Rev. 1)

The objective of this short report is to document the application of our 3D geologic modeling workflow to an argillite (shale) host rock. Over the past four years, our team at Los Alamos National Laboratory has developed a geologic modeling workflow that can be applied to generic alluvial basins such as those found in the western United States. In “frontier” or “exploratory” basins where data are sparse, the first steps are to collect, evaluate and integrate available subsurface data into conceptual geologic models. Those models form the basis for constructing the geologic framework model, a 3D geocellular model ideally constrained by seismic and borehole data. To date we have constructed our models using “synthetic” well data derived from conceptual models, without the prospect of validating our workflow using “real” subsurface data. We were tasked to investigate whether our workflow designed for alluvial basin sediments could be applied to other potential repository host rocks. This task also provided the opportunity to work with high-quality subsurface data collected specifically for siting and evaluating a nuclear waste repository. Nagra, the Swiss governmental agency responsible for the disposal of the nation’s radioactive waste, generously provided us with data from two deep boreholes drilled through their argillaceous target formation. The aim of our proof-of-concept demonstration is to evaluate whether geostatistical methods offer a viable approach to property modeling in argillaceous rocks. Nagra provided us with the well data on the condition that we maintain confidentiality with all transferred information and results. Fortunately, Nagra posts numerous technical reports on its public website that describe the subsurface geology in great detail. All of the information and illustrations in this report related to the Swiss repository enterprise are taken from the Nagra public website.

58 GEOSCIENCES↗
DOE OSTI2024

A Complete Machine-Learning-Based Workflow to Illuminate Earthquake Processes

Under this grant we developed, tested, and made available, machine learning models to improve the tasks in the earthquake monitoring workflow (Figure 1). We implemented these models as a part of an end-to-end workflow for seismic network processing and demonstrated, in a variety of settings, that these methods generalize and that they result in dramatically more comprehensive earthquake catalogs. These catalogs illuminate earthquake processes in detail and to an extent that had previously not been possible, and they do so for both tectonic seismicity and seismicity induced by fluid injection related to unconventional hydrocarbon development. This report summarizes the results from the 15 publications that resulted from this grant. Those contributes are divided into: (1) the development of specific tasks related to monitoring (7 publications), (2) the organization of those tasks into workflows for seismic monitoring (2 publications), and (3) applications of those workflows to data (6 publications).

58 GEOSCIENCES↗
DOE OSTI2024

Advancements in Multiphysics Microdepletion Analysis of an eVinci TM -like Microreactor Leveraging OpenMC-CRAB Workflow

Nuclear microreactors (MRs) are a class of nuclear reactor technology, characterized by reduced dimensions, modular design, and reduced power output in contrast to conventional Light Water Reactors (LWRs). MRs are proposed for supplying electricity and eventual process heat to remote locations, such as military installations and disaster-affected areas. Current research work sponsored by the US Department of Energy Microreactor Program (MRP) is devoted to the development of novel modeling and simulation tools to better support MR vendors and regulatory bodies. Notably, the NRC is projected to utilize the CRAB multiphysics software driver for executing both design and beyond-design-basis accident analyses. Furthermore, the NRC has been utilizing the MELCOR code to calculate mechanistic source terms during accidents. Since MELCOR relies on isotopic inventory and reactor temperature/power profiles under accident conditions, which theoretically can be derived from CRAB, the goal is to establish a comprehensive CRAB-MELCOR computational framework. Past work was focused on testing and demonstrating CRAB's capability to generate results that can be used to inform mechanistic source term calculations in MELCOR. In particular, a computational workflow leveraging OpenMC-generated microscopic cross sections and CRAB was first applied to perform multiphysics microscopic depletion calculation followed by an accident scenario for a stylized microreactor problem. In fiscal year 2024, the research work has been focused on applying the OpenMC-CRAB workflow, which was first tested in fiscal year 2023, to a realistic 3D heat-pipe cooled MR problem representative of the eVinci TM design. The latter computational problem was developed with inputs from WEC to conserve selected neutronic and thermal characteristics of the eVinci TM design without releasing proprietary data. The results of this simulation, encompassing isotopic inventory, power density distribution, and kinetic parameters, will inform both MELCOR and the WEC-developed FATE code for mechanistic source terms calculations. The results from the two codes will then be compared for code verification purposes. This report contains the design characteristics of the realist heat pipe cooled microreactor developed as a use-case for the verification exercise, and the current results for the multiphysics microscopic depletion performed with the OpenMC-CRAB workflow. The results include eigenvalue as a function of time, power distribution at EOL, in addition to nuclides inventory's time evolution and spatial distribution. Finally, we report improvements to the workflow efficiency achieved through a collaboration with the NEAMS programs. Through this collaborative effort, we were able to strongly decrease the computational time for the multiphysics microdepletion calculation (i.e., from 17.4 hours to 5.7 hours on 280 processors) in addition to simplifying the interface to generate isotopics spatial distribution utilizable by FATE and MELCOR. Future work, including the improvement of the current microscopic cross-sections' library and the simulation of an accident scenario at EOL, is also discussed.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Transmission Operator Workflows for Real-Time Reliability Studies: A Review of Control Room Practices and Naturalistic Decision Making

This report provides an overview of real-time reliability study tools and their use by power system operators in the control room environment. After introducing some of the nuances of the control room environment and the differences in perspectives between power system engineers and operators, the roles and responsibilities of key entities involved in RTCA workflows are introduced. These are specifically the transmission system operator (TOP) and reliability coordinator (RC), which are required to run tools such as real-time contingency analysis (RTCA) as part of a real-time reliability assessment every 30 minutes, as dictated by a series of standards issued by the North American Electric Reliability Corporation (NERC). The process by which power systems operators operate the grid is discussed in terms of naturalistic decision making (NDM) and the recognition-primed decision-making (RPD) model. This cognitive model describe how experts working in high-risk, high-stress environments make safety-critical decisions under uncertainty and time pressure. For power system operators, the mental simulations involved in the traditional RPD model are supplemented by physics-based simulations using numerical tools, such as RTCA, to improve situational awareness and effectiveness of control actions. Next, a generic workflow is introduced to describe operator decision making for running RTCA tools and responding to system violations on a pre-contingent basis. The types of analysis performed and control actions chosen by power system operators are described in detail. The overall high-level workflow is then expanded in subsequent sections, with special attention given to high-voltage violations, low-voltage violations, and thermal overloads. Each type of violation is described in detail, with explanations of common causes, impacts on equipment and customers, and mitigation strategies. An additional workflow diagram is provided for each type of violation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Workflow for Developing and Operating Subsurface Hydrogen Storage Facilities in Porous Reservoirs

Long-duration (seasonal) storage of natural gas (NG), which primarily consists of methane (CH 4 ), has been practiced for more than a hundred years at underground gas storage (UGS) facilities that use depleted hydrocarbon reservoirs, saline aquifers, and salt caverns. To enable hydrogen (H 2 ) to be used as a long-duration, energy-storage medium, similar facilities are envisioned for underground H 2 storage (UHS) of either H 2 or H 2 /NG mixtures. Experience with UGS can be used to guide recommended practices for developing and operating UHS facilities in porous reservoirs. The most important factors (formation/fluid properties and engineering choices) that influence the performance of UHS reservoirs have been identified and quantified in previous studies. These factors and choices influence phenomena that determine the sweep efficiency of the stored working gas. These phenomena include viscous fingering, hysteretic capillary trapping, and gravity override of the working gas, as well as the upconing of nonproductive fluid that determine the sweep efficiency of the stored working gas. This report describes initial recommended-practices and a project-development workflow for UHS facilities that utilize porous reservoirs, based on the current state-of-knowledge about H 2 behavior in the subsurface. The workflow sequentially addresses all aspects of UHS project development, including the identification of H 2 sources and users, site ranking and down-selection, geologic and reservoir-engineering characterization, reservoir design, testing, risk management, commissioning, operations, and monitoring for a UHS facility. The goal is to enable UHS facilities to be developed in an efficient and timely manner, while carefully managing project risks. This workflow is similar to that which has been developed for UGS facilities (see Figure 1 of API, 2022), with the addition of tasks and subtasks specific to H 2 and UHS. The project-development workflow is broken down into three major stages: (1) define the H 2 use case; (2) rank, down-select, and characterize potential, candidate UHS sites; and (3) reservoir design, integrity testing, risk assessment, commissioning, operations, and monitoring for selected UHS sites. Each major stage is further broken down into tasks and subtasks, which are described at a high level. This report also provides more detailed descriptions of all tasks and subtasks that involve reservoir analysis and testing.

08 HYDROGEN↗
DOE OSTI2026

The Art of Automation: Translating Electron Microscopy Workflows Into Automated Processes

Acquiring data using a scanning transmission electron microscope (STEM) is a complex, multi-step process. The intricacy of the process depends on the type of sample, composition of the material, desired results of the experiment, resolution requirement and other experimental factors. Each experiment presents unique complications, such as sample drift and contamination, that the microscopist must consider when acquiring data. All these challenges are handled fluidly and expertly by experienced microscopists, but to reach new levels of innovation in material development, including greater reproducibility, throughput, and precision, the automation of these workflows is essential. The initial phase of this work involved translating intuition-based workflows into discrete, programmable steps. Some common key stages in STEM workflows are the initial tuning, scanning the sample for areas of interest, and then acquiring the data. Each stage can be broken further into specific parameter adjustments, such as aberration correction and dwell time optimization, depending on the experiment. When deconstructing various experiments each step was assessed for automation feasibility based on the amount of real time operator decisions. There are steps that lend themselves to automation more readily than others, such as course focusing and sample screening, but there is potential for full automation of all stages with time. As an initial step, an automated montage routine was developed, allowing for the efficient acquisition of large portions of the sample without requiring continuous intervention from the operator. The automation of this small process of the procedure demonstrates the value of this capability. A major challenge in automation arises from discrepancies between commanded, reported and actual stage movements. Using systematic tests, stage movement was quantified. This error can be corrected algorithmically for more accurate workflows in the future. Expanding automation capabilities would result in larger, more efficient data acquisition which allows for more robust statistical analysis. Additionally, this work lays the groundwork for a closed loop system where machine learning algorithms would intake automatically acquired data and make real time decisions. By progressively automating this instrument, this work establishes the foundation for fully automated experimentation in transmission electron microscopy.

97 MATHEMATICS AND COMPUTING↗