SEARCH · Engineering Papers

Results for “Workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Automated ICRF heating surrogate modeling via machine learning

This work introduces automated machine learning workflows that address critical bottlenecks in surrogate model development for Ion Cyclotron Range of Frequencies (ICRF) heating applications. The automated framework includes data analysis tools that transform raw datasets into actionable insights in seconds, replacing weeks of manual exploratory effort and ensuring consistent, reproducible dataset characterization. By integrating advanced hyperparameter optimization (HPO) methods including Bayesian optimization via BoTorch and Tree-structured Parzen Estimators (TPE), the framework significantly reduces model development time from weeks to hours, decreasing computational cost and required expertise, while enabling high-accuracy surrogate models. Compared to traditional hyperparameter scanning (HPS) techniques such as methodical, randomized, and grid searches, HPO methods achieve superior convergence and predictive performance, even when compared to already well-tuned reference models. On NSTX High Harmonic Fast Wave (HHFW) heating datasets, both Random Forest Regressor (RFR) and neural network surrogates demonstrate improved accuracy, achieving R 2 values beyond 0.97 and 0.98, respectively. The results show that while HPO gains are modest for robust architectures like RFR, they become essential for more sensitive models such as neural networks, highlighting the trade-offs across optimization strategies. Through automated workflows that eliminate manual hyperparameter tuning and require minimal ML expertise, this work enables widespread adoption of high-fidelity surrogate models across the fusion community for real-time plasma control, uncertainty quantification, rapid experimental scenario development, and integrated system optimization.

Sanchez-Villar, Alvaro [Princeton Plasma Physics L↗

DOE OSTI2023

High-throughput ab initio reaction mechanism exploration in the cloud with automated multi-reference validation

Quantum chemical calculations on atomistic systems have evolved into a standard approach to studying molecular matter. These calculations often involve a significant amount of manual input and expertise, although most of this effort could be automated, which would alleviate the need for expertise in software and hardware accessibility. Here, we present the AutoRXN workflow, an automated workflow for exploratory high-throughput electronic structure calculations of molecular systems, in which (i) density functional theory methods are exploited to deliver minimum and transition-state structures and corresponding energies and properties, (ii) coupled cluster calculations are then launched for optimized structures to provide more accurate energy and property estimates, and (iii) multi-reference diagnostics are evaluated to back check the coupled cluster results and subject them to automated multi-configurational calculations for potential multi-configurational cases. All calculations are carried out in a cloud environment and support massive computational campaigns. Key features of all components of the AutoRXN workflow are autonomy, stability, and minimum operator interference. We highlight the AutoRXN workflow with the example of an autonomous reaction mechanism exploration of the mode of action of a homogeneous catalyst for the asymmetric reduction of ketones.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Elevating zero dimensional global scaling predictions to self-consistent theory-based simulations

In this work, we have developed an innovative workflow, Stability, Transport, Equilibrium, and Pedestal (STEP)-zero-dimensional (0D), within the OMFIT integrated modeling framework. Through systematic validation against the International Tokamak Physics Activity global H-mode confinement database, we demonstrated that STEP-0D, on average, predicts the energy confinement time with a mean relative error of less than 19%. Moreover, this workflow showed promising potential in predicting plasmas for proposed fusion reactors such as the affordable, robust, compact (ARC) reactor, the European demonstration power plant (EU-DEMO), and the China fusion engineering test reactor (CFETR) indicating moderate H-factors between 0.9 and 1.2. STEP-0D allows theory-based prediction of tokamak scenarios, beginning with 0D quantities. The workflow initiates with the PRO-create module, generating physically consistent plasma profiles and equilibrium using the same 0D quantities as the IPB98(y,2) confinement scaling. This sets the starting point for the STEP module, which further iterates between theory-based physics models of equilibrium, core transport, and pedestal to yield a self-consistent solution. Given these attributes, STEP-0D not only improves the accuracy of predicting plasma performance but also provides a path toward a novel fusion power plant design workflow. When integrated with engineering and costing models within an optimization, this new approach could eliminate the iterative reconciliation between plasma models of varying fidelity. This potential for a more efficient design process underpins STEP-0D's significant contribution to future fusion power plant development.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Electronic structure simulations in the cloud computing environment

The transformative impact of modern computational paradigms and technologies, such as high-performance computing, quantum computing, and cloud computing, has opened up profound new opportunities for scientific simulations. Scalable computational chemistry is one beneficiary of this technological progress. The main focus of this paper is on the performance of various quantum chemical formulations, ranging from low-order methods to high-accuracy approaches, implemented in different computational chemistry packages, such as NWChem, NWChemEx, SPEC, ExaChem, and FLOSIC codes on the Azure Quantum Element (AQE) Microsoft cloud services. We pay particular attention to the intricate workflows for performing composite chemistry simulations, associated data curation, and mechanisms for accuracy assessment, as defined by the enabling cloud Computational Chemistry as a Service (CCaaS). Our focus also extends to Arrows' automated workflow for high throughput simulations. Finally, we provide a perspective on the role of cloud computing in supporting the mission of leadership computational facilities (LCFs).

computational chemistry, electronic structure, Clo↗

DOE OSTI2020

Veridical data science

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, composed of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. As part of the PCS workflow, we develop PCS inference procedures, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others. Moreover, we demonstrate its favorable performance over existing methods in terms of receiver operating characteristic (ROC) curves in high-dimensional, sparse linear model simulations, including a wide range of misspecified models. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Separatrix-to-Wall Simulations of Impurity Transport with a Fully Three-Dimensional Wall in DIII-D

A novel multi-code workflow to interpret collector probe deposition patterns in DIII-D has been developed. The components of the workflow consist of a detailed computer-aided design (CAD) file of the vessel wall and the scrape-off layer (SOL) codes MAFOT, OSM, DIVIMP and 3DLIM. A special-purpose toolkit enables passing the output of these codes between each other to provide a full-SOL picture of impurity transport. A demonstration of the workflow is described to support evidence of near-SOL tungsten parallel accumulation during trace W impurity experiments on DIII-D. Iteration between simulated deposition patterns in 3DLIM and DIVIMP predicts a region of elevated W density near the separatrix about halfway between the outboard midplane and the top of the plasma. Furthermore, this workflow will be used to better interpret collector probe experiments on DIII-D.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

The ASHRAE Great Energy Predictor III competition: Overview and results

In late 2019, ASHRAE hosted the Great Energy Predictor III (GEPIII) machine learning competition on the Kaggle platform. This launch marked the third energy prediction competition from ASHRAE and the first since the mid-1990s. In this updated version, the competitors were provided with over 20 million points of training data from 2,380 energy meters collected for 1,448 buildings from 16 sources. This competition’s overall objective was to find the most accurate modeling solutions for the prediction of over 41 million private and public test data points. Furthermore, the competition had 4,370 participants, split across 3,614 teams from 94 countries who submitted 39,403 predictions. In addition to the top five winning workflows, the competitors publicly shared 415 reproducible online machine learning workflow examples (notebooks), including over 40 additional, full solutions. This paper gives a high-level overview of the competition preparation and dataset, competitors and their discussions, machine learning workflows and models generated, winners and their submissions, discussion of lessons learned, and competition outputs and next steps. The most popular and accurate machine learning workflows used large ensembles of mostly gradient boosting tree models, such as LightGBM. Similar to the first predictor competition, preprocessing of the data sets emerged as a key differentiator.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2024

From minimum-viable-products to full models: a step-wise development of diagnostic forward models in support of design, analysis and modelling on the ST40 tokamak

Like most magnetic confined fusion experiments, the ST40 tokamak started off with a small subset of diagnostics and gradually increased the diagnostic set to include more complex and comprehensive systems. To make the most of each operational phase, forward models of various diagnostics are used and developed to aid design, provide consistency-checks during commissioning, test analysis methods, and build workflows to constrain high-level parameters to inform interpretation, theory and modelling. For new models and new analysis workflows, minimum-viable-products are released early, and their complexity is increased in a step-wise manner, facilitating the support of all programme phases on multiple parallel applications, while enabling learning opportunities and feedback loops. In this contribution we review the philosophy, scope and architecture of the framework under development. We discuss the details of some forward models, with examples on how they are used to aid diagnostic design, to investigate analysis methodologies through synthetic data, and how they are embedded in experimental analysis workflows. We compare previously published experimental results with new, more advanced analysis workflows employing more recent, detailed models and new diagnostic data, providing confirmation of the published material from the 2021–22 experimental campaign.

integrated data analysis↗

DOE OSTI2026

Statistical inference of anomalous thermal transport with uncertainty quantification for interpretive 2D SOL models

The critical task of inferring anomalous cross-field transport coefficients is addressed in simulations of boundary plasmas with fluid models. A workflow for parameter inference in the UEDGE fluid code is developed using Bayesian optimization with parallelized sampling and integrated uncertainty quantification. In this workflow, transport coefficients are inferred by maximizing their posterior probability distribution, which is generally multidimensional and non-Gaussian. Uncertainty quantification is integrated throughout the optimization within the Bayesian framework that combines diagnostic uncertainties and model limitations. As a concrete example, we infer the anomalous electron thermal diffusivity $\chi_\perp$ from an interpretive 2D model describing electron heat transport in the conduction-limited region with radiative power loss. The workflow is first benchmarked against synthetic data and then tested on H-, L-, and I-mode discharges to match their midplane temperature and divertor heat flux profiles. We demonstrate that the workflow efficiently infers diffusivity and its associated uncertainty, generating 2D profiles that match 1D measurements. Future efforts will focus on incorporating more complicated fluid models and analyzing transport coefficients inferred from a large database of experimental results.

Bayesian optimization↗

DOE OSTI2021

Bragg coherent diffraction imaging by simultaneous reconstruction of multiple diffraction peaks

Bragg coherent diffractive imaging (BCDI) is a non-invasive microscopy technique that can visualize the morphology and internal lattice deviations of crystals with nanoscale spatial resolution and picometer deformation sensitivity. While BCDI has been successfully applied in various studies of materials, it is less successful for highly-strained crystals. Specifically, it is difficult to correctly reconstruct the electron density of a highly-strained object using conventional phase retrieval algorithms. Although various algorithms have been developed to overcome this challenge, most of them require a priori knowledge that is not always available in practice. Here we report on a new phase retrieval workflow that can invert diffraction patterns from multiple Bragg peaks simultaneously. The new workflow is explored via simulated diffraction from crystals with various strain conditions. Furthermore, reconstructions from the new workflow consistently demonstrate more accurate electron density maps, in comparison with the conventional method. For highly-strained crystals, the new workflow improves the reliability and consistency of BCDI phase retrieval significantly.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Normality of I-V Measurements Using ML

There is an increased interest in instrument-computing ecosystems (ICEs) that support science workflows empowered by AI-automated experiments and computations in diverse areas. In particular, electrochemistry ICEs are promising for accelerating the design and discovery of electrochemical systems for energy storage and conversion, by automating significant parts of workflows that combine synthesis and characterization experiments with computations. They require the integration of flow controllers, solvent containers, pumps, fraction collectors, and potentiostats, all connected to an electrochemical cell, as illustrated in Fig. 1. These are specialized instruments with custom software that is not originally designed for network integration. We developed network and software solutions for electrochemical workflows that adapt system and instrument settings in real-time for multiple rounds of experiments. In particular, we developed Python wrappers for Application Programming Interfaces (APIs) of instrument commands and Pyro client-server modules that enable them to be executed from remote computers. The entire workflow is orchestrated by a Jupyter notebook running on a remote computer.

Al Najjar, Anees↗

DOE OSTI2024

Autonomous Electrochemistry Platform with Real-Time Normality Testing of Voltammetry Measurements Using ML

Electrochemistry workflows utilize various instruments and computing systems to execute workflows consisting of electrocatalyst synthesis, testing and evaluation tasks. The heterogeneity of the software and hardware of these ecosystems makes it challenging to orchestrate a complete workflow from production to characterization by automating its tasks. We propose an autonomous electrochemistry computing platform for a multi-site ecosystem that provides the services for remote experiment steering, real-time measurement transfer, and AI/ML-driven analytics. We describe the integration of a mobile robot and synthesis workstation into the ecosystem by developing custom hub-networks and software modules to support remote operations over the ecosystem’s wireless and wired networks. We describe a workflow task for generating I-V voltammetry measurements using a potentiostat, and a machine learning framework to ensure their normality by detecting abnormal conditions such as disconnected electrodes. We study a number of machine learning methods for the underlying detection problem, including smooth, non-smooth, structural and statistical methods, and their fusers. We present experimental results to illustrate the effectiveness of this platform, and also validate the proposed ML method by deriving its rigorous generalization equations.

Alnajjar, Anees↗

DOE OSTI2024

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

The use of convolutional neural networks (CNNs) for building extraction from remote sensing images has been widely studied and many public datasets have been made available for accelerating development of these CNN models. Yet adapting pretrained models at scale in real-world scenarios remains a challenging task. The main barrier is that certain new labels are still needed to compensate for domain shifting between the labeled data and new images that potentially cover new geographic locations or that are from a different sensor. In this article, we propose to add informatively labeled samples from a new image pool under the paradigm of active learning. To select the most useful samples based on model uncertainty, we first tackle the problem of uncalibrated uncertainty estimation due to distribution shifting by adapting feature extractors with boundary-based adversarial learning. Calibrated uncertainty is used as the query criterion in the active learning process, where the most uncertain samples are selected for annotation and included for model retraining. The proposed workflow was tested with three data pairs in which each workflow represents a scenario often encountered in real-world applications, including adapting pretrained models to new images collected with different sensors or to new geographic areas where appearances and types of buildings are very different. Compared to several baselines, including random sampling, temperature scaling (a well-known uncertainty calibration technique), different query strategies, and active domain adaptation methods, the proposed workflow shows that strategically querying a smaller set of samples for labeling achieves comparable or better building extraction performance. The proposed method reduces the number of labeled samples required to achieve sufficient model accuracy, thus significantly reducing hundreds of person-hours for labeled data creation. In addition, we include a few considerations when deploying this workflow in a GPU cluster that can be easily adapted to achieve operational building extraction model retraining.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

A Methodology for Assessing Risk to Inform Technology Integration

When new technology, such as artificial intelligence (AI), is introduced into an existing workflow it may impact risk by mitigating some vulnerabilities and threats in the workflow while introducing others. We present a versatile methodology for assessing the vulnerabilities and threats that impact overall risk in a workflow to inform technology integration. Our method involves both qualitative and quantitative assessment of risk and includes a formula for generating a risk score to guide technology integration. We describe our methodology and demonstrate its application to a specific workflow (the Derivative Classification review process). This work was funded by the Department of Energy (DOE) Automated Classification Tools for the Identification of Classified Information (ACTICI) program.

risk, technology integration↗

DOE OSTI2020

Processing Full-Scale Square Kilometre Array Data on the Summit Supercomputer

This work presents a workflow for simulating and processing the full-scale low-frequency telescope data of the Square Kilometre Array (SKA) Phase 1. The SKA project will enter the construction phase soon, and once completed, it will be the world’s largest radio telescope and one of the world’s largest data generators. The authors used Summit to mimic an endto-end SKA workflow, simulating a dataset of a typical 6 hour observation and then processing that dataset with an imaging pipeline. This workflow was deployed and run on 4,560 compute nodes, and used 27,360 GPUs to generate 2.6 PB of data. This was the first time that radio astronomical data were processed at this scale. Results show that the workflow has the capability to process one of the key SKA science cases, an Epoch of Reionization observation. This analysis also helps reveal critical design factors for the next-generation radio telescopes and the required dedicated processing facilities.

Wang, Ruonan↗

DOE OSTI2024

Visual Analytics of Multivariate Networks With Representation Learning and Composite Variable Construction

Multivariate networks are commonly found in real-world data-driven applications. Uncovering and understanding the relations of interest in multivariate networks is not a trivial task. This article presents a visual analytics workflow for studying multivariate networks to extract associations between different structural and semantic characteristics of the networks (e.g., what are the combinations of attributes largely relating to the density of a social network?). The workflow consists of a neural-network-based learning phase to classify the data based on the chosen input and output attributes, a dimensionality reduction and optimization phase to produce a simplified set of results for examination, and finally an interpreting phase conducted by the user through an interactive visualization interface. A key part of our design is a composite variable construction step that remodels nonlinear features obtained by neural networks into linear features that are intuitive to interpret. We demonstrate the capabilities of this workflow with multiple case studies on networks derived from social media usage and also evaluate the workflow with qualitative feedback from experts.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

3D modeling of deep borehole electromagnetic measurements with energized casing source for fracture mapping at the Utah Frontier Observatory for Research in Geothermal Energy

Here, we present a 3D numerical modelling analysis evaluating the deployment of a borehole electromagnetic measurement tool to detect and image a stimulated zone at the Utah Frontier Observatory for Research in Geothermal Energy geothermal site. As the depth to the geothermal reservoir is several kilometres and the size of the stimulated zone is limited to several 100 m, surface-based controlled-source electromagnetic measurements lack the sensitivity for detecting changes in electrical resistivity caused by the stimulation. To overcome the limitation, the study evaluates the feasibility of using a three-component borehole magnetic receiver system at the Frontier Observatory for Research in Geothermal Energy site. To provide sufficient currents inside and around the enhanced geothermal reservoir, we use an injection well as an energized casing source. To efficiently simulate energizing the injection well in a realistic 3D resistivity model, we introduce a novel modelling workflow that leverages the strengths of both 3D cylindrical-mesh-based electromagnetic modelling code and 3D tetrahedral-mesh-based electromagnetic modelling code. The former is particularly well-suited for modelling hollow cylindrical objects like casings, whereas the latter excels at representing more complex 3D geological structures. In this workflow, our initial step involves computing current densities along a vertical steel-cased well using a 3D cylindrical electromagnetic modelling code. Subsequently, we distribute a series of equivalent current sources along the well's trajectory within a complex 3D resistivity model. We then discretize this model using a tetrahedral mesh and simulate the borehole electromagnetic responses excited by the casing source using a 3D finite-element electromagnetic code. This multi-step approach enables us to simulate 3D casing source electromagnetic responses within a complex 3D resistivity model, without the need for explicit discretization of the well using an excessive number of fine cells. We discuss the applicability and limitations of this proposed workflow within an electromagnetic modelling scenario where an energized well is deviated, such as at the Frontier Observatory for Research in Geothermal Energy site. Using the workflow, we demonstrate that the combined use of the energized casing source and the borehole electromagnetic receiver system offer measurable magnetic field amplitudes and sensitivity to the deep localized stimulated zone. The measurements can also distinguish between parallel-fracture anisotropic reservoirs and isotropic cases, providing valuable insights into the fracture system of the stimulated zone. Besides the magnetic field measurements, vertical electric field measurements in the open well sections are also highly sensitive to the stimulated zone and can be used as additional data for detecting and imaging the target. We can also acquire additional multiple-source data by grounding the surface electrode at various locations and repeating borehole electromagnetic measurements. This approach can increase the number of monitoring data by several factors, providing a more comprehensive dataset for analysing the deep-localized stimulated zone. The numerical analysis indicates that it is feasible to use the combination of the energized casing and downhole electromagnetic measurements in monitoring localized stimulated zone at large depths.

58 GEOSCIENCES↗

DOE OSTI2023

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗