Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning and data science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Correlator convolutional neural networks as an interpretable architecture for image-like quantum matter data

Image-like data from quantum systems promises to offer greater insight into the physics of correlated quantum matter. However, the traditional framework of condensed matter physics lacks principled approaches for analyzing such data. Machine learning models are a powerful theoretical tool for analyzing image-like data including many-body snapshots from quantum simulators. Recently, they have successfully distinguished between simulated snapshots that are indistinguishable from one and two point correlation functions. Thus far, the complexity of these models has inhibited new physical insights from such approaches. Here, we develop a set of nonlinearities for use in a neural network architecture that discovers features in the data which are directly interpretable in terms of physical observables. Applied to simulated snapshots produced by two candidate theories approximating the doped Fermi-Hubbard model, we uncover that the key distinguishing features are fourth-order spin-charge correlators. Our approach lends itself well to the construction of simple, versatile, end-to-end interpretable architectures, thus paving the way for new physical insights from machine learning studies of experimental and numerical data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Micropulse Lidar Cloud Mask Machine-Learning Value-Added Product Report

Cloud detection algorithms of various techniques have been developed and applied to atmospheric ground-based lidar data to identify cloud boundaries and produce clouds masks. While these algorithms are able to identify a wide variety of cloud types and conditions, it is often observed that the algorithms can still fail to accurately detect clouds that are readily discernible when inspecting the lidar imagery. Based on this observation, an alternative approach for cloud detection is to take advantage of machine-learning capabilities and the trained human eye as an interpreter of lidar images, and in turn, to train a neural network to recognize the desired features in the lidar data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Hybrid (PDE+ML) models in the context of land ice modeling

Focal Area(s): Primary focal areas are: predictive modeling through the use of AI-derived model components; advanced methods including network design/optimization/deep learning. Science Challenge: The atmospheric, ocean and ice dynamics components of the Energy Exascale Earth System Model (E3SM) are governed by Partial Differential Equations (PDEs) and significant efforts have been made during the last decades to develop such computational models. Here we propose to fundamentally improve these PDE-based codes by enhancing them with Machine Learning (ML) sub-models for complex, poorly understood physical processes in the context of ice sheet modeling. We propose to train these models with a novel approach that allows the assimilation of the different sources of data available (direct/indirect observations and possibly simulation data), improving on existing simplified models. We also highlight computational challenges originating from the coexistence of PDE-based and ML-based models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Quantifying the Dynamics of Protein Self-Organization Using Deep Learning Analysis of Atomic Force Microscopy Data

The dynamics of protein self-assembly on the inorganic surface and the resultant geometric patterns are visualized using high-speed atomic force microscopy. The time dynamics of the classical macroscopic descriptors such as 2D fast Fourier transforms, correlation, and pair distribution functions are explored using the unsupervised linear unmixing, demonstrating the presence of static ordered and dynamic disordered phases and establishing their time dynamics. Here, the deep learning (DL)-based workflow is developed to analyze detailed particle dynamics and explore the evolution of local geometries. Finally, we use a combination of DL feature extraction and mixture modeling to define particle neighborhoods free of physics constraints, allowing for a separation of possible classes of particle behavior and identification of the associated transitions. Overall, this work establishes the workflow for the analysis of the self-organization processes in complex systems from observational data and provides insight into the fundamental mechanisms.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Novel Proposals for FAIR, Automated, Recommendable, and Robust Workflows

Lightning talks of the Workflows in Support of Large-Scale Science (WORKS) workshop are a venue where the workflow community (researchers, developers, and users) can discuss work in progress, emerging technologies and frameworks, and training and education materials. This paper summarizes the WORKS 2022 lightning talks, which cover five broad topics: data integrity of scientific workflows; a machine learning-based recommendation system; a Python toolkit for running dynamic ensembles of simulations; a cross-platform, high-performance computing utility for processing shell commands; and a meta(data) framework for reproducing hybrid workflows.

Abhinit, Ishan↗
DOE OSTI2023

Multi-fidelity physics-informed machine learning for probabilistic damage diagnosis

Machine learning (ML) models are gaining popularity in structural health monitoring (SHM) because of their ability to learn the complex relationship between damage and sensor data. However, the lack of sufficient experimental data for structures with different degrees of damage is a key problem in training ML models for SHM. This problem can be alleviated by using physics-based models to generate the required training data to build physics-informed ML (PIML) models for SHM. However, it takes significant computational effort to perform enough high-fidelity simulations of the diagnostic test. It is thus desirable to know whether the available computational resource budget should be expended on numerous low-fidelity physics simulations, or a small number of high-fidelity simulations, or their combination. In this paper, we investigate this aspect of generating adequate training data for PIML, by constructing multi-fidelity PIML models. We evaluate the performance of several PIML models, trained with different amounts of low-fidelity and high-fidelity data, in locating hidden cracks in concrete structures using a nonlinear dynamics-based diagnosis technique. Here, we find that high-fidelity physics simulations that do not cover the (test and damage) parameter space do not improve the performance of diagnostic PIML models built using data from many low-fidelity physics simulations.

42 ENGINEERING↗
DOE OSTI2023

Physics-informed machine-learning model of temperature evolution under solid phase processes

We model temperature dynamics during Shear Assisted Proccess Extrusion (ShAPE), a solid phase process that plasticizes feedstock with a rotating tool and subsequently extrudes it into a consolidated tube, rod, or wire. Control of temperature is critical during ShAPE processing to avoid liquefaction, ensure smooth extrusion, and develop desired material properties in the extruded products. Accurate modeling of the complicated thermo-mechanical feedbacks between process inputs, material temperature, and heat generation presents a significant barrier to predictive modeling and process design. In particular, connecting micro-structural scale mechanisms of heat generation to macro-scale predictions of temperature can become computationally intractable. In this work we use a neural network (NN) model of heat generation to bridge this gap, by combining it with a simplified model of the temperature dynamics due to conduction and convection to capture the macro scale evolution of temperature. We inform the construction of the NN heat generation model using crystal plasticity simulations at the micro-structural scale to model the effects of process inputs on generation of heat. We achieved close fits of the temperature dynamics model to a diverse experimental data-set. Further, the relationships learned by the NN model between process inputs and heat generation showed qualitative agreement with those predicted by crystal plasticity simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

In the alloy design and development process, a wealth of atomically resolved structural high-resolution transmission electron microscopy (HRTEM) images are produced. Identifying the different nano-precipitate phases and tracking their evolution under various compositions and during manufacturing or post-processing requires hundreds of HRTEM images and thousands of precipitates. The nanoscopic phase information labeling and analysis purely relies on humans are prohibitively costly and time-consuming, sometimes not reliable because of the lack of authoritative knowledge. Here, in this work, we develop a novel unsupervised machine learning approach coupled with adaptive computer vision techniques with features in the Fourier space to automatically determine the number of phases and segment/quantify the phases with nanoscale resolution, allowing for quantitative correlation between nanostructure formation, processing and functional properties. To automate the phase extraction/quantification and ascertain its applicability, we have applied the developed framework to the HRTEM images from several alloy systems, processing conditions, image magnifications, and phase types and morphologies (precipitates, nano-twins, stacking faults, crystalline matrix, and amorphous structures) for verification. This study paves the road for compression, visualization, and translation of raw image structural data into physically relevant information in real-time with minimal human supervision. It shows the promise of enabling high-throughput materials characterization for the acceleration of alloy manufacturing and design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Using Machine Learning to Develop a Predictive Understanding of the Impacts of Extreme Water Cycle Perturbations on River Water Quality

This whitepaper addresses to two focal areas – (3) Insight gleaned from complex data using Artificial Intelligence (AI), and other advanced techniques (primary), and (2) Predictive modeling through the use of AI techniques and AI-derived model components (secondary). This topic is directly relevant to four DOE Earth and Environmental Systems Science Division Grand Challenges: integrated water cycle, biogeochemistry, drivers and responses in the Earth system, and data-model integration.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks and Operators in Scientific Computing: Fluid and Solid Mechanics

Abstract Advancements in computing power have recently made it possible to utilize machine learning and deep learning to push scientific computing forward in a range of disciplines, such as fluid mechanics, solid mechanics, materials science, etc. The incorporation of neural networks is particularly crucial in this hybridization process. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data are sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints during training. Generally speaking, there are three distinct neural network frameworks to enforce the underlying physics: (i) physics-guided neural networks (PgNNs), (ii) physics-informed neural networks (PiNNs), and (iii) physics-encoded neural networks (PeNNs). These methods provide distinct advantages for accelerating the numerical modeling of complex multiscale multiphysics phenomena. In addition, the recent developments in neural operators (NOs) add another dimension to these new simulation paradigms, especially when the real-time prediction of complex multiphysics systems is required. All these models also come with their own unique drawbacks and limitations that call for further fundamental research. This study aims to present a review of the four neural network frameworks (i.e., PgNNs, PiNNs, PeNNs, and NOs) used in scientific computing research. The state-of-the-art architectures and their applications are reviewed, limitations are discussed, and future research opportunities are presented in terms of improving algorithms, considering causalities, expanding applications, and coupling scientific and deep learning solvers.

Computer Science↗
DOE OSTI2024

Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning

Despite the dramatic progress in Large Language Model (LLM) development, LLMs often provide seemingly plausible but not factual information, often referred as hallucinations. Retrieval-augmented LLMs provide a non-parametric approach to solve these issues by retrieving relevant information from external data sources and augment the training process. These models helps to trace evidence from an externally provided knowledge base allowing the model predictions to be better interpreted and verified. In this work, we critically evaluate these models in their ability to perform in scientific document reasoning tasks. To this end, we tuned multiple such model variants with science-focused instructions and evaluated them on a scientific document reasoning benchmark for the usefulness of the retrieved document passages. Our findings suggest that models justify predictions in science tasks with fabricated evidence and leveraging scientific corpus as pretraining data does not alleviate the risk of evidence fabrication.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2023

Coolant Pump Predictive Data Analytics from Signatures Generated by the Recursive Short Time Fast Fourier Transform

Although a nuclear reactor is a hostile environment for sensors and signal transmissions, the reactor core is amenable to acoustic communication. An acoustic measurement infrastructure installed at the Advanced Test Reactor (ATR) nozzle trench area records acoustic signals that can capture reactor operating states. The distinct states produce unique signatures that can be identified and tracked using data processing and data analytics. The infrastructure relies on acoustic transmission through ATR in-pile structural components, piping, and coolant that transmit acoustically modified signals generated by the coolant pumps. This paper will discuss results from using the Recursive Short Time Fast Fourier Transform (RSTFFT) technique used to process acoustic signals and provide signatures that are identified and monitored by analytics. The RSTFFT is applied to ATR data to understand the vibration levels and signatures for different operating regimes as displayed by the spectrogram. The combination of coolant pumps for normal and high-power operation generate unique signatures. These acoustic signatures are used to develop machine learning approaches to automatically classify operating regimes. Two machine-learning models, Support Vector Machines and Linear Discriminant Analysis, were developed to classify two event classes. Class 1 is a normal steady-state operation, and Class 2 is any event that is due to start up, shut down, or other actions. Both types of machine learning models had over a 96% prediction accuracy for the two classes. These results lay the foundation for predictive analytic frameworks that can be leveraged by ATR to optimize operations and maintenance.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Advancing Fusion with Machine Learning Research Needs Workshop Report

Abstract Machine learning and artificial intelligence (ML/AI) methods have been used successfully in recent years to solve problems in many areas, including image recognition, unsupervised and supervised classification, game-playing, system identification and prediction, and autonomous vehicle control. Data-driven machine learning methods have also been applied to fusion energy research for over 2 decades, including significant advances in the areas of disruption prediction, surrogate model generation, and experimental planning. The advent of powerful and dedicated computers specialized for large-scale parallel computation, as well as advances in statistical inference algorithms, have greatly enhanced the capabilities of these computational approaches to extract scientific knowledge and bridge gaps between theoretical models and practical implementations. Large-scale commercial success of various ML/AI applications in recent years, including robotics, industrial processes, online image recognition, financial system prediction, and autonomous vehicles, have further demonstrated the potential for data-driven methods to produce dramatic transformations in many fields. These advances, along with the urgency of need to bridge key gaps in knowledge for design and operation of reactors such as ITER, have driven planned expansion of efforts in ML/AI within the US government and around the world. The Department of Energy (DOE) Office of Science programs in Fusion Energy Sciences (FES) and Advanced Scientific Computing Research (ASCR) have organized several activities to identify best strategies and approaches for applying ML/AI methods to fusion energy research. This paper describes the results of a joint FES/ASCR DOE-sponsored Research Needs Workshop on Advancing Fusion with Machine Learning, held April 30–May 2, 2019, in Gaithersburg, MD (full report available at https://science.osti.gov/-/media/fes/pdf/workshop-reports/FES_ASCR_Machine_Learning_Report.pdf ). The workshop drew on broad representation from both FES and ASCR scientific communities, and identified seven Priority Research Opportunities (PRO’s) with high potential for advancing fusion energy. In addition to the PRO topics themselves, the workshop identified research guidelines to maximize the effectiveness of ML/AI methods in fusion energy science, which include focusing on uncertainty quantification, methods for quantifying regions of validity of models and algorithms, and applying highly integrated teams of ML/AI mathematicians, computer scientists, and fusion energy scientists with domain expertise in the relevant areas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

A neural network for beam background decomposition in Belle II at SuperKEKB

Here, we describe a neural network for predicting the background hit rate in the Belle II detector produced by the SuperKEKB electron-positron collider. The neural network, BGNet, learns to predict the individual contributions of different physical background sources, such as beam-gas scattering or continuous top-up injections into the collider, to Belle II sub-detector rates. The samples for learning are archived 1 Hz time series of diagnostic variables from the SuperKEKB collider subsystems and measured hit rates of Belle II used as regression targets. We test the learned model by predicting detector hit rates on archived data from different run periods not used during training. We show that a feature attribution method can help interpret the source of changes in the background level over time.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

A review on machine learning-guided design of energy materials

Abstract The development and design of energy materials are essential for improving the efficiency, sustainability, and durability of energy systems to address climate change issues. However, optimizing and developing energy materials can be challenging due to large and complex search spaces. With the advancements in computational power and algorithms over the past decade, machine learning (ML) techniques are being widely applied in various industrial and research areas for different purposes. The energy material community has increasingly leveraged ML to accelerate property predictions and design processes. This article aims to provide a comprehensive review of research in different energy material fields that employ ML techniques. It begins with foundational concepts and a broad overview of ML applications in energy material research, followed by examples of successful ML applications in energy material design. We also discuss the current challenges of ML in energy material design and our perspectives. Our viewpoint is that ML will be an integral component of energy materials research, but data scarcity, lack of tailored ML algorithms, and challenges in experimentally realizing ML-predicted candidates are major barriers that still need to be overcome.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems

In recent years, the field of machine learning (ML), specifically neural networks, has grown significantly and has spurred research in its applicability to digital instrumentation and control systems (DI&C). While ML models have shown promise in operational contexts, the trustworthiness of using such algorithms has not been adequately assessed. Failures of ML integrated systems are not well understood, and the lack of comprehensive risk modeling can degrade the trustworthiness in these systems. In recent reports by the National Institute for Standards and Technology (NIST) [1] and the Nuclear Regulatory Commission (NRC) [2], they indicate that trustworthiness in ML is a critical barrier and will play a vital role in the safe, accountable, and secure operation of intelligent systems. Thus, in this work, we demonstrate a dynamic model-agnostic method to quantify the relative reliability of AI/ML predictions by incorporating out-of-distribution (OOD) detection on the training dataset. It is well documented that most ML algorithms excel at interpolation (or near-interpolation) tasks but experience significant performance degradation at extrapolation. The method, referenced as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets which can used to the relative reliability of AI/ML predictions. LADDR is then demonstrated on a feedforward neural network based digital twin used for the prediction of safety significant factors during a loss-of-flow transient. LADDR is used to demonstrate how training data can be used as evidence to support the relative reliability of ML/AI predictions enhancing the overall trustworthiness of the system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Multi-fidelity information fusion with concatenated neural networks

Recently, computational modeling has shifted towards the use of statistical inference, deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and leads to poorly generalizable data-driven models. This gap can be furnished by leveraging information from physics-based models. Exploiting prior knowledge about the problem at hand, this study puts forth a physics-guided machine learning (PGML) approach to build more tailored, effective, and efficient surrogate models. For our analysis, without losing its generalizability and modularity, we focus on the development of predictive models for laminar and turbulent boundary layer flows. In particular, we combine the self-similarity solution and power-law velocity profile (low-fidelity models) with the noisy data obtained either from experiments or computational fluid dynamics simulations (high-fidelity models) through a concatenated neural network. We illustrate how the knowledge from these simplified models results in reducing uncertainties associated with deep learning models applied to boundary layer flow prediction problems. The proposed multi-fidelity information fusion framework produces physically consistent models that attempt to achieve better generalization than data-driven models obtained purely based on data. While we demonstrate our framework for a problem relevant to fluid mechanics, its workflow and principles can be adopted for many scientific problems where empirical, analytical, or simplified models are prevalent. In line with grand demands in novel PGML principles, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid physics and machine learning modeling approaches for next-generation digital twin technologies.

42 ENGINEERING↗