Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learned priors”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Preventing Reverse Engineering of Critical Industrial Data with DIOD

Business analytics augmented by artificial intelligence and machine learning (AI/ML) have revolutionized the role of data in the modern world. In recent years, businesses have incorporated data into their decision-making process for better prediction, risk-assessment, content creation, etc. While such businesses often seek to leverage the full use of their data through third-party AI/ML services, they are often hampered by the risks of data leaks, reverse-engineering, stolen technology, etc. that often have disastrous consequences for businesses and their stakeholders alike. Thus, there arises a need for data masking prior to its transmission that obfuscates proprietary information while preserving the information relevant for AI/ML applications. In order to meet the needs of industrial data which are significantly different from those of data warehouses, previous work proposed an efficient time and space-scalable data masking paradigm known as the deceptive infusion of data (DIOD) methodology. The present work expands upon this work by leveraging existing reverse-engineering capabilities to facilitate the decomposition of industrial data into its proprietary and AI/ML-relevant parts, referred to as fundamental and inference metadata respectively. Both sets of metadata are further obfuscated in accordance with the DIOD methodology to create the DIOD rendition of the industrial data, which is rendered immune to reverse-engineering by discarding proprietary information and only preserving AI/ML-relevant information. Additionally, constraints of the original DIOD manuscript are relaxed using mutual information by configuring the methodology to the target AI/ML application to unlock the full potential of the DIOD methodology. As an example, data from a nuclear reactor is transformed into that from a nonlinear spring-mass system with different levels of data masking as required by the generic system and the target application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Multi-fidelity Bayesian neural networks: Algorithms and applications

Here we propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity, and we apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs). These multi-fidelity BNNs consist of three neural networks: The first is a fully connected neural network, which is trained following the maximum a posteriori probability (MAP) method to fit the low-fidelity data; the second is a Bayesian neural network employed to capture the cross-correlation with uncertainty quantification between the low- and high-fidelity data; and the last one is the physics-informed neural network, which encodes the physical laws described by PDEs. For the training of the last two neural networks, we first employ the mean-field variational inference (VI) to maximize the evidence lower bound (ELBO) to obtain informative prior distributions for the hyperparameters in the BNNs, and subsequently we use the Hamiltonian Monte Carlo (HMC) method to estimate accurately the posterior distributions for the corresponding hyperparameters. We demonstrate the accuracy of the present method using synthetic data as well as real measurements. Specifically, we first approximate a one- and four-dimensional function, and then infer the reaction rates in one- and two-dimensional diffusion-reaction systems. Moreover, we infer the sea surface temperature (SST) in the Massachusetts and Cape Cod Bays using satellite images and in-situ measurements. Taken together, our results demonstrate that the present method can capture both linear and nonlinear correlation between the low- and high-fidelity data adaptively, identify unknown parameters in PDEs, and quantify uncertainties in predictions, given a few scattered noisy high-fidelity data. Finally, we demonstrate that we can effectively and efficiently reduce the uncertainties and hence enhance the prediction accuracy with an active learning approach, using as examples a specific one-dimensional function approximation and an inverse PDE problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Recursive Use of the Short-Time Fast Fourier Transform for Signature Analysis in Continuous Processes

Although a nuclear reactor is a hostile environment for sensing and electrical communications, the reactor core is amenable to acoustic communication. An acoustic measurement infrastructure (AMI) has been installed in the Advanced Test Reactor (ATR) to record acoustic signals that can capture its different operating regimes. AMI uses coolant pumps as continuous signal sources, coolant and structural components as transmission lines, and accelerometers to capture system motion. A recursive signal processing technique based on the short-time fast Fourier transform (STFFT) for continuous processes provides unique signatures for diagnostic and prognostic analyses from the system motion data. Here this article presents a recursive STFFT methodology that processes acoustic signals from continuous industrial processes. The article first discusses the initial STFFT use with simulated data to elucidate the basic principles necessary to understand and interpret the STFFT results from actual pump vibration data. Each repetitive use of the STFFT on pump vibration data using the results from the prior STFFT processing will generate additional complimentary time-frequency-based signatures. These signatures are generated by the coolant pumps operating under different process conditions. After each use of the STFFT, the resulting signatures provide exemplary examples of the diversity and intuitive nature of recursively using the STFFT. This article focuses on recursively using the STFFT to provide numerous complimentary and diverse signatures that will ultimately be inputs for machine learning algorithms that provide predictive data analytics. The intuitive nature of the information and signatures from recursive STFFT processing will also bring intuitive interpretation capabilities to machine learning and predictive data analytic techniques.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

An exponential build-up in seismic energy suggests a months-long nucleation of slow slip in Cascadia

Slow slip events result from the spontaneous weakening of the subduction megathrust and bear strong resemblance to earthquakes, only slower. This resemblance allows us to study fundamental aspects of nucleation that remain elusive for classic, fast earthquakes. We rely on machine learning algorithms to infer slow slip timing from statistics of seismic waveforms. We find that patterns in seismic power follow the 14-month slow slip cycle in Cascadia, arguing in favor of the predictability of slow slip rupture. Here, we show that seismic power exponentially increases as the slowly slipping portion of the subduction zone approaches failure, a behavior that shares a striking similarity with the increase in acoustic power observed prior to laboratory slow slip events. Our results suggest that the nucleation phase of Cascadia slow slip events may last from several weeks up to several months.

58 GEOSCIENCES↗
DOE OSTI2024

Flux REaction TArget Prioritization (Flux RETAP) v1

Metabolic engineering is evolving rapidly as a result of new advances in synthetic biology and automation, as well as the irruption of machine learning (ML). ML has been shown to provide the predictive power synthetic biology lacked and needed, and to be able to effectively guide the metabolic engineering process. However, current technical limitations prevent the independent application of ML approaches to metabolic engineering without the use of previous biological knowledge in the form of a prioritized list of desirable engineering targets. Here, we present FluxRETAP, a simple and computationally inexpensive method that leverages the prior mechanistic knowledge embedded in genome-scale metabolic models (GSMs) for suggesting targets for genetic overexpression, downregulation or deletion, with the final goal of increasing metabolite production. FluxRETAP captured 100% of reaction targets experimentally verified to improve Escherichia coli isoprenol production in the literature accessible to us, 50% of targets that experimentally improved taxadiene production in E. coli and ~60% of genetic targets from a verified minimal constrained cut-set in Pseudomonas putida while providing additional high priority targets that could be tested. Overall, FluxRETAP is an efficient algorithm for identifying a prioritized list of testable genetic and reaction targets which can also be utilized in ML pipelines.

Czajka, Jeffrey [Battelle Memorial Institute, Paci↗
DOE OSTI2024

Quantifying Capital Cost Reduction Pathways for Advanced Nuclear Reactors

The framework developed in this study is provided both as an excel sheet (https://inl.gov/content/uploads/2023/11/Nuclear-Reactor-Cost-Reduction-Pathway-Spreadsheet-Tool.xlsx) and a Python (Jupyter) Notebook (link: https://github.com/accert-dev/ACCERT/tree/main/Cost%20Reduction). Capital cost considerations are one of the primary inhibitors to the large-scale deployment of nuclear power plants. While it is widely accepted that first units will likely be expensive and relatively uncompetitive, it is reasonable to expect that subsequent units, built in relative quick succession, will be cheaper as they benefit from the so-called “learning effects”. However, the large degree of uncertainty associated with this parameter renders it challenging for first movers to invest in the first few expensive units. To resolve this impasse, the U.S. Department of Energy’s Advanced Nuclear Liftoff study advocated for the formation of large, committed order books of plants of the same technology to spread the costs across several units and kickstart the nuclear supply chain. The study also advocated best practices for avoiding overruns and keeping reactors on budget. This report builds on these key recommendations by attempting to quantify specific pathways toward cost reduction for nuclear energy. A capital cost estimation framework was built to untangle the effect of learning into a subset of key cost drivers, referred to as “levers”. Collectively, the choice of these levers is intended to reflect the decision-making of high-level stakeholders like plant owners and the government. In addition to the size of the firm orderbook, these levers included (a) cost drivers that are most often attributed to cost overruns such as architect/engineering (A/E) proficiency, construction proficiency, procurement service proficiency, design completion prior to the start of construction, and design maturity, and (b) cost reduction strategies such as modular construction, cross-site standardization, safety classification of the reactor building, and of the balance of plant. Two advanced reactor designs were leveraged as use cases and bottom-up cost estimates made with assumptions consistent with a well-executed first-of-a-kind project (WE-FOAK, i.e., almost no overruns) were used as baselines for the models. Cost correlations were surveyed from the literature to determine the impact of important variables on projected timelines and costs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Detecting and Characterizing Fracture Zones Using a Convolutional Neural Network

This project directly supports the Geothermal Technologies Office (GTO) objectives outlined in the Multi-Year Program Plan (MYPP) by advancing two key research areas: “Exploration and Characterization” and “Data, Modeling, and Analysis.” This project has successfully demonstrated a pre-drilling ability to image and characterize the distribution and connectivity of subsurface faults and fractures, key parameters for identifying permeable pathways that enable geothermal fluids to circulate and produce energy. Specifically, we developed and implemented innovative machine learning methodologies to enhance geothermal exploration. Large-scale faults were detected using a Convolutional Neural Network (CNN), while small-scale fractures were characterized using a novel Double-Beam Neural Network (DBNN). These tools have proven both technically effective and cost-efficient by reducing reliance on expensive exploratory drilling. Through collaboration with our geothermal industry partner, this research has significantly advanced techniques for identifying hidden geothermal systems and extending the productive lifespan of existing geothermal fields. We applied our methods to two geothermal fields—Soda Lake (Nevada) and Lightning Dock (New Mexico)—to identify shallow steam-charged fracture zones and characterize deep faults at depths of 1.5-2 km. The steam zone identified at the Soda Lake geothermal field showed excellent agreement with prior drilling data, validating the effectiveness of our approaches. In addition, the analysis revealed three new prospective drilling targets for further development and verification. The outcomes of this project improve our scientific understanding of geothermal reservoir behavior, enhance exploration efficiency, extend the economic life of existing geothermal plants. Ultimately, these advancements contribute to GTO’s goal of achieving more sustainable, affordable, and data-driven geothermal energy development across the United States.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Bayesian differential programming for robust systems identification under uncertainty

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator–prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj .

Science & Technology - Other Topics↗
DOE OSTI2021

A generalized machine learning workflow to visualize mechanical discontinuity

Accurate detection and mapping of mechanical discontinuity in materials has widespread industrial and research applications. Herein, we developed a generalized machine-learning framework for visualizing single mechanical discontinuity embedded in material of any composition, velocity, density, porosity, and size with limited data. The proposed visualization of discontinuity requires accurate estimations of the length, location, and orientation of the embedded discontinuity by processing multipoint wave-transmission measurements. k-Wave simulator is used to create a large dataset of elastic waveforms recorded during multi-point wave-transmission measurements through materials containing single mechanical discontinuity. k-Wave simulator considers the wave attenuation, dispersion, and mode conversion in wave motion. Discrete wavelet transform (DWT) and statistical feature extraction are essential for data preprocessing prior to the data-driven model development. DWT also minimizes the effect of noise. Using hyper-parameter tuning and cross validation, gradient boosting regression can visualize the mechanical discontinuity with an accuracy of 0.85, in terms of coefficient of determination. A double-layered neural network-based regression has better performance with an accuracy of 0.95. Use of convolutional neural network converts the predictive task from a waveform processing to an image processing problem. Convolutional neural network achieved a generalization performance of 0.91. The proposed generalized workflow requires robust simulation of wave propagation, signal processing, feature engineering, and model evaluation. Sensors closest to the source and those located opposite the source are the most significant for the desired visualization. Notably, the sensors closest to the source capture the non-linear associations, whereas the sensor on the border opposite to the source capture the linear associations between the measured waveforms and the properties of the mechanical discontinuity.

42 ENGINEERING↗
DOE OSTI2021

Identifying Key Drivers of Wildfires in the Contiguous US Using Machine Learning and Game Theory Interpretation

Abstract Understanding the complex interrelationships between wildfire and its environmental and anthropogenic controls is crucial for wildfire modeling and management. Although machine learning (ML) models have yielded significant improvements in wildfire predictions, their limited interpretability has been an obstacle for their use in advancing understanding of wildfires. This study builds an ML model incorporating predictors of local meteorology, land‐surface characteristics, and socioeconomic variables to predict monthly burned area at grid cells of 0.25° × 0.25° resolution over the contiguous United States. Besides these predictors, we construct and include predictors representing the large‐scale circulation patterns conducive to wildfires, which largely improves the temporal correlations in several regions by 14%–44%. The Shapley additive explanation is introduced to quantify the contributions of the predictors to burned area. Results show a key role of longitude and latitude in delineating fire regimes with different temporal patterns of burned area. The model captures the physical relationship between burned area and vapor pressure deficit, relative humidity (RH), and energy release component (ERC), in agreement with the prior findings. Aggregating the contribution of predictor variables of all the grids by region, analyses show that ERC is the major contributor accounting for 14%–27% to large burned areas in the western US. In contrast, there is no leading factor contributing to large burned areas in the eastern US, although large‐scale circulation patterns featuring less active upper‐level ridge‐trough and low RH two months earlier in winter contribute relatively more to large burned areas in spring in the southeastern US.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Data-driven predictions of complex organic mixture permeation in polymer membranes

Membrane-based organic solvent separations are rapidly emerging as a promising class of technologies for enhancing the energy efficiency of existing separation and purification systems. Polymeric membranes have shown promise in the fractionation or splitting of complex mixtures of organic molecules such as crude oil. Determining the separation performance of a polymer membrane when challenged with a complex mixture has thus far occurred in an ad hoc manner, and methods to predict the performance based on mixture composition and polymer chemistry are unavailable. Here, we combine physics-informed machine learning algorithms (ML) and mass transport simulations to create an integrated predictive model for the separation of complex mixtures containing up to 400 components via any arbitrary linear polymer membrane. We experimentally demonstrate the effectiveness of the model by predicting the separation of two crude oils within 6-7% of the measurements. Integration of ML predictors of diffusion and sorption properties of molecules with transport simulators enables for the rapid screening of polymer membranes prior to physical experimentation for the separation of complex liquid mixtures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Toward the Detection of Polyglot Files

Standardized file types play a key role in the development and use of computer software. However, it is possible to confound standardized file type processing by creating a file that is valid in multiple file types. The resulting polyglot (many languages) file can confuse file type identification, allowing elements of the file to evade analysis. This is especially problematic for malware detection systems that rely on file type identification for feature extraction. Although work has been done to identify file types using more comprehensive methods than file signatures, accurate identification of polyglot files remains an open problem. Since malware detection systems routinely perform file type-specific feature extraction, polyglot files need to be filtered out prior to ingestion by these systems. Otherwise, malicious content could pass through undetected. To address the problem of polyglot detection we assembled a data set using the mitra tool. We then evaluated the performance of the most commonly used file identification tools, including file, polydet, binwalk, and TrID. Our analysis demonstrates that existing file type detection tools fail to provide reliable polyglot detection. We then evaluated the ability of a range of machine and deep learning models to detect polyglot files. The most performant models were MalConv2 and Catboost, which demonstrated the highest recall on our data set with 95.16% and 95.45%, respectively. These models outperformed existing methods and could be incorporated into a malware detector’s file processing pipeline to filter out potentially malicious polyglots before file type-dependent feature extraction takes place.

Koch, Luke↗
DOE OSTI2022

Toward the Detection of Polyglot Files

Standardized file types play a key role in the development and use of computer software. However, it is possible to confound standardized file type processing by creating a file that is valid in multiple file types. The resulting polyglot (many languages) file can confuse file type identification, allowing elements of the file to evade analysis. This is especially problematic for malware detection systems that rely on file type identification for feature extraction. Although work has been done to identify file types using more comprehensive methods than file signatures, accurate identification of polyglot files remains an open problem. Since malware detection systems routinely perform file type-specific feature extraction, polyglot files need to be filtered out prior to ingestion by these systems. Otherwise, malicious content could pass through undetected. To address the problem of polyglot detection we assembled a data set using the mitra tool. We then evaluated the performance of the most commonly used file identification tools, including file, polydet, binwalk, and TrID. Our analysis demonstrates that existing file type detection tools fail to provide reliable polyglot detection. We then evaluated the ability of a range of machine and deep learning models to detect polyglot files. The most performant models were MalConv2 and Catboost, which demonstrated the highest recall on our data set with 95.16% and 95.45%, respectively. These models outperformed existing methods and could be incorporated into a malware detector’s file processing pipeline to filter out potentially malicious polyglots before file type-dependent feature extraction takes place.

Koch, Lucas↗
DOE OSTI2025

Building Datasets and Training Methods for ML Based Magnet Quench Detection

Detecting quenches in superconducting (SC) magnets during training is a challenging process that involves capturing physical events that occur at different frequencies and appear as various signal features. These events may be correlated across instrumentation type, thermal cycle, and ramp. These events together build a more complete picture of continuous processes occurring in the magnet, and may allow us to flag potential precursors for quench detection. We present our work on building an automatic machine learning (ML) based quench detection system. We build upon our existing work on unsupervised auto-encoders for acoustic sensors and quench antenna (QA) by first establishing a supervised ML training pipeline. We show the results of an event tagging, analysis, and simulation framework on our QA and acoustic data which are used concurrently to build a training dataset for a supervised implementation. We then show how this supervised training can be used as a prior in a semi-supervised framework and compare this to the unsupervised neural network auto-encoder performance.This allows us to have a more concrete understanding of the performance of our algorithms relative to physical events occurring in the magnet, and also provides a baseline software tool to generically evaluate our quench prediction autoencoders under completely unsupervised, supervised, and semi-supervised training conditions.

Khan, Maira [Fermilab]↗
DOE OSTI2026

Learned adaptive properties for mitigation of weight perturbations in embedded spiking networks

Recent years have seen an increased importance of neural network inference in edge-based scenarios, which impose size and power constraints requiring novel computing devices. These same edge scenarios may require operating over long periods of time, or exposure to extreme environments, resulting in a drift of neural network weights that cause degraded performance. In searching for ways to develop neural network approaches that perform robustly under these conditions, we propose a biologically-inspired mechanism for the dynamic adaptation of within-neuron parameters that is guided by a global context signal carrying information about perturbations and variability in incoming stimuli. Specifically, we demonstrate that adaptive voltage thresholds or neuronal time constants, when informed by a global context signal, can enable network-level mechanisms to recover from perturbed synaptic weights. Consistent with prior literature, the context-modulated approach is effective for recurrent, but not feedforward networks, by modulating network level dynamics. We demonstrate this approach successfully recovers performance in image classification tasks and spatiotemporal tracking tasks under idealized and Gaussian noise as well as for realistic perturbations from a memristive device when exposed to ionizing radiation. Finally, we discuss how this approach enables the design of robust and energy-efficient neuromorphic systems that perform well, even in resource-constrained scenarios with extreme environments such as edge processing.

context modulation↗
DOE OSTI2023

Probe microscopy is all you need *

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALL·E and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogenous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to application programming interfaces (APIs) facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to create novel set of development targets for the ML community by accelerating both real world ML applications and scientific progress.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Computer Vision Pipeline for Image Analysis for Freeze‐Fracture Electron Microscopy: Rosette Cellulose Synthase Complexes Case

In materials science, plant biology, agriculture, and environmental research, the automated analysis of high-magnification, complex microscopy images, such as those generated by freeze-fracture electron microscopy (FF-TEM), remains a critical challenge that limits the scalability of data interpretation. We present a deep learning computer vision pipeline for high-throughput detection and morphological characterization analysis of cellulose synthase complexes (CSCs, or rosettes) in FF-TEM images. The pipeline integrates preprocessing, detection, human-in-the-loop verification, and semantic segmentation to quantify features such as rosette diameter and inter-lobe spacing. The approach was trained and tested on a curated dataset of high-resolution FF-TEM micrographs of Physcomitrium patens, expanded via strategic tiling and augmentation to over 650 images. We compare YOLOv8 and YOLOv9 architectures and demonstrate that YOLOv9 achieves superior performance in both localization accuracy (mAP50-95 = 0.854) and inference speed. The resulting distributions revealed biological variability consistent with prior manual studies, validating the approach for high-throughput applications. Our results show that the pipeline achieves human-expert level accuracy while dramatically reducing analysis time, enabling scalable, reproducible structural characterization of intramembrane protein complexes. The pipeline is broadly applicable to other domains requiring precise interpretation of complex microscopy data and establishes a foundation for future artificial intelligence (AI)-assisted workflows in biological imaging.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Machine learning for neutron reflectometry data analysis of two-layer thin films

Neutron reflectometry (NR) is a powerful tool for probing thin films at length scales down to nanometers. We investigated the use of a neural network to predict a two-layer thin film structure to model a given measured reflectivity curve. Application of this neural network to predict a thin film structure revealed that it was accurate and could provide an excellent starting point for traditional fitting methods. Employing prediction-guided fitting has considerable potential for more rapidly producing a result compared to the labor-intensive but commonly-used approach of trial and error searches prior to refinement. A deeper look at the stability of the predictive power of the neural network against statistical fluctuations of measured reflectivity profiles showed that the predictions are stable. We conclude that the approach presented here can provide valuable assistance to users of NR and should be further extended for use in studies of more complex n-layer thin film systems. This result also opens up the possibility of developing adaptive measurement systems in the future.

36 MATERIALS SCIENCE↗