Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Boosting background suppression in the NEXT experiment through Richardson-Lucy deconvolution

Next-generation neutrinoless double beta decay experiments aim for half-life sensitivities of ~ 10 27 yr, requiring suppressing backgrounds to < 1 count/tonne/yr. For this, any extra background rejection handle, beyond excellent energy resolution and the use of extremely radiopure materials, is of utmost importance. The NEXT experiment exploits differences in the spatial ionization patterns of double beta decay and single-electron events to discriminate signal from background. While the former display two Bragg peak dense ionization regions at the opposite ends of the track, the latter typically have only one such feature. Thus, comparing the energies at the track extremes provides an additional rejection tool. The unique combination of the topology-based background discrimination and excellent energy resolution (1% FWHM at the Q-value of the decay) is the distinguishing feature of NEXT. Previous studies demonstrated a topological background rejection factor of ~ 5 when reconstructing electron-positron pairs in the 208Tl 1.6 MeV double escape peak (with Compton events as background), recorded in the NEXT-White demonstrator at the Laboratorio Subterráneo de Canfranc, with 72% signal efficiency. This was recently improved through the use of a deep convolutional neural network to yield a background rejection factor of ~ 10 with 65% signal efficiency. Here, we present a new reconstruction method, based on the Richardson-Lucy deconvolution algorithm, which allows reversing the blurring induced by electron diffusion and electroluminescence light production in the NEXT TPC. The new method yields highly refined 3D images of reconstructed events, and, as a result, significantly improves the topological background discrimination. When applied to real-data 1.6 MeV e - e + pairs, it leads to a background rejection factor of 27 at 57% signal efficiency.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Using Machine Learning to Predict Future Temperature Outputs in Geothermal Systems

Optimizing the power output, and economic value, of geothermal power plants over decades of operation is a major challenge in renewable energy. Optimizing the output requires the ability to predict the mass flow rates and the output temperatures of production wells based on the inputs of injection wells, as well as the time history of the system. Machine Learning (ML) that incorporates the known physics of geothermal systems is one possible solution to this challenge. In this work, we explore the ability of ML algorithms to predict future temperature outputs based on historical data. Considering the challenges with obtaining an empirical dataset from field data that is large enough to enable reliable ML, we propose an alternate approach: developing a high-fidelity reservoir model and using computational resources to build a dataset that enables ML. As a first step towards achieving this goal, we present preliminary results from applying ML to predict the temperature timeseries of simple modeled geothermal systems. We describe the application of relevant state-of-the-art ML approaches, such as the Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNN), to extract temporal structures in the model data. We assess the accuracy of the forecasts we obtain, compare the selected approaches, and share the lessons learned that would inform the process of training and utilizing ML algorithms for larger and more complex geothermal systems.

GEOTHERMAL ENERGY↗
DOE OSTI2023

Reconstruction of fast neutron direction in segmented organic detectors using deep learning

A method for reconstructing the direction of a fast neutron source using a segmented organic scintillator-based detector and deep learning model is proposed and analyzed. Here, the model is based on recurrent neural network, which can be trained by a sequence of data obtained from an event recorded in the detector and suitably pre-processed. The performance of deep learning-based model is compared with the conventional double-scatter detection algorithm in reconstructing the direction of a fast neutron source. With the deep learning model, the uncertainty in source direction of 0.301 rad is achieved with 100 neutron detection events in a segmented cubic organic scintillator detector with a side length of 46 mm. To reconstruct the source direction with the same angular resolution as the double-scatter algorithm, the deep learning method requires 75% fewer events. Application of this method could augment the operation of segmented detectors operated in the neutron scatter camera configuration for applications such as special nuclear material detection.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Enhancing Nanoparticle Detection in Interferometric Scattering (iSCAT) Microscopy Using a Mask R-CNN

Interferometric scattering microscopy (iSCAT) is a label-free optical microscopy technique that enables imaging of individual nano-objects such as nanoparticles, viruses, and proteins. Essential to this technique is the suppression of background scattering and identification of signals from nano-objects. In the presence of substrates with high roughness, scattering heterogeneities in the background, when coupled with tiny stage movements, cause features in the background to be manifested in background-suppressed iSCAT images. Traditional computer vision algorithms detect these background features as particles, limiting the accuracy of object detection in iSCAT experiments. Here, in this paper, we present a pathway to improve particle detection in such situations using supervised machine learning via a mask region-based convolutional neural network (mask R-CNN). Using a model iSCAT experiment of 19.2 nm gold nanoparticles adsorbing to a rough layer-by-layer polyelectrolyte film, we develop a method to generate labeled datasets using experimental background images and simulated particle signals and train the mask R-CNN using limited computational resources via transfer learning. We then compare the performance of the mask R-CNN trained with and without inclusion of experimental backgrounds in the dataset against that of a traditional computer vision object detection algorithm, Haar-like feature detection, by analyzing data from the model experiment. Results demonstrate that including representative backgrounds in training datasets improved the mask R-CNN in differentiating between background and particle signals and elevated performance by markedly reducing false positives. The methodology for creating a labeled dataset with representative experimental backgrounds and simulated signals facilitates the application of machine learning in iSCAT experiments with strong background scattering and thus provides a useful workflow for future researchers to improve their image processing capabilities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Explainable Bayesian Neural Network for Probabilistic Transient Stability Analysis Considering Wind Energy

While several data-driven models have been developed for transient stability assessment, how to consider the uncertainties from load and renewable generations and provide interpretation of data-driven assessment results are still open. This paper proposes an explainable Bayesian Neural Network (BNN) for probabilistic transient stability assessment (TSA). By extracting the uncertainties from loads and wind farms, the BNN model can make a reliable prediction and quantify the prediction uncertainties. We also develop the Gradient Shap algorithm to make the global and local explanations for the probabilistic TSA model, a significant advantage over existing black-box data-driven methods. Numerical results on the modified IEEE 39-bus system show that the proposed method outperforms the existing methods in terms of prediction accuracy and uncertainty quantification capabilities. The explainability of the proposed method allows system operators to design preventive controls for enhancing system stability.

Bayesian Neural Network↗
DOE OSTI2020

Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms [Slides]

Deep learning is used in many applications in the physical sciences. In those sciences we are used to having uncertainty on every measurement or prediction. In the last years, many methods have been put forth for uncertainty quantification: Bayesian Neural Networks, Concrete Dropout, and Deep Ensembles. Uncertainty in deep learning is often divided into aleatoric or irreducible and epistemic or reducible. The problem remains: Which uncertainty quantification method should be chosen? How are those results interpreted?

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine learning-assisted crystal engineering of a zeolite

Abstract It is shown that Machine Learning (ML) algorithms can usefully capture the effect of crystallization composition and conditions (inputs) on key microstructural characteristics (outputs) of faujasite type zeolites (structure types FAU, EMT, and their intergrowths), which are widely used zeolite catalysts and adsorbents. The utility of ML (in particular, Geometric Harmonics) toward learning input-output relationships of interest is demonstrated, and a comparison with Neural Networks and Gaussian Process Regression, as alternative approaches, is provided. Through ML, synthesis conditions were identified to enhance the Si/Al ratio of high purity FAU zeolite to the hitherto highest level (i.e., Si/Al = 3.5) achieved via direct (not seeded), and organic structure-directing-agent-free synthesis from sodium aluminosilicate sols. The analysis of the ML algorithms’ results offers the insight that reduced Na 2 O content is key to formulating FAU materials with high Si/Al ratio. An acid catalyst prepared by partial ion exchange of the high-Si/Al-ratio FAU (Si/Al = 3.5) exhibits improved proton reactivity (as well as specific activity, per unit mass of catalyst) in propane cracking and dehydrogenation compared to the catalyst prepared from the previously reported highest Si/Al ratio (Si/Al = 2.8).

36 MATERIALS SCIENCE↗
DOE OSTI2021

Evaluation of the PhaseNet Model Applied to the IMS Seismic Network

Producing a complete and accurate set of signal detections is essential for automatically building and characterizing seismic events of interest for nuclear explosion monitoring. Signal detection algorithms have been an area of research for decades, but still produce large quantities of false detections and misidentify real signals that must be detected to produce a complete global catalog of events of interest. Deep learning methods have shown promising capabilities in effectively characterizing seismic signals for complex tasks such as identifying phase arrival times. We use the PhaseNet model, a UNet-based Neural Network, trained on local distance data from northern California to predict seismic arrivals on data from the International Monitoring System (IMS) global network. We use an analyst-curated bulletin generated from this data set to compare the performance of PhaseNet to that of the Short-Term Average/Long-Term Average (STA/LTA) algorithm. We find that PhaseNet has the potential of outperforming traditional processing methods and recommend the training of a new model with the IMS data to achieve optimal performance.

58 GEOSCIENCES↗
DOE OSTI2025

Bayesian Optimized Deep Ensemble for Uncertainty Quantification of Deep Neural Networks: a System Safety Case Study on Sodium Fast Reactor Thermal Stratification Modeling

Deep neural networks (DNNs) are increasingly important to scientific computing and engineering system simulations. Accurate uncertainty quantification (UQ) for DNNs is critical in safety-sensitive engineering domains. Traditional Deep Ensemble (DE) methods, while easy to implement, frequently suffer from poorly calibrated uncertainty estimates and limited predictive accuracy due to reliance on fixed architectures with varied weight initializations. To address these issues, we introduce a workflow that combines Bayesian Optimization (BO) and DE. The workflow is modular, scalable, and integrates parallel BO initialized with Sobol sequences to individually optimize the hyperparameters of each ensemble member. This method enhances ensemble diversity, improves predictive accuracy, and provides reliable uncertainty estimates. We evaluate the proposed BODE approach in a sodium fast reactor thermal stratification modeling case study, where we used a densely connected convolutional neural network to predict turbulent viscosity during the reactor transient with consideration of data noise. We benchmark its performance against several optimization approaches, including baseline deep ensemble, evolutionary algorithm-optimized ensemble, ensemble formed via random search combined with greedy selection, and a BO ensemble using random initialization. Here, our results demonstrate superior performance of the developed BODE approach. In noise-free scenarios, BODE notably reduces incorrect aleatoric uncertainty and significantly enhances predictive accuracy. Under conditions of 5% and 10% Gaussian noise, BODE adaptively quantifies uncertainty proportional to data noise, achieving up to an 80% reduction in root mean square error compared to baseline methods and producing well-calibrated prediction intervals.

Bayesian optimization↗
DOE OSTI2021

Pushing automated morphological classifications to their limits with the Dark Energy Survey

ABSTRACT We present morphological classifications of ∼27 million galaxies from the Dark Energy Survey (DES) Data Release 1 (DR1) using a supervised deep learning algorithm. The classification scheme separates: (a) early-type galaxies (ETGs) from late-type galaxies (LTGs); and (b) face-on galaxies from edge-on. Our convolutional neural networks (CNNs) are trained on a small subset of DES objects with previously known classifications. These typically have mr ≲ 17.7 mag; we model fainter objects to mr < 21.5 mag by simulating what the brighter objects with well-determined classifications would look like if they were at higher redshifts. The CNNs reach 97 per cent accuracy to mr < 21.5 on their training sets, suggesting that they are able to recover features more accurately than the human eye. We then used the trained CNNs to classify the vast majority of the other DES images. The final catalogue comprises five independent CNN predictions for each classification scheme, helping to determine if the CNN predictions are robust or not. We obtain secure classifications for ∼87 per cent and 73 per cent of the catalogue for the ETG versus LTG and edge-on versus face-on models, respectively. Combining the two classifications (a) and (b) helps to increase the purity of the ETG sample and to identify edge-on lenticular galaxies (as ETGs with high ellipticity). Where a comparison is possible, our classifications correlate very well with Sérsic index (n), ellipticity (ϵ), and spectral type, even for the fainter galaxies. This is the largest multiband catalogue of automated galaxy morphologies to date.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

A machine learning approach to photometric metallicities of giant stars

ABSTRACT Despite the advances provided by large-scale photometric surveys, stellar features – such as metallicity – generally remain limited to spectroscopic observations often of bright, nearby low-extinction stars. To rectify this, we present a neural network approach for estimating the metallicities and distances of red giant stars with 8-band photometry and parallaxes from Gaia EDR3 and the 2MASS and WISE surveys. The algorithm accounts for uncertainties in the predictions arising from the range of possible outputs at each input and from the range of models compatible with the training set (through drop-out). A two-stage procedure is adopted where an initial network to estimate photoastrometric parallaxes is trained using a large sample of noisy parallax data from Gaia EDR3 and then a secondary network is trained using spectroscopic metallicities from the APOGEE and LAMOST surveys and an augmented feature space utilizing the first-stage parallax estimates. The algorithm produces metallicity predictions with an average uncertainty of $\pm 0.19\, \mathrm{dex}$. The methodology is applied to stars within the Galactic bar/bulge with particular focus on a sample of 1.69 million objects with Gaia radial velocities. We demonstrate the use and validity of our approach by inspecting both spatial and kinematic gradients with metallicity in the Galactic bar/bulge recovering previous results on the vertical metallicity gradient (−0.528 ± 0.002 dex kpc−1) and the vertex deviation of the bar ($-21.29\pm 2.74\, \mathrm{deg}$).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Learning-Based Real-Time Event Identification Using Rich Real PMU Data

A large-scale deployment of phasor measurement units (PMUs) that reveal the inherent physical laws of power systems from a data perspective enables an enhanced awareness of power system operation. However, the high-granularity and non-stationary nature of PMU data and imperfect data quality could bring great technical challenges for real-time system event identification. To address these challenges, this paper proposes a two-stage learning-based framework. In the first stage, a Markov transition field (MTF) algorithm is exploited to extract the latent data features by encoding temporal dependency and transition statistics of PMU data in graphs. Then, a spatial pyramid pooling (SPP)-aided convolutional neural network (CNN) is established to efficiently and accurately identify power events. The proposed method fully builds on and is also tested on a large real-world dataset from several tens of PMU sources (and the corresponding event logs), located across the U.S., with a time span of two consecutive years. We report the numerical results validate that our method has high identification accuracy while showing good robustness against poor data quality.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Two-Step Hyperparameter Optimization Method: Accelerating Hyperparameter Search by Using a Fraction of a Training Dataset

Abstract Hyperparameter optimization (HPO) is an important step in machine learning (ML) model development, but common practices are archaic—primarily relying on manual or grid searches. This is partly because adopting advanced HPO algorithms introduces added complexity to the workflow, leading to longer computation times. This poses a notable challenge to ML applications, as suboptimal hyperparameter selections curtail the potential of ML model performance, ultimately obstructing the full exploitation of ML techniques. In this article, we present a two-step HPO method as a strategic solution to curbing computational demands and wait times, gleaned from practical experiences in applied ML parameterization work. The initial phase involves a preliminary evaluation of hyperparameters on a small subset of the training dataset, followed by a reevaluation of the top-performing candidate models postretraining with the entire training dataset. This two-step HPO method is universally applicable across HPO search algorithms, and we argue it has attractive efficiency gains. As a case study, we present our recent application of the two-step HPO method to the development of neural network emulators for aerosol activation. Although our primary use case is a data-rich limit with many millions of samples, we also find that using up to 0.0025% of the data—a few thousand samples—in the initial step is sufficient to find optimal hyperparameter configurations from much more extensive sampling, achieving up to 135× speedup. The benefits of this method materialize through an assessment of hyperparameters and model performance, revealing the minimal model complexity required to achieve the best performance. The assortment of top-performing models harvested from the HPO process allows us to choose a high-performing model with a low inference cost for efficient use in global climate models (GCMs).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

FALCON: Framework for Anomaly Detection in Industrial Control Systems

Industrial Control Systems (ICS) are used to control physical processes in critical infrastructure. These systems are used in a wide variety of operations such as water treatment, power generation and distribution, and manufacturing. While the safety and security of these systems are of serious concern, recent reports have shown an increase in targeted attacks aimed at manipulating physical processes to cause catastrophic consequences. This trend emphasizes the need for algorithms and tools that provide resilient and smart attack detection mechanisms to protect ICS. In this paper, we propose an anomaly detection framework for ICS based on a deep neural network. The proposed methodology uses dilated convolution and long short-term memory (LSTM) layers to learn temporal as well as long term dependencies within sensor and actuator data in an ICS. The sensor/actuator data are passed through a unique feature engineering pipeline where wavelet transformation is applied to the sensor signals to extract features that are fed into the model. Additionally, this paper explores four variations of supervised deep learning models, as well as an unsupervised support vector machine (SVM) model for this problem. The proposed framework is validated on Secure Water Treatment testbed results. This framework detects more attacks in a shorter period of time than previously published methods.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Data-Driven Prediction and Optimization of Energy Use for Transit Fleets of Electric and ICE Vehicles

Due to the high upfront cost of electric vehicles, many public transit agencies can afford only mixed fleets of internal combustion and electric vehicles. Optimizing the operation of such mixed fleets is challenging because it requires accurate trip-level predictions of electricity and fuel use as well as efficient algorithms for assigning vehicles to transit routes. We present a novel framework for the data-driven prediction of trip-level energy use for mixed-vehicle transit fleets and for the optimization of vehicle assignments, which we evaluate using data collected from the bus fleet of CARTA, the public transit agency of Chattanooga, TN. We first introduce a data collection, storage, and processing framework for system-level and high-frequency vehicle-level transit data, including domain-specific data cleansing methods. We train and evaluate machine learning models for energy prediction, demonstrating that deep neural networks attain the highest accuracy. Based on these predictions, we formulate the problem of minimizing energy use through assigning vehicles to fixed-route transit trips. We propose an optimal integer program as well as efficient heuristic and meta-heuristic algorithms, demonstrating the scalability and performance of these algorithms numerically using the transit network of CARTA.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Active sampling for neural network potentials: Accelerated simulations of shear-induced deformation in Cu–Ni multilayers

Neural network potentials (NNPs) can greatly accelerate atomistic simulations relative to ab initio methods, allowing one to sample a broader range of structural outcomes and transformation pathways. In this work, we demonstrate an active sampling algorithm that trains an NNP that is able to produce microstructural evolutions with accuracy comparable to those obtained by density functional theory, exemplified during structure optimizations for a model Cu–Ni multilayer system. We then use the NNP, in conjunction with a perturbation scheme, to stochastically sample structural and energetic changes caused by shear-induced deformation, demonstrating the range of possible intermixing and vacancy migration pathways that can be obtained as a result of the speedups provided by the NNP. The code to implement our active learning strategy and NNP-driven stochastic shear simulations is openly available at https://github.com/pnnl/Active-Sampling-for-Atomistic-Potentials .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Classification of Semiconductors Using Photoluminescence Spectroscopy and Machine Learning

Photoluminescence spectroscopy is a nondestructive optical method that is widely used to characterize semiconductors. In the photoluminescence process, a substance absorbs photons and emits light with longer wavelengths via electronic transitions. This paper discusses a method for identifying substances from their photoluminescence spectra using machine learning, a technique that is efficient in making classifications. Neural networks were constructed by taking simulated photoluminescence spectra as the input and the identity of the substance as the output. Here, six different semiconductors were chosen as categories: gallium oxide (Ga 2 O 3 ), zinc oxide (ZnO), gallium nitride (GaN), cadmium sulfide (CdS), tungsten disulfide (WS 2 ), and cesium lead bromide (CsPbBr 3 ). The developed algorithm has a high accuracy (>90%) for assigning a substance to one of these six categories from its photoluminescence spectrum and correctly identified a mixed Ga 2 O 3 /ZnO sample.

47 OTHER INSTRUMENTATION↗