Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

SIDDA: SInkhorn Dynamic Domain Adaptation

Modern neural networks (NNs) often do not generalize well in the presence of a "covariate shift"; that is, in situations where the training and test data distributions differ, but the conditional distribution of classification labels remains unchanged. In such cases, NN generalization can be reduced to a problem of learning more domain-invariant features. Domain adaptation (DA) methods include a range of techniques aimed at achieving this; however, these methods have struggled with the need for extensive hyperparameter tuning, which then incurs significant computational costs. In this work, we introduce SIDDA, an out-of-the-box DA training algorithm built upon the Sinkhorn divergence, that can achieve effective domain alignment with minimal hyperparameter tuning and computational overhead. We demonstrate the efficacy of our method on multiple simulated and real datasets of varying complexity, including simple shapes, handwritten digits, and real astronomical observations. SIDDA is compatible with a variety of NN architectures, and it works particularly well in improving classification accuracy and model calibration when paired with equivariant neural networks (ENNs). We find that SIDDA enhances the generalization capabilities of NNs, achieving up to a ≈40% improvement in classification accuracy on unlabeled target data. We also study the efficacy of DA on ENNs with respect to the varying group orders of the dihedral group DN, and find that the model performance improves as the degree of equivariance increases. Finally, we find that SIDDA enhances model calibration on both source and target data--achieving over an order of magnitude improvement in the ECE and Brier score. SIDDA's versatility, combined with its automated approach to domain alignment, has the potential to advance multi-dataset studies by enabling the development of highly generalizable models.

Pandya, Sneh [Northeastern U.]↗
DOE OSTI2023

Efficient Subset Simulation using Hamiltonian Neural Network enhanced Markov Chain Monte Carlo Methods

The Monte Carlo method delivers an unbiased estimate of the probability of failure. However, the variance of the estimate depends on the number of evaluated samples. This number must be very large for estimations of a low probability of failure. If the evaluation of each sample is computationally expensive, the crude Monte Carlo simulation strategy is impracticable. Therefore, subset simulations are used to reduce the required number of evaluations. Subset simulations require a Markov Chain Monte Carlo sampler, such as the random walk Metropolis-Hastings algorithm. The algorithm, however, struggles with sampling in low-probability regions, especially if they are narrow. As a consequence, advanced Markov Chain Monte Carlo simulations have been developed. In particular, the Hamiltonian Monte Carlo method explores the target distribution rapidly. Driven by the idea of Hamiltonian dynamics, this sampler provides a non-random walk through the target distribution. The incorporation of subset simulation and Hamiltonian Monte Carlo methods has shown promising results for reliability analysis. One downside of the Hamiltonian Monte Carlo method is that gradient evaluations are computationally expensive, especially when dealing with high-dimensional problems and evaluating long trajectories. We show that integrating Hamiltonian neural networks in Hamiltonian Monte Carlo simulations significantly speeds up the sampling task. Furthermore, the enhancement of adaptive trajectory length within the Hamiltonian Monte Carlo results in the efficient proposal of the following states. Based on this recent enhancement, we provide a fast sampling strategy for subset simulations using Hamiltonian neural networks to replace the evaluation of the gradient and significantly speed up the Hamiltonian Monte Carlo simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

An autonomous robot for shell and tube heat exchanger inspection

Shell and tube heat exchangers (STHEs) are critical to energy conversion efficiency of power plants. Eddy current examination is a way to evaluate working conditions of these tubes. However, the current testing apparatus requires human to manually insert an eddy current testing (ECT) probe into and extract it out of individual tubes, and meanwhile monitor measurement results for diagnosis. It is a time-consuming and labor-intensive procedure even for an experienced technician. To tackle this challenge, in this study, we developed a robot enabled ECT system for autonomous inspection of STHEs. The robotic platform employs Mecanum wheeled chassis for high mobility, machine vision to locate tube bundle and tube inlets, a rotational Cartesian mechanism to operate at planes with all possible inclinations, and a task-specific mechanism for ECT probe delivery. Machine vision locates tube bundle and tube inlets by an April tag detection algorithm and a Circle Hough Transform algorithm, respectively. Assisted by a guiding cone, the ECT probe is continuously fed into the tubes with a fill factor of 0.819. In conclusion, during this process, the eddy current data are automatically collected and real-time analyzed by convolutional neural networks, showing accuracy of nearly 100% for identifying defective and nondefective tubes and 85% for four types of defective tubes and nondefective tubes.

42 ENGINEERING↗
DOE OSTI2018

Deep neural networks for energy and position reconstruction in EXO-200

Here, we apply deep neural networks (DNN) to data from the EXO-200 experiment. In the studied cases, the DNN is able to reconstruct the relevant parameters—total energy and position—directly from raw digitized waveforms, with minimal exceptions. For the first time, the developed algorithms are evaluated on real detector calibration data. The accuracy of reconstruction either reaches or exceeds what was achieved by the conventional approaches developed by EXO-200 over the course of the experiment. Most existing DNN approaches to event reconstruction and classification in particle physics are trained on Monte Carlo simulated events. Such algorithms are inherently limited by the accuracy of the simulation. We describe a unique approach that, in an experiment such as EXO-200, allows to successfully perform certain reconstruction and analysis tasks by training the network on waveforms from experimental data, either reducing or eliminating the reliance on the Monte Carlo.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Integrated Model, Batch and Domain Parallelism in Training Neural Networks

We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD). Our goal is to find an efficient parallelization strategy for a fixed batch size using $P$ processes. Our method is inspired by the communication-avoiding algorithms in numerical linear algebra. We see $P$ processes as logically divided into a $P_r \times P_c$ grid where the $P_r$ dimension is implicitly responsible for model/domain parallelism and the $P_c$ dimension is implicitly responsible for batch parallelism. In practice, the integrated matrix-based parallel algorithm encapsulates these types of parallelism automatically. We analyze the communication complexity and analytically demonstrate that the lowest communication costs are often achieved neither with pure model nor with pure data parallelism. We also show how the domain parallel approach can help in extending the theoretical scaling limit of the typical batch parallel method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Automatic Differentiation for Adjoint Stencil Loops

Stencil loops are a common motif in computations including convolutional neural networks, structured-mesh solvers for partial differential equations, and image processing. Stencil loops are easy to parallelise, and their fast execution is aided by compilers, libraries, and domain-specific languages. Reverse-mode automatic differentiation, also known as algorithmic differentiation, autodiff, adjoint differentiation, or back-propagation, is sometimes used to obtain gradients of programs that contain stencil loops. Unfortunately, conventional automatic differentiation results in a memory access pattern that is not stencil-like and not easily parallelisable. In this paper we present a novel combination of automatic differentiation and loop transformations that preserves the structure and memory access pattern of stencil loops, while computing fully consistent derivatives. The generated loops can be parallelised and optimised for performance in the same way and using the same tools as the original computation. We have implemented this new technique in the Python tool PerforAD, which we release with this paper along with test cases derived from seismic imaging and computational fluid dynamics applications.

Automatic Differentiation↗
DOE OSTI2022

Physics-driven learning of Wasserstein GAN for density reconstruction in dynamic tomography

Object density reconstruction from projections containing scattered radiation and noise is of critical importance in many applications. Existing scatter correction and density reconstruction methods may not provide the high accuracy needed in many applications and can break down in the presence of unmodeled or anomalous scatter and other experimental artifacts. Incorporating machine-learning models could prove beneficial for accurate density reconstruction, particularly in dynamic imaging, where the time evolution of the density fields could be captured by partial differential equations or by learning from hydrodynamics simulations. In this work, we demonstrate the ability of learned deep neural networks to perform artifact removal in noisy density reconstructions, where the noise is imperfectly characterized. Here, we use a Wasserstein generative adversarial network (WGAN), where the generator serves as a denoiser that removes artifacts in densities obtained from traditional reconstruction algorithms. We train the networks from large density time-series datasets, with noise simulated according to parametric random distributions that may mimic noise in experiments. The WGAN is trained with noisy density frames as generator inputs, to match the generator outputs to the distribution of clean densities (time series) from simulations. A supervised loss is also included in the training, which leads to an improved density restoration performance. In addition, we employ physics-based constraints such as mass conservation during the network training and application to further enable highly accurate density reconstructions. Our preliminary numerical results show that the models trained in our frameworks can remove significant portions of unknown noise in density time-series data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Gaussian Process Regression for Aggregate Baseline Load Forecasting

Demand response (DR) is one of the most effective ways to maintain the reliability and improve the flexibility of power systems. Accurate forecasts of baseline loads are essential for DR programs. In the era of big data, machine learning-based approaches present a unique opportunity for baseline load forecasting. Thus, this paper presents a machine learning-based approach using a relatively less explored algorithm, Gaussian process regression (GPR), to forecast aggregate baseline loads. As such, a dataset was generated using a set of EnergyPlus simulations. Using the generated dataset, a GPR-based forecasting model was developed. In addition, support vector regression (SVR)-, artificial neural network (ANN)-, and averaging-based models were developed as baseline models for comparison. These models were compared in terms of accuracy, simplicity, and integrity. The prediction performance of the models showed that the GPR-based model is more accurate and reliable than the others. Such high performance shows the potential of the GPR in baseline load forecasting. GPR, therefore, can be used for DR applications.

Amasyali, Kadir↗
DOE OSTI2024

Improving Neutrino Energy Reconstruction with Machine Learning

Faithful energy reconstruction is foundational for precision neutrino experiments like DUNE, but is hindered by uncertainties in our understanding of neutrino--nucleus interactions. Here, we demonstrate that dense neural networks are very effective in overcoming these uncertainties by estimating inaccessible kinematic variables based on the observable part of the final state. We find improvements in the energy resolution by up to a factor of two compared to conventional reconstruction algorithms, which translates into an improved physics performance equivalent to a 10-30% increase in the exposure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Power System Frequency Dynamics Modeling, State Estimation, and Control using Neural Ordinary Differential Equations (NODEs) and Soft Actor-Critic (SAC) Machine Learning Approaches

With the global energy transition of the electric power system, grid control, supervision, and protection is becoming more challenging. With the increasing integration of renewable energy sources (RES), the system dynamics are changing, causing traditional power system dynamic modeling with swing equation-based modeling approaches to fail. Additionally, the converter-dominated power grid is decreasing the system inertia, making the power system more fragile to the frequency swings. This paper first investigates and compares the application of a model-based Kalman filter state estimation approach with (i) a model-free machine learning approach --- neural ordinary differential equations (NODEs) --- and (ii) a data-driven system identification (SysId) approach to model and infer critical state values of the power system frequency dynamics. Then a model predictive control (MPC) framework is compared to a model-free Soft Actor-Critic (SAC) reinforcement learning (RL) control algorithm in providing efficient fast frequency response (FFR) to the power system frequency dynamics. The approaches are compared in terms of their performance goals as well as their per-timestep computational efficiency. Furthermore, the comparative study for state estimation shows that for the model-free requirement, both NODEs and SysId can provide accurate state estimates; however, with increasing model complexity, NODEs can be a better choice for model identification. Similarly, the results from the FFR comparative study show that the SAC RL-based FFR, once trained, outperforms MPC with better control signals and faster computation time, making the SAC RL-based FFR better option for providing FFR to the power system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Solving key challenges in collider physics with foundation models

Foundation models are neural networks that are capable of simultaneously solving many problems. Large language foundation models like ChatGPT have revolutionized many aspects of daily life, but their impact for science is not yet clear. In this paper, we use a new foundation model for hadronic jets to solve three key challenges in collider physics. In particular, we show how experiments can (1) save significant computing power when developing reconstruction algorithms, (2) perform a complete uncertainty quantification for high-dimensional measurements, and (3) search for new physics with model agnostic methods using low-level inputs. In each case, there are significant computational or methodological challenges with current methods that limit the science potential of deep learning algorithms. By solving each problem, we take jet foundation models beyond proof-of-principle studies and into the toolkit of practitioners.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Automated Approach to Accurate, Precise, and Fast Detector Simulation and Reconstruction

Detector simulation and reconstruction are a significant computational bottleneck in particle physics. Here, we develop particle-flow neural-assisted simulations (parnassus) to address this challenge. Our deep learning model takes as input a point cloud (particles impinging on a detector) and produces a point cloud (reconstructed particles). By combining detector simulations and reconstruction into one step, we aim to minimize resource utilization and enable fast surrogate models suitable for application both inside and outside large collaborations. We demonstrate this approach using a publicly available dataset of jets passed through the full simulation and reconstruction pipeline of the Compact Muon Solenoid (CMS) experiment. We show that parnassus accurately mimics the CMS particle flow algorithm on the (statistically) same events it was trained on and can generalize to jet momentum and type outside of the training distribution.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Deep learning and radiomics: the utility of Google TensorFlow™ Inception in classifying clear cell renal cell carcinoma and oncocytoma on multiphasic CT

Purpose: Currently, all solid enhancing renal masses without microscopic fat are considered malignant until proven otherwise and there is substantial overlap in the imaging findings of benign and malignant renal masses, particularly between clear cell RCC (ccRCC) and benign oncocytoma (ONC). Radiomics has attracted increased attention for its utility in pre-operative work-up on routine clinical images. Radiomics based approaches have converted medical images into mineable data and identified prognostic imaging signatures that machine learning algorithms can use to construct predictive models by learning the decision boundaries of the underlying data distribution. The TensorFlow™ framework from Google is a state-of-the-art open-source software library that can be used for training deep learning neural networks for performing machine learning tasks. The purpose of this study was to investigate the diagnostic value and feasibility of a deep learning-based renal lesion classifier using open-source Google TensorFlow™ Inception in differentiating ccRCC from ONC on routine four-phase MDCT in patients with pathologically confirmed renal masses. Methods: With institutional review board approval for this 1996 Health Insurance Portability and Accountability Act compliant retrospective study and a waiver of informed consent, we queried our institution’s pathology, clinical, and radiology databases for histologically proven cases of ccRCC and ONC obtained between January 2000 and January 2016 scanned with a an intravenous contrast-enhanced four-phase renal mass protocol (unenhanced (UN), corticomedullary (CM), nephrographic (NP), and excretory (EX) phases). To extract features to be used for the machine learning model, the entire renal mass was contoured in the axial plane in each of the four phases, resulting in a 3D volume of interest (VOI) representative of the entire renal mass. We investigated thirteen different approaches to convert the acquired VOI data into a set of images that adequately represented each tumor which was used to train the final layer of the neural network model. Training was performed over 4000 iterations. In each iteration, 90% of the data were designated as training data and the remaining 10% served as validation data and a leave-one-out cross-validation scheme was implemented. Accuracy, sensitivity, specificity, positive (PPV) and negative predictive (NPV) values, and CIs were calculated for the classification of the thirteen processing modes. Results: We analyzed 179 consecutive patients with 179 lesions (128 ccRCC and 51 ONC). The ccRCC cohort had a mean size of 3.8 cm (range 0.8–14.6 cm) and the ONC cohort had a mean lesion size of 3.9 cm (range 1.0–13.1 cm). The highest specificity and PPV (52.9% and 80.3%, respectively) were achieved in the EX phase when we analyzed the single mid-slice of the tumor in the axial, coronal and sagittal plane, and when we increased the number of mid-slices of the tumor to three, with an accuracy of 75.4%, which also increased the sensitivity to 88.3% and the PPV to 79.6%. Using the entire tumor volume also showed that classification performance was best in the EX phase with an accuracy of 74.4%, a sensitivity of 85.8% and a PPV of 80.1%. When the entire tumor volume, plus mid-slices from all phases and all planes presented as tiled images, were submitted to the final layer of the neural network we achieved a PPV of 82.5%. Conclusions: The best classification result was obtained in the EX phase among the thirteen classification methods tested. Our proof of concept study is the first step towards understanding the utility of machine learning in the differentiation of ccRCC from ONC on routine CT images. We hope this could lead to future investigation into the development of a multivariate machine learning model which may augment our ability to accurately predict renal lesion histology on imaging.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2024

Terrain-Relative Navigation with Neuro-Inspired Elevation Encoding

Terrain-relative navigation (TRN) encompasses a wide variety of algorithms that perform localization with respect to the terrain below a flying vehicle. In traditional approaches, measurements of the terrain are matched to a map carried onboard. This work presents a terrain-relative navigation filter with a position measurement inspired by neural activity associated with positioning in nature. Here, the filter is shown to produce accurate position measurements that outperform popular optimization and template matching methods given poor prior knowledge of the position. The proposed method is also better-suited to distributed implementation than optimization-based methods.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Machine Learning-Assisted Stability Boundary Determination of Multiport Autonomous Reconfigurable Solar Power Plants

The multiport autonomous reconfigurable solar (MARS) power plant is a promising solution to integrate renewable resources and energy storage systems into the alternating current (ac) power grid and an high-voltage direct current (HVdc) link. In the MARS system, various input power sources are connected to the individual submodules (SMs) through direct current (dc)–dc converters. However, the presence of external power sources can result in unbalanced capacitor voltages of SMs, thereby violating stability constraints under multiple/diverse operating conditions. This article aims to address the gap by accurately determining the stability boundary of the MARS system. As such, a novel machine learning (ML)-assisted energy balancing control (EBC) criterion is proposed. Further, in conjunction with a refined EBC, this approach ensures balanced capacitor voltages across various types of SMs, significantly enhancing the overall system efficiency. The proposed EBC criterion effectively controls EBC activation and deactivation, achieving remarkable accuracy. Both power systems computer aided design (PSCAD)/electromagnetic transients including direct current (EMTDC) simulations and control hardware-in-the-loop (cHIL) tests are conducted to validate the feasibility and efficiency of the proposed method. By combining the EBC and ML-assisted EBC criterion, efficient energy management is achieved for systems featuring multiple input power sources, such as MARS. This approach enables the system to fully exploit its potential across an expanded operational range while upholding high-efficiency standards.

14 SOLAR ENERGY↗
DOE OSTI2021

Stabilization of the 81-channel coherent beam combination using machine learning

We develop a rapidly converging algorithm for stabilizing a large channel-count diffractive optical coherent beam combination. An 81-beam combiner is controlled by a novel, machine-learning based, iterative method to correct the optical phases, operating on an experimentally calibrated numerical model. A neural-network is trained to detect phase errors based on interference pattern recognition of uncombined beams adjacent to the combined one. Due to the non-uniqueness of solutions in the full space of possible phases, the network is trained within a limited phase perturbation/error range. This also reduces the number of samples needed for training. Simulations have proven that the network can converge in one step for small phase perturbations. When the trained neural-network is applied to a realistic case of 360 degree full range, an iterative scheme exploits random walking at the beginning, with the accuracy of prediction on phase feedback direction, to allow the neural-network to step into the training range for fast convergence. This neural-network-based iterative method of phase detection works tens of times faster than the commonly used stochastic parallel gradient descent approach (SPGD) using a single-detector and random dither when both are tested with random phase perturbations.

Wang, Dan↗
DOE OSTI2021

Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials and Components. Third Annual Progress Report

Additive manufacturing (AM) of high-strength corrosion resistance alloys for nuclear energy applications, such as stainless steel and Inconel, is currently based on laser powder bed fusion (LPBF) process. Some of the challenges with using LPBF method for nuclear manufacturing include the possibility of introducing pores into metallic structures. Probability of crack initiation at the pore depends on size, shape, and orientation of the defect. Pulsed Infrared Thermography Imaging (PIT) provides a capability for non-destructive evaluation (NDE) of sub-surface defects in arbitrary size structures. The PIT method is based on recording material surface temperature transients with infrared (IR) camera following thermal pulse delivered on material surface with flash light. The PIT method has advantages for NDE of actual AM structures because the method involves one-sided non-contact measurements and fast processing of large sample areas captured in one image. Following initial qualification of an AM component for deployment in a nuclear reactor, a PIT system can also be used for in-service nondestructive evaluation (NDE) applications. In this report, we describe recent progress in enhancing PIT capabilities in detecting microscopic subsurface defects in metals, and classifying shapes and orientation of pores in thermal images. For detection of microscopic defects in PIT imaging data, we have developed Spatial Temporal Denoised Thermal Source Separation (STDTSS) unsupervised machine learning (ML) image processing algorithm. We show that flat bottom hole (FBH) defects as small as 200µm in SS316 and IN718 specimens, can be detected with STDTSS algorithm. To the best of our knowledge, these are the smallest detected defects which are reported in literature. For classification of defects shapes, we have previously developed thermal tomography (TT) algorithm to obtain depth reconstructions of material defects from data cube of sequentially recorded surface temperatures. However, interpretation of TT images is non-trivial because of blurring with increasing depth. To address this challenge, we have developed a deep learning convolutional neural network (CNN) to classify size and orientation subsurface defects in simulated TT images.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1999

The Prospect for Remote Sensing of Cirrus Clouds with a Submillimeter-Wave Spectrometer

Given the substantial radiative effects of cirrus clouds and the need to validate cirrus cloud mass in climate models, it is important to measure the global distribution of cirrus properties with satellite remote sensing. Existing cirrus remote sensing techniques, such as solar reflectance methods, measure cirrus ice water path (IWP) rather indirectly and with limited accuracy. Submillimeter/wave radiometry is an independent method of cirrus remote sensing based on ice particles scattering the upwelling radiance emitted by the lower atmosphere. A new aircraft instrument, the Far Infrared Sensor for Cirrus (FIRSC), is described. The FIRSC employs a Fourier Transform Spectrometer (FTS). which measures the upwelling radiance across the whole submillimeter region (0.1 1.0-mm wavelength). This wide spectral coverage gives high sensitivity to most cirrus particle sizes and allows accurate determination of the characteristic particle size. Radiative transfer modeling is performed to analyze the capabilities of the submillimeter FTS technique. A linear inversion analysis is done to show that cirrus IWP, particle size, and upper-tropospheric temperature and water vapor may be accurately measured, A nonlinear statistical algorithm is developed using a database of 20000 spectra simulated by randomly varying most relevant cirrus and atmospheric parameters. An empirical orthogonal function analysis reduces the 500-point spectrum (20 - 70/cm) to 15 "pseudo-channels" that are then input to a neural network to retrieve cirrus IWP and median particle diameter. A Monte Carlo accuracy study is performed with simulated spectra having realistic noise. The retrieval errors are low for IWP (rms less than a factor of 1.5) and for particle sizes (rins less than 30%) for IWP greater than 5 g/sq m and a wide range of median particle sizes. This detailed modeling indicates that there is good potential to accurately measure cirrus properties with a submillimeter FTS.

Evans, K. Franklin↗