Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Digital twin framework for PIP-II linac: AI-driven multi-scale modeling from ion source to 800 MeV

The PIP-II superconducting linac at Fermilab is designed to deliver multi-megawatt proton beams for neutrino physics and other high-intensity applications. To expedite commissioning and enhance operational reliability, we have developed an EPICS-based data flow framework that seamlessly integrates digital twins (DT) with physical twins (PT). These digital twins comprise high-fidelity beam dynamics models or data-driven surrogate models connected to their physical counterparts through real-time diagnostics and advanced machine-learning algorithms.Central to this framework is Linac_Gen, an accelerated simulation tool that incorporates convolutional neural networks, random forests, and genetic algorithms to provide up to a tenfold speedup in optimizing the accelerator geometry model. An EPICS translator layer ensures interoperability by efficiently mapping lattice parameters across diverse simulation platforms.Our EPICS-based framework supports multiple operational modes—monitoring, passive learning, closed-loop control, and online learning—covering the entire machine lifecycle. By leveraging HPC resources and multi-objective optimization techniques, the digital twin enables adaptive trajectory correction, real-time fault detection, and predictive modeling of beam stability. This comprehensive approach paves the way for robust, high-intensity operation and data-driven accelerator R&D at Fermilab.

Pathak, Abhishek [Fermilab]↗
DOE OSTI2021

Primordial non-Gaussianity from the completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey – I: Catalogue preparation and systematic mitigation

ABSTRACT We investigate the large-scale clustering of the final spectroscopic sample of quasars from the recently completed extended Baryon Oscillation Spectroscopic Survey (eBOSS). The sample contains 343 708 objects in the redshift range 0.8 < z < 2.2 and 72 667 objects with redshifts 2.2 < z < 3.5, covering an effective area of $4699\, {\rm deg}^{2}$. We develop a neural network-based approach to mitigate spurious fluctuations in the density field caused by spatial variations in the quality of the imaging data used to select targets for follow-up spectroscopy. Simulations are used with the same angular and radial distributions as the real data to estimate covariance matrices, perform error analyses, and assess residual systematic uncertainties. We measure the mean density contrast and cross-correlations of the eBOSS quasars against maps of potential sources of imaging systematics to address algorithm effectiveness, finding that the neural network-based approach outperforms standard linear regression. Stellar density is one of the most important sources of spurious fluctuations, and a new template constructed using data from the Gaia spacecraft provides the best match to the observed quasar clustering. The end-product from this work is a new value-added quasar catalogue with the improved weights to correct for non-linear imaging systematic effects, which will be made public. Our quasar catalogue is used to measure the local-type primordial non-Gaussianity in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Bridging Hydrological Ensemble Simulation and Learning Using Deep Neural Operators

Ensemble-based simulation and learning (ESnL) has long been used in hydrology for parameter inference, but computational demands of process-based ESnL can be quite high. To address this issue, we propose a deep neural operator learning approach. Neural operators are generic machine learning algorithms that can learn functional mappings between infinite-dimensional spaces, providing a highly flexible tool for scientific machine learning. Our approach is built upon DeepONet, a specific deep neural operator, and is designed to address several common problems in hydrology, namely, model parameter estimation, prediction at ungaged locations, and uncertainty quantification. Here we demonstrate the effectiveness of our DeepONet-based workflow using an existing large model ensemble created for an eastern U.S. watershed that is instrumented with 10 streamflow gages. Results suggest DeepONet achieves high efficiency in learning an ML surrogate model from the model ensemble, with the modified Kling-Gupta Efficiency exceeding 0.9 on holdout test sets. Parameter inference, carried out using the trained DeepONet surrogate model and genetic algorithm, also yields robust results. Additionally, we formulate and train a separate DeepONet model for physics-informed, seq-to-seq streamflow forecasting, which further reduces biases in the pre-trained DeepONet surrogate model. While this study focuses primarily on a single watershed, our approach is general and may be extended to enable learning from model ensembles across multiple basins or models. Thus, this research represents a significant contribution to the application of hybrid machine learning in hydrology.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Here using a model heat engine, we show that neural-network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Development of Hopfield Artificial Neural Network for Anomaly Detection in Environmental Gamma Radiation Background: Consortium on Nuclear Security Technologies (CONNECT) (Q2 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore supervised machine learning (ML) algorithms for development of a Hopfield Neural Network (HNN) in conjunction with an image processing algorithm for detection and identification of weak nuisances and anomalies events in the presence of a highly fluctuating background.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Reducing Communication in Graph Neural Network Training

Graph Neural Networks (GNNs) are powerful and flexible neural networks that use the naturally sparse connectivity information of the data. GNNs represent this connectivity as sparse matrices, which have lower arithmetic intensity and thus higher communication costs compared to dense matrices, making GNNs harder to scale to high concurrencies than convolutional or fully-connected neural networks. Here, we introduce a family of parallel algorithms for training GNNs and show that they can asymptotically reduce communication compared to previous parallel GNN training methods. We implement these algorithms, which are based on 1D, 1. 5D, 2D, and 3D sparse-dense matrix multiplication, using torch.distributed on GPU-equipped clusters. Our algorithms optimize communication across the full GNN training pipeline. We train GNNs on over a hundred GPUs on multiple datasets, including a protein network with over a billion edges.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Search for the pair production of long-lived supersymmetric partners of the tau lepton in proton-proton collisions at $\sqrt{s}=13$ TeV

Gauge-mediated supersymmetry-breaking models provide a strong motivation to search for a supersymmetric partner of the tau lepton (stau) with a macroscopic lifetime. Long-lived stau decays produce tau leptons that are displaced from the primary proton-proton interaction vertex, leading to an unconventional signature. This paper presents a search for the direct production of long-lived staus decaying within the CMS tracker volume in proton-proton collisions at $\sqrt{s}=13$ TeV, performed for the first time with an identification algorithm based on a graph neural network dedicated to displaced tau leptons. The data sample, corresponding to an integrated luminosity of 138 fb −1 , was recorded with the CMS experiment at the CERN LHC between 2016 and 2018. This search excludes, at 95% confidence level, stau masses, m~ τ , in the 126–260 (90–425) GeV range for a proper decay length of 50 mm in the maximally mixed (mass-degenerate) scenario, while for m~ τ = 200 GeV, stau proper decay lengths are excluded in the range 21–94 (6–333) mm. These results improve the exclusion limits compared to previous searches, and extend the parameter space explored in the context of supersymmetry.

Hadron-Hadron Scattering↗
DOE OSTI2023

Improved calorimetric particle identification in NA62 using machine learning techniques

Measurement of the ultra-rare $K^+$ → $π$ +$ν$$\overline{ν}$ decay at the NA62 experiment at CERN requires high-performance particle identification to distinguish muons from pions. Calorimetric identification currently in use, based on a boosted decision tree algorithm, achieves a muon misidentification probability of 1.2×10 -5 for a pion identification efficiency of 75% in the momentum range of 15–40 GeV/c. In this work, calorimetric identification performance is improved by developing an algorithm based on a convolutional neural network classifier augmented by a filter. Muon misidentification probability is reduced by a factor of six with respect to the current value for a fixed pion-identification efficiency of 75%. Alternatively, pion identification efficiency is improved from 72% to 91% for a fixed muon misidentification probability of 10 -5 .

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

First measurement of the cross section for top quark pair production with additional charm jets using dileptonic final states in pp collisions at $\sqrt{s}$= 13 TeV

The first measurement of the inclusive cross section for top quark pairs ($t\bar{t}$) produced in association with two additional charm jets is presented. The analysis uses the dileptonic final states of $t\bar{t}$ events produced in proton-proton collisions at a centre-of-mass energy of 13 TeV. The data correspond to an integrated luminosity of 41.5 fb -1 recorded by the CMS experiment at the LHC. A new charm jet identification algorithm provides input to a neural network that is trained to distinguish among $t\bar{t}$ events with two additional charm ($t\bar{t}c\bar{c}$), bottom ($t\bar{t}b\bar{b}$), and light-flavour or gluon ($t\bar{t}LL$) jets. By means of a template fitting procedure, the inclusive $t\bar{t}c\bar{c}$, $t\bar{t}b\bar{b}$, and $t\bar{t}LL$ cross sections are simultaneously measured, together with their ratios to the inclusive $t\bar{t}$ + two jets cross section. This provides measurements of the $t\bar{t}c\bar{c}$ and $t\bar{t}b\bar{b}$ cross sections of 10.1 ± 1.2 (stat) ± 1.4 (syst) pb and 4.54 ± 0.35 (stat) ± 0.56 (syst) pb, respectively, in the full phase space. The results are compared and found to be consistent with predictions from two different matrix element generators with next-to-leading order accuracy in quantum chromodynamics, interfaced with a parton shower simulation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

A Generalized Transformer-Based Pulse Detection Algorithm

Pulse-like signals are ubiquitous in the field of single molecule analysis, e.g., electrical or optical pulses caused by analyte translocations in nanopores. The primary challenge in processing pulse-like signals is to capture the pulses in noisy backgrounds, but current methods are subjectively based on a user-defined threshold for pulse recognition. Here, we propose a generalized machine-learning based method, named pulse detection transformer (PETR), for pulse detection. PETR determines the start and end time points of individual pulses, thereby singling out pulse segments in a time-sequential trace. It is objective without needing to specify any threshold. It provides a generalized interface for downstream algorithms for specific application scenarios. PETR is validated using both simulated and experimental nanopore translocation data. It returns a competitive performance in detecting pulses through assessing them with several standard metrics. Finally, the generalization nature of the PETR output is demonstrated using two representative algorithms for feature extraction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Leveraging prior mean models for faster Bayesian optimization of particle accelerators

Tuning particle accelerators is a challenging and time-consuming task that can be automated and carried out efficiently using suitable optimization algorithms, such as model-based Bayesian optimization techniques. One of the major advantages of Bayesian algorithms is the ability to incorporate prior information about beam physics and historical behavior into the model used to make control decisions. In this work, we examine incorporating prior accelerator physics information into Bayesian optimization algorithms by utilizing fast executing, neural network models trained on simulated or historical datasets as prior mean functions in Gaussian process models. We show that in ideal cases, this technique substantially increases convergence speed to optimal solutions in high-dimensional tuning parameter spaces. Additionally, we demonstrate that even in non-ideal cases, where prior models of beam dynamics do not exactly match experimental conditions, the use of this technique can still enhance convergence speed. Finally, we demonstrate how these methods can be used to improve optimization in practical applications, such as transferring information gained from beam dynamics simulations to online control of the LCLS injector, and transferring knowledge gained from experimental measurements across different operating modes, such as accelerating different ion species at the ATLAS heavy ion accelerator.

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

Reionization history constraints from neural network based predictions of high-redshift quasar continua

ABSTRACT Observations of the early Universe suggest that reionization was complete by z ∼ 6, however, the exact history of this process is still unknown. One method for measuring the evolution of the neutral fraction throughout this epoch is via observing the Lyα damping wings of high-redshift quasars. In order to constrain the neutral fraction from quasar observations, one needs an accurate model of the quasar spectrum around Lyα, after the spectrum has been processed by its host galaxy but before it is altered by absorption and damping in the intervening intergalactic medium (IGM). In this paper, we present a novel machine learning approach, using artificial neural networks, to reconstruct quasar continua around Lyα. Our Quasar Spectra from Artificial Neural Network based predictive Regression Algorithm(QSANNdRA) improves the error in this reconstruction compared to the state-of-the-art principal component analysis (PCA) based model in the literature by 14.2 per cent on average, and provides an improvement of 6.1 per cent on average when compared to an extension thereof. In comparison with the extended PCA model, QSANNdRA further achieves an improvement of 22.1 per cent and 16.8 per cent when evaluated on low-redshift quasars most similar to the two high-redshift quasars under consideration, ULAS J1120+0641 at z = 7.0851 and ULAS J1342+0928 at z = 7.5413, respectively. Using our more accurate reconstructions of these two z > 7 quasars, we estimate the neutral fraction of the IGM using a homogeneous reionization model and find $\bar{x}_\mathrm{H\, \small{I}} = 0.25^{+0.05}_{-0.05}$ at z = 7.0851 and $\bar{x}_\mathrm{H\, \small{I}} = 0.60^{+0.11}_{-0.11}$ at z = 7.5413. Our results are consistent with the literature and favour a rapid end to reionization.

Ďurovčíková, Dominika↗
DOE OSTI2023

Search for narrow resonances in the b -tagged dijet mass spectrum in proton-proton collisions at s = 13 TeV

A search is performed for narrow resonances decaying to final states of two jets, with at least one jet originating from a b quark, in proton-proton collisions at s = 13 TeV . The data set corresponds to an integrated luminosity of 138 fb - 1 collected with the CMS detector at the LHC. Jets originating from energetic b hadrons are identified through a b -tagging algorithm that utilizes a deep neural network or the presence of a muon inside a jet. The invariant mass spectrum of jet pairs is well described by a smooth parametrization and no evidence for the production of new particles is observed. Upper limits on the production cross section are set for excited b quarks and other resonances decaying to dijet final states containing b quarks. These limits exclude at 95% confidence level models of Z ' bosons with masses from 1.8 TeV to 2.4 TeV and of excited b quarks with masses from 1.8 TeV to 4.0 TeV. This is the most stringent exclusion of excited b quarks to date.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

High–throughput measurement of plant fitness traits with an object detection method using Faster R–CNN

Revealing the contributions of genes to plant phenotype is frequently challenging because loss-of-function effects may be subtle or masked by varying degrees of genetic redundancy. Such effects can potentially be detected by measuring plant fitness, which reflects the cumulative effects of genetic changes over the lifetime of a plant. However, fitness is challenging to measure accurately, particularly in species with high fecundity and relatively small propagule sizes such as Arabidopsis thaliana. An image segmentation-based method using the software ImageJ and an object detection-based method using the Faster Region-based Convolutional Neural Network (R-CNN) algorithm were used for measuring two Arabidopsis fitness traits: seed and fruit counts. The segmentation-based method was error-prone (correlation between true and predicted seed counts, r 2 = 0.849) because seeds touching each other were undercounted. By contrast, the object detection-based algorithm yielded near perfect seed counts (r 2 = 0.9996) and highly accurate fruit counts (r 2 = 0.980). Comparing seed counts for wild-type and 12 mutant lines revealed fitness effects for three genes; fruit counts revealed the same effects for two genes. Our study provides analysis pipelines and models to facilitate the investigation of Arabidopsis fitness traits and demonstrates the importance of examining fitness traits when studying gene functions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Deep Learning Image Segmentation for Atmospheric Rivers

Abstract The identification of atmospheric rivers (ARs) is crucial for weather and climate predictions as they are often associated with severe storm systems and extreme precipitation, which can cause large impacts on society. This study presents a deep learning model, termed ARDetect, for image segmentation of ARs using ERA5 data from 1960 to 2020 with labels obtained from the TempestExtremes tracking algorithm. ARDetect is a convolutional neural network (CNN)-based U-Net model, with its structure having been optimized using automatic hyperparameter tuning. Inputs to ARDetect were selected to be the integrated water vapor transport (IVT) and total column water (TCW) fields, as well as the AR mask from TempestExtremes from the previous time step to the one being considered. ARDetect achieved a mean intersection-over-union (mIoU) rate of 89.04% for ARs, indicating its high accuracy in identifying these weather patterns and a superior performance than most deep learning–based models for AR detection. In addition, ARDetect can be executed faster than the TempestExtremes method (seconds vs minutes) for the same period. This provides a significant benefit for online AR detection, especially for high-resolution global models. An ensemble of 10 models, each trained on the same dataset but having different starting weights, was used to further improve on the performance produced by ARDetect, thus demonstrating the importance of model diversity in improving performance. ARDetect provides an effective and fast deep learning–based model for researchers and weather forecasters to better detect and understand ARs, which have significant impacts on weather-related events such as floods and droughts.

Galea, Daniel↗
DOE OSTI2022

Non-trivial symmetries in quantum landscapes and their resilience to quantum noise

Very little is known about the cost landscape for parametrized Quantum Circuits (PQCs). Nevertheless, PQCs are employed in Quantum Neural Networks and Variational Quantum Algorithms, which may allow for near-term quantum advantage. Such applications require good optimizers to train PQCs. Recent works have focused on quantum-aware optimizers specifically tailored for PQCs. However, ignorance of the cost landscape could hinder progress towards such optimizers. In this work, we analytically prove two results for PQCs: (1) We find an exponentially large symmetry in PQCs, yielding an exponentially large degeneracy of the minima in the cost landscape. Alternatively, this can be cast as an exponential reduction in the volume of relevant hyperparameter space. (2) We study the resilience of the symmetries under noise, and show that while it is conserved under unital noise, non-unital channels can break these symmetries and lift the degeneracy of minima, leading to multiple new local minima. Based on these results, we introduce an optimization method called Symmetry-based Minima Hopping (SYMH), which exploits the underlying symmetries in PQCs. Our numerical simulations show that SYMH improves the overall optimizer performance in the presence of non-unital noise at a level comparable to current hardware. Overall, this work derives large-scale circuit symmetries from local gate transformations, and uses them to construct a noise-aware optimization method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Incorporating polar field data for improved solar flare prediction

In this paper, we consider incorporating data associated with the sun’s north and south polar field strengths to improve solar flare prediction performance using machine learning models. When used to supplement local data from active regions on the photospheric magnetic field of the sun, the polar field data provides global information to the predictor. While such global features have been previously proposed for predicting the next solar cycle’s intensity, in this paper we propose using them to help classify individual solar flares. We conduct experiments using HMI data employing four different machine learning algorithms that can exploit polar field information. Additionally, we propose a novel probabilistic mixture of experts model that can simply and effectively incorporate polar field data and provide on-par prediction performance with state-of-the-art solar flare prediction algorithms such as the Recurrent Neural Network (RNN). Our experimental results indicate the usefulness of the polar field data for solar flare prediction, which can improve Heidke Skill Score (HSS2) by as much as 10.1%.

79 ASTRONOMY AND ASTROPHYSICS↗