Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep energy method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Addressing GPU memory limitations for Graph Neural Networks in High-Energy Physics applications

Introduction Reconstructing low-level particle tracks in neutrino physics can address some of the most fundamental questions about the universe. However, processing petabytes of raw data using deep learning techniques poses a challenging problem in the field of High Energy Physics (HEP). In the Exa.TrkX Project, an illustrative HEP application, preprocessed simulation data is fed into a state-of-art Graph Neural Network (GNN) model, accelerated by GPUs. However, limited GPU memory often leads to Out-of-Memory (OOM) exceptions during training, due to the large size of models and datasets. This problem is exacerbated when deploying models on High-Performance Computing (HPC) systems designed for large-scale applications. Methods We observe a high workload imbalance issue during GNN model training caused by the irregular sizes of input graph samples in HEP datasets, contributing to OOM exceptions. We aim to scale GNNs on HPC systems, by prioritizing workload balance in graph inputs while maintaining model accuracy. Our paper introduces diverse balancing strategies aimed at decreasing the maximum GPU memory footprint and avoiding the OOM exception, across various datasets. Results Our experiments showcase memory reduction of up to 32.14% compared to the baseline. We also demonstrate the proposed strategies can avoid OOM in application. Additionally, we create a distributed multi-GPU implementation using these samplers to demonstrate the scalability of these techniques on the HEP dataset. Discussion By assessing the performance of these strategies as data loading samplers across multiple datasets, we can gauge their effectiveness in both single-GPU and distributed environments. Our experiments, conducted on datasets of varying sizes and across multiple GPUs, broaden the applicability of our work to various GNN applications that handle input datasets with irregular graph sizes.

Lee, Claire Songhyun↗
DOE OSTI2024

Advancing the Prediction of MS/MS Spectra Using Machine Learning

Tandem mass spectrometry (MS/MS) is an important tool for the identification of small molecules and metabolites where resultant spectra are most commonly identified by matching them with spectra in MS/MS reference libraries. While popular, this strategy is limited by the contents of existing reference libraries. In response to this limitation, various methods are being developed for the in silico generation of spectra to augment existing libraries. Recently, machine learning and deep learning techniques have been applied to predict spectra with greater speed and accuracy. Here, in this work, we investigate the challenges these algorithms face in achieving fast and accurate predictions on a wide range of small molecules. The challenges are often amplified by the use of generic machine learning benchmarking tactics, which lead to misleading accuracy scores. Curating data sets, only predicting spectra for sufficiently high collision energies, and working more closely with experimental mass spectrometrists are recommended strategies to improve overall prediction accuracy in this nuanced field.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Surrogate Model Based Optimization for Finding Robust Deep Learning Model Architectures

Deep Learning (DL) models are increasingly used throughout the sciences. However, their performance and usefulness depend greatly on their architecture which is defined by hyperparameters such as the number of nodes, layers, the learning rate, etc. Tuning these hyperparameters is time-consuming because evaluating their performance requires a lengthy training step. Stochastic optimizers used in training lead to performance variability and potentially prediction reliability issues. In this talk, we will describe an automated optimization method based on surrogate models and active learning strategies for tuning DL model architectures. We take into account the prediction variability with the goal to identify architectures that make reliable and robust predictions. We demonstrate our developments on an application arising in particle physics.

deep learning↗
DOE OSTI2025

Applications of Nickelate perovskites for neuromorphic computing from electronic structure and Machine Learning

While the limit of Moore's law is presently being reached with current microelectronic technologies, we need to develop new paradigms that overcome this limitation. In that respect, neuromorphic computing is a concept that emulates the neural behavior and response of the human brain, and it has been recognized as a promising alternative approach. In this research project, we will perform multi-fidelity scale bridging to explore the potential use of materials with metal to insulator transition for neuromorphic applications. In particular, rare earth nickelates are promising for such purposes, as the transition in these materials is quite sensitive to a broad set of different external stimuli. Our multi-fidelity approach will bridge the high-fidelity electronic structure calculations with classical potentials. We will bridge dynamical mean field theory with a classical atomistic representation via a deep learning force field. The neural network is trained with energies, charges, and forces obtained by accurate electronic structure theories based on Dynamical Mean Field Theory. The configurational space is generated from known crystal phases, ab initio molecular dynamics with exchange-correlation functionals corrected with the Hubbard model, disordered phases with different concentrations of oxygen vacancies, and nonsymmetrical positions and induced strain by grain interfaces or contact with a substrate. Strategies to train the model with a reduced number of training examples are obtained from active learning methods, and new structures for improving the learning process are generated by using machine learning autoencoders. This classical potential will be validated through a diversity of electronic structure methods and represents an important step to combine the flexibility and accuracy of first-principles with the speed of classical potentials. The generated multi-fidelity surrogate model will be used to understand the role of strain, oxygen vacancies, proton doping, the variation of the crystal phase, substrate effects, vibrational effects as the octahedral rotation, grain boundaries and defect effects on the response of a Metal to Insulator Transition (MIT) in correlated materials. Long time and large-scale simulations will help understand the role of different stimuli to control the hysteresis of the MIT, as it has been experimentally suggested. Selected configurations will be analyzed with higher-level theories to provide an accurate electronic description and to study how the orbitals and charges are rearranged under different conditions.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Abstract This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

LSBGnet: an improved detection model for low-surface brightness galaxies

ABSTRACT The Chinese Space Station Telescope (CSST) is scheduled to launch soon, which is expected to provide a vast amount of image potentially containing low-surface brightness galaxies (LSBGs). However, detecting and characterizing LSBGs is known to be challenging due to their faint surface brightness, posing a significant hurdle for traditional detection methods. In this paper, we propose LSBGnet, a deep neural network specifically designed for automatic detection of LSBGs. We established LSBGnet-SDSS model using data set from the Sloan Digital Sky Survey (SDSS). The results demonstrate a significant improvement compared to our previous work, achieving a recall of 97.22 per cent and a precision of 97.27 per cent on the SDSS test set. Furthermore, we use the LSBGnet-SDSS model as a pre-training model, employing transfer learning to retrain the model with LSBGs from Dark Energy Survey (DES), and establish the LSBGnet-DES model. Remarkably, after retraining the model on a small DES sample, it achieves over 90 per cent precision and recall. To validate the model’s capabilities, we utilize the trained LSBGnet-DES model to detect LSBG candidates within a selected 5 sq. deg area in the DES footprint. Our analysis reveals the detection of 204 LSBG candidates, characterized by a mean surface brightness range of $23.5\ \mathrm{ mag}\ \mathrm{ arcsec}^{-2}\le \bar{\mu }_{\text{eff}}(g)\le 26.8\ \mathrm{ mag}\ \mathrm{ arcsec}^{-2}$ and a half-light radius range of 1.4 arcsec ≤ r1/2 ≤ 8.3 arcsec. Notably, 116 LSBG candidates exhibit a half-light radius ≥2.5 arcsec. These results affirm the remarkable performance of our model in detecting LSBGs, making it a promising tool for the upcoming CSST.

Su, Hao (ORCID:0009000778888301)↗
DOE OSTI2020

Doped NiO: The mottness of a charge transfer insulator

The evolution of the electronic structures of strongly correlated insulators with doping has long been a central fundamental question in condensed matter physics; it is also of great practical relevance for applications. We have studied the evolution of NiO under hole and electron doping at low doping levels such that the system remains insulating using high-quality thin film and a wide range of experimental and theoretical methods. The evolution is in both cases very smooth with dopant concentration. The band gap is asymmetric under electron and hole doping, consistent with a charge-transfer insulator picture, and is reduced faster under hole doping than under electron doping. For both electron and hole doping, occupied states are introduced at the top of the valence band. The formation of deep donor levels under electron doping and the inability to pin otherwise empty states near the conduction-band edge are indicative that local electron addition and removal energies are dominated by a Mott-like Hubbard U interaction even though the global band gap is predominantly a charge-transfer-type gap.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

A Gigaparsec-scale Hydrodynamic Volume Reconstructed with Deep Learning

The next generation of spectroscopic surveys will map the large-scale structure of the Universe at high redshifts (2 ≤ z ≤ 5) using millions of quasar spectra, enabling major advances in constraining both the standard cosmological model and its extensions. Robust cosmological analyses of these data sets require numerical simulations that both cover gigaparsec volumes and resolve features on ∼10 kpc scales and smaller. However, running such large-volume, high-resolution hydrodynamic simulations is computationally prohibitive. We present a generative deep learning model that enhances a low-resolution, gigaparsec-scale (960 h −1 Mpc) hydrodynamic simulation using a smaller (80 h −1 Mpc) high-resolution input hydrodynamic simulation as training data. The resulting enhanced simulation reproduces the line-of-sight power spectrum to within ∼10% and the three-dimensional power spectrum at the ∼20% level at intermediate to small scales (k ≲ 2 h Mpc −1 ). Our method shows strong promise for producing realistic simulations for cosmological analyses with current surveys such as the Dark Energy Spectroscopic Instrument and upcoming next-generation experiments, but further improvements are needed to accurately recover the large-scale modes. We publicly release the enhanced hydrodynamic simulation, along with a halo catalog from a companion N-body dark matter simulation to support the calibration of data analysis pipelines for these large-scale surveys.

Convolutional neural networks↗
DOE OSTI2020

Characterizing Baselines and Change in Gas Hydrate Systems using EM Methods

The objective of this project was to advance our understanding of gas hydrate systems in nature by characterizing their electrical properties in the field and in the laboratory. In the laboratory measurements, methane hydrate was synthesized from pure water ice and flash frozen seawater, with varying amounts of sand or silt added. Electrical conductivity was determined by impedance spectroscopy, using equivalent circuit modeling to separate the effects of electrodes and to gain insight into conduction mechanisms. Silt and sand increase the conductivity of pure hydrate, inferred to be contaminant NaCl contributing to conduction in hydrate, to a peak conductivity in agreement with peak resistivities observed in well logs through massive hydrate (3,000--10,000 Ωm). The addition of silt and sand lowers the conductivity of hydrate synthesized from seawater, by an amount consistent with Archie's Law. All samples were characterized using cryogenic scanning electron microscopy and energy dispersive spectroscopy, which shows good connectivity of salt and brine phases. Electrical conductivity measurements of pure hydrate and hydrate mixed with silt during pressure-induced dissociation supports previous conclusions that sediment increases dissociation rate. In order to characterize gas hydrate systems in the field, we collected 360 line kilometers of controlled-source electromagnetic data on Walker Ridge 313, Orca Basin (WR100), Mad Dog (GC781), and Green Canyon 955 in the Gulf of Mexico, all areas with known or seismically inferred gas hydrate deposits and which have be drilled or targeted for future drilling. We deep-towed an EM transmitter that generates an alternating electric field which propagates through the seafloor geology. Data were recorded on 6 receivers towed behind the transmitter at distances between 550 and 1550 m. In the presence of conductive geology, the electric fields will be attenuated, and conversely, in resistive geology the fields will be preserved. Our data were inverted using a 2D inversion method that first optimizes the model-data misfit, then finds the smoothest model fitting the data. This ensures that resistivity structures present in the final model are likely necessary. At each of the proposed drilling sites we found increased resistivity, interpreted as increased hydrate concentrations. However, not only were the primary drilling sites not always more resistive than the alternate sites, at WR313 the strongest resistors were not at the locations targeted for drilling.

03 NATURAL GAS↗
DOE OSTI2025

Boosting Noise2Inverse via enhanced model selection for denoising computed tomography data

Synchrotron-based x-ray tomographic imaging enables the examination of the internal structure of materials at high spatial and temporal resolution. Experimental constraints can impose dose and time limits on the measurements, introducing a higher level of noise and artifacts in the reconstructed images. Deep learning has emerged as a powerful tool to remove noise from reconstructed images. Recently, the Noise2Inverse method was designed specifically for denoising reconstructed images without requiring paired noisy and clean images. This method creates multiple statistically independent reconstructions used to pair the data in which training involves transforming one reconstruction into the other, and vice versa. Originally designed to be used after a fixed number of epochs, we see in practice that this approach may not produce the optimal model and may unnecessarily waste computational resources. Therefore, we propose an alternative method of identifying the best model during training that aligns with the Noise2Inverse method. During validation, we compare the model output of the multiple reconstructions among each other. We hypothesize that the best model is the one that produces images with the highest similarity, implying a convergence in the predicted material properties and absorption values. To compare model outputs, we consider the absolute error, square error, structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and cosine similarity. We evaluate our method on two simulated tomography datasets and two, real-world, low-contrast, high-energy x-ray tomography datasets. We show our approach is more effective at determining the best model, up to an increase of 12.50% and 12.53% in SSIM and PSNR, respectively, while only requiring a fifth of the training time compared to the original approach.

CT↗
DOE OSTI2025

Deep image reconstruction for background subtraction in heavy-ion collisions

Jet reconstruction in an ultrarelativistic heavy-ion collision suffers from a notoriously large, fluctuating thermal background. Traditional background‐subtraction methods struggle to remove this soft background while preserving the jet's hard substructure. In this Letter, we present DeepSub, a machine-learning-based approach for full-event background‐subtraction. DeepSub utilizes a model based on Swin Transformer layers to denoise jet images and disentangle hard jets from the heavy-ion background. DeepSub significantly outperforms existing subtraction techniques by reproducing key jet observables such as jet 𝑝 T and mass, and substructure observables such as girth and the energy correlation function, at the subpercent to percent level. As such, DeepSub paves the way for precision heavy-ion measurements in hitherto inaccessible kinematic regimes.

Qureshi, Umar Sohail [Stanford University, CA (Uni↗
DOE OSTI2024

Feature selection with distance correlation

Choosing which properties of the data to use as input to multivariate decision algorithms—also known as feature selection—is an important step in solving any problem with machine learning. While there is a clear trend towards training sophisticated deep networks on large numbers of relatively unprocessed inputs (so-called automated feature engineering), for many tasks in physics, sets of theoretically well-motivated and well-understood features already exist. Working with such features can bring many benefits, including greater interpretability, reduced training and run time, and enhanced stability and robustness. We develop a new feature selection method based on distance correlation, and demonstrate its effectiveness on the tasks of boosted top- and W -tagging. Using our method to select features from a set of over 7,000 energy flow polynomials, we show that we can match the performance of much deeper architectures, by using only ten features and two orders-of-magnitude fewer model parameters. Published by the American Physical Society 2024

Astronomy & Astrophysics↗
DOE OSTI2026

Improving neutrino oscillation measurements through event classification

Precise neutrino energy reconstruction is essential for next-generation long-baseline oscillation experiments, yet current methods remain limited by large uncertainties in neutrino-nucleus interaction modeling. Even so, it is well established that different interaction channels produce systematically varying amounts of missing energy and therefore yield different reconstruction performance–information that standard calorimetric approaches do not exploit. We introduce a strategy that incorporates this structure by classifying events according to their underlying interaction type prior to energy reconstruction. Using supervised machine-learning techniques trained on labeled generator events, we leverage intrinsic kinematic differences among quasielastic scattering, meson-exchange current, resonance production, and deep-inelastic scattering processes. A cross-generator testing framework demonstrates that this classification approach is robust to microphysics mismodeling and, when applied to a simulated DUNE 𝜈 𝜇 disappearance analysis, yields improved accuracy and sensitivity at the 10%–20% level. These results highlight a practical path toward reducing reconstruction-driven systematics in future oscillation measurements.

Ellis, Sebastian A. R. [King's College, London (Un↗
DOE OSTI2021

Deep Learning for Automated Detection and Identification of Migrating American Eel Anguilla rostrata from Imaging Sonar Data

Adult American eels (Anguilla rostrata) are vulnerable to hydropower turbine mortality during outmigration from growth habitat in inland waters to the ocean where they spawn. Imaging sonar is a reliable and proven technology for monitoring of fish passage and migration; however, there is no efficient automated method for eel detection. We designed a deep learning model for automated detection of adult American eels from sonar data. The method employs convolution neural network (CNN) to distinguish between 14 images of eels and non-eel objects. Prior to image classification with CNN, background subtraction and wavelet denoising were applied to enhance sonar images. The CNN model was first trained and tested on data obtained from a laboratory experiment, which yielded overall accuracies of >98% for image-based classification. Then, the model was trained and tested on field data that were obtained near the Iroquois Dam located on the St. Lawrence River; the accuracy achieved was commensurate with that of human experts.

13 HYDRO ENERGY↗
DOE OSTI2020

DCTRGAN: improving the precision of generative models with reweighting

Significant advances in deep learning have led to more widely used and precise neural network-based generative models such as Generative Adversarial Networks (Gans). We introduce a post-hoc correction to deep generative models to further improve their fidelity, based on the Deep neural networks using the Classification for Tuning and Reweighting (Dctr) protocol. The correction takes the form of a reweighting function that can be applied to generated examples when making predictions from the simulation. We illustrate this approach using Gans trained on standard multimodal probability densities as well as calorimeter simulations from high energy physics. We show that the weighted Gan examples significantly improve the accuracy of the generated samples without a large loss in statistical power. This approach could be applied to any generative model and is a promising refinement method for high energy physics applications and beyond.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

A Non-Invasive Approach for Elucidating the Spatial Distribution of In-Situ Stress in Deep Subsurface Geologic Formations Considered for CO 2 Storage (Final Report)

The primary objectives of this project were to: develop a new, non-invasive method to define the orientation and magnitude of in-situ stresses in deep geologic formations (depths that exceed 1500 meters) considered for carbon dioxide (CO 2 ) sequestration; to demonstrate the method at a real field site; and, to extend current technical capabilities and reduce costs of acquiring information about deep-stress fields. This investigation was segmented into five tasks, including Project Management (Task 1) and four technical tasks (Tasks 2 through 5). The results of the technical tasks are described in this document.

02 PETROLEUM↗
DOE OSTI2024

Synthesis of Thick Hg x Cd 1– x Se Nanoplatelets by Cation Exchange Catalyzed by Silver Ions, Showing Amplified Spontaneous Emission

II-VI semiconductor nanoplatelets have emerged as promising candidates for various applications, owing to their tunable optical properties dictated by their thickness and compositions. In the realm of infrared technology, mercury chalcogenides stand out as particularly promising materials for optoelectronic applications. However, the direct synthesis of 2D particles in this category remains challenging, thus prompting the exploration of alternative methods such as cation exchange. Here, we demonstrate that the cation exchange process from cadmium to mercury can be effectively catalyzed by monovalent Ag + cations. Further, this catalysis facilitates the formation of alloyed Hg x Cd 1-x Se nanoplatelets with tunable optical properties, with the photoluminescence peak ranging from 1.23 eV for the thinnest three-monolayer (ML) nanoplatelets to 0.92 eV for the thickest 7 ML nanoplatelets. The Ag + ions reduce the activation energy of the cation exchange process by a factor of 2, enabling enhanced penetration of mercury atoms deep into the native CdSe nanoplatelets. Moreover, these nanoplatelets exhibit optical gain in the infrared spectrum, including the 1.3 μm telecommunication band, with a fluence threshold at 80 K of 50 μJ∙cm -2 .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Understanding the High Energy Higgs Sector with the CMS Experiment and Artificial Intelligence

This dissertation describes efforts towards understanding the Higgs boson at the highest energies humanly accessible, using the CMS experiment at the Large Hadron Collider and advances in artificial intelligence (AI) and machine learning (ML). We present searches for resonant and nonresonant Higgs-boson (H) pair production in the all-hadronic two beauty-quark and two vector boson (V) final state, using a novel strategy to measure the quartic HHVV coupling and search for new Higgs-like bosons. By targeting highly Lorentz-boosted Higgs pairs, we probe effects of potential new physics in the high energy Higgs sector, which could hold answers to fundamental mysteries of nature such as baryon asymmetry. To enable these and future searches, we introduce as well significant developments in AI/ML, including in the identification of boosted H$\rightarrow$VV decays with deep transformer networks and advances in AI-accelerated fast simulations of the CMS detector. The latter notably includes the development of the first, highly performant generative models for point-cloud data in high energy physics, which have the potential to improve CMS' computational efficiency by up to three orders of magnitude. We also highlight novel solutions to the important and challenging problems of calibrating and validating these ML techniques. Finally, we present new approaches to search for new physics in a model-agnostic manner, using physics-informed ML methods equivariant to Lorentz transformations. The quartic HHVV coupling is observed (expected) to be constrained to $[-0.04, 2.05]$ ($[0.05, 1.98]$) at the 95% confidence level relative to the standard model prediction, representing the second-most sensitive measurement of this coupling by CMS to date. Exclusion limits on the production cross section of new heavy resonances decaying to two Higgs-like bosons are expected to be as low as 0.3 fb for high resonance masses.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗