Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

De Novo Design of High‐Affinity Miniprotein Binders Targeting Francisella Tularensis Virulence Factor

Abstract Francisella tularensis poses considerable public health risk due to its high infectivity and potential for bioterrorism. Francisella‐like lipoprotein (Flpp3), a key virulence factor unique to Francisella, plays critical roles in infection and immune evasion, making it a promising target for therapeutic development. However, the lack of well‐defined binding pockets and structural information on native interactions has hindered structure‐guided ligand discovery against Flpp3. Here, we used a combination of physics‐based and deep‐learning methods to design high‐affinity miniprotein binders targeting two distinct sites on Flpp3. We identified four binders for site I with binding affinities ranging between 24–110 nM. For the second site, an initial binder showed a dissociation constant ( K D ) of 81 nM, and subsequent site saturation mutagenesis yielded variants with sub‐nanomolar affinities. Circular dichroism confirmed the topology of designed miniproteins. The X‐ray crystal structure of Flpp3 in complex with a site I binder is nearly identical to the design model (Cα root‐mean‐square deviation (RMSD): 0.9 Å). These designed miniproteins provide research tools to explore the roles of Flpp3 in tularemia and should enable the development of new therapeutic candidates.

Gokce‐Alpkilic, Gizem [Molecular Engineering and S↗
DOE OSTI2022

Understanding and Leveraging the I/O Patterns of Emerging Machine Learning Analytics

The scientific community is currently experiencing unprecedented amounts of data generated by cutting-edge science facilities. Soon facilities will be producing up to 1 PB/s which will force scientist to use more autonomous techniques to learn from the data. The adoption of machine learning methods, like deep learning techniques, in large-scale workflows comes with a shift in the workflow’s computational and I/O patterns. These changes often include iterative processes and model architecture searches, in which datasets are analyzed multiple times in different formats with different model configurations in order to find accurate, reliable and efficient learning models. This shift in behavior brings changes in I/O patterns at the application level as well at the system level. These changes also bring new challenges for the HPC I/O teams, since these patterns contain more complex I/O workloads. In this paper we discuss the I/O patterns experienced by emerging analytical codes that rely on machine learning algorithms and highlight the challenges in designing efficient I/O transfers for such workflows. We comment on how to leverage the data access patterns in order to fetch in a more efficient way the required input data in the format and order given by the needs of the application and how to optimize the data path between collaborative processes. We will motivate our work and show performance gains with a study case of medical applications.

Gainaru, Ana↗
DOE OSTI2021

Charged Particle Tracking via Edge-Classifying Interaction Networks

Recent work has demonstrated that geometric deep learning methods such as graph neural networks (GNNs) are well suited to address a variety of reconstruction problems in high-energy particle physics. In particular, particle tracking data are naturally represented as a graph by identifying silicon tracker hits as nodes and particle trajectories as edges, given a set of hypothesized edges, edge-classifying GNNs identify those corresponding to real particle trajectories. In this work, we adapt the physics-motivated interaction network (IN) GNN toward the problem of particle tracking in pileup conditions similar to those expected at the high-luminosity Large Hadron Collider. Assuming idealized hit filtering at various particle momenta thresholds, we demonstrate the IN’s excellent edge-classification accuracy and tracking efficiency through a suite of measurements at each stage of GNN-based tracking: graph construction, edge classification, and track building. The proposed IN architecture is substantially smaller than previously studied GNN tracking architectures; this is particularly promising as a reduction in size is critical for enabling GNN-based tracking in constrained computing environments. Furthermore, the IN may be represented as either a set of explicit matrix operations or a message passing GNN. Efforts are underway to accelerate each representation via heterogeneous computing resources towards both high-level and low-latency triggering applications.

accelerator physics↗
DOE OSTI2021

An adaptive adversarial domain adaptation approach for corn yield prediction

Recently, statistical machine learning and deep learning methods have been widely explored for corn yield prediction. Though successful, machine learning models generated within a specific spatial domain often lose their validity when directly applied to new regions. To address this issue, we designed an unsupervised adaptive domain adversarial neural network (ADANN). Specifically, through domain adversarial training, the ADANN model reduced the impact of domain shift by projecting data from different domains into the same subspace. Also, the ADANN model was designed to be trained in an adaptive way, which guaranteed the model can learn the domain-invariant features and perform accurate yield prediction simultaneously. Informative variables including time-series vegetation indices and sequential weather observations were first collected from multiple data sources and aggregated to the county level. Then, we trained the ADANN model with the extracted features and corresponding reported county-level corn yield from the U.S. Department of Agriculture (USDA). Finally, the trained model was evaluated in four testing years 2016–2019. The U.S. corn belt was used as the study area and counties under study were grouped into two diverse ecological regions. Overall, the experimental results showed that the developed ADANN model had better performance than three other state-of-the-art machine learning models in both local experiments (train and test in the same region) and transfer experiments (train and test in different regions). As the first study using adversarial learning for crop yield prediction, this research demonstrates a novel solution for improving model transferability on crop yield prediction.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Sequence2Self: Self-supervised image sequence denoising of pixel-level spray breakup morphology

Optical imaging of fast and transient phenomena such as the turbulent breakup of liquid sprays exhibit low signal-to-noise ratios due to the limited illumination intensity relative to the short exposure time. Image denoising is required to facilitate physical studies over these data but is challenging due to the absence of clean ground-truths and the stringency of the denoising task (e.g., strong and complex noise, limited resolution, preserving physical fidelity), preventing supervised and existing un-/self-supervised deep learning methods. To this end, Sequence2Self (Seq2S) is proposed, an extension of Self2Self (S2S) to image sequences that leverages both the signal’s spatial and temporal correlation. Seq2S is demonstrated on time-resolved x-ray phase contrast imaging of liquid jet fuel sprays in a gas turbine combustor, which possesses all of challenges detailed above. Experiments are conducted across four fuels with different breakup morphology using various state-of-the-art methods. Overall, many of the methods failed and Seq2S was most successful: (1) Accurate spray structures were reconstructed with consistent evolution across frames void of artifacts. (2) The performance was robust, invariant to the hyperparameter choice. (3) Computational time is short and can be made eligible for real-time denoising. In particular, the images denoised by Seq2S showed spray droplet diameter distributions with near-zero Kullback–Leibler divergence (0.01 ± 0.01) to a cleaner reference, whereas the second best method yielded 0.06 ± 0.03. In conclusion, this suggests that Seq2S can be reliably used prior to subsequent quantitative spray analyses as it retains (if not, improves) the statistical physical properties of the data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

AI-Based Quantification of Planned Radiation Therapy Dose to Cardiac Structures and Coronary Arteries in Patients With Breast Cancer

The purpose of this work is to develop and evaluate an automatic deep learning method for segmentation of cardiac chambers and large arteries, and localization of the 3 main coronary arteries in radiation therapy planning on computed tomography (CT). In addition, a second purpose is to determine the planned radiation therapy dose to cardiac structures for breast cancer therapy.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2021

A data-driven global soil heterotrophic respiration dataset and the drivers of its inter-annual variability

Soil heterotrophic respiration (SHR), one of the primary carbon fluxes from terrestrial ecosystems to the atmosphere, is important for carbon-climate feedbacks because of its sensitivity to available litter and soil carbon, climatic conditions, and nutrient availability. However, until recently limited SHR data were available, and most published global SHR estimates have either a short time span, coarse spatial resolution, or reply on overly-simple model formulations. To better understand and quantify the global distribution of SHR and its sensitivity to climate variability, we produced a new global SHR dataset using Random Forest algorithms, up-scaling 455 point data from the Global Soil Respiration Database (SRDB 4.0) with gridded fields of climatic, edaphic and productivity as explanatory variables. We estimated a global total SHR of 46.8 Pg C yr-1 over 1985-2013 (95% confidence interval: 38.6-56.3 Pg C yr-1), with a significant increasing trend of 0.03 Pg C yr-2 during this period. We found that the choice of soil moisture datasets contributes more to the difference among these data-driven SHR members rather than that of productivity, temperature and precipitation data sources. We also analyzed the influence of climatic variables on the inter-annual variability (IAV) of our SHR product. Water availability was the dominant driver of IAV at global scales, although the inferred sensitivity depends on the choice of the soil moisture gridded dataset. At the ecosystem scale, temperature strongly controls the IAV of SHR in tropical forests, while water availability dominates in extra-tropical forest and semi-arid regions. Our machine-learning gridded SHR dataset and outputs from process-based land surface models (TRENDYv6) show agreement for a strong association between water variability and SHR IAV at the global scale, but the two approaches lead to different temporal trend globally and different controlling variables for IAV at the ecosystem scale. Our study provides evidence for the pervasive and important role of water availability in driving SHR, indicating both a direct effect limiting decomposition rates and an indirect effect through the amount of fresh organic matter made available to SHR from productivity. In consideration of potential limitations and uncertainties remaining in our data-driven SHR datasets, we call for a more scientifically designed observation network for SHR, more observation data compilation, and increased use of deep learning methods making maximum use of observation data in hand. This will benefit process-based models, and improve our understanding of SHR response to future anomalous environmental conditions.

Yao, Yitong↗
DOE OSTI2024

Machine Learning‐Assisted Microearthquake Location Workflow for Monitoring the Newberry Enhanced Geothermal System

Abstract Enhanced geothermal systems (EGS) offer a sustainable energy source but face challenges in accurately locating microearthquakes induced during reservoir stimulation. Locating these microearthquakes provides reliable feedback on the stimulation progress. Current deep learning methods for locating earthquakes require extensive data sets for training, which is problematic as detected microearthquakes are often limited. To address the scarcity of training data, we propose a practical workflow using probabilistic multilayer perceptron (PMLP) which predicts microearthquake locations from cross‐correlation time lags in waveforms. Utilizing a 3D velocity model of Newberry site derived from ambient noise interferometry, we generate numerous synthetic microearthquakes and 3D acoustic waveforms for PMLP training. Accurate synthetic tests prompt us to apply the trained network to the 2012 and 2014 stimulation field waveforms. To enhance the accuracy of source localization, we carefully handpick the P‐arrival times. Predictions on the 2012 stimulation data set show major microseismic activity at depths of 0.5–1.2 km, correlating with a known casing leakage scenario. In the 2014 data set, the majority of predictions concentrate at 2.0–2.9 km depths, consistent with results obtained from conventional physics‐based inversion, and align with the presence of natural fractures from 2.0 to 2.7 km. We validate our findings by comparing the synthetic and field picks, demonstrating a satisfactory match for the first arrivals. By combining the benefits of quick inference speeds and accurate location predictions, we demonstrate the feasibility of using realistic synthetic data set to locate microseismicity for EGS monitoring.

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

Off-the-shelf deep learning is not enough, and requires parsimony, Bayesianity, and causality

Abstract Deep neural networks (‘deep learning’) have emerged as a technology of choice to tackle problems in speech recognition, computer vision, finance, etc. However, adoption of deep learning in physical domains brings substantial challenges stemming from the correlative nature of deep learning methods compared to the causal, hypothesis driven nature of modern science. We argue that the broad adoption of Bayesian methods incorporating prior knowledge, development of solutions with incorporated physical constraints and parsimonious structural descriptors and generative models, and ultimately adoption of causal models, offers a path forward for fundamental and applied research.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Neural network representations of multiphase Equations of State

Abstract Equations of State model relations between thermodynamic variables and are ubiquitous in scientific modelling, appearing in modern day applications ranging from Astrophysics to Climate Science. The three desired properties of a general Equation of State model are adherence to the Laws of Thermodynamics, incorporation of phase transitions, and multiscale accuracy. Analytic models that adhere to all three are hard to develop and cumbersome to work with, often resulting in sacrificing one of these elements for the sake of efficiency. In this work, two deep-learning methods are proposed that provably satisfy the first and second conditions on a large-enough region of thermodynamic variable space. The first is based on learning the generating function (thermodynamic potential) while the second is based on structure-preserving, symplectic neural networks, respectively allowing modifications near or on phase transition regions. They can be used either “from scratch” to learn a full Equation of State, or in conjunction with a pre-existing consistent model, functioning as a modification that better adheres to experimental data. We formulate the theory and provide several computational examples to justify both approaches, highlighting their advantages and shortcomings.

Science & Technology - Other Topics↗
DOE OSTI2024

Geometry-complete diffusion for 3D molecule generation and optimization

Abstract Generative deep learning methods have recently been proposed for generating 3D molecules using equivariant graph neural networks (GNNs) within a denoising diffusion framework. However, such methods are unable to learn important geometric properties of 3D molecules, as they adopt molecule-agnostic and non-geometric GNNs as their 3D graph denoising networks, which notably hinders their ability to generate valid large 3D molecules. In this work, we address these gaps by introducing the Geometry-Complete Diffusion Model (GCDM) for 3D molecule generation, which outperforms existing 3D molecular diffusion models by significant margins across conditional and unconditional settings for the QM9 dataset and the larger GEOM-Drugs dataset, respectively. Importantly, we demonstrate that GCDM’s generative denoising process enables the model to generate a significant proportion of valid and energetically-stable large molecules at the scale of GEOM-Drugs, whereas previous methods fail to do so with the features they learn. Additionally, we show that extensions of GCDM can not only effectively design 3D molecules for specific protein pockets but can be repurposed to consistently optimize the geometry and chemical composition of existing 3D molecules for molecular stability and property specificity, demonstrating new versatility of molecular diffusion models. Code and data are freely available on GitHub .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Physics-constrained deep learning of nonlinear normal modes of spatiotemporal fluid flow dynamics

In this study, we present a physics-constrained deep learning method to discover and visualize from data the invariant nonlinear normal modes (NNMs) which contain the spatiotemporal dynamics of the fluid flow potentially containing strong nonlinearity. Specifically, we develop a NNM-physics-constrained convolutional autoencoder (NNM-CNN-AE) integrated with a multi-temporal-step dynamics prediction block to learn the nonlinear modal transformation, the NNMs containing the spatiotemporal dynamics of the flow, and reduced-order reconstruction and long-time future-state prediction of the flow fields, simultaneously. In test cases, we apply the developed method to analyze different flow regimes past a cylinder, including laminar flows with low Reynolds number in transient and steady states (RD = 100) and high Reynolds number flow (RD = 1000), respectively. The results indicate that the identified NNMs are able to reveal the nonlinear spatiotemporal dynamics of these flows, and the NNMs-based reduced-order modeling consistently achieves better accuracy with orders of magnitudes smaller errors in construction and prediction of the nonlinear velocity and vorticity fields, compared to the linear proper orthogonal decomposition (POD) method and the Koopman-constrained-CNN-AE using the same number or dimension of modes. We perform an analysis of the modal energy distribution of NNMs and find that compared to POD modes, the few fundamental NNMs capture a very high level of total energy of the flow, which is advantageous for reduced-order modeling and representation of the complex flows. Finally, we discuss the potentials and limitations of the presented method.

Mechanics↗
DOE OSTI2024

Improved deep learning prediction of antigen–antibody interactions

Identifying antibodies that neutralize specific antigens is crucial for developing effective immunotherapies, but this task remains challenging for many target antigens. The rise of deep learning–based computational approaches presents a promising avenue to address this challenge. Here, we assess the performance of a deep learning approach through two benchmark tests aimed at predicting antibodies for the receptor-binding domain of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spike protein. Three different strategies for constructing input sequence alignments are employed for predicting structural models of antigen–antibody complexes. In our initial testing set, which comprises known experimental structures, these strategies collectively yield a significant top-ranked prediction for 61% of cases and a success rate of 47%. Notably, one strategy that utilizes the sequences of known antigen binders outperforms the other two, achieving a precision of 90% in a subsequent test set of ~1,000 antibodies, balanced between true and control antibodies for the antigen, albeit with a lower recall of 25%. Our results underscore the potential of integrating deep learning methods with single B cell sequencing techniques to enhance the prediction accuracy of antigen–antibody interactions.

Science & Technology - Other Topics↗
DOE OSTI2022

A Conditional Autoencoder for Galaxy Photometric Parameter Estimation

Astronomical photometric surveys routinely image billions of galaxies, and traditionally infer the parameters of a parametric model for each galaxy. This approach has served us well, but the computational expense of deriving a full posterior probability distribution function is a challenge for increasingly ambitious surveys. In this paper, we use deep learning methods to characterize galaxy images, training a conditional autoencoder on mock data. The autoencoder can reconstruct and denoise galaxy images via a latent space engineered to include semantically meaningful parameters, such as brightness, location, size, and shape. Our model recovers galaxy fluxes and shapes on mock data with a lower variance than the Hyper Suprime-Cam photometry pipeline, and returns reasonable answers even for inputs outside the range of its training data. When applied to data in the training range, the regression errors on all extracted parameters are nearly unbiased with a variance near the Cramr-Rao bound.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Automatic Detection and Classification of Radio Galaxy Images by Deep Learning

Abstract Surveys conducted by radio astronomy observatories, such as SKA, MeerKAT, Very Large Array, and ASKAP, have generated massive astronomical images containing radio galaxies (RGs). This generation of massive RG images has imposed strict requirements on the detection and classification of RGs and makes manual classification and detection increasingly difficult, even impossible. Rapid classification and detection of images of different types of RGs help astronomers make full use of the observed astronomical image data for further processing and analysis. The classification of FRI and FRII is relatively easy, and there are more studies and literature on them at present, but FR0 and FRI are similar, so it is difficult to distinguish them. It poses a greater challenge to image processing. At present, deep learning has made breakthrough progress in the field of image analysis and processing and has preliminary applications in astronomical data processing. Compared with classification algorithms that can only classify galaxies, object detection algorithms that can locate and classify RGs simultaneously are preferred. In target detection algorithms, YOLOv5 has outstanding advantages in the classification and positioning of small targets. Therefore, we propose a deep-learning method based on an improved YOLOv5 object detection model that makes full use of multisource data, combining FIRST radio with SDSS optical image data, and realizes the automatic detection of FR0, FRI, and FRII RGs. The innovation of our work is that on the basis of the original YOLOv5 object detection model, we introduce the SE Net attention mechanism, increase the number of preset anchors, adjust the network structure of the feature pyramid, and modify the network structure, thereby allowing our model to demonstrate galaxy classification and position detection effects. Our improved model produces satisfactory results, as evidenced by experiments. Overall, the mean average precision (mAP@0.5) of our improved model on the test set reaches 89.4%, which can determine the position (R.A. and decl.) and automatically detect and classify FR0s, FRIs, and FRIIs. Our work contributes to astronomy because it allows astronomers to locate FR0, FRI, and FRII galaxies in a relatively short time and can be further combined with other astronomically generated data to study the properties of these galaxies. The target detection model can also help astronomers find FR0s, FRIs, and FRIIs in future surveys and build a large-scale star RG catalog. Moreover, our work is also useful for the detection of other types of galaxies.

Astronomy & Astrophysics↗
DOE OSTI2025

MHD, disruptions and control physics: Chapter 4 of the special issue: on the path to tokamak burning plasma operation

In this chapter, we review the progress in MHD stability, disruptions and control in magnetic fusion research that has occurred over the past (more than) one and a half decades since the publication by Hender et al in 2007 on the same topic as part of the update of ITER Physics Basis. During this period, remarkable progress has been achieved in the understanding of the basic physics and overall control of MHD instabilities through a wide spectrum of dedicated experiments, theory and modeling. The sawtooth activities are probably today one of the best understood of MHD events and very robust control schemes have been developed for reliable operation of tokamaks through core heating. Similarly, significant improvements have been achieved in understanding and control of neoclassical tearing modes, resistive wall modes or locked modes and their control through ECCD or error field control. The field of disruption prediction through application of artificial intelligence, machine learning or deep learning methods, which had already started at the time of the 2007 review, has progressed significantly due to general progress in these fields and application of newer, more sophisticated algorithms. However, although remarkable progress has been achieved in the field of Disruptions, their understanding, prediction, possible avoidance and mitigation still remain probably the most active fields of R&D globally in this field. This is especially because reactor grade machines like ITER and DEMO will be much less tolerant in respect of disruptions and runaway currents, and their occurrences must be either avoided altogether or minimized to an acceptable value without causing any significant hindrance to robust machine operations. This review is intended to present a broad spectrum of the R&D that has occurred in this field in support of ITER, which will also be of immense significance for all future machines, especially reactors like DEMO.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

First application of a liquid argon time projection chamber for the search for intranuclear neutron-antineutron transitions and annihilation in 40 Ar using the MicroBooNE detector

We present a novel methodology to search for intranuclear neutron-antineutron transition (n→n̅) followed by n̅-nucleon annihilation within an 40 Ar nucleus, using the MicroBooNE liquid argon time projection chamber (LArTPC) detector. A discovery of n→n̅ transition or a new best limit on the lifetime of this process would either constitute physics beyond the Standard Model or greatly constrain theories of baryogenesis, respectively. The approach presented in this paper makes use of deep learning methods to select n→n̅ events based on their unique features and differentiate them from cosmogenic backgrounds. The achieved signal and background efficiencies are (70.22 ± 6.04)% and (0.0020 ± 0.0003)%, respectively. A demonstration of a search is performed with a data set corresponding to an exposure of 3.32 ×10 26 neutron-years, and where the background rate is constrained through direct measurement, assuming the presence of a negligible signal. With this approach, no excess of events over the background prediction is observed, setting a demonstrative lower bound on the n→n̅ lifetime in 40 Ar of τ m ≳ 1.1×10 26 years, and on the free n→n̅ transition time of τ n→n̅ ≳ 2.6×10 5 s, each at the 90% confidence level. This analysis represents a first-ever proof-of-principle demonstration of the ability to search for this rare process in LArTPCs with high efficiency and low background.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Transforming the bootstrap: using Transformers to compute scattering amplitudes in planar N = 4 Super Yang-Mills theory

Abstract We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.

Cai, Tianji (ORCID:0000000232359486)↗