Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Atomic protein structure refinement using all-atom graph representations and SE(3)-equivariant graph transformer

Abstract Motivation The state-of-art protein structure prediction methods such as AlphaFold are being widely used to predict structures of uncharacterized proteins in biomedical research. There is a significant need to further improve the quality and nativeness of the predicted structures to enhance their usability. In this work, we develop ATOMRefine, a deep learning-based, end-to-end, all-atom protein structural model refinement method. It uses a SE(3)-equivariant graph transformer network to directly refine protein atomic coordinates in a predicted tertiary structure represented as a molecular graph. Results The method is first trained and tested on the structural models in AlphaFoldDB whose experimental structures are known, and then blindly tested on 69 CASP14 regular targets and 7 CASP14 refinement targets. ATOMRefine improves the quality of both backbone atoms and all-atom conformation of the initial structural models generated by AlphaFold. It also performs better than two state-of-the-art refinement methods in multiple evaluation metrics including an all-atom model quality score—the MolProbity score based on the analysis of all-atom contacts, bond length, atom clashes, torsion angles, and side-chain rotamers. As ATOMRefine can refine a protein structure quickly, it provides a viable, fast solution for improving protein geometry and fixing structural errors of predicted structures through direct coordinate refinement. Availability and implementation The source code of ATOMRefine is available in the GitHub repository (https://github.com/BioinfoMachineLearning/ATOMRefine). All the required data for training and testing are available at https://doi.org/10.5281/zenodo.6944368.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Turbulence model reduction by deep learning

A defining problem of turbulence theory is to produce a predictive model for turbulent fluxes. These have profound implications for virtually all aspects of the turbulence dynamics. In magnetic confinement devices, drift-wave turbulence produces anomalous fluxes via cross-correlations between fluctuations. In this work, we introduce an alternative, data-driven method for parametrizing these fluxes. The method uses deep supervised learning to infer a reduced mean-field model from a set of numerical simulations. We apply the method to a simple drift-wave turbulence system and find a significant new effect which couples the particle flux to the local gradient of vorticity. Notably, here, this effect is much stronger than the oft-invoked shear suppression effect. We also recover the result via a simple calculation. The vorticity gradient effect tends to modulate the density profile. In addition, our method recovers a model for spontaneous zonal flow generation by negative viscosity, stabilized by nonlinear and hyperviscous terms. We highlight the important role of symmetry to implementation of the new method.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Calorimetry with deep learning: particle simulation and reconstruction for collider physics

Using detailed simulations of calorimeter showers as training data, we investigate the use of deep learning algorithms for the simulation and reconstruction of single isolated particles produced in high-energy physics collisions. We train neural networks on single-particle shower data at the calorimeter-cell level, and show significant improvements for simulation and reconstruction when using these networks compared to methods which rely on currently-used state-of-the-art algorithms. We define two models: an end-to-end reconstruction network which performs simultaneous particle identification and energy regression of particles when given calorimeter shower data, and a generative network which can provide reasonable modeling of calorimeter showers for different particle types at specified angles and energies. We investigate the optimization of our models with hyperparameter scans. Furthermore, we demonstrate the applicability of the reconstruction model to shower inputs from other detector geometries, specifically ATLAS-like and CMS-like geometries. These networks can serve as fast and computationally light methods for particle shower simulation and reconstruction for current and future experiments at particle colliders.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

An evaluation of GPT models for phenotype concept recognition

Clinical deep phenotyping and phenotype annotation play a critical role in both the diagnosis of patients with rare disorders as well as in building computationally-tractable knowledge in the rare disorders field. These processes rely on using ontology concepts, often from the Human Phenotype Ontology, in conjunction with a phenotype concept recognition task (supported usually by machine learning methods) to curate patient profiles or existing scientific literature. With the significant shift in the use of large language models (LLMs) for most NLP tasks, we examine the performance of the latest Generative Pre-trained Transformer (GPT) models underpinning ChatGPT as a foundation for the tasks of clinical phenotyping and phenotype annotation. The experimental setup of the study included seven prompts of various levels of specificity, two GPT models (gpt-3.5-turbo and gpt-4.0) and two established gold standard corpora for phenotype recognition, one consisting of publication abstracts and the other clinical observations. The best run, using in-context learning, achieved 0.58 document-level F1 score on publication abstracts and 0.75 document-level F1 score on clinical observations, as well as a mention-level F1 score of 0.7, which surpasses the current best in class tool. Without in-context learning, however, performance is significantly below the existing approaches. Our experiments show that gpt-4.0 surpasses the state of the art performance if the task is constrained to a subset of the target ontology where there is prior knowledge of the terms that are expected to be matched. While the results are promising, the non-deterministic nature of the outcomes, the high cost and the lack of concordance between different runs using the same prompt and input make the use of these LLMs challenging for this particular task.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Micropolar deep material network

This study extends the Deep Material Network (DMN), a physics-informed machine learning framework, to predict the homogenized mechanical response of composite materials with micropolar (Cosserat-type) constitutive behavior. This extension incorporates microstructure-dependent size effects, enabling accurate, efficient, and size-aware predictions for composites with complex internal architectures. While traditional, direct numerical simulation micropolar models effectively capture size effects by introducing extra local degrees of freedom, they bring significant computational challenges, particularly for multiscale analyses relevant to engineering applications. The micropolar DMN developed in this paper achieves high accuracy while significantly reducing computation time compared to micropolar direct numerical simulations. This advancement enables multiscale analyses and parameter studies that were previously impractical, such as high-cycle fatigue simulations and comprehensive investigations of internal length scale effects notably in size-dependent plastic response and the optimization of lattice structures. By uniting microstructure-sensitive modeling, physics-driven learning, and scalable surrogate modeling, the micropolar DMN paves the way for accelerated material design, large-scale parametric studies, and the reliable incorporation of size-dependent effects across a wide range of engineering applications, including optimization and next-generation composite design.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Network of epistatic interactions in an enzyme active site revealed by large-scale deep mutational scanning

Cooperative interactions between amino acids are critical for protein function. A genetic reflection of cooperativity is epistasis, which is when a change in the amino acid at one position changes the sequence requirements at another position. To assess epistasis within an enzyme active site, we utilized CTX-M β-lactamase as a model system. CTX-M hydrolyzes β-lactam antibiotics to provide antibiotic resistance, allowing a simple functional selection for rapid sorting of modified enzymes. We created all pairwise mutations across 17 active site positions in the β-lactamase enzyme and quantitated the function of variants against two β-lactam antibiotics using next-generation sequencing. Context-dependent sequence requirements were determined by comparing the antibiotic resistance function of double mutations across the CTX-M active site to their predicted function based on the constituent single mutations, revealing both positive epistasis (synergistic interactions) and negative epistasis (antagonistic interactions) between amino acid substitutions. The resulting trends demonstrate that positive epistasis is present throughout the active site, that epistasis between residues is mediated through substrate interactions, and that residues more tolerant to substitutions serve as generic compensators which are responsible for many cases of positive epistasis. Additionally, we show that a key catalytic residue (Glu166) is amenable to compensatory mutations, and we characterize one such double mutant (E166Y/N170G) that acts by an altered catalytic mechanism. These findings shed light on the unique biochemical factors that drive epistasis within an enzyme active site and will inform enzyme engineering efforts by bridging the gap between amino acid sequence and catalytic function.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Exploring QCD Factorization at Moderate Energy Scales

Asymptotic freedom in QCD facilitates the use of partonic degrees of freedom over short distances, but physical processes are sensitive to a wide range of scales. Thus, it is necessary in QCD calculations to utilize a factorization scheme to separate a process into perturbative and non-perturbative factors. This separation relies on an assumption that one energy scale is infinitely larger than the other scales involved in the process. However, much experimental research in areas such as nucleon structure and quark-hadron duality occur at more moderate energy scales where that basic assumption may not be true but perturbative calculations should still be useful. Thus, an exploration of the limits of factorization at more moderate energy scales is needed. This dissertation examines various aspects of factorization at these energy scales first by applying the necessary approximations to a simple model where exact calculations are possible and so the effects of these approximations can be quantified. This is followed by examining areas where corrections are known to be needed. First is an exploration of target mass corrections (TMCs) in the case of deep inelastic scattering (DIS), including a discussion of what large corrections imply about the target structure. Second, is a general examination of PDFs and FFs at moderate scales. Third, I will discuss how this fits into the long-term effort to study the transition between small transverse momentum (generated non-perturbatively) and large transverse momentum (generated in the hard process)

Moffat, Eric↗
DOE OSTI2023

Estimating the Meridional Extent of Adiabatic Mixing in the Stratosphere Using Age-Of-Air

Wave-induced adiabatic mixing in the winter midlatitudes is one of the key processes impacting stratospheric transport. Understanding its strength and structure is vital to understanding the distribution of trace gases and their modulation under a changing climate. Age-of-air is often used to understand stratospheric transport, and this study proposes refinements to the vertical age gradient theory of Linz et al. (2021), https://doi.org/10.1029/2021JD035199. The theory assumes exchange of air between a well-mixed tropics and a well-mixed extratropics, separated by a transport barrier, quantifying the adiabatic mixing flux across the interface using age-based measures. These assumptions are re-evaluated and a refined framework that includes the effects of meridional tracer gradients is established to quantify the mixing flux. This is achieved, in part, by computing a circulation streamfunction in age-potential temperature coordinates to generate a complete distribution of parcel ages being mixed in the midlatitudes. The streamfunction quantifies the “true” age of parcels mixed between the tropics and the extratropics. Applying the revised theory to an idealized and a comprehensive climate model reveals that ignoring the meridional gradients in age leads to an underestimation of the wave-driven mixing flux. Stronger, and qualitatively similar fluxes are obtained in both models, especially in the lower-to-middle stratosphere. While the meridional span of adiabatic mixing in the two models exhibits some differences, they show that the deep tropical pipe, that is, latitudes equatorward of 15° barely mix with older midlatitude air. The novel age-potential temperature circulation can be used to quantify additional aspects of stratospheric transport.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Manifestation of postcollision interaction in Krypton LMN Auger spectra following K-shell photoionization

In this work, we report on an experimental and theoretical study of post-collision interaction (PCI) effects on L 2 -M 4,5 N 2,3 Auger electrons measured above the Kr K-edge in which L 2 vacancies are primarily generated by KL 2 x-ray emission. Such cascade processes, in which a deep inner-shell vacancy decays first by x-ray emission followed by Auger electron emission, is a strong decay mode in heavy atoms. The L 2 -M 4,5 N 2,3 Auger electron peak is observed to become increasingly asymmetric with a shifting peak maximum as the absorbed x-ray energy approaches the K-shell ionization threshold. This is attributed to PCI energy exchanges of the Auger electron with the 1s photoelectron. To model the PCI effects, we have applied a semiclassical approach modified to account for the combined lifetimes of the K and L 2 hole states. In addition, our analysis treats several closely spaced Auger transitions with final ionic states having different terms and angular momenta.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2025

High-precision monitoring of outgassing species in model extreme ultraviolet photoresists with a cavity ring-down spectrometer

Background Extreme ultraviolet (EUV) photoresists play a pivotal role in advancing nanopatterning technologies by balancing image quality and sensitivity. The outgassing behavior of photoresist thin films under EUV and deep ultraviolet (DUV) exposure reveals chemical details relevant to their performance. Aim Here, we focus on utilizing an analytical technique not previously used in photolithography, the tabletop cavity ring-down spectrometer, to investigate outgassing dynamics in EUV photoresists, enabling precise chemical identification and deeper insights into resist processing. Approach The spectrometer’s enhanced laser path length (∼ 20 km) and broadband absorption capabilities in the C–H overtone region allow for sensitive and temporally resolved detection of mixtures of outgassed species. Using a model resist comprising a polymer matrix with a photoacid generator and quencher, we analyzed the influence of time delays between exposure and post-exposure bake (PEB) as well as storage under varying environmental conditions. Results Suppression of isobutylene outgassing and thickness loss was observed with extended delays between exposure and PEB, potentially linked to water absorption and acid deactivation. The technique proved highly effective in distinguishing subtle chemical differences between processing stages. Conclusions Delay times and their environmental conditions, particularly humidity, reduce outgassing and thickness loss of photoresists during PEB, suggesting decreased acid-driven deprotection. This can potentially impact sensitivity, defectivity, and roughness of resist patterns, necessitating precise monitoring and control.

Extreme ultraviolet photoresists↗
DOE OSTI2022

Fast and Accurate Predictions of Total Energy for Solid Solution Alloys with Graph Convolutional Neural Networks

We use graph convolutional neural networks (GCNNs) to produce fast and accurate predictions of the total energy of solid solution binary alloys. GCNNs allow us to abstract the lattice structure of a solid material as a graph, whereby atoms are modeled as nodes and metallic bonds as edges. This representation naturally incorporates information about the structure of the material, thereby eliminating the need for computationally expensive data pre-processing which would be required with standard neural network (NN) approaches. We train GCNNs on ab-initio density functional theory (DFT) for copper-gold (CuAu) and iron-platinum (FePt) data that has been generated by running the LSMS-3 code, which implements a locally self-consistent multiple scattering method, on OLCF supercomputers Titan and Summit. GCNN outperforms the ab-initio DFT simulation by orders of magnitude in terms of computational time to produce the estimate of the total energy for a given atomic configuration of the lattice structure. We compare the predictive performance of GCNN models against a standard NN such as dense feedforward multi-layer perceptron (MLP) by using the root-mean-squared errors to quantify the predictive quality of the deep learning (DL) models. We find that the attainable accuracy of GCNNs is at least an order of magnitude better than that of the MLP.

Lupo Pasini, Massimiliano↗
DOE OSTI2023

Physics-informed neural network with transfer learning (TL-PINN) based on domain similarity measure for prediction of nuclear reactor transients

Nuclear reactor safety and efficiency can be enhanced through the development of accurate and fast methods for prediction of reactor transient (RT) states. Physics informed neural networks (PINNs) leverage deep learning methods to provide an alternative approach to RT modeling. Applications of PINNs in monitoring of RTs for operator support requires near real-time model performance. However, as with all machine learning models, development of a PINN involves time-consuming model training. Here, we show that a transfer learning (TL-PINN) approach achieves significant performance gain, as measured by reduction of the number of iterations for model training. Using point kinetic equations (PKEs) model with six neutron precursor groups, constructed with experimental parameters of the Purdue University Reactor One (PUR-1) research reactor, we generated different RTs with experimentally relevant range of variables. The RTs were characterized using Hausdorff and Fréchet distance. We have demonstrated that pre-training TL-PINN on one RT results in up to two orders of magnitude acceleration in prediction of a different RT. The mean error for conventional PINN and TL-PINN models prediction of neutron densities is smaller than 1%. We have developed a correlation between TL-PINN performance acceleration and similarity measure of RTs, which can be used as a guide for application of TL-PINNs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Physics-informed machine learning

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Furthermore, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Denoising diffusion models with geometry adaptation for high fidelity calorimeter simulation

Simulation is crucial for all aspects of collider data analysis, but the available computing budget in the High Luminosity LHC era will be severely constrained. Generative machine learning models may act as surrogates to replace physics-based full simulation of particle detectors, and diffusion models have recently emerged as the state of the art for other generative tasks. Here, we introduce CaloDiffusion, a denoising diffusion model trained on the public CaloChallenge datasets to generate calorimeter showers. Our algorithm employs 3D cylindrical convolutions, which take advantage of symmetries of the underlying data representation. To handle irregular detector geometries, we augment the diffusion model with a new geometry latent mapping (GLaM) layer to learn forward and reverse transformations to a regular geometry that is suitable for cylindrical convolutions. The showers generated by our approach are nearly indistinguishable from the full simulation, as measured by several different metrics.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

The LSST DESC DC2 Simulated Sky Survey

Here, we describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep area of approximately 300 deg 2 , as well as a deep drilling field of approximately 1 deg 2 . We simulate 5 yr of the planned 10 yr survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the data set to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic test bed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time domain cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Modeling Nanoconfinement Effects Using Active Learning

Predicting the spatial configuration of gas in nanopores of is relevant in applications such as fluid flow forecasting and hydrocarbon reserves estimation. For example, shale reservoirs have suffered from computationally intractable multiscale problems, since fluid properties such as viscosity, density, and adsorption must be calculated by using expensive molecular dynamics (MD) simulations within each nanopore, whereas flow through these connected nanopores must be simulated at the micrometer scale. We utilize machine learning techniques to quickly and accurately model nanoscale confinement effects as an important step toward bridging the nano and micro scales. Our workflow is based on building and training physics-based deep-neural-networks models by learning from a database of MD calculations. The model accounts for the adsorption phenomenon by predicting the statistical distribution of gas inside nanopores. Because large databases of MD calculations are expensive to create, we investigate active learning (AL) as a data set construction strategy. In this workflow, new data are selected based on the model uncertainty via the query-by-committee approach. We show that our workflow obtains accurate models that generalize to real scanning electron microscopy geometries with 1/10th of the number of MD calculations required vs random data set generation. Our method enables the possibility of modeling nanoconfinement effects at the mesoscale, where complex connected sets of nanopores affect flow.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Neural network reconstruction of the dense matter equation of state from neutron star observables

The Equation of State (EoS) of strongly interacting cold and hot ultra-dense QCD matter remains a major challenge in the field of nuclear astrophysics. With the advancements in measurements of neutron star masses, radii, and tidal deformabilities, from electromagnetic and gravitational wave observations, neutron stars play an important role in constraining the ultra-dense QCD matter EoS. Here, in this work, we present a novel method that exploits deep learning techniques to reconstruct the neutron star EoS from mass-radius (M-R) observations. We employ neural networks (NNs) to represent the EoS in a model-independent way, within the range of ~1-7 times the nuclear saturation density. The unsupervised Automatic Differentiation (AD) framework is implemented to optimize the EoS, so as to yield through TOV equations, an M-R curve that best fits the observations. We demonstrate that this method works by rebuilding the EoS on mock data, i.e., mass-radius pairs derived from a randomly generated polytropic EoS. The reconstructed EoS fits the mock data with reasonable accuracy, using just 11 mock M-R pairs observations, close to the current number of actual observations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

hi_class background evolution, initial conditions and approximation schemes

Cosmological datasets have great potential to elucidate the nature of dark energy and test gravity on the largest scales available to observation. Theoretical predictions can be computed with hi_class (www.hiclass-code.net), an accurate, fast and flexible code for linear cosmology, incorporating a wide range of dark energy theories and modifications to general relativity. We introduce three new functionalities into hi_class: (1) Support for models based on covariant Lagrangians, including a constraint-preserving integration scheme for the background evolution and a series of worked-out examples: Galileon, nKGB, quintessence (monomial, tracker) and Brans-Dicke. (2) Consistent initial conditions for the scalar-field perturbations in the deep radiation era, identifying the conditions under which modified-gravity isocurvature perturbations may grow faster than adiabatic modes leading to a loss of predictivity. (3) An automated quasistatic approximation scheme allowing order-of-magnitude improvement in computing performance without sacrificing accuracy for wide classes of models. These enhancements bring the treatment of dark energy and modified gravity models to the level of detail comparable to software tools restricted to standard $\Lambda$CDM cosmologies. Finally, the hi_class code is publicly available (https://github.com/miguelzuma/hi_class_public), ready to explore current data and prepare for next-generation experiments.

79 ASTRONOMY AND ASTROPHYSICS↗