Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “embedding model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A new biogeochemical modelling framework (FLaMe-v1.0) for lake methane emissions on the regional scale: development and application to the European domain

This study presents a new physical-biogeochemical modelling framework for simulating lake methane (CH 4 ) emissions at regional scales. The new model, FLaMe-v1.0 (Fluxes of Lake Methane), rests on an innovative, computationally efficient lake clustering approach that enables the simulation of CH 4 emissions across a large number of lakes. Building on the Canadian Small Lake Model (CSLM) that simulates the lake physics, we develop a suite of biogeochemical modules to simulate transient dynamics of organic Carbon (C), Oxygen (O 2 ), and CH 4 . We first test the performance of FLaMe-v1.0 by analyzing physical and biogeochemical processes in two theoretical lakes with characteristics that can be considered representative for many lakes (an oligotrophic, deep lake driven by cold climate versus a eutrophic, shallow lake driven by warm climate). Next, we evaluate the model by comparing simulated and observed timeseries of CH 4 emissions in four well-surveyed lakes. We then apply FLaMe-v1.0 at the European scale to evaluate simulated diffusive and ebullitive lake CH 4 fluxes against in-situ measurements in both boreal and central European regions. Finally, we provide a first assessment of the spatio-temporal variability in CH 4 emissions from European lakes with a surface area comprised between 0.1–1000 km 2 (n= 108 407, total area = 1.33 × 105 km 2 ), indicating a total emission of 0.97 ± 0.23 Tg CH 4 yr −1 , with the uncertainty constrained by combining FLaMe-v1.0 and machine learning techniques. Moreover, 30 % and 70 % of these CH 4 emissions are through diffusive and ebullitive pathways, respectively. Annually averaged CH 4 emission rates per unit lake area during 2010–2016 have a South-to-North decreasing gradient, resulting in a mean over the European domain as 7.39 g CH 4 m −2 yr −1 . Our simulations reveal a strong seasonality (with ice-blocking effects accounted for) in European lake CH 4 emissions, with nearly ten times higher emissions during late summer than during winter. This pronounced seasonal variation highlights the importance of accounting for the sub-annual variability in CH 4 emissions to accurately constrain regional CH 4 budgets. In the future, FLaMe-v1.0 could be embedded into Earth System Models to investigate the feedback between climate warming and global lake CH 4 emissions.

Maisonnier, Manon [Free Univ. of Brussels (Belgium↗
DOE OSTI2024

Baselining the Indirect Effect by Improving Quantification of Sea Spray and Marine Sources at Ascension Island (Final Report)

Oceans cover two-thirds of the Earth and understanding the interactions of aerosols with clouds in these large marine regions requires quantifying the man-made contributions to the budget of cloud-drop forming particles (known as cloud condensation nuclei, or CCN) relative to the non-manmade “baseline” conditions and understanding the meteorology of boundary layer clouds. Modeling studies have shown substantial uncertainties and sensitivities to natural marine CCN sources, meaning that to reduce uncertainties in indirect effects we must be able to better quantify the CCN budget in ocean regions. While models provide important constraints on these uncertainties, actually reducing uncertainties requires substantial observations in open-ocean and coastal regions in order to establish the baseline on which manmade emissions are added. The tropical South Atlantic Ocean is one of the least-sampled regions of the planet, making the comprehensive measurements of the Department of Energy Atmospheric Radiation Measurement Layered Atlantic Smoke Interactions with Clouds (LASIC) campaign provides the longest record of aerosol size distribution measurements from a differential mobility analyzer in a cloud-influenced marine location in the ARM database, with 17 months of ground-based aerosol size distribution measurements. This project addressed the research topic of Aerosol-Cloud Interactions (ACI) by supporting three publications from the LASIC measurements: 1. The ARM measurements were combined with a new value-added technique that was pioneered by the Russell group for quantifying sea salt. This quantification of sea salt uses supermicron scattering measurements for retrieving reasonable sea spray mass concentrations, providing the best-available, observationally-constrained estimate of the sea spray mode properties when supermicron size distribution measurements are not available. 2. The fitted modes of the distribution were used to investigate the signatures of cloud processing for very clean to very smoky aerosol conditions, revealing not only differences in the particles that activate in clouds but also in the mechanisms that control that droplet formation process. In clean air, the size required to form a cloud droplet is influenced by the number of particles, as well as how quickly particles take up water during growth in cloud. 3. Building on this aerosol characterization, cloud and meteorological conditions were used to evaluate aerosol-related changes in cloud albedo and optical depth. We introduced a new method of decomposing the impact of aerosols on clouds known as the Twomey effect by incorporating retrieved supersaturation from two independent sets of observations to constrain the feedback of aerosol particles on cloud properties. The method quantifies the reduction of the Twomey effect at high aerosol concentrations, which has never been explained quantitatively by observations. In addition, the results provide the first direct validation for the conditions observed in the tropical South Atlantic of a parcel-based approach that is embedded in many climate models. Together these findings illustrate how ARM extended field campaigns in stratocumulus-covered regions can be used to constrain ACI processes with direct observations, providing specific radiative effects without models. By making such process-specific constraints available to improve ACI in global climate models, ARM observations play a key role in supporting model development.

58 GEOSCIENCES↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Coupled Climate Simulations With E3SM‐MMF

Simulations of the recent historical period from 1950 to 2014 are conducted with E3SM‐MMF, which uses an embedded 2D cloud resolving model that runs efficiently on GPUs in place of traditional parameterizations for cloud and turbulence. Analysis of the climate and variability reveal several aspects where E3SM‐MMF produces smaller biases compared to E3SMv2, including better agreement with the observed evolution of global mean surface temperature, although the representation of ENSO is too weak and fast. Three idealized abrupt CO 2 experiments were also conducted to assess climate sensitivity and feedbacks. These yield three estimates of effective climate sensitivity (4.38, 5.21, and 6.06 K), with a corresponding spread in the shortwave cloud feedbacks. These estimates are on the higher end of sensitivity estimates from CMIP ensembles, and the spread indicates substantial state‐dependent feedbacks. These results demonstrate how multiscale modeling framework (MMF) models can be used for climate relevant experiments and projections by leveraging modern GPU enabled computational platforms. The unique qualities of E3SM‐MMF shown in previous literature are largely still present, but various instances of reduced biases suggest that MMF models have utility in improving future projections.

E3SM↗
DOE OSTI2025

A Hybrid Dynamic/Steady-State Tool With Protection Simulation for Cascading-Outage Analysis of Extreme Events in Power Systems

The bulk electric power grid is subject to vulnerabilities from component outages, which in certain combinations (extreme events) might lead to cascading outages. Some of these outages can be severe enough to trigger brownouts and blackouts. Much is known about mitigating the first few failures near the beginning of a cascade, but there are few established methods and tools for directly analyzing the risks of cascading component outages over a longer time scale. Current power system tools have limited ability to perform detailed and accurate cascading-outage analysis, which could be computationally intensive. The Dynamic Contingency Analysis Tool (DCAT) enables power system planning engineers to more realistically assess the consequences of extreme contingencies and potential cascading events across their systems and interconnections. DCAT has several unique features: (i) detailed hybrid dynamic and steady-state analysis of power systems to mimic real-world cascading outages, (ii) detailed modeling of protection systems embedded in the dynamic simulation, (iii) simulation of corrective action after transients, (iv) simulation of islanding , and (v) high-performance computing capability to simulate a large number of contingencies in a reasonable time. DCAT outputs will help find technically sound solutions to reduce the risk of cascading outages. This paper provides details of DCAT methodology and shows its capabilities with extreme events on real-world cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Debiasing with Diffusion: Probabilistic Reconstruction of Dark Matter Fields from Galaxies with CAMELS

Abstract Galaxies are biased tracers of the underlying cosmic web, which is dominated by dark matter (DM) components that cannot be directly observed. Galaxy formation simulations can be used to study the relationship between DM density fields and galaxy distributions. However, this relationship can be sensitive to assumptions in cosmology and astrophysical processes embedded in galaxy formation models, which remain uncertain in many aspects. In this work, we develop a diffusion generative model to reconstruct DM fields from galaxies. The diffusion model is trained on the CAMELS simulation suite that contains thousands of state-of-the-art galaxy formation simulations with varying cosmological parameters and subgrid astrophysics. We demonstrate that the diffusion model can predict the unbiased posterior distribution of the underlying DM fields from the given stellar density fields while being able to marginalize over uncertainties in cosmological and astrophysical models. Interestingly, the model generalizes to simulation volumes ≈500 times larger than those it was trained on and across different galaxy formation models. The code for reproducing these results can be found athttps://github.com/victoriaono/variational-diffusion-cdm✎.

Astronomy & Astrophysics↗
DOE OSTI2026

Quantum solver for single-impurity Anderson models with particle-hole symmetry

Quantum embedding methods, such as dynamical mean-field theory (DMFT), provide a powerful framework for investigating strongly correlated materials. A central computational bottleneck in DMFT is in solving the Anderson impurity model (AIM), whose exact solution is classically intractable for large bath sizes. In this work, we benchmark a quantum-classical hybrid solver tailored for particle-hole symmetric AIMs, using the variational quantum eigensolver to prepare the ground state of the model with shallow quantum circuits. The solver uses shallow quantum ansätze and one set of variational parameters to prepare the ground state and its particle and hole excitations, enabling the construction of the impurity Green’s function through a continued-fraction expansion. We evaluate the performance of this approach across a few bath sizes and interaction strengths under noisy, shot-limited conditions. We compare three optimization routines (COBYLA, Adam, and L-BFGS-B) in terms of convergence and fidelity, assess the benefits of estimating a quantum-computed moment correction to the variational energies, and benchmark the approach by comparing the density of states computed from the impurity Green’s function against that obtained using a classical pipeline. Our results demonstrate the feasibility of Green’s function construction on near-term devices and establish practical benchmarks for quantum impurity solvers embedded within self-consistent DMFT loops.

Karabin, Mariia [ORNL]↗
DOE OSTI2024

Integrating the Design of Desalination Technologies into Produced Water Network Optimization

The oil and gas energy sector uses billions of gallons of water for hydraulic fracturing each year to extract oil and gas. The water injected into the ground for fracturing along with naturally occurring formation water from oil wells surfaces back in the form of produced water. Produced water can contain high concentrations of total dissolved solids and is unfit for reuse outside the oil and gas industry without desalination. In semi-arid shale plays, produced water desalination for beneficial reuse could play a crucial role in alleviating water shortages and addressing extreme drought conditions. In this poster we co-optimize the design and operation of desalination technologies along with operational decisions across produced water networks. A multi-period produced water network model with simplified split-fraction-based desalination nodes is developed. Rigorous steady-state desalination mathematical models based on mechanical vapor recompression are developed and embedded at the desalination sites in the network model. An optimal common design across all periods is ensured using global capacity constraints. The solution approach is demonstrated for multi-period planning problems on networks from the PARETO open-source library. Model formulation and challenges associated with scalability are discussed.

Naik, Sakshi↗
DOE OSTI2024

Integrating the Design of Desalination Technologies into Produced Water Network Optimization

The oil and gas energy sector uses billions of gallons of water for hydraulic fracturing each year to extract oil and gas. The water injected into the ground for fracturing along with naturally occur-ring formation water from the oil wells surfaces back in the form of produced water. Produced water can contain high concentrations of total dissolved solids and is unfit for reuse outside the oil and gas industry without desalination. In semi-arid shale plays, produced water desalination for beneficial reuse could play a crucial role in alleviating water shortages and addressing extreme drought conditions. In this paper we co-optimize the design and operation of desalination technologies along with operational decisions across produced water networks. A multi-period produced water network model with simplified split-fraction-based desalination nodes is developed. Rigorous steady-state desalination mathematical models based on mechanical vapor recompression are developed and embedded at the desalination sites in the network model. An optimal common design is ensured across all periods using global capacity constraints. The solution approach is demonstrated for multi-period planning problems on networks from the PARETO open-source library. Model formulation and challenges associated with scalability are discussed.

Naik, Sakshi↗
DOE OSTI2025

ChemEcho v1.0

ChemEcho is a tool that converts tandem mass spectra into embeddings used to build machine learning (ML) models with fully explainable predictions. It provides an API for transforming raw tandem mass spectral data into embeddings, along with functions for training and validating ML models. Additionally, it includes utilities for retrieving and cleaning training data. ChemEcho is broadly applicable in ML pipelines that use tandem mass spectra for a variety of tasks, such as chemical classification or bioactivity mining. While there are existing methods to generate embeddings from fragmentation data, ChemEcho's approach ensures that predictions remain interpretable, enabling experts to evaluate results and generate hypotheses about the underlying data.

Harwood, Thomas [Lawrence Berkeley National Labora↗
DOE OSTI2026

Leveraging large language models to address data scarcity in machine learning for graphene synthesis

Machine learning in experimental materials science faces significant challenges due to the scarcity of data, which are costly and time-consuming to generate, particularly when relying on in-house experiments. Literature data mining offers a potential solution but introduces issues like mixed data quality, inconsistent formats, and non-uniform reporting of synthesis parameters, resulting in partially missing and heterogeneous features across the dataset. Here, we propose data imputation and feature engineering methods that employ pre-trained large language models (LLMs) to enhance machine learning performance on scarce, heterogeneous datasets, demonstrated on graphene CVD synthesis data and the ML-HydPARK hydrogen storage dataset. GPT models perform data imputation via tailored prompting and semantic normalization of inconsistently reported features through embeddings, for example, to harmonize the complex nomenclature of CVD substrates. Beyond yielding more diverse and richer feature representations than traditional methods such as K-nearest neighbors (KNN) and Multivariate Imputation by Chained Equations (MICE), LLM-based data imputation is evaluated against dataset characteristics and prompting strategies. We vary the level of autonomy granted to the LLM, from generic prompting that leverages pre-trained knowledge for autonomous data generation to data-informed prompting that constrains outputs using target-specific information, and demonstrate which level of autonomy yields superior imputation performance across datasets and feature types. The proposed data engineering methods markedly improve downstream performance; for example, in graphene layer number classification using a support vector machine (SVM), binary accuracy increases from 39% to 65% and ternary accuracy from 52% to 72%. Fine-tuning experiments on both datasets show that combining our proposed LLM-based data imputation and feature encoding methods with numerical machine learning predictors outperforms standalone fine-tuned LLM predictors in data-scarce settings. The proposed strategies emphasize data enhancement techniques rather than refining learning architectures or regularizing loss functions, offering a broadly applicable framework for improving machine learning performance on scarce, inhomogeneous datasets.

Chemical vapor deposition↗
DOE OSTI2026

A Kinetic Model of the Long-Term Corrosion of Glass-Ceramic Materials

Multiphase waste forms show promise for increased waste loading and for the ability to dispose of contaminated solid and particulate waste through direct densification. However, achieving predictive capability for long-term durability of multiphase waste forms, and thus assessing their possible deployment, requires expanding the current, limited knowledge base. Here, we describe the development of a corrosion model of a two-phase waste form consisting of crystals of known volume fraction embedded in a glass matrix. This model accounts for the dissolution of both the crystalline and glass phases as well as the hydration of the glass phase through an ion exchange reaction. Because of the large difference in solubility between the two phases, the reactive surface of the crystalline phase is a function of the extent of dissolution of the glass phase in this model. Model parameterization was performed using corrosion data, such as from single-pass flow-through tests, for the individual phases. The parameterized corrosion model was evaluated against static dissolution test data for a glass-ceramic multiphase waste form. This evaluation demonstrated the model’s ability to reproduce the time-dependent release of key tracers of glass and crystalline phase dissolution. Hence, the development of a kinetic model provides a pathway for long-term durability predictions and thus the use of multiphase waste forms in nuclear cleanup missions.

Kerisit, Sebastien N.↗
DOE OSTI2024

FTL: Transfer Learning Nonlinear Plasma Dynamic Transitions in Low Dimensional Embeddings (FTL) v1.0

Fusion Transfer Learning (FTL) model provides a new paradigm to study high-dimensional dynamical behaviors, such as those in fusion plasma systems. The knowledge transfer process leverages a pre-trained neural encoder-decoder network, initially trained on linear simulations, to effectively capture nonlinear dynamics. The low-dimensional embeddings extract the coherent structures of interest, while preserving the inherent dynamics of the complex system. Experimental results highlight FTL's capacity to capture transitional behaviors and dynamical features in plasma dynamics -- a task often challenging for conventional methods. The model developed in this study is generalizable and can be extended broadly through transfer learning to address various magnetohydrodynamics (MHD) modes.

Bai, Zhe↗
DOE OSTI2025

Finch: Toxicity Dose Response Curve Prediction of Chemical Compounds and Mixtures

A paradigm shift in chemical risk assessment is emphasizing mixture testing over single compound analysis, eliminating animal testing, and adopting advanced modeling approaches to understand mixture activity profiles. However, existing computational models largely focus on single chemicals, with few effective solutions for modeling complex mixtures that account for synergistic or antagonistic effects and multiple Modes of Action (MoA). Conventional methods like concentration addition (CA) and independent action (IA) are insufficient for this task as they are designed for simplistic interactions and struggle to account for the dynamic and multifaceted nature of chemical mixtures, such as overlapping MoA and non-linear interactions. Finch offers a novel approach utilizing deep learning (DL) embeddings and multi-task quantitative structure-activity relationship (QSAR) models to improve chemical exposure prediction. By leveraging molecular descriptors, physiochemical properties, and large language model (LLM) embeddings from SMILES inputs, Finch preserves critical information in a latent space thereby enhancing predictive accuracy. The multi-task learning aspect of Finch is highly advantageous, as it simultaneously optimizes multiple loss functions, leveraging all available data across tasks to develop generalized representations that effectively capture complex ingredient interactions within mixtures.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

RINO: Renormalization Group Invariance with No Labels

A common challenge with supervised machine learning (ML) in high energy physics (HEP) is the reliance on simulations for labeled data, which can often mismodel the underlying collision or detector response. To help mitigate this problem of domain shift, we propose RINO (Renormalization Group Invariance with No Labels), a self-supervised learning approach that can instead pretrain models directly on collision data, learning embeddings invariant to renormalization group flow scales. In this work, we pretrain a transformer-based model on jets originating from quantum chromodynamic (QCD) interactions from the JetClass dataset, emulating real QCD-dominated experimental data, and then finetune on the JetNet dataset -- emulating simulations -- for the task of identifying jets originating from top quark decays. RINO demonstrates improved generalization from the JetNet training data to JetClass data compared to supervised training on JetNet from scratch, demonstrating the potential for RINO pretraining on real collision data followed by fine-tuning on small, high-quality MC datasets, to improve the robustness of ML models in HEP.

Hao, Zichun [Caltech] (ORCID:0000000256244907)↗
DOE OSTI2025

Simulating Atmospheric Processes in Earth System Models and Quantifying Uncertainties With Deep Learning Multi‐Member and Stochastic Parameterizations

Abstract Deep learning is a powerful tool to represent subgrid processes in climate models, but many application cases have so far used idealized settings and deterministic approaches. Here, we develop stochastic parameterizations with calibrated uncertainty quantification to learn subgrid convective and turbulent processes and surface radiative fluxes of a superparameterization embedded in an Earth System Model (ESM). We explore three methods to construct stochastic parameterizations: (a) a single Deep Neural Network (DNN) with Monte Carlo Dropout; (b) a multi‐member parameterization; and (c) a Variational Encoder Decoder with latent space perturbation. We show that the multi‐member parameterization improves the representation of convective processes, especially in the planetary boundary layer, compared to individual DNNs. The respective uncertainty quantification illustrates that methods (b) and (c) are advantageous compared to a dropout‐based DNN parameterization regarding the spread of convective processes. Hybrid simulations with our best‐performing multi‐member parameterizations remained challenging and crash within the first days. Therefore, we develop a pragmatic partial coupling strategy relying on the superparameterization for condensate emulation. Partial coupling reduces the computational efficiency of hybrid Earth‐like simulations but enables model stability over 5 months with our multi‐member parameterizations. However, our hybrid simulations exhibit biases in thermodynamic fields and differences in precipitation patterns. Despite this, the multi‐member parameterizations enable improvements in reproducing tropical extreme precipitation compared to a traditional convection parameterization. Despite these challenges, our results indicate the potential of a new generation of multi‐member machine learning parameterizations leveraging uncertainty quantification to improve the representation of stochasticity of subgrid effects.

Behrens, Gunnar [Deutsches Zentrum für Luft‐ und R↗
DOE OSTI2024

Graph-learning approach to combine multiresolution seismic velocity models

SUMMARY The resolution of velocity models obtained by tomography varies due to multiple factors and variables, such as the inversion approach, ray coverage, data quality, etc. Combining velocity models with different resolutions can enable more accurate ground motion simulations. Toward this goal, we present a novel methodology to fuse multiresolution seismic velocity maps with probabilistic graphical models (PGMs). The PGMs provide segmentation results, corresponding to various velocity intervals, in seismic velocity models with different resolutions. Further, by considering physical information (such as ray path density), we introduce physics-informed probabilistic graphical models (PIPGMs). These models provide data-driven relations between subdomains with low (LR) and high (HR) resolutions. Transferring (segmented) distribution information from the HR regions enhances the details in the LR regions by solving a maximum likelihood problem with prior knowledge from HR models. When updating areas bordering HR and LR regions, a patch-scanning policy is adopted to consider local patterns and avoid sharp boundaries. To evaluate the efficacy of the proposed PGM fusion method, we tested the fusion approach on both a synthetic checkerboard model and a fault zone structure imaged from the 2019 Ridgecrest, CA, earthquake sequence. The Ridgecrest fault zone image consists of a shallow (top 1 km) high-resolution shear-wave velocity model obtained from ambient noise tomography, which is embedded into the coarser Statewide California Earthquake Center Community Velocity Model version S4.26-M01. The model efficacy is underscored by the deviation between observed and calculated traveltimes along the boundaries between HR and LR regions, 38 per cent less than obtained by conventional Gaussian interpolation. The proposed PGM fusion method can merge any gridded multiresolution velocity model, a valuable tool for computational seismology and ground motion estimation.

Geochemistry & Geophysics↗