Uncertainty Quantification and Calibration of the E3SM Land Model using an Embedded Model Error Approach
Explore the source record for details and available documents.
Iyengar, Nikhil [Sandia National Laboratories (SNL↗
SEARCH · Engineering Papers
Results for “embedding model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Intern Poster: Reducing AI Hallucination Using Instruction Tuned Embedding Models
Analysis on the effectiveness of instruction tuned embedding models for the reduction of Artificial Intelligence hallucinations compared to the standard embedding models.
97 MATHEMATICS AND COMPUTING↗
PhysBERT: A text embedding model for physics scientific literature
The specialized language and complex concepts in physics pose significant challenges for information extraction through Natural Language Processing (NLP). Central to effective NLP applications is the text embedding model, which converts text into dense vector representations for efficient information retrieval and semantic analysis. In this work, we introduce PhysBERT, the first physics-specific text embedding model. Pre-trained on a curated corpus of 1.2 × 106 arXiv physics papers and fine-tuned with supervised data, PhysBERT outperforms leading general-purpose models on physics-specific tasks, including the effectiveness in fine-tuning for specific physics subdomains.
Hellert, Thorsten (ORCID:0000000227970926)↗
Domain-specific text embedding model for accelerator physics
Accelerator physics presents unique challenges for natural language processing (NLP) due to its specialized terminology and complex concepts. A key component in overcoming these challenges is the development of robust text embedding models that transform textual data into dense vector representations, facilitating efficient information retrieval and semantic understanding. In this work, we introduce AccPhysBERT, a sentence embedding model fine-tuned specifically for accelerator physics. Our model demonstrates superior performance across a range of downstream NLP tasks, surpassing existing models in capturing the domain-specific nuances of the field. We further showcase its practical applications, including semantic paper-reviewer matching and integration into retrieval-augmented generation systems, highlighting its potential to enhance information retrieval and knowledge discovery in accelerator physics. Published by the American Physical Society 2025
Hellert, Thorsten (ORCID:0000000227970926)↗
Data and Code for Understanding Generative AI Content with Embedding Models
This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).
Vargas, Max [Pacific Northwest National Laboratory↗
Logic flowgraph methodology - A tool for modeling embedded systems
The logic flowgraph methodology (LFM), a method for modeling hardware in terms of its process parameters, has been extended to form an analytical tool for the analysis of integrated (hardware/software) embedded systems. In the software part of a given embedded system model, timing and the control flow among different software components are modeled by augmenting LFM with modified Petrinet structures. The objective of the use of such an augmented LFM model is to uncover possible errors and the potential for unanticipated software/hardware interactions. This is done by backtracking through the augmented LFM mode according to established procedures which allow the semiautomated construction of fault trees for any chosen state of the embedded system (top event). These fault trees, in turn, produce the possible combinations of lower-level states (events) that may lead to the top event.
Muthukumar, C. T.↗
Understanding Generative AI Content with Embedding Models
The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.
Vargas, Max↗
Symbolic diagnostics to interpret and analyze neural network models
Embedded machine-learned models (EMLMs) have the promise to improve the predictive accuracy of engineering simulators in environments of national interest. EMLMs often comprise complex input-output maps (e.g., neural networks), which make them unamenable to rigorous analysis and generally difficult to interpret. In the face of decades of theory, this lack of interpretability is a significant barrier to building confidence in these models. This work outlines an approach to interpret EMLMs using sparse polynomial regression for comparison with theoretical understanding. To do so, we build on the concept of Locally Interpretable Model-agnostic Explanations (LIME) using physics-informed clustering, prototype selection, and library construction. While general, we demonstrate our method on tensor-basis neural networks used in Reynolds-Averaged Navier-Stokes simulations of hypersonic fluid flows. Results are presented for a simulated toy model and for direct numerical simulations (DNS) of turbulent flows over a flat plate.
97 MATHEMATICS AND COMPUTING↗
Explaining word embeddings with perfect fidelity: a case study in predicting research impact
The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.
Coarse-grained models↗
The embedded feature model for the interpretation of chromospheric contrast profiles
Contrast profiles obtained from chromospheric filtergrams and spectra of bright and dark mottles have to date been interpreted almost exclusively in terms of Becker's cloud model. Here we demonstrate the failure of this model to account in a physically consistent way for the observed contrasts. As an alternative, we introduce an embedded-feature model, restricting our discussion in this paper to stationary features. Our model is then characterized by three independent parameters: the density of absorbing atoms, the geometrical depth, and the profile of the absorption coefficient. An analytic approximation to the contrast resulting from such a model reproduces well the observed behavior of all types of contrast profiles.
Steinitz, R.↗
Observational constraints on an embedded cloud model for the soft X-ray diffuse background
The reason for the observed anticorrelation between soft X-ray diffuse background (SXRB) and neutral hydrogen column densities are considered. Using the B and C band data from an all-sky survey of the SXRB, as well as data in the Be band, it is shown that the Jakobsen and Kahn (1986) embedded cloud model, in which several statistical properties of the SXRB in the B and C bands were used to constrain their model parameters, cannot fit the observed band ratios, except in the limit in which virtually all of the emission originates in the near side of the absorbing gas. It was also shown that model parameters corresponding to appreciably mixed emission and absorption, as proposed by Bunner et al. (1969) and Davidsen et al. (1972), do not fit the data. Model parameters corresponding to the absorption models of Bowyer et al. (1968), Bunner et al. (1969), Davidsen et al. (1972), and Yentis et al. (1972), cannot fit the data unless the absorbing material is more strongly clumped than is permitted by 21-cm observations.
Burrows, D. N.↗
Dispatch Manager for NEML2 Constitutive Model Calculations Embedded in MOOSE
This report describes the extended capabilities of the NEML2 constitutive modeling library, including a flexible and efficient work dispatching system designed to leverage both CPU and GPU resources. This enhancement addresses one of the primary computational challenges in large-scale simulations: the ability to distribute and execute batches of material model evaluations across heterogeneous computing devices. The new dispatch system introduces a modular set of dispatcher and scheduler classes that coordinate the flow of data and execution between devices. The dispatcher is responsible for efficiently packaging work, managing device-specific memory operations, and synchronizing results. This modularity allows for extensibility, making it straightforward to integrate additional computing backends in the future. From an implementation standpoint, the dispatcher system interfaces seamlessly with NEML2's existing models. They handle device-aware tensor operations, optimize memory transfers, and support asynchronous execution when applicable. This design ensures that batches of material points can be evaluated concurrently, substantially improving throughput compared to previous single-device or serial implementations. These improvements not only enhance the raw performance of NEML2 but also improve its usability in multiscale and high-fidelity simulations, where the simultaneous evaluation of large material point batches is critical. Benchmarks included in the report demonstrate the system’s scalability, highlighting its effectiveness when leveraging modern GPU architectures.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
A liquid drop model for embedded atom method cluster energies
Minimum energy configurations for homonuclear clusters containing from two to twenty-two atoms of six metals, Ag, Au, Cu, Ni, Pd, and Pt have been calculated using the Embedded Atom Method (EAM). The average energy per atom as a function of cluster size has been fit to a liquid drop model, giving estimates of the surface and curvature energies. The liquid drop model gives a good representation of the relationship between average energy and cluster size. As a test the resulting surface energies are compared to EAM surface energy calculations for various low-index crystal faces with reasonable agreement.
Finley, C. W.↗
Group-Additivity–Embedded Multiscale Modeling for Electric Field-Enhanced Nanocatalysis
Elucidating structure-performance relationships remains a central challenge in field-enhanced catalysis, where nanoparticles exhibit nonuniform surface sites with site-dependent responses to electric fields. Low-coordination sites (edges, corners, and tips) are particularly electric field-sensitive (EF), leading to nonuniform charge distribution, adsorption energies, and catalytic activity. Here, using ammonia decomposition on a ruthenium cluster as a model system, we develop a transferable multiscale framework integrating density functional theory, group additivity (GA), Brønsted-Evans-Polanyi scaling, and microkinetic modeling to predict EF-dependent activity across nonuniform cluster sites. Across sites and fields, the nitrogen adsorption energy (E N ) emerges as the governing descriptor, yielding robust volcano relationships whose optimum shifts systematically with field: negative fields strengthen N binding via electron accumulation, while positive fields weaken N binding via charge depletion, moving the optimal E N toward weaker binding. Microkinetic analysis shows that N≡N bond formation remains the key kinetic bottleneck over most conditions; positive fields lower the effective barrier and, critically, increase the fraction of near-optimal active sites, leading to a net enhancement in overall activity relative to zero-field and negative-field cases. By capturing EF- and site-dependent energetics with high accuracy and low computational cost, this GA-embedded multi-scale simulation workflow provides a physically interpretable route to predict and design field-enhanced nanocatalysis.
ammonia decomposition↗
Impact of the continuum on the γ decay of the lowest 2 + states in 14 C
The half-life, T 1/2 =14.6(33) fs, of the 7012-keV 2$^{+}_{1}$ state in 14 C was measured in an experiment employing the 9 Be( 6 Li,pγ) fusion-evaporation reaction and the GODDESS setup. The γ decay from the 2$^{+}_{2}$ near-threshold resonance, located 142 keV above the 8176-keV neutron-emission threshold, was also investigated. An upper limit of 4.0 × 10 –5 was established for the γ-decay branching ratio with respect to the neutron-decay channel. The B(E2) transition probabilities for these 2$^{+}_{1}$ and 2$^{+}_{2}$ states were compared to predictions from the Shell Model Embedded in the Continuum (SMEC). Significant modifications of these B(E2) probabilities, with respect to the standard shell model, are expected due to the coupling to the continuum. For calculations using the YSOX interaction, agreement was found for large negative values of V 0 , the coupling constant to the continuum. The central value V 0 = −645 MeV fm 3 results in a retardation by a factor ∼ 3.5 for the 2$^{+}_{1}$ → 0$^{+}_{1}$ transition, and an enhanced probability by a factor ∼ 2.5 for the γ-decay out of the 2$^{+}_{2}$ state. The latter factor reflects the effect of collectivization of the 2$^{+}_{2}$ excitation when the coupling to the continuum is taken into account.
14C↗
An unsupervised machine learning based approach to identify efficient spin-orbit torque materials
Materials with large spin–orbit torque (SOT) hold considerable significance for many spintronic applications because of their potential for energy-efficient magnetization switching. Unfortunately, most of the existing materials exhibit an SOT efficiency factor that is much less than unity, requiring a large current for magnetization switching. The search for new materials that can exhibit an SOT efficiency much greater than unity is a topic of active research, and only a few such materials have been identified using conventional approaches. In this paper, we present a machine learning-based approach using a word embedding model that can identify new results by deciphering non-trivial correlations among various items in a specialized scientific text corpus. We show that such a model can be used to identify materials likely to exhibit high SOT and rank them according to their expected SOT strengths. The model captured the essential spintronics knowledge embedded in scientific abstracts within various materials science, physics, and engineering journals and identified 97 new materials to exhibit high SOT. Among them, 16 candidate materials are expected to exhibit an SOT efficiency greater than unity, and one of them has recently been confirmed with experiments with quantitative agreement with the model prediction.
Sayed, Shehrin↗
Beyond Point Estimates: Benchmarking Uncertainty Quantification Methods on the AION-1 Astronomical Foundation Model
Foundation models for astronomical surveys offer powerful learned representations that can be transferred to downstream regression tasks such as galaxy property estimation. However, point predictions alone are insufficient for scientific inference; reliable uncertainty quantification (UQ) is essential. We compare seven UQ methods on galaxy property regression using frozen AION-1 foundation-model embeddings, predicting redshift, stellar mass, stellar-population age, gas-phase metallicity, and specific star-formation rate, from Legacy Survey photometry/imaging and DESI spectra, with PROVABGS-derived labels. Distribution-free conformal methods achieve marginal coverage within $\sim$1 pp of the nominal 90% across all properties, while non-conformal baselines (Deep Ensembles, MC~Dropout) fail to calibrate reliably. Among conformal approaches, Conformalized Quantile Regression (CQR) delivers the best coverage in the bin with the poorest model predictions. More importantly, only the Locally Valid and Discriminative (LVD) framework -- particularly when operating on AION-1 embeddings -- also provides finite-sample \emph{local validity}, producing intervals that adapt to each galaxy's local prediction difficulty rather than relying on marginal guarantees alone. These results establish conformal prediction, and LVD in particular, as the preferred UQ framework for uncertainty-aware inference on foundation-model embeddings in astrophysics.
Tame-Narvaez, Karla [Fermilab] (ORCID:000000022249↗
Global Circulation and Impact of Plasmaspheric Plumes
We report results from the global circulation model of Lyon, Fedder, and Mobarry with an embedded model of the inner magnetosphere including the plasmasphere. The combination is used to initiate large numbers of representative protons on the geosynchronous orbit L shell, to assign particle weightings, to track their: subsequent trajectories in the 3D fields. This permits us to study the global circulation of plasmaspheric plumes and to compare these with Polar observations from the dayside magnetopause region . A range of events is studied from an isolated period of SBz in the solar wind,to a large storm sequence. We consider effects on circulating plasma reaching the dayside reconnection X-line, the population of the plasma sheet with ionospheric protons and the generation of ring current pressure from this source, compared with solar wind, polar wind, and auroral wind sources. We find that the transient plasmaspheric plume source is large in terms of total fluence, but of modest proportions in terms of contribution to the ring current. Implications of this and other results for improved space weather modeling and prediction will be discussed.
Moore, Thomas E.↗