Uncertainty Quantification and Calibration of the E3SM Land Model using an Embedded Model Error Approach
Explore the source record for details and available documents.
Iyengar, Nikhil [Sandia National Laboratories (SNL↗
SEARCH · Engineering Papers
Results for “embedding model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Intern Poster: Reducing AI Hallucination Using Instruction Tuned Embedding Models
Analysis on the effectiveness of instruction tuned embedding models for the reduction of Artificial Intelligence hallucinations compared to the standard embedding models.
97 MATHEMATICS AND COMPUTING↗
PhysBERT: A text embedding model for physics scientific literature
The specialized language and complex concepts in physics pose significant challenges for information extraction through Natural Language Processing (NLP). Central to effective NLP applications is the text embedding model, which converts text into dense vector representations for efficient information retrieval and semantic analysis. In this work, we introduce PhysBERT, the first physics-specific text embedding model. Pre-trained on a curated corpus of 1.2 × 106 arXiv physics papers and fine-tuned with supervised data, PhysBERT outperforms leading general-purpose models on physics-specific tasks, including the effectiveness in fine-tuning for specific physics subdomains.
Hellert, Thorsten (ORCID:0000000227970926)↗
Domain-specific text embedding model for accelerator physics
Accelerator physics presents unique challenges for natural language processing (NLP) due to its specialized terminology and complex concepts. A key component in overcoming these challenges is the development of robust text embedding models that transform textual data into dense vector representations, facilitating efficient information retrieval and semantic understanding. In this work, we introduce AccPhysBERT, a sentence embedding model fine-tuned specifically for accelerator physics. Our model demonstrates superior performance across a range of downstream NLP tasks, surpassing existing models in capturing the domain-specific nuances of the field. We further showcase its practical applications, including semantic paper-reviewer matching and integration into retrieval-augmented generation systems, highlighting its potential to enhance information retrieval and knowledge discovery in accelerator physics. Published by the American Physical Society 2025
Hellert, Thorsten (ORCID:0000000227970926)↗
Data and Code for Understanding Generative AI Content with Embedding Models
This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).
Vargas, Max [Pacific Northwest National Laboratory↗
Understanding Generative AI Content with Embedding Models
The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.
Vargas, Max↗
Symbolic diagnostics to interpret and analyze neural network models
Embedded machine-learned models (EMLMs) have the promise to improve the predictive accuracy of engineering simulators in environments of national interest. EMLMs often comprise complex input-output maps (e.g., neural networks), which make them unamenable to rigorous analysis and generally difficult to interpret. In the face of decades of theory, this lack of interpretability is a significant barrier to building confidence in these models. This work outlines an approach to interpret EMLMs using sparse polynomial regression for comparison with theoretical understanding. To do so, we build on the concept of Locally Interpretable Model-agnostic Explanations (LIME) using physics-informed clustering, prototype selection, and library construction. While general, we demonstrate our method on tensor-basis neural networks used in Reynolds-Averaged Navier-Stokes simulations of hypersonic fluid flows. Results are presented for a simulated toy model and for direct numerical simulations (DNS) of turbulent flows over a flat plate.
97 MATHEMATICS AND COMPUTING↗
Explaining word embeddings with perfect fidelity: a case study in predicting research impact
The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.
Coarse-grained models↗
Dispatch Manager for NEML2 Constitutive Model Calculations Embedded in MOOSE
This report describes the extended capabilities of the NEML2 constitutive modeling library, including a flexible and efficient work dispatching system designed to leverage both CPU and GPU resources. This enhancement addresses one of the primary computational challenges in large-scale simulations: the ability to distribute and execute batches of material model evaluations across heterogeneous computing devices. The new dispatch system introduces a modular set of dispatcher and scheduler classes that coordinate the flow of data and execution between devices. The dispatcher is responsible for efficiently packaging work, managing device-specific memory operations, and synchronizing results. This modularity allows for extensibility, making it straightforward to integrate additional computing backends in the future. From an implementation standpoint, the dispatcher system interfaces seamlessly with NEML2's existing models. They handle device-aware tensor operations, optimize memory transfers, and support asynchronous execution when applicable. This design ensures that batches of material points can be evaluated concurrently, substantially improving throughput compared to previous single-device or serial implementations. These improvements not only enhance the raw performance of NEML2 but also improve its usability in multiscale and high-fidelity simulations, where the simultaneous evaluation of large material point batches is critical. Benchmarks included in the report demonstrate the system’s scalability, highlighting its effectiveness when leveraging modern GPU architectures.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Group-Additivity–Embedded Multiscale Modeling for Electric Field-Enhanced Nanocatalysis
Elucidating structure-performance relationships remains a central challenge in field-enhanced catalysis, where nanoparticles exhibit nonuniform surface sites with site-dependent responses to electric fields. Low-coordination sites (edges, corners, and tips) are particularly electric field-sensitive (EF), leading to nonuniform charge distribution, adsorption energies, and catalytic activity. Here, using ammonia decomposition on a ruthenium cluster as a model system, we develop a transferable multiscale framework integrating density functional theory, group additivity (GA), Brønsted-Evans-Polanyi scaling, and microkinetic modeling to predict EF-dependent activity across nonuniform cluster sites. Across sites and fields, the nitrogen adsorption energy (E N ) emerges as the governing descriptor, yielding robust volcano relationships whose optimum shifts systematically with field: negative fields strengthen N binding via electron accumulation, while positive fields weaken N binding via charge depletion, moving the optimal E N toward weaker binding. Microkinetic analysis shows that N≡N bond formation remains the key kinetic bottleneck over most conditions; positive fields lower the effective barrier and, critically, increase the fraction of near-optimal active sites, leading to a net enhancement in overall activity relative to zero-field and negative-field cases. By capturing EF- and site-dependent energetics with high accuracy and low computational cost, this GA-embedded multi-scale simulation workflow provides a physically interpretable route to predict and design field-enhanced nanocatalysis.
ammonia decomposition↗
Impact of the continuum on the γ decay of the lowest 2 + states in 14 C
The half-life, T 1/2 =14.6(33) fs, of the 7012-keV 2$^{+}_{1}$ state in 14 C was measured in an experiment employing the 9 Be( 6 Li,pγ) fusion-evaporation reaction and the GODDESS setup. The γ decay from the 2$^{+}_{2}$ near-threshold resonance, located 142 keV above the 8176-keV neutron-emission threshold, was also investigated. An upper limit of 4.0 × 10 –5 was established for the γ-decay branching ratio with respect to the neutron-decay channel. The B(E2) transition probabilities for these 2$^{+}_{1}$ and 2$^{+}_{2}$ states were compared to predictions from the Shell Model Embedded in the Continuum (SMEC). Significant modifications of these B(E2) probabilities, with respect to the standard shell model, are expected due to the coupling to the continuum. For calculations using the YSOX interaction, agreement was found for large negative values of V 0 , the coupling constant to the continuum. The central value V 0 = −645 MeV fm 3 results in a retardation by a factor ∼ 3.5 for the 2$^{+}_{1}$ → 0$^{+}_{1}$ transition, and an enhanced probability by a factor ∼ 2.5 for the γ-decay out of the 2$^{+}_{2}$ state. The latter factor reflects the effect of collectivization of the 2$^{+}_{2}$ excitation when the coupling to the continuum is taken into account.
14C↗
An unsupervised machine learning based approach to identify efficient spin-orbit torque materials
Materials with large spin–orbit torque (SOT) hold considerable significance for many spintronic applications because of their potential for energy-efficient magnetization switching. Unfortunately, most of the existing materials exhibit an SOT efficiency factor that is much less than unity, requiring a large current for magnetization switching. The search for new materials that can exhibit an SOT efficiency much greater than unity is a topic of active research, and only a few such materials have been identified using conventional approaches. In this paper, we present a machine learning-based approach using a word embedding model that can identify new results by deciphering non-trivial correlations among various items in a specialized scientific text corpus. We show that such a model can be used to identify materials likely to exhibit high SOT and rank them according to their expected SOT strengths. The model captured the essential spintronics knowledge embedded in scientific abstracts within various materials science, physics, and engineering journals and identified 97 new materials to exhibit high SOT. Among them, 16 candidate materials are expected to exhibit an SOT efficiency greater than unity, and one of them has recently been confirmed with experiments with quantitative agreement with the model prediction.
Sayed, Shehrin↗
Beyond Point Estimates: Benchmarking Uncertainty Quantification Methods on the AION-1 Astronomical Foundation Model
Foundation models for astronomical surveys offer powerful learned representations that can be transferred to downstream regression tasks such as galaxy property estimation. However, point predictions alone are insufficient for scientific inference; reliable uncertainty quantification (UQ) is essential. We compare seven UQ methods on galaxy property regression using frozen AION-1 foundation-model embeddings, predicting redshift, stellar mass, stellar-population age, gas-phase metallicity, and specific star-formation rate, from Legacy Survey photometry/imaging and DESI spectra, with PROVABGS-derived labels. Distribution-free conformal methods achieve marginal coverage within $\sim$1 pp of the nominal 90% across all properties, while non-conformal baselines (Deep Ensembles, MC~Dropout) fail to calibrate reliably. Among conformal approaches, Conformalized Quantile Regression (CQR) delivers the best coverage in the bin with the poorest model predictions. More importantly, only the Locally Valid and Discriminative (LVD) framework -- particularly when operating on AION-1 embeddings -- also provides finite-sample \emph{local validity}, producing intervals that adapt to each galaxy's local prediction difficulty rather than relying on marginal guarantees alone. These results establish conformal prediction, and LVD in particular, as the preferred UQ framework for uncertainty-aware inference on foundation-model embeddings in astrophysics.
Tame-Narvaez, Karla [Fermilab] (ORCID:000000022249↗
model-of-perception
Algorithms for perceptually grounded evaluation of scientific visualizations, using neural reconstruction and embedding models to quantify clarity and interpretability.
Bujack, Roxana↗
Machine learning prediction of enzyme optimum pH
The relationship between pH and enzyme catalytic activity, especially the optimal pH (pH opt ) at which enzymes function, is critical for biotechnological applications. Hence, computational methods to predict pH opt will enhance enzyme discovery and design by facilitating accurate identification of enzymes that function optimally at specific pH levels, and by elucidating sequence-function relationships. Here, in this study, we proposed and evaluated various machine learning methods for predicting pH opt , conducting extensive hyperparameter optimization and training over 11,000 model instances. Our results demonstrate that models utilizing language model embeddings markedly outperform other methods in predicting pHopt. We present EpHod, the best-performing model, to predict pHopt, making it publicly available to researchers. From sequence data, EpHod directly learns structural and biophysical features that relate to pH opt , including proximity of residues to the catalytic centre and the accessibility of solvent molecules. Overall, EpHod presents a promising advancement in pH opt prediction and will potentially speed up the development of enzyme technologies.
97 MATHEMATICS AND COMPUTING↗
Plastics Environmental Risk Calculator
SF-24-074 The Plastics Environmental Risk Calculator (PERC) was developed in Microsoft Excel and estimates the environmental distribution and lifetime of new biobased and conventional plastics from commonly measured properties of the plastics. A Random Forest regression model embedded in the calculator calculates plastic degradation rates and lifetimes as a proxy for environmental risk. Model default assumptions may be overwritten by the user.
Beckman, Kevin [Argonne National Laboratory (ANL),↗
Processing MCNP Elemental Edit Outputs
The Monte Carlo N-Particle (MCNP) transport code version 6 (also known as MCNP6) has the capability for tracking particles on unstructured mesh (UM) geometry models embedded into constructive solid geometry (CSG) cells. A UM geometry is a collection of elements representing a solid geometry. The first step of MCNP UM modeling is using other software packages to create a finite element mesh representation of a solid 3D geometry. Computer-aided design (CAD) or computer-aided manufacturing (CAM) software is typically used to create a solid geometry model, which is later imported into mesh generation software to create a UM model. The MCNP UM feature was originally designed for models generated by the Abaqus/CAE software. The MCNP code version 6.0 and later can process UM models formatted as Abaqus input files. MCNP can process a UM model consisting of several different element types including linear tetrahedral or hexahedral elements and calculate quantities of interest such as flux and energy deposition at elements. An MCNP UM simulation provides high-fidelity elemental edit (i.e., tally) outputs, which can be further used in multiphysics calculations. The MCNP UM feature was used for multiphysics simulations where quantities of interest calculated by MCNP are used as inputs for heat transfer calculations in Abaqus. MCNP6.3 can produce two types of elemental edit output (EEOUT) file formats: ASCII and HDF5. An EEOUT file type must be requested on an EMBED card while output type (flux or energy deposition) must be requested on an EMBEE card. We wrote Python3 scripts to extract energy deposition values in an ASCII or HDF5 EEOUT file and compute a heat flux profile for an Abaqus heat transfer calculation.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Convergent Manufacturing of Large-Scale Components for Nuclear Applications, via Additive Manufacturing and Powder Metallurgy Hot Isostatic Pressing
Powder metallurgy (PM)–hot isostatic pressing (PM-HIP) has long been recognized as a powerful route for producing fully dense, near net shape metallic components. By consolidating powders under high temperature and pressure, HIP provides isotropic properties, uniform microstructures, and scalability to complex geometries that are vital for sectors such as aerospace, energy, and nuclear power. Yet despite these advantages, the technology has remained constrained by costly trial and error canister fabrication, limitations of conventional forging, and incomplete knowledge about how the canister design influences final part properties. Additive manufacturing (AM), by contrast, thrives on design freedom and geometric flexibility but struggles with speed, scalability, and cost when applied to very large structures. The research presented in this report investigated how a convergent manufacturing approach, combining AM with PM-HIP, can merge the strengths of both technologies, leveraging AM’s flexibility for canister design and HIP’s consolidation capability to deliver reliable, large, and complex parts. The work progressed through three case studies that built on one another in scale and complexity. Small cylindrical canisters fabricated by conventional methods, laser powder bed fusion, and directed energy deposition were filled with stainless steel powders and subjected to HIP. The resulting parts demonstrated near-full density and mechanical properties on par with wrought stainless steel, showing for the first time that AM canisters can be a direct substitute for conventional ones without sacrificing quality. The next step involved a medium-scale, noncentrosymmetric T-valve, which is an enclosed, multibranch geometry that tested the limits of AM + PM-HIP integration. The T-valve achieved predictable shrinkage and uniform densification, confirming feasibility for enclosed designs. However, this study also revealed oxide inclusions and interfacial challenges at the AM + HIP boundary, underscoring the critical importance of controlling interface chemistry and employing robust, in situ strategies, such as melt pool monitoring and thermal monitoring, coupled with nondestructive evaluation techniques such as x-ray computed tomography. Finally, the effort culminated in fabricating a large-scale impeller weighing nearly 2000 lb and spanning 5 ft in diameter. Produced via multirobot wire arc AM and hot isostatic pressed to near-full density, the impeller validated industrial-scale feasibility. Predictive models closely matched experimental shrinkage, tensile properties were spatially uniform across the component, and the AM + PM-HIP interface proved mechanically sound despite the presence of oxide-decorated prior particle boundaries. This large-scale demonstration is a major milestone, showing that hybrid AM + PM‑HIP can reliably deliver components at reactor-relevant scales. Collectively, these studies charted a logical pathway: small-scale work built scientific confidence, medium-scale work highlighted opportunities and challenges, and large-scale work proved industrial impact. The overarching conclusion of this report is that AM + PM-HIP should not be seen as a replacement for forging but as a complementary pathway that provides the US with flexibility, resilience, and new options for manufacturing nuclear-grade components. Looking ahead, several directions emerge as critical to sustaining progress. Predictive modeling must become faster, more accessible, and more accurate, with digital twins and machine learning reducing reliance on trial and error. Powders and alloys must be optimized for HIP, with improved cleanliness, reduced oxides, and tailored chemistries that enhance creep, fatigue, and irradiation resistance. Interfaces between AM and HIP regions must be better engineered through coatings, machining strategies, and surface treatments to mitigate oxide formation and ensure reliable bonding to explore opportunities for HIP of targeted compositional parts, as well as multimaterial HIP cladding applications. Monitoring and nondestructive evaluation need to expand, incorporating multimodal sensors, x-ray computed tomography, and real-time data integration through platforms such as Pelican. At the same time, the pathway to industrial adoption requires techno-economic analysis, machinability studies, and qualification frameworks aligned with industry and regulatory standards. Finally, workforce and academic engagement must be strengthened. Programs that train technicians and engineers for US Navy and US Department of Energy manufacturing challenges should be paired with academic partnerships to support fundamental research, with open sharing of non-export-controlled data to accelerate innovation and build the next generation of experts. In conclusion, this report demonstrates that hybrid AM + PM-HIP is scientifically viable and strategically important. By combining the design agility of AM with the consolidation strength of HIP and embedding modeling, monitoring, and workforce development, this approach provided a transformative new capability for US manufacturing. The path forward is clear: hybrid AM + PM-HIP is not just a promising research direction but is also potentially an industrially relevant pathway that can reshape how nuclear-grade components are designed, qualified, and deployed.
36 MATERIALS SCIENCE↗