Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “embedding model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Masked Symbol Modeling for Demodulation of Oversampled Baseband Communication Signals in Impulsive Noise-Dominated Channels

Recent breakthroughs in natural language processing show that attention mech- anism in Transformer networks, trained via masked-token prediction, enables models to capture the semantic context of the tokens and internalize the grammar of language. While the application of Transformers to communication systems is a burgeoning field, the notion of context within physical waveforms remains under-explored. This paper addresses that gap by re-examining inter-symbol con- tribution (ISC) caused by pulse-shaping overlap. Rather than treating ISC as a nuisance, we view it as a deterministic source of contextual information embedded in oversampled complex baseband signals. We propose Masked Symbol Model- ing (MSM), a framework for the physical (PHY) layer inspired by Bidirectional Encoder Representations from Transformers methodology. In MSM, a subset of symbol-aligned samples is randomly masked, and a Transformer predicts the missing symbol identifiers using the surrounding “in-between” samples. Through this objective, the model learns the latent syntax of complex baseband waveforms. We illustrate MSM’s potential by applying it to the task of demodulating sig- nals corrupted by impulsive noise, where the model infers corrupted segments by leveraging the learned context. Our results suggest a path toward receivers that interpret, rather than merely detect communication signals, opening new avenues for context-aware PHY layer design.

Bedir, Oguz↗
DOE OSTI2024

Thermal partition function of $$ {J}_3{\overline{J}}_3 $$ deformed AdS3

Abstract We derive a compact formula for the one-loop, bosonic string partition function of Euclideanized$$ {J}_3{\overline{J}}_3 $$ J 3 J ¯ 3 deformedAdS 3 with periodic Euclidean time as an integral transform of the partition function of the undeformed EuclideanizedAdS 3 . Such a deformation is interpretable as an irrelevant “single-trace$$ T\overline{T} $$ T T ¯ deformation” of the boundary. We will do this by first establishing a formal procedure to compute a worldsheet torus zero point function for an exactly marginal$$ J\overline{J} $$ J J ¯ deformation of a sigma model with U(1) L × U(1) R global symmetry. We then describe how this procedure is implemented on SL(2,R) sigma model and its Euclidean continuation. Finally, we describe the embedding of the deformed SL(2,R) torus amplitude into critical string theory and interpret the result as the leading perturbative contribution to the thermal partition function of the deformed theory.

Physics↗
DOE OSTI2025

Field space geometry and nonlinear supersymmetry

We propose a geometric formulation of effective field theories via nonlinear supersymmetry. Nonsupersymmetric particles are embedded in constrained superfields governed by a nonlinear σ model, and operators are collected into potentials on the target space. The use of chiral superfields standardizes the treatment of flavor across scalars and fermions, and the minimal jet bundle extension makes invariance under derivative field redefinitions manifest. Published by the American Physical Society 2025

Lee, Yu-Tse (ORCID:0009000190324206)↗
DOE OSTI2024

Semantic Stealth: Crafting Covert Adversarial Patches for Sentiment Classifiers Using Large Language Models

Deep learning models have been shown to be vulnerable to adversarial attacks, in which perturbations to their inputs cause the model to produce incorrect predictions. As opposed to adversarial attacks in computer vision, where small changes introduced to pixel values can drastically alter a model's output while remaining imperceptible to humans, text-based attacks are difficult to conceal due to the discrete nature of tokens. Consequently, unconstrained gradient-based attacks often produce adversarial examples that lack semantic meaning, rendering them detectable through visual inspection or perplexity filters. In contrast to methods that rely on gradient-based optimization in the embedding space, we propose an approach that leverages a Large Language Model's ability to generate grammatically correct and semantically meaningful text to craft adversarial patches that seamlessly blend in with the original input text. These patches can be used to alter the behavior of a target model, such as a text classifier. Since our approach does not rely on gradient backpropagation, it only requires access to the target model's confidence scores, making it a grey-box attack. We demonstrate the feasibility of our approach using open-source LLMs, including Intel's Neural Chat, Llama2, and Mistral-Instruct, to generate adversarial patches capable of altering the predictions of a distilBERT model fine-tuned on the IMDB reviews dataset for sentiment classification.

Roa Carvajal, Maria↗
DOE OSTI2026

Hybrid Quantum–Classical Graph Transformers for Efficient Sentiment Analysis

Quantum Machine Learning (QML) offers a promising paradigm that leverages quantum computing principles to develop efficient and expressive models for learning from complex and structured data. Recent advances in natural language processing (NLP) and artificial intelligence (AI) have demonstrated capabilities in understanding, generating, and reasoning over linguistic and multimodal information. In this work, we present the Quantum Graph Transformer (QGT), a hybrid quantum–classical architecture that extends graph transformer capabilities through quantum self-attention. The QGT models variable-length sentences as token graphs, where both the embedding encoding and the self-attention mechanisms are implemented using parameterized quantum circuits (PQCs), enabling efficient contextual learning with significantly fewer trainable parameters. We train QGT using both fully connected and 𝑘 -nearest-neighbor graph structures and evaluate it on five benchmark sentiment-classification datasets. Experimental results show that QGT consistently achieves higher or comparable accuracy to existing quantum NLP models and outperforms a Classical Graph Transformer (CGT) baseline with identical architecture, achieving 29.4 × fewer parameters while requiring 3–5 × fewer samples to reach comparable performance. These findings highlight the potential of graph-based quantum models as scalable and data-efficient architectures for natural language understanding.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Leveraging large language models to automate the identification of healthcare access barriers for veterans

Objective: To develop and evaluate an automated system for identifying healthcare barriers focusing on transportation issues in veterans’ clinical notes using large language models (LLMs) and to assess the impact of different prompting strategies on classification performance and explanation consistency. Methods: We developed a hybrid system combining pattern matching for templated notes with LLM analysis for free-text notes. Using 2000 manually annotated clinical notes, we compared four prompting strategies (dual-role short, dual-role long, analysis-first, analysis-only) across Mistral-7B and Llama-3.1 models. We evaluated classification performance using standard metrics and assessed explanation consistency through embedding similarity analysis. Results: The analysis-first strategy achieved superior performance, with Mistral-7B reaching an F1 score of 0.914, outperforming traditional machine learning approaches (GBM: 0.786, BERT: 0.811). LLMs demonstrated higher explanation consistency within models (mean cosine similarity 0.887–0.908) compared to cross-model similarities (0.767–0.872). Pattern matching successfully handled 6.7% of templated notes deterministically. Mistral-7B showed greater internal consistency but higher abstention rates compared to Llama-3.1. Conclusion: Requiring LLMs to analyze evidence before classification improves both accuracy and explanation consistency for identifying transportation barriers in clinical notes. This approach enables automated barrier detection at scale while providing clinically relevant explanations, supporting both population-level healthcare planning and individual patient care decisions.

Healthcare access barriers↗
DOE OSTI2026

Quantum-classical embedding via ghost Gutzwiller approximation for enhanced simulations of correlated electron systems

Simulating correlated materials on present-day quantum hardware remains challenging due to limited quantum resources. Quantum embedding methods offer a promising route by reducing computational complexity through the mapping of bulk systems onto effective impurity models, allowing more feasible simulations on pre- and early-fault-tolerant quantum devices. Here, this work develops a quantum-classical embedding framework based on the ghost Gutzwiller approximation to enable quantum-enhanced simulations of ground-state properties and spectral functions of correlated electron systems. Circuit complexity is analyzed using an adaptive variational quantum algorithm on a statevector simulator, applied to the infinite-dimensional Hubbard model with increasing ghost mode numbers from 3 to 5, resulting in circuit depths growing from 16 to 104. Noise effects are examined using a realistic error model, revealing significant impact on the spectral weight of the Hubbard bands. To mitigate these effects, the Iceberg quantum error detection code is employed, achieving up to 40% error reduction in simulations. Finally, the accuracy of the density matrix estimation and the derived spectral function is benchmarked on IBM and Quantinuum quantum hardware, featuring distinct qubit-connectivity and employing multiple levels of error mitigation techniques.

Chen, I-Chi [Ames Laboratory (AMES), Ames, IA (Uni↗
DOE OSTI2025

Increasing Mosquito Abundance Under Global Warming

Mosquitoes are a key virus vector that poses significant health threats globally, affecting 700 million individuals and causing 1 million deaths annually. Accurately predicting mosquito abundance and dispersion remains a challenge. Complex interactions between mosquito dynamics and various environmental factors, notably hydrology, contribute to this challenge. Existing models typically focus on precipitation and temperature and often overlook further impacts of hydrological variables within mosquito modeling. In this study, we developed an artificial intelligence‐based model for mosquito dynamics, explicitly accounting for different hydrological variables, such as precipitation, soil moisture and streamflow. Using Toronto, Canada, as a case study, we identified causal relationships between changes in mosquito populations, hydrological factors, vegetation (e.g., leaf area index), and climate variables (e.g., daylight length, precipitation, and temperature). We embedded these relationships into a Long Short‐Term Memory (LSTM) Neural Network Model capable of accurately detecting mosquito dynamics across annual, seasonal, and monthly time scales. The LSTM is able to explain, on average, approximately 40% of the variance in the observed mosquito abundance data. Using the calibrated model, we predicted that the summer season mosquito abundance would increase by ∼16% and ∼19% under an intermediate greenhouse emission scenario, Shared Socioeconomic Pathway (SSP) 2–4.5, and a high greenhouse emission scenario, SSP5‐8.5, respectively. We expect that this model can serve as a valuable tool and inform science‐based decisions affecting mosquito dynamics and public health. It can also build a foundation for future risk analysis at the regional and larger scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Hierarchical Reinforcement Learning of a Short-Range Bond-Order Potential for Silica: Analytic Embedding of Coordination with Classical Efficiency

Reinforcement learning (RL) has recently emerged as a data-efficient strategy to parametrize short-range interatomic potentials. Building on our past RL optimization of pairwise silica models, we extend the framework to a bond-order (Tersoff-type) potential that provides an analytic embedding of local coordination through a three-body term. A hierarchical RL workflow combining continuous-action Monte Carlo Tree Search and property-based rewards efficiently explores the 26-dimensional parameter space, sequentially optimizing lattice parameters, densities, angles, and cohesive energies of 21 silica polymorphs. The resulting models, Q-Tersoff and ML-Tersoff, reproduce the energetic ordering of low-energy phases and capture the angular correlations and amorphous structure factors of silica with improved fidelity over pairwise force fields, while remaining orders of magnitude faster than high-dimensional machine-learned potentials. Both models underperform for elastic constants and high-energy frameworks, delineating the limits of the current analytic form. The approach establishes a general and interpretable route to angle-aware, short-range potentials that bridge physics-based and machine-learned descriptions of silicate materials.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Comparative life cycle assessment of woody biomass processing: air classification, drying, and size reduction powered by bioelectricity versus grid electricity

Sulfur accumulation during biofuel production is pollutive and toxic to conversion catalysts and causes the premature breakdown of processing equipment. Air classification is an effective preprocessing technology for ash and sulfur reduction from biomass feedstocks. Here, a life cycle assessment (LCA) sought to understand the environmental impact of implementing air classification as a sulfur-mitigation technique to improve feedstock quality for pine residues using a grid electricity scenario (GES) versus a bioelectricity scenario (BES). Global warming potential (GWP) for preprocessing was simulated using inventory databases embedded in SimaPro and the Argonne National Laboratory’s GREET model, specifically focusing on comparing the GWP of a GES versus a BES. Overall, the GES had a GWP impact over seven times that of the BES (136 versus 18 kg CO 2 equivalent per tonne of usable feedstock), with steam generation during rotary drying accounting for 57% of the GES’s GWP. Air classification represents 0.4% and 1.6% of the total GWP impact for the GES and BES, respectively. Therefore, air classification can facilitate a 30% reduction in feedstock sulfur content to improve feedstock quality for biofuel conversion and lessen corrosion of equipment while contributing minimal GWP impact during preprocessing.

Air classification↗
DOE OSTI2024

Maven: a multimodal foundation model for supernova science

Abstract A common setting in astronomy is the availability of a small number of high-quality observations, and larger amounts of either lower-quality observations or synthetic data from simplified models. Time-domain astrophysics is a canonical example of this imbalance, with the number of supernovae observed photometrically outpacing the number observed spectroscopically by multiple orders of magnitude. At the same time, no data-driven models exist to understand these photometric and spectroscopic observables in a common context. Contrastive learning objectives, which have grown in popularity for aligning distinct data modalities in a shared embedding space, provide a potential solution to extract information from these modalities. We present Maven, the first foundation model for supernova science. To construct Maven, we first pre-train our model to align photometry and spectroscopy from 0.5 M synthetic supernovae using a contrastive objective. We then fine-tune the model on 4702 observed supernovae from the Zwicky transient facility. Maven reaches state-of-the-art performance on both classification and redshift estimation, despite the embeddings not being explicitly optimized for these tasks. Through ablation studies, we show that pre-training with synthetic data improves overall performance. In the upcoming era of the Vera C. Rubin observatory, Maven will serve as a valuable tool for leveraging large, unlabeled and multimodal time-domain datasets.

Zhang, Gemma (ORCID:0000000280198082)↗
DOE OSTI2025

Multiphysics simulation of recent experiments on alkali‐silica reaction expansion in reinforced concrete members

Alkali‐silica reaction (ASR) is an important degradation process that causes volumetric expansion and damage in concrete, and is affected significantly by the local temperature, moisture and stress conditions that often vary across the regions of a structure. Numerical simulation is essential to predict the progression and effects of ASR on the performance of structures. Because of the interactions between thermal and moisture transport and mechanical deformation, it is important for numerical models to represent all these physical phenomena and the coupling between them. Simulations of ASR in reinforced concrete (RC) structures are further complicated by the need to capture interactions between concrete and embedded reinforcing bars. Here, this paper describes the implementation of a scalable, coupled‐physics ASR model for simulating RC structures and assesses the ability of that model to predict ASR‐induced expansion in recent laboratory tests on RC block and beam specimens. These laboratory tests and the simulation approach were selected because of their applicability to RC structural‐scale simulations. This validation study helps builds confidence the ability of this approach to model ASR expansion in large, complex RC structures, which is a current high‐priority need.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Distribution Grid Model Publication Investigation

Interest in the external exchange of distribution grid model data is growing around the world, driven largely by the challenges and opportunities presented by the increasing amount of generation, storage, and flexible load being embedded within the distribution grid. This report provides an overview of the current state of distribution grid model data sharing, with a focus on the industry-leading activities currently underway in Great Britain (GB). A second report will explore opportunities for external distribution grid model sharing in the United States.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Residual symmetries and scalar multiplet vacuum alignment in non-Abelian flavour models

We demonstrate that, upon minimizing a renormalizable, single-scalar potential invariant under a non-Abelian symmetry, special orientations in the associated vacuum alignment of the scalar multiplet correspond to the preservation of a discrete residual flavour symmetry in the broken phase of the theory. Conversely, we show that these special scalar alignments are perturbed when additional Lagrangian operators (e.g. renormalizable, multi-flavon operators and/or effective, higher-dimensional operators) are present that break said residual symmetry, leading to a vacuum reorientation and phenomenological consequences. We therefore construct a one-to-one correspondence principle between broken residual symmetries and vacuum alignment corrections, providing a mechanism to identify (and correct) a subtle but persistent form of phenomenologically relevant fine-tuning embedded in — but often ignored by — most successful non-Abelian flavour models. We first establish this correspondence in a set of toy models based on the S4 permutation symmetry, and then apply the lessons learned to the more realistic A4 Altarelli-Feruglio and ∆(27) Universal Texture Zero models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Multi‐Scale Seismic Imaging of the Ridgecrest, CA, Region With Waveform Inversion of Regional and Dense Array Data

Abstract We develop an inversion procedure for deriving multi‐scale velocity models with waveform inversions of earthquake and ambient noise data at multi‐frequency bands recorded by regional and dense sensor configurations. The method is applied for the area around the 2019 Ridgecrest earthquake rupture zones, utilizing data recorded by regional stations and dense 2D and 1D arrays with station spacings of ∼5 km and ∼100 m, respectively. Starting with regional Vp, Vs models and locations of Ridgecrest aftershocks, the velocity models and event locations are improved iteratively by inversions of waveforms recorded by regional stations and the 2D array, using a minimum spectral element size of ∼600 m. Waveforms from local events recorded by dense 1D arrays across the M7.1 rupture zone with frequencies of up to 10 Hz are used to resolve small‐scale features of the rupture zone and shallow crust with a local spectral element size of 80 m. The refined models provide self‐consistent descriptions of the rupture zone and the shallow crust embedded in the regional structures. The results reveal pronounced low Vs and high Vp/Vs in the M6.4 and M7.1 rupture zones coinciding with concentrations of seismicity, and also around the Garlock fault and in several local basins. We also observe clear velocity contrasts across the Garlock fault with polarity reversals along strike and with depth. The obtained multi‐scale velocity models can be used to improve derivations of earthquake source properties, simulations of dynamic ruptures and ground motions, and the understanding of fault and tectonic processes in the region.

Geochemistry & Geophysics↗
DOE OSTI2024

Investigating resource-efficient neutron/gamma classification ML models targeting eFPGAs

There has been considerable interest and resulting progress in implementing machine learning (ML) models in hardware over the last several years from the particle and nuclear physics communities. A big driver has been the release of the Python package, hls4ml, which has enabled porting models specified and trained using Python ML libraries to register transfer level (RTL) code. So far, the primary end targets have been commercial field-programmable gate arrays (FPGAs) or synthesized custom blocks on application specific integrated circuits (ASICs). However, recent developments in open-source embedded FPGA (eFPGA) frameworks now provide an alternate, more flexible pathway for implementing ML models in hardware. These customized eFPGA fabrics can be integrated as part of an overall chip design. In general, the decision between a fully custom, eFPGA, or commercial FPGA ML implementation will depend on the details of the end-use application. In this work, we explored the parameter space for eFPGA implementations of fully-connected neural network (fcNN) and boosted decision tree (BDT) models using the task of neutron/gamma classification with a specific focus on resource efficiency. We used data collected using an AmBe sealed source incident on Stilbene, which was optically coupled to an OnSemi J-series silicon photomultiplier (SiPM) to generate training and test data for this study. We investigated relevant input features and the effects of bit-resolution and sampling rate as well as trade-offs in hyperparameters for both ML architectures while tracking total resource usage. The performance metric used to track model performance was the calculated neutron efficiency at a gamma leakage of 10 -3 . The results of the study will be used to aid the specification of an eFPGA fabric, which will be integrated as part of a test chip.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Insights from a coupled thermo-hydro-mechanical analysis of a layered high-temperature thermal energy storage reservoir

Coupled thermal-hydraulic-mechanical (THM) modeling is applied to investigate the performance of a seasonal high-temperature aquifer thermal energy storage operation based on data and conditions from current site investigations at the Geostorage Forsthaus pilot project in Bern (Switzerland). The model includes subhorizontal sand lenses of various lengths and dips that are embedded in a low permeability clay matrix. Thermal energy storage is simulated by seasonal injection and withdrawal of hot (up to 90 °C) water from a main well, with reservoir pressure regulated by two auxiliary wells at a distance of about 70 m from the main well. The results show how targeted injection into deeper permeable storage formations, along with active deep well pressure control, can effectively minimize geomechanical impact and the potential risk of damaging subsurface storage and sealing formations, or even surface facilities. With such pressure control, the subsurface mechanical responses are dominated by thermal strain and stress, which can be monitored with subsurface fiber optics. The study demonstrates how coupled THM modeling can be applied for the design of a safe and efficient thermal energy storage operation, and how subsurface fiber optic monitoring can be applied for performance confirmation, allowing for more confident operational forecasting.

Rutqvist, Jonny↗
DOE OSTI2025

PDF Entity Annotation Tool (PEAT)

While different text mining approaches – including the use of Artificial Intelligence (AI) and other machine based methods - continue to expand at a rapid pace, the tools used by researchers to create the labeled datasets required for training, modeling, and evaluation remain rudimentary. Labeled datasets contain the target attributes the machine is going to learn; for example, training an algorithm to delineate between images of a car or truck would generally require a set of images with a quantitative description of the underlying features of each vehicle type. Development of labeled textual data that can be used to build natural language machine learning models for scientific literature is not currently integrated into existing manual workflows used by domain experts. Published literature is rich with important information, such as different types of embedded text, plots, and tables that can all be used as inputs to train ML/natural language processing (NLP) models, when extracted and prepared in machine readable formats. Currently, both normalized data extraction of use to domain experts and extraction to support development of ML/NLP models are labor intensive and cumbersome manual processes. Automatic extraction of data and information from formats such as PDFs that are optimized for layout and human readability, not machine readability. The PDF (Portable Document Format) Entity Annotation Tool (PEAT) was developed with the goal of allowing users to annotate publications within their current print format, while also allowing those annotations to be captured in a machine-readable format. One of the main issues with traditional annotation tools is that they require transforming the PDF into plain text to facilitate the annotation process. While doing so lessens the technical challenges of annotating data, the user loses all structure and provenance that was inherent in the underlying PDF. Also, textual data extraction from PDFs can be an error prone process. Challenges include identifying sequential blocks of text and a multitude of document formats (multiple columns, font encodings, etc.). As a result of these challenges, using existing tools for development of NLP/ML models directly from PDFs is difficult because the generated outputs are not interoperable. We created a system that allows annotations to be completed on the original PDF document structure, with no plain text extraction. The result is an application that allows for easier and more accurate annotations. In addition, by including a feature that grants the user the ability to easily create a schema, we have developed a system that can be used to annotate text for different domain-centric schemas of relevance to subject matter experts. Different knowledge domains require distinct schemas and annotation tags to support machine learning.

97 MATHEMATICS AND COMPUTING↗