Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Embedded computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2022

Teaching AI when to care about gender

Natural Language Processing (NLP) is a branch of Artificial Intelligence (AI) concerned with solving language tasks by modeling large amounts of textual data. Some NLP techniques use word embeddings which are semantic models where machine learning (ML) is used to learn to cluster semantically related words by learning about word co-occurrences in the original training text. Unfortunately, these models tend to reflect or even exaggerate biases that are present in the training corpus. Here we describe the Word Embedding Navigator (WEN), which is a tool for exploring word embedding models. We examine a specific potential use case for this tool: interactive discovery and neutralization of gender bias in word embedding models, and compare this human-in-the-loop approach to reducing bias in word embeddings with a debiasing post-processing technique.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

AlignOT: An Optimal Transport Based Algorithm for Fast 3D Alignment With Applications to Cryogenic Electron Microscopy Density Maps

Aligning electron density maps from Cryogenic electron microscopy (cryo-EM) is a first key step for studying multiple conformations of a biomolecule. As this step remains costly and challenging, with standard alignment tools being potentially stuck in local minima, we propose here a new procedure, called AlignOT, which relies on the use of computational optimal transport (OT) to align EM maps in 3D space. By embedding a fast estimation of OT maps within a stochastic gradient descent algorithm, our method searches for a rotation that minimizes the Wasserstein distance between two maps, represented as point clouds. Here, we quantify the impact of various parameters on the precision and accuracy of the alignment, and show that AlignOT can outperform the standard local alignment methods, with an increased range of rotation angles leading to proper alignment. We further benchmark AlignOT on various pairs of experimental maps, which account for different types of conformational heterogeneities and geometric properties. As our experiments show good performance, we anticipate that our method can be broadly applied to align 3D EM maps.

3D alignment↗
DOE OSTI2021

Projector-Based Quantum Embedding for Molecular Systems: An Investigation of Three Partitioning Approaches

Projector-based embedding is a relatively recent addition to the collection of methods that seek to utilize chemical locality to provide improved computational efficiency. This work considers the interactions between the different proposed procedures for this method and their effects on the accuracy of the results. The interplay between the embedded background, projector type, partitioning scheme, and level of atomic orbital (AO) truncation are investigated on a selection of reactions from the literature. The Huzinaga projection approach proves to be more reliable than the level-shift projection when paired with other procedural options. Active subsystem partitioning from the subsystem projected AO decomposition (SPADE) procedure proves slightly better than the combination of Pipek-Mezey localization and Mulliken population screening (PMM). Along with these two options, a new partitioning criteria is proposed based on subsystem von Neumann entropy and the related subsystem orbital occupancy. This new method overlaps with the previous PMM method, but the screening process is computationally simpler. Finally, AO truncation proves to be a robust option for the tested systems when paired with the Huzinaga projection, with satisfactory results being acquired at even the most severe truncation level.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Cement sensors with acoustic bandgaps using carbon nanotubes

Cement is widely used in wellbores to stabilize the steel casing used in wellbore operations for oil and gas production, enhanced geothermal systems and carbon sequestration, and to limit fluid movement between sub-surface strata. Flaws such as microcracks in wellbore cement can lead to leakage along the wellbore compromising wellbore integrity. There is an increasing need for methods to monitor cement crack propagation in wellbore environments. In this study, we develop and report the first cementitious sensors capable of exhibiting high frequency acoustic bandgaps (ABGs) using carbon nanotubes (CNTs). Computational simulations of a sensor unit cell are used to design cement-multi walled carbon nanotubes (MWCNTs) sensors that show a wide bandgap. When the cement-MWCNTs sensors is embedded in cement specimens, bandgaps were measured experimentally under 300 kHz and under 600 kHz, consistent with the computationally predicted bandgaps in the range of 290–360 kHz, 410–460 kHz and 515–585 kHz. These bandgap features were absent in homogeneous cement specimens. X-ray tomographic reconstructions showed microscopic debonding at cement-MWCNTs sensor interface. Frequency response analysis of a three-dimensional computational model indicated a shift of frequency of minimum transmission due to the interface debonding, but no perturbation of bandgap response was observed. Here, the cement-MWCNTs sensors developed in this study show the potential of a packed CNT inclusion material in cementitious matrix to create ABGs in a cement matrix.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Integrating fiber optic sensors into metallic components for sensing in harsh environments

The integration of fiber optic sensors into high-temperature materials is critical for real-time monitoring and autonomous operation of engineering systems. This study demonstrated a spark plasma sintering (SPS)-assisted embedding process for integrating sapphire fiber optic sensors into stainless steel components during part fabrication. Optical fibers were encapsulated in stainless steel 316L powders which were sintered at different fabrication conditions using SPS to investigate the effects of sintering parameters on the embedment. Measurements of optical transmittance, combined with microstructural analysis (X-ray computed tomography and scanning electron microscopy) and mechanical testing (tensile and microhardness), were conducted to examine the fiber functionality, fiber–matrix bonding quality, and properties of the sintered materials. Here, the results show that under suitable fabrication conditions, intact optical fibers can be encapsulated in highly-densified (>98 % relative density) stainless steel components. These conditions also led to a good bond at the fiber–matrix interface with micron-sized material interdiffusion across the interface. The sintering parameters were observed to affect fiber optical attenuation, where high temperature, pressure, and hold time during SPS enhanced fiber–matrix bonding and adversely affected optical transmission. Tensile testing confirmed the superior tensile strength and ductility of the matrix fabricated by SPS. Furthermore, the materials exhibited limited strength reduction (~70 MPa) upon the integration of fibers. This study demonstrates the effectiveness of SPS for fiber-material integration for high-temperature applications.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Projective embedding of dynamical systems: Uniform mean field equations

Herein we study embeddings of continuous dynamical systems in larger dimensions via projector operators. We call this technique PEDS, projective embedding of dynamical systems, as the stable fixed point of the original system dynamics are recovered via projection from the higher dimensional space. In this paper we provide a general definition and prove that for a particular type of rank-1 projector operator, the uniform mean field projector, the equations of motion become a mean field approximation of the dynamical system. While in general the embedding depends on a specified variable ordering, the same is not true for the uniform mean field projector. We prove a variety of results on the relationship between the spectrum of the Jacobian for fixed points in the original and in the embedded system. Direct applications of PEDS can be non-convex optimization and machine learning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Fusing Edge Computing with Transport Security by Leveraging the Controller Area Network Transport Security Tracking and Reporting (C-STAR) Unit

Rapid advances in embedded system complexity and capability provides exciting opportunities for transportation security deployment. Manufacturers and developers of these embedded systems continue to provide lower cost and more powerful solutions that can be leveraged by researchers and engineers. Furthermore, deploying these devices at the “edge” of the Internet-of-Things (IoT) infrastructure provides opportunities for highly capable applications in transport security. In an edge computation architecture, the device is co-located at the source of the data in the larger IoT structure – this provides computational capability at the location directly where the data is collected. For shipment transport security, this provides a direct compute node for digestion of data and mitigation actions in real-time. In our application, the vehicle provides a significant amount of this data that can be processed in real-time via the Controller Area Network Transport Security Tracking and Reporting (C-STAR) edge device. Utilization of a computational node located on the vehicle, such as the C-STAR, capitalizes on previously discussed opportunities of edge architectures. In this paper, we will discuss this security solution’s usability, current deployments, and scalability to further applications in transport security. First, we will cover the supported vehicle platforms that can leverage the C-STAR technology. This will be particularly relevant to medium- and heavy-duty vehicles transporting high-risk shipments. Second, we will speak to current deployments of the C-STAR that are ongoing. Finally, we will discuss additional areas for expansion such as maturing the onboard algorithms through continuing collaborations.

Cook, Adian [ORNL] (ORCID:0000000160825395)↗
DOE OSTI2021

C3PU:Real-Time Predictive Charge Control Software for Battery Management Systems (Code for Charge Control and Predictive Unit) [SWR-21-48]

C3PU sets up a model predictive control formulation for the optimal charging control of battery packs and can be deployed for real-time operation on an embedded microprocessor. C3PU uses electrochemical and thermal models of battery packs to predict the charging trajectories of the battery pack over a time horizon under different operating conditions. It then selects the optimal charging trajectory such that a pre-defined objective is minimized, which in the present state of the code is to minimize the charging duration of the battery pack. The optimal charging trajectory is obtained by solving an underlying mathematical problem. However, C3PU is flexible to incorporate other charging objectives. The underlying battery models can be swapped as well, as long as they follow certain mathematical properties. Mathematically, optimal control problems (in this case, optimally controlling the charging current of the battery pack) are computationally expensive. C3PU implements advanced numerical techniques, namely pseudo-spectral optimization, to reduce the computational burden of the underlying problem to solve. This allows for the problem to be solved in real-time in an embedded system. Once the optimal charging trajectory is computed over a time horizon, C3PU can package such data and send it out via appropriate communication protocols.

Mishra, Partha↗
DOE OSTI2023

Compression of tokamak boundary plasma simulation data using a maximum volume algorithm for matrix skeleton decomposition

This report demonstrates satisfactory data compression of SOLPS-ITER simulation output ranging from 2D fields, 1D profiles, and 0D scalar variables with a novel matrix decomposition approach. The singular value decomposition (SVD) scales poorly for large matrix sizes and is unsuited to the application on high dimensional data common to fusion plasma physics simulation. In this work, we employ the columns-submatrix-rows (CUR) matrix factorization technique in order to compute a low-rank approximation up to two orders of magnitude faster than the SVD, but within a nominal L2-norm relative error of ε = 10 –2 . In addition, the CUR approach maintains the original format of the data, in its extracted columns and rows, allowing for interpretable data storage at the original resolution of the simulation. We utilize an iterative algorithm to compute the CUR decomposition of simulation output by maximizing the volume, or linearly independent information content, of a low-rank submatrix contained within the data. Experiments over $\textit{n} × \textit{n}$ randomized test matrices with embedded rank-deficient features show that this maximum volume implementation of CUR matrix approximation has reduced asymptotic computational complexity on the order of n compared to the SVD, which scales approximately as $n^3$. These results show that the CUR technique can be used to effectively select time step snapshots (columns) of over 140 SOLPS-ITER output variables and the associated discretized coordinate timeseries (rows) allowing for reconstruction of the complete simulation dynamics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Generative Vulnerability Assessment for Cyber-Physical Systems

Cyber-physical systems (CPS) are highly susceptible to malicious attacks due to their complex dynamics and interconnectivity. A comprehensive understanding of their vulnerabilities is essential for designing effective resilience measures. This paper presents a data-driven attack generative system for evaluating the vulnerability of CPS. The proposed approach formulates the vulnerability assessment problem as determining the feasibility of a specific attack set based on two boundary functions that represent the effectiveness and stealthiness of attacks. The attack generative model is trained using a custom loss function, with two universal approximators designed to learn the effectiveness and stealthiness functions simultaneously. Theoretical results for successful generation and asymptotic convergence of the resulting training algorithm are given. As a result, the proposed approach is evaluated via numerical simulation of an IEEE 14-bus system and gas pipeline systems, demonstrating its viability in learning how to attack nonlinear CPS and identify potential vulnerabilities.

Computer systems organization↗
DOE OSTI2024

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗
DOE OSTI2020

Selecting the independent coordinates of manifolds with large aspect ratios

Many manifold embedding algorithms fail apparently when the data manifold has a large aspect ratio (such as a long, thin strip). Here, we formulate success and failure in terms of finding a smooth embedding, showing also that the problem is pervasive and more complex than previously recognized. Mathematically, success is possible under very broad conditions, provided that embedding is done by carefully selected eigenfunctions of the Laplace-Beltrami operator Δ. Hence, we propose a bicriterial Independent Eigencoordinate Selection (IES) algorithm that selects smooth embeddings with few eigenvectors. The algorithm is grounded in theory, has low computational overhead, and is successful on synthetic and large real data.

Meila, Marina↗
DOE OSTI2025

Accelerating Embedding Potential Optimization by Reconstructing the Pseudo-Valence Electron Density

Density functional embedding theory (DFET) enables use of electronic structure methods with higher accuracy than density functional theory in a local region, with applications thus far ranging from (photo/electro)catalysis to reactions in solution. DFET partitions a large collection of atoms into smaller groups that interact via a shared embedding (interaction) potential V emb , determined via functional optimization. The optimized effective potential (OEP) process used to optimize V emb is time-consuming and becomes a computational bottleneck due to sharp, oscillating features of V emb near nuclei. Here, similar to pseudopotential theory, by reconstructing electron densities used in the OEP process from smoother pseudo-valence-only (PVO) electron densities as proxies for total densities of the full system and subsystems, we can retain accuracy in the embedded electronic structure calculations while potentially reducing the overhead of V emb construction, within the projector augmented-wave (PAW) formalism. We explore three different chemical reactions as exemplars to test PVO–DFET, namely, H 2 dissociative adsorption on a Cu(111) surface, H 2 O adsorption on a Pt(111) surface, and aqueous [Ca 2+ –SO 4 2– ] ion-pair formation. The PVO approximation works well for all three systems with minimal loss of accuracy (∼10–70 meV error relative to the original exact-derivative (ED) approach) while accelerating V emb generation for the Cu and Pt systems respectively by 20× and 5×. Given proper numerical convergence parameters, the spatial distributions of differences between PVO- and ED-based V emb outside the core regions are small, explaining the exceptional agreement between the two approaches. Finally, we anticipate that this more efficient PVO–DFET approximation will be useful whenever computation of V emb is much more expensive than subsequent embedded high-level electron correlation calculations.

approximation↗
DOE OSTI2026

Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models

Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs. To better understand LLM scaling, we study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in smaller models. We observe a geometric phenomenon which we term embedding condensation, where token embeddings collapse into a narrow cone-like subspace in some language models. Through systematic analyses across multiple Transformer families, we show that small models such as GPT2 and Qwen3-0.6B exhibit severe condensation, whereas larger models such as GPT2-x1 and Qwen3-32B are more resistant to this phenomenon. Additional observations show that embedding condensation is not reliably mitigated by knowledge distillation from larger models. To fight against it, we formulate a dispersion loss that explicitly encourages embedding dispersion during training. Experiments demonstrate that it mitigates condensation, recovers dispersion patterns seen in larger models, and yields performance gains across 10 benchmarks. We believe this work offers a principled path toward improving smaller Transformers without additional parameters.

Xiao, Xi [ORNL] (ORCID:0009000009316982)↗
DOE OSTI2024

Understanding Event Trajectories Across Massive Temporal Datasets with Word Embeddings and Visualization

In collaboration with researchers from Virginia Tech, Savannah River National Laboratory has continued development of a natural language processing pipeline to identify and extract events of interest from massive open data sources in the domain of worldwide state-sponsored civil nuclear energy. The foundation of the pipeline is built on compass aligned temporal word embedding models, whereby contextual shifts are automatically identified by comparing keyword embedding vectors across successive time windows. Within the approach, a contextual shift indicates the occurrence of a potential event of interest. However, in such a broad topical domain that captures events at a global scale, across various life cycle stages, and across numerous different technology types, a user that is monitoring events may have broad interests in capturing many different event types with varying degrees of signal. As such, the quantity of information that may be returned from an automated event extraction pipeline can be substantial, requiring manual effort to sift through the information to identify any relevant bits of information. Therefore, a more streamlined workflow that aids in directing a user toward specific information at different points in time is necessary. The workflow presented here has been developed with this concept in mind, built on top of the initial prototype event extraction pipeline, whereby a user can analyze temporal text-based data sources at multiple different contextual levels to isolate key points in time and key subdomains captured within a data corpus. Using multiple corpuses that consist of approximately 7 million Tweets and 7 million news articles, the team has extended compass aligned temporal word embedding models to establish an interconnected and hierarchical structure that relates known key words of interest to documents, local topics (i.e., within a time window), and global topics across the corpuses. All of this information is packaged into a visual analytics system that is linked to the information extraction pipeline and enables a user to identify contextual information that describes the evolution of a high dimensional embedding space across time to isolate changes of interest and explore associated events. This report demonstrates the use of these analytics and a means to fuse information across multiple datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

First Principles Simulations of Electrified Interfaces in Electrochemistry

This chapter discusses some of the recent advances made in the first principles modeling of electrochemical catalysts. It also discusses the key development, namely the ability to explicitly treat the effects of surface electrification due to electrochemical processes and applied voltages in a computationally efficient manner. The chapter introduces the thermodynamics and statistical mechanics of electrified metal-solution interfaces. It then discusses the structure of the electrode-electrolyte interface and the effects of applied voltages, followed by an overview of the first-principles model and a motivating example. The chapter also provides a brief summary of classical thermodynamics and describes the basics of thermodynamic detour. It considers the thermodynamics of macroscopic systems and introduces several useful fundamental relations. Here, the chapter also considers a system to exist in a certain macrostate, which is a particular thermodynamic state specified by a set of fixed properties such as constant particle number, constant volume, and constant temperature.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Operator inference for non-intrusive model reduction with quadratic manifolds

This paper proposes a novel approach for learning a data-driven quadratic manifold from high-dimensional data, then employing this quadratic manifold to derive efficient physics-based reduced-order models. The key ingredient of the approach is a polynomial mapping between high-dimensional states and a low-dimensional embedding. This mapping consists of two parts: a representation in a linear subspace (computed in this work using the proper orthogonal decomposition) and a quadratic component. The approach can be viewed as a form of data-driven closure modeling, since the quadratic component introduces directions into the approximation that lie in the orthogonal complement of the linear subspace, but without introducing any additional degrees of freedom to the low-dimensional representation. Combining the quadratic manifold approximation with the operator inference method for projection-based model reduction leads to a scalable non-intrusive approach for learning reduced-order models of dynamical systems. Further, applying the new approach to transport-dominated systems of partial differential equations illustrates the gains in efficiency that can be achieved over approximation in a linear subspace.

42 ENGINEERING↗