Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “knowledge representation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Three-Channel Sunphotometer Cloud Mode Value-Added Product Report

A primary source of uncertainty in Earth system model (ESM) predictions is the representation of cloud processes and associated cloud feedback. Several fundamental cloud properties critical to the understanding of aerosol-cloud interactions are poorly constrained by observations, with key deficiencies in our observations of cloud and precipitation droplet sizes and cloud optical depth. Observations of these cloud properties are often challenging to estimate from remote-sensing platforms and costly to obtain from in situ aircraft. Nevertheless, observations of boundary-layer clouds, and improved knowledge of stratocumulus cloud (Sc) processes, are especially important to ESM advancement. This is because these clouds have extensive coverage and exert controls on boundary-layer dynamics and the global radiative energy balance.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

Abstract There is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Domain-specific text embedding model for accelerator physics

Accelerator physics presents unique challenges for natural language processing (NLP) due to its specialized terminology and complex concepts. A key component in overcoming these challenges is the development of robust text embedding models that transform textual data into dense vector representations, facilitating efficient information retrieval and semantic understanding. In this work, we introduce AccPhysBERT, a sentence embedding model fine-tuned specifically for accelerator physics. Our model demonstrates superior performance across a range of downstream NLP tasks, surpassing existing models in capturing the domain-specific nuances of the field. We further showcase its practical applications, including semantic paper-reviewer matching and integration into retrieval-augmented generation systems, highlighting its potential to enhance information retrieval and knowledge discovery in accelerator physics. Published by the American Physical Society 2025

Hellert, Thorsten (ORCID:0000000227970926)↗
DOE OSTI2023

Knowledge-Informed Uncertainty-Aware Machine Learning for Time Series Forecasting of Dynamical Engineered Systems

The high complexity and multiscale nature of many engineered systems—such as those in nuclear power plants—make representing and forecasting their dynamic behavior challenging. Physics-based models can be overly complex and computationally intractable, whereas machine learning (ML) tools are often data-hungry and prone to unphysical solutions. This study proposes a knowledge-informed ML-aided hybrid residual modeling approach that offers accurate and efficient time series forecasting for the operation of dynamical engineered systems. Hybrid residual modeling entails a baseline solution from domain knowledge and known physics expressions about the system dynamics integrated with an ML model to capture undiscovered information from the mismatch (i.e., residuals) between true states from measurements and baseline-predicted outputs. This study further quantifies the ML model uncertainty to provide trustworthy solutions. Real-time operational data from thermal-hydraulic flow loops of the cryogenic moderator system in Oak Ridge National Laboratory’s Spallation Neutron Source facility were used to demonstrate the potential of knowledge-informed uncertainty-aware ML in real-world applications. The state variables of the cryogenic helium loop were modeled with (1) first principles–based system identification (sysID), (2) long short-term memory (LSTM) neural network, and (3) hybrid sysID (baseline) + LSTM (residual). The superior predictive capability of the sysID+LSTM model versus stand-alone sysID and LSTM is confirmed by average performance metrics and individual data points across different prediction horizons. By creating a robust representation of the underlying physical system, the widely applicable hybrid residual modeling approach will enable the future development of digital twins for performance prediction, prognostics, and operation control.

Zhao, Xingang↗
DOE OSTI2022

Bayesian State-Space Modeling Framework for Understanding and Predicting Golden Eagle Movements Using Telemetry Data

Predicting raptor movements through a wind power plant under given atmospheric and topographical conditions is a crucial first step in the overall goal of quantifying the risk of turbine-related collisions and mortalities. Extracting behavioral traits of golden eagles (Aquila chrysaetos) from telemetry data requires the fusion of noisy and sparse movement data (location, heading, velocity) with a stochastic mathematical representation of the eagles' decision-making processes. In this study, we framed this problem in a Bayesian state-space framework where both observations and decision-making are assumed to be stochastic processes connected through hidden states (mode of flight, intent), and the unknown model parameters are assumed to be random variables that are calibrated using the available telemetry data. This framework allowed for rigorous consideration of underlying uncertainties while allowing for both data and prior biological knowledge to contribute to a probabilistic and predictive agent-based movement model. We implemented and applied the Bayesian framework to understand movement behavior of 23 GPS-tagged golden eagles travelling in the western US for years 2019 and 2020. Our preliminary findings show that the Bayesian state-space framework provides a robust inverse modeling apparatus to decode eagle behavioral characteristics from telemetry data. This study was primarily aimed at verifying and validating the framework with selected golden eagle tracks (both long- and short-ranged), with future research aimed at extending the framework to include multi-mode flight, consideration of atmospheric data and uplift mechanisms, eagle-to-eagle interaction, and eagle-to-turbine interaction.

Bayesian modeling↗
DOE OSTI2024

Updates to the Alliance of Genome Resources central infrastructure

The Alliance of Genome Resources (Alliance) is an extensible coalition of knowledgebases focused on the genetics and genomics of intensively studied model organisms. The Alliance is organized as individual knowledge centers with strong connections to their research communities and a centralized software infrastructure, discussed here. Model organisms currently represented in the Alliance are budding yeast, Caenorhabditis elegans, Drosophila, zebrafish, frog, laboratory mouse, laboratory rat, and the Gene Ontology Consortium. The project is in a rapid development phase to harmonize knowledge, store it, analyze it, and present it to the community through a web portal, direct downloads, and application programming interfaces (APIs). Here, we focus on developments over the last 2 years. Specifically, we added and enhanced tools for browsing the genome (JBrowse), downloading sequences, mining complex data (AllianceMine), visualizing pathways, full-text searching of the literature (Textpresso), and sequence similarity searching (SequenceServer). We enhanced existing interactive data tables and added an interactive table of paralogs to complement our representation of orthology. To support individual model organism communities, we implemented species-specific “landing pages” and will add disease-specific portals soon; in addition, we support a common community forum implemented in Discourse software. We describe our progress toward a central persistent database to support curation, the data modeling that underpins harmonization, and progress toward a state-of-the-art literature curation system with integrated artificial intelligence and machine learning (AI/ML).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Structure-Informed Graph Learning of Networked Dependencies for Online Prediction of Power System Transient Dynamics

Online transient analysis plays an increasingly important role in dynamic power grids as the renewable generation continues growing. Traditional numerical methods for transient analysis not only are computationally intensive but also require precise contingency information as input, and therefore, are not suitable for online applications. Existing online transient assessment studies focus on the determination of post-contingency system stability or stability margin. Here, this paper develops a novel graph-learning framework, Deep-learning Neural Representation or DNR, for online prediction, of the time-series trajectories of the system states using initial system responses that can be measured by phasor measurement units (PMUs). The proposed DNR framework consists of two sequential modules: a Network Constructor that captures network dependencies among generators, and a Dynamics Predictor that predicts the system trajectories. The key to improved prediction performance is the introduction of the spatio-temporal message-passing operations into graph neural networks with structural knowledge. Its effectiveness and scalability are validated through comparative studies, demonstrating the prediction performance under different contingency scenarios for systems of different sizes. This framework provides a solution to online predicting post-fault system dynamics based on real-time PMU measurements. Additionally, it can also be applied to facilitate the offline transient simulation without simulating the entire trajectories.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

I Can’t Read All That! Improving the Usability of Semantic Models Using Concise, Ontology-Agnostic, Building-Specific Schemas

Semantic ontologies have enabled the creation of formalized, machine-readable descriptions of heterogenous building systems by providing dictionaries of well defined concepts that can be applied to model them. Within a semantic model of a particular building, a subset of an ontology's concepts may be applied in different ways to represent a particular perspective of the building's systems. How the concepts were applied can only be understood by examining the large amount of instance data within a semantic model, which leads to usability challenges. We propose a concise, ontology-agnostic method for defining building-specific schema (b-schema) graphs that summarize the structure and content of a semantic model. This approach provides a queryable and concise representation of the model's contents, separate from the instance data within a model, that can mitigate the challenges posed by the size and complexity of semantic models in processes such as visualization, querying, validation, and the use of large language models (LLMs). We validate our approach on semantic models based on the Brick and ASHRAE S223 ontologies. Results demonstrate that b-schemas significantly reduce the complexity of visual interpretation, accelerate SPARQL queries and SHACL validation, and improve LLM-based knowledge graph question answering.

Paul, Lazlo [Lawrence Berkeley National Laboratory↗
DOE OSTI2023

Learning subgrid-scale models with neural ordinary differential equations

We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are generally difficult to tune. Machine learning approaches have increased the accuracy and efficiency of computational fluid dynamics solvers. In this approach neural networks are used to learn the coarse- to fine-grid map, which can be viewed as subgrid-scale parameterization. We propose a strategy that uses the NODE and partial knowledge to learn the source dynamics at a continuous level. Our method inherits the advantages of NODEs and can be used to parameterize subgrid scales, approximate coupling operators, and improve the efficiency of low-order solvers. Numerical results with the two-scale Lorenz 96 ODE, the convection-diffusion PDE, and the viscous Burgers' PDE are used to illustrate this approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

OSTI Semantic Thesaurus v1

The OSTI Semantic Thesaurus is a reference for scientific and technical terms and relationships. The term data is used to implement keyword-to-concept mapping in OSTI.GOV searches, where searched term(s) will be mapped to related scientific concepts, allowing the end user to retrieve results related to their search terms and also explore narrower and similar concepts. The OSTI Semantic Thesaurus originally inherited data and structure from the INIS/ETDE Thesaurus (https://www.etde.org/edb/IAEA-INIS-ETDE-01-2013-08.pdf), which was a controlled terminology for indexing information within the subject scopes of the International Nuclear Information System (INIS) and the Energy Technology Data Exchange (ETDE). The data has been expanded upon since that time with the inclusion of scientific concepts from sources like Wikidata, and through manual curation. This dataset is an export from the OSTI Semantic Thesaurus in RDF/SKOS format, which is specifically suited for representation of controlled vocabularies like thesauri. While the full thesaurus includes additional relation types which may be included in future revisions, this export is limited to broader, narrower, and related term relations. Definitions and scope notes are included where available.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models

Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs. To better understand LLM scaling, we study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in smaller models. We observe a geometric phenomenon which we term embedding condensation, where token embeddings collapse into a narrow cone-like subspace in some language models. Through systematic analyses across multiple Transformer families, we show that small models such as GPT2 and Qwen3-0.6B exhibit severe condensation, whereas larger models such as GPT2-x1 and Qwen3-32B are more resistant to this phenomenon. Additional observations show that embedding condensation is not reliably mitigated by knowledge distillation from larger models. To fight against it, we formulate a dispersion loss that explicitly encourages embedding dispersion during training. Experiments demonstrate that it mitigates condensation, recovers dispersion patterns seen in larger models, and yields performance gains across 10 benchmarks. We believe this work offers a principled path toward improving smaller Transformers without additional parameters.

Xiao, Xi [ORNL] (ORCID:0009000009316982)↗
DOE OSTI2022

Latent Mechanisms of Polarization Switching from In Situ Electron Microscopy Observations

In situ scanning transmission electron microscopy enables observation of the domain dynamics in ferroelectric materials as a function of externally applied bias and temperature. The resultant data sets contain a wealth of information on polarization switching and phase transition mechanisms. However, identification of these mechanisms from observational data sets has remained a problem due to a large variety of possible configurations, many of which are degenerate. Here, an approach based on a combination of deep learning-based semantic segmentation, rotationally invariant variational autoencoder (VAE), and non-negative matrix factorization to enable learning of a latent space representation of the data with multiple real-space rotationally equivalent variants mapped to the same latent space descriptors is introduced. By varying the size of training sub-images in the VAE, the degree of complexity in the structural descriptors is tuned from simple domain wall detection to the identification of switching pathways. Importantly, this yields a powerful tool for the exploration of the dynamic data in mesoscopic electron, scanning probe, optical, and chemical imaging. Moreover, this work adds to the growing body of knowledge of incorporating physical constraints into the machine and deep-learning methods to improve learned descriptors of physical phenomena.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Er Al :Al 2 ⁢O 3 for telecom-band photonics: Electronic structure and optical properties

Er-doped Al 2 ⁢O 3 is a promising host for telecom-band integrated photonics. Here, in this study, we combine ab initio calculations with a symmetry-resolved analysis to elucidate substitutional Er on the Al site (Er Al ) in 𝛼−Al 2 ⁢O 3 . First-principles relaxations confirm the structural stability of Er Al . We then use the local trigonal crystal-field symmetry to classify the Er-derived impurity levels by irreducible representations and to derive polarization-resolved electric-dipole selection rules, explicitly identifying the symmetry-allowed 𝑓−𝑑 hybridization channels. Kubo-Greenwood absorption spectra computed from Kohn-Sham states quantitatively corroborate these symmetry predictions. Furthermore, we connect the calculated intra-4⁢𝑓 line strengths to Judd-Ofelt theory, clarifying the role of 4⁢𝑓−5⁢𝑑 admixture in enabling optical activity. Notably, we predict a characteristic absorption near 1.47 µ⁢m (telecom band), relevant for on-chip amplification and emission. To our knowledge, a symmetry-resolved first-principles treatment of Er:Al 2 ⁢O 3 with an explicit Judd-Ofelt interpretation has not been reported, providing a transferable framework for tailoring rare-earth dopants in wide-band-gap oxides for integrated photonics. Our results for the optical spectra are in good agreement with experimental data. The resulting symmetry-based selection rules translate directly to polarization-dependent coupling in Al 2 ⁢O 3 integrated photonic waveguides and resonators, enabling device-level design of TE/TM-mode interaction with Er emitters in the 1.5-µ⁢m telecom band.

Er-doped Al2O3↗
DOE OSTI2026

Reduced‐Order Modeling of Energetic Materials Using Physics‐Aware Recurrent Convolutional Neural Networks in a Latent Space (LatentPARC)

Physics-aware deep learning (PADL) has gained popularity for use in spatiotemporal dynamics simulations, such as those in computational modeling of energetic materials (EM). We show that the challenge PADL methods face while learning complex field evolution problems can be simplified and accelerated by decoupling it into two tasks: learning complex geometric features in evolving fields and modeling dynamics over these features in a lower-dimensional feature space. We build upon our previous work on physics-aware recurrent convolutional neural networks (PARC). PARC embeds knowledge of underlying physics into its neural network architecture for more robust and accurate prediction of evolving physical fields. PARC was shown to effectively learn complex nonlinear features such as the formation of hotspots and coupled shock fronts in various initiation scenarios of EMs, as a function of microstructures, serving effectively as a microstructure-aware burn model. Here, we further accelerate PARC and reduce its computational cost by projecting the original dynamics onto a lower-dimensional invariant manifold, or “latent space.” The projected latent representation encodes the complex geometry of evolving fields (e.g., temperature and pressure) in a set of data-driven features. The reduced dimension of this latent space allows us to learn the dynamics during the initiation of EM with a lighter and more efficient model. We observe a significant decrease in training and inference time while maintaining results comparable to PARC at inference. This work takes steps towards enabling rapid prediction of EM thermomechanics at larger scales and characterization of EM structure–property–performance linkages at a full application scale.

Mathematics and Computing↗
DOE OSTI2023

Architecture-Preserving Provable Repair of Deep Neural Networks

Deep neural networks (DNNs) are becoming increasingly important components of software, and are considered the state-of-the-art solution for a number of problems, such as image recognition. However, DNNs are far from infallible, and incorrect behavior of DNNs can have disastrous real-world consequences. This paper addresses the problem of architecture-preserving V-polytope provable repair of DNNs. A V-polytope defines a convex bounded polytope using its vertex representation. V-polytope provable repair guarantees that the repaired DNN satisfies the given specification on the infinite set of points in the given V-polytope. An architecture-preserving repair only modifies the parameters of the DNN, without modifying its architecture. The repair has the flexibility to modify multiple layers of the DNN, and runs in polynomial time. It supports DNNs with activation functions that have some linear pieces, as well as fully-connected, convolutional, pooling and residual layers. To the best our knowledge, this is the first provable repair approach that has all of these features. We implement our approach in a tool called APRNN. Using MNIST, ImageNet, and ACAS Xu DNNs, we show that it has better efficiency, scalability, and generalization compared to PRDNN and REASSURE, prior provable repair methods that are not architecture preserving.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Aerosol Hygroscopic Growth, Mixing State, and Cloud Condensation Nuclei Activity during TRACER (Field Campaign Report)

Convective clouds play a critical role in the Earth’s climate system. Recent research has shown that a realistic representation of convective processes is critical to constraining climate sensitivity in global climate models. Theoretical and modeling studies showed that aerosols could have strong dynamic feedback to convection in warm and humid environments through enhancing ice-related processes and condensational growth. A few observation-based studies also suggested the influence of aerosols on convective cloud and precipitation properties. However, robust observational quantification of an aerosol effect on convective clouds isolated from other factors remains elusive. Understanding the impact of aerosol on convective clouds requires knowledge of the cloud condensation nuclei (CCN) spectrum, which represents the number of particles that uptake water and form cloud droplets as a function of supersaturation. The water uptake by aerosol is also of critical importance for the direct interaction of aerosol with radiation (i.e., aerosol direct effect) due to light scattering and absorption by aerosol. While both droplet activation under supersaturated conditions (i.e., relative humidity RH > 100%) and the hygroscopic growth under sub-saturated conditions (i.e., RH < 100%) are strongly influenced by particle hygroscopicity, the thermodynamic regimes and measurement methods are quite different. Aerosol particles, especially organic particles, can exhibit higher hygroscopicity for droplet activation than that for hygroscopic growth. In the subsaturated regime, the hygroscopicity of organic particles can also vary strongly with RH. However, global climate models usually treat organic species in aerosols with a constant hygroscopicity, potentially introducing substantial uncertainties in the quantification of aerosol radiative effects.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Development of a Convolutional Neural Network Classifier for Data Starved Spectra - 20199

The Institute for Clean Energy Technology (ICET) at Mississippi State University is exploring the utility of machine learning in augmenting its mobile radiation surveying platforms, which are currently being developed as means to survey depleted uranium contaminated areas in support of remediation and decommissioning efforts. Mobile survey platforms provide a means to efficiently scan large areas of interest while reducing human exposure to radiation and other hazards. The survey platforms can also be used for scanning for any gamma emitting isotope in addition to depleted uranium. The spectral data that the platforms collect may be data starved with relatively low counts and poorly defined spectral features depending on the speed of the platforms and scintillation detector selection. Such data-starved spectra are difficult to use for isotope identification, requiring advanced knowledge of the possible radionuclides that could be present and environmental factors that could attenuate signals or introduce background noise. These factors in combination with the volume of survey data increases the time it takes to perform analysis of survey data when the source type is unknown. There are a number of algorithms in the field of machine learning that can be used to classify data that would be challenging and time-consuming for a human to identify. Supervised machine learning algorithms train models based on extensive amounts of human-labeled training data. Once sufficiently trained, these models can be used to quickly make high-fidelity predictions on new data. Convolutional neural networks are machine learning algorithms that excel in learning representations of 'shapes'. They do this by taking numerical input data and convolving them with spatial feature detectors referred to as filters. These filters are incrementally adjusted to reduce the prediction error on the data during the backpropagation step of training. Discussed in this paper is the development of a convolutional neural network classifier (CNNC) that can utilize spectral survey data for source discrimination and isotope identification. Bench-top laboratory experiments data using LaBr{sub 3}(Ce) scintillation detectors were used to train and evaluate the performance of the developed CNNC. The CNNC is capable of discriminating a variety of gamma emitting source types, differentiating different forms of uranium (depleted vs. natural), and estimating the amount of uranium for a known geometry. The discussed CNNC may be useful in scenarios where survey systems are deployed in situations where hazardous radioactive material maybe present, but the type is unknown. When used in remediation applications the CNNC can be used to screen-out false positives, helping reduce remediation costs. (authors)

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2021

Assimilating Scanning Radar Data into Cloud-Scale Models

The aim of this project is to improve the initial conditions of large eddy simulations (LES) of deep convective storms by assimilating scanning radar reflectivity and radial velocity data as well as local environmental observations. Accomplishing this goal often results in more accurate numerical forecasts of clouds and precipitation than if the background initial meteorological conditions are solely relied upon for convection initiation and intensification, as is typical of most operational and research models. Conducted in a storm-scale LES framework, these more accurate numerical analyses can serve as tool to better understand and parameterize land-atmosphere coupling, boundary layer turbulence, and cloud processes, each used for model parameterization development of convection. Our work examined the sensitivity of the numerical representation of cloud-scale wind and microphysical structures to: i) the assimilation of radiosonde observations characterizing the near-storm inflow, ii) the assimilation of radar data of varying resolution for model simulations with grid spacings of 1-km and 0.25-km, and iii) to the assimilation of three-dimensional plan-projection-indicator (PPI) and range-height indicator (RHI) radar volumes. Few studies have explored deep convective storm analyses and forecasts produced by assimilation of radar observations to cloud-scale LES. Furthermore, to our knowledge, no storm-scale DA study has explored the value of assimilating high-resolution sector RHI volumes to determine if they can improve the retrieval of the vertical kinematic and microphysical storm structure aloft over the use of traditionally utilized PPI scans.

54 ENVIRONMENTAL SCIENCES↗