Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65

KG-Hub—building and exchanging biological knowledge graphs

Knowledge graphs (KGs) are a powerful approach for integrating heterogeneous data and making inferences in biology and many other domains, but a coherent solution for constructing, exchanging, and facilitating the downstream use of KGs is lacking. Here we present KG-Hub, a platform that enables standardized construction, exchange, and reuse of KGs. Features include a simple, modular extract–transform–load pattern for producing graphs compliant with Biolink Model (a high-level data model for standardizing biological data), easy integration of any OBO (Open Biological and Biomedical Ontologies) ontology, cached downloads of upstream data sources, versioned and automatically updated builds with stable URLs, web-browsable storage of KG artifacts on cloud infrastructure, and easy reuse of transformed subgraphs across projects. Current KG-Hub projects span use cases including COVID-19 research, drug repurposing, microbial–environmental interactions, and rare disease research. KG-Hub is equipped with tooling to easily analyze and manipulate KGs. KG-Hub is also tightly integrated with graph machine learning (ML) tools which allow automated graph ML, including node embeddings and training of models for link prediction and node classification.

59 BASIC BIOLOGICAL SCIENCES↗

Predicting industrial building energy consumption with statistical and machine-learning models informed by physical system parameters

The industrial sector consumes about one-third of global energy, making them a frequent target for energy use reduction. Variation in energy usage is observed with weather conditions, as space conditioning needs to change seasonally, and with production, energy-using equipment is directly tied to production rate. Previous models were based on engineering analyses of equipment and relied on site-specific details. Others consisted of single-variable regressors that did not capture all contributions to energy consumption. Further, new modeling techniques could be applied to rectify these weaknesses. Applying data from 45 different manufacturing plants obtained from industrial energy audits, a supervised machine-learning model is developed to create a general predictor for industrial building energy consumption. The model uses features of air enthalpy, solar radiation, and wind speed to predict weather-dependency; motor, steam, and compressed air system parameters to capture support equipment contributions; and operating schedule, production rate, number of employees, and floor area to determine production-dependency. Results showed that a model that used a linear regressor over a transformed feature space could outperform a support vector machine and utilize features more representative of physical systems. Using informed parameters to build a reliable predictor will more accurately characterize a manufacturing facility's energy savings opportunities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Root genetics in the field to understand drought adaptation and carbon sequestration (Final Scientific/Technical Report)

For all crop plants, roots play a critical role in growth. Roots anchor the plants, and are the primary site of nutrient and water uptake. Roots are also the main source of C to soil in the form of root tissues and exudates, and thus greatly influence SOM stocks. To perform these functions, primary roots extend into soil, producing a network of branching roots of characteristic form, known as its root system architecture (RSA). RSA varies among species, and among varieties within a species that are adapted to different environments. Root traits are major targets for the second green revolution because of their potential to improve crop productivity, increase drought tolerance and nutrient acquisition, and increase C capture of soil. Improving the quality of roots in maize will be particularly valuable, since this crop is planted on over 92 million acres annually in the US. The future sustainability of agricultural systems relies on their ability to enhance soil organic matter (SOM) storage and reduce GHG emissions, while maintaining or enhancing productivity. This program had two components, Sensors and Models. For the first component, we designed and built a high-throughput phenotyping platform for root pulling of maize plants. This eliminated the physical labor of manually pulling up plants and reduced the number of personnel required down to one. The standardized pulling mechanism allowed recording force curves during the pulling process, providing additional information. We validated that the maximum force for pulling the root system was well-correlated with the root system mass and provided root crowns for further RSA analysis. These root crowns identified significant correlations with 2D root area and root depth, along with 3D root volume, total root length and number of root tips. We then used this system for field-based studies in maize on the genetics of root system architecture and its relation to nitrogen-use efficiency (NUE), including using lines relevant to the Corteva breeding program. Varieties were also evaluated at Corteva sites in the cornbelt and Danforth farm in Missouri, to establish responses across sites. From these studies we have identified genetic loci associated with root traits and created mutant lines for these loci and correlations of root traits with NUE. For the Models component, we worked to incorporate root and soil characteristics into the MEMS 2.0 soil and ecosystem biogeochemical model. Existing soil C models, such as Century, are unable to represent specific root trait interactions with the soil environment and therefore to accurately forecast the potential C sequestration benefits of root breeding under different climatic and soil type conditions. We have developed the MEMS 2.0 ecosystem biogeochemical model to improve quantification of farm-scale soil carbon and greenhouse gas emissions. The new knowledge and large datasets produced by this project will be used to develop and drive an innovative model capable of forecasting the impacts on soil C stocks and nutrient dynamics. An innovation was to use the empirical data from the field studies (in 1, above) to model genetic variation in nitrogen use efficiencies and soil C input. Our work demonstrated that maize root-derived C rapidly replaces existing soil C and after 3 years of continuous maize, up to 20% of soil organic C in the topsoil (0-15cm) and 3% in the subsoil (15-30cm) was contributed by maize. However, this contribution did not entirely represent a net increase. Root C contribution to soil was affected by maize genetics. We have analyzed soils derived from the CSU field trials for C and N stocks, in the different soil physical fractions represented by the MEMS model, using both physical fractionation with elemental analyses, and Fourier transformed infrared spectroscopy. Data will be used to link crop nitrogen use efficiencies with soil C sequestration and provide data to bridge the field trials with the model development, for verification of model predictions. The project had a number of successful outcomes: we have used the new phenotyping platform to identify new genetic loci that can enhance root phenotypes; we have partnered with multiple maize seed companies phenotype varieties in their breeding programs; we have developed the MEMS model that can help inform industry on the potential for carbon sequestration in the agricultural sector, and which is now available at the CSU Soil Carbon Solutions Center for use.

59 BASIC BIOLOGICAL SCIENCES↗

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state (t) and previous context (over timestamps [t–1,t–l], l is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp t. We consider diverse benchmarks with varying levels of "novelty" as measured by the TEA (Temporal Edge Appearance) plots. Here, our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

97 MATHEMATICS AND COMPUTING↗

2D Ising Model for Adsorption–induced Enantiopurification of Racemates

Mechanisms for the spontaneous transformation of achiral chemical systems into states of enantiomeric purity have important ramifications in modern pharmacology and potential relevance to the origins of homochirality in life on Earth. Such mechanisms for enantiopurification are needed for production of chiral pharmaceuticals and other bioactive compounds. Previously proposed chemical mechanisms leading from achiral systems to near homochirality are initiated by a symmetry–breaking step resulting in a minor excess of one enantiomer via statistical fluctuations in enantiomer concentrations. Subsequent irreversible processes then amplify the majority enantiomer concentration while simultaneously suppressing minority enantiomer production. Herein, equilibrium adsorption of amino acid enantiomer mixtures onto chiral and achiral surfaces reveals amplification of surface enantiomeric excess relative to the gas phase; i. e. enantiopurification of chiral adsorbates by adsorption. Furthermore, this adsorption–induced amplification of enantiomeric excess is shown to be well–describe by the 2D Ising model. More importantly, the 2D–Ising model predicts formation of homochiral monolayers from adsorption of racemic mixtures or prochiral molecules on achiral surfaces; i.e. enantiopurification with no apparent chiral driving force.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

General Bayesian Inference over the Stiefel Manifold via the Givens Transform

We introduce here an approach based on the Givens representation for posterior inference in statistical models with orthogonal matrix parameters, such as factor models and probabilistic principal component analysis (PPCA). We show how the Givens representation can be used to develop practical methods for transforming densities over the Stiefel manifold into densities over subsets of Euclidean space. We demonstrate how to deal with issues arising from the topology of the Stiefel manifold and how to inexpensively compute the change-of-measure terms. We introduce an auxiliary parameter approach that limits the impact of topological issues. We provide both analysis of our methods and numerical examples demonstrating the effectiveness of the approach. We also discuss how our Givens representation can be used to define general classes of distributions over the space of orthogonal matrices. We then give demonstrations on several examples showing how the Givens approach performs in practice in comparison with other methods.

97 MATHEMATICS AND COMPUTING↗

Application of nonlinear transformations to automatic flight control

The theory of transformations of nonlinear systems to linear ones is applied to the design of an automatic flight controller for the UH-1H helicopter. The helicopter mathematical model is described and it is shown to satisfy the necessary and sufficient conditions for transformability. The mapping is constructed, taking the nonlinear model to canonical form. The performance of the automatic control system in a detailed simulation on the flight computer is summarized.

Meyer, G.↗

A Deep State Space Model for Rainfall‐Runoff Simulations

The classical way of studying the rainfall‐runoff processes in the water cycle relies on conceptual or physically‐based hydrologic models. Deep learning (DL) has recently emerged as an alternative and blossomed in the hydrology community for rainfall‐runoff simulations. However, the decades‐old Long Short‐Term Memory (LSTM) network remains the benchmark for this task, outperforming newer architectures like Transformers. In this work, we propose a State Space Model (SSM), specifically the Frequency Tuned Diagonal State Space Sequence (S4D‐FT) model, for rainfall‐runoff simulations. The proposed S4D‐FT is benchmarked against the established LSTM and a physically‐based Sacramento Soil Moisture Accounting model under in‐sample and out‐of‐sample simulation setups across 531 watersheds in the contiguous United States (CONUS). Results show that S4D‐FT is able to outperform the LSTM model across diverse regions under both simulation setups, especially for regions that feature snowmelt‐driven or intermittent flow regimes. In contrast, S4D‐FT tends to underperform in flashier, high‐magnitude flow regimes, likely due to its global state‐space convolution computation that emphasizes slow, storage‐driven dynamics, which makes it less effective at picking up short bursts and noisy spikes in the data. In summary, our pioneering introduction of the S4D‐FT for rainfall‐runoff simulations challenges the dominance of LSTM in the hydrology community and expands the arsenal of DL tools available for hydrological modeling.

Wang, Yihan [Univ. of Oklahoma, Norman, OK (United↗

Short‐Term Hourly Weather Forecasting Using PredRNN With Image Preprocessing

Global weather forecast models are vital tools with numerous applications, including public safety, agriculture, and transportation. Recent advancements in artificial intelligence (AI) and deep learning (DL) have shown the potential to enhance weather forecasting accuracy and speed. In this study, we developed a short-term hourly weather forecast framework with a wavelet transform function for data preprocessing and a spatiotemporal DL model, PredRNN, for predicting five surface atmospheric variables, including wind speed and direction, mean sea level pressure (MSLP), temperature, and precipitation. The framework demonstrated promising results. It produces global forecasts at 0.25° (∼25 km) with a 1-day lead time RMSE of 1.8 m/s for wind components, 180 Pa for MSLP, and 1.8 K for temperature. Although our model does not surpass state-of-the-art AI weather forecast models across all metrics, it outperforms these models in precipitation forecasting and wind prediction at short lead times and achieves comparable accuracy for MSLP. Its native hourly forecasting capability, together with training on widely accessible GPU hardware, contributes meaningfully to the advancement of accessible DL weather forecasting methods. Our work highlights the importance of integrating temporal components and data transformation techniques to improve the predictability and accuracy of weather forecasts.

Tran, Hoang [Pacific Northwest National Laboratory↗

Fault diagnosis based on continuous simulation models

The results are described of an investigation of techniques for using continuous simulation models as basis for reasoning about physical systems, with emphasis on the diagnosis of system faults. It is assumed that a continuous simulation model of the properly operating system is available. Malfunctions are diagnosed by posing the question: how can we make the model behave like that. The adjustments that must be made to the model to produce the observed behavior usually provide definitive clues to the nature of the malfunction. A novel application of Dijkstra's weakest precondition predicate transformer is used to derive the preconditions for producing the required model behavior. To minimize the size of the search space, an envisionment generator based on interval mathematics was developed. In addition to its intended application, the ability to generate qualitative state spaces automatically from quantitative simulations proved to be a fruitful avenue of investigation in its own right. Implementations of the Dijkstra transform and the envisionment generator are reproduced in the Appendix.

Feyock, Stefan↗

Benchmarking the exponential ansatz for the Holstein model

Polarons are quasiparticles formed as a result of lattice distortions induced by charge carriers. The single-electron Holstein model captures the fundamentals of single polaron physics. We examine the power of the exponential ansatz for the polaron ground-state wavefunction in its coupled cluster, canonical transformation, and (canonically transformed) perturbative variants across the parameter space of the Holstein model. Our benchmark serves to guide future developments of polaron wavefunctions beyond the single-electron Holstein model.

Chemistry↗

Kondo Lattice Model of Magic-Angle Twisted-Bilayer Graphene: Hund’s Rule, Local-Moment Fluctuations, and Low-Energy Effective Theory

For this work, we apply a generalized Schrieffer-Wolff transformation to the extended Anderson-like topological heavy fermion (THF) model for the magic-angle (θ=1.05°) twisted bilayer graphene (MATBLG) [Phys. Rev. Lett. 129, 047601 (2022)], to obtain its Kondo lattice limit. In this limit localized $\mathcal{f}$ electrons on a triangular lattice interact with topological conduction $\mathcal{c}$ electrons. By solving the exact limit of the THF model, we show that the integer fillings ν=0, ±1, ±2 are controlled by the heavy $\mathcal{f}$ electrons, while ν=±3 is at the border of a phase transition between two $\mathcal{f}$-electron fillings. For ν=0, ±1, ±2, we then calculate the Ruderman-Kittel-Kasuya-Yosida (RKKY) interactions between the $\mathcal{f}$ moments in the full model and analytically prove the SU(4) Hund’s rule for the ground state which maintains that two $\mathcal{f}$ electrons fill the same valley-spin flavor. Our (ferromagnetic interactions in the) spin model dramatically differ from the usual Heisenberg antiferromagnetic interactions expected at strong coupling. We show the ground state in some limits can be found exactly by employing a positive semidefinite “bond-operators” method. We then compute the excitation spectrum of the $\mathcal{f}$ moments in the ordered ground state, prove the stability of the ground state favored by RKKY interactions, and discuss the properties of the Goldstone modes, the (reason for the accidental) degeneracy of (some of) the excitation modes, and the physics of their phase stiffness. We develop a low-energy effective theory for the $\mathcal{f}$ moments and obtain analytic expressions for the dispersion of the collective modes. We discuss the relevance of our results to the spin-entropy experiments in TBG.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Modeling Framework for Bulk Electric Grid Impacts from HEMP E1 and E3 Effects (Tasks 3.1 Final Report)

This report presents a framework to evaluate the impact of a high-altitude electromagnetic pulse (HEMP) event on a bulk electric power grid. This report limits itself to modeling the impact of EMP E1 and E3 components. The co-simulation of E1 and E3 is presented in detail, and the focus of the paper is on the framework rather than actual results. This approach is highly conservative as E1 and E3 are not maximized with the same event characteristics and may only slightly overlap. The actual results shown in this report are based on a synthetic grid with synthetic data and a limited exemplary EMP model. The framework presented can be leveraged and used to analyze the impact of other threat scenarios, both manmade and natural disasters. This report d escribes a Monte-Carlo based methodology to probabilistically quantify the transient response of the power grid to a HEMP event. The approach uses multiple fundamental steps to characterize the system response to HEMP events, focused on the E1 and E3 components of the event. 1) Obtain component failure data related to HEMP events testing of components and creating component failure models. Use the component failure model to create component failure conditional probability density function (PDF) that is a function of the HEMP induced terminal voltage. 2) Model HEMP scenarios and calculate the E1 coupled voltage profiles seen by all system components. Model the same HEMP scenarios and calculate the transformer reactive power consumption profiles due to E3. 3) Sample each component failure PDF to determine which grid components will fail, due to the E1 voltage spike, for each scenario. 4) Perform dynamic simulations that incorporate the predicted component failures from E1 and reactive power consumption at each transformer affected by E3. These simulations allow for secondary transients to affect the relays/protection remaining in service which can lead to cascading outages. 5) Identify the locations and amount of load lost for each scenario through grid dynamic simulation. This can be an indication of the immediate grid impacts from a HEMP event. In addition, perform more detailed analysis to determine critical nodes and system trends. 6) To help realize the longer-term impacts, a security constrained alternating current optimal power flow (ACOPF) is run to maximize critical load served. This report describes a modeling framework to assess the systemic grid impacts due to a HEMP event. This stochastic simulation framework generates a large amount of data for each Monte Carlo replication, including HEMP location and characteristics, relay and component failures, E3 GIC profiles, cascading dynamics including voltage and frequency over time, and final system state. This data can then be analyzed to identify trends, e.g., unique system behavior modes or critical components whose failure is more likely to cause serious systemic effects. The proposed analysis process is demonstrated on a representative system. In order to draw realistic conclusions of the impact of a HEMP event on the grid, a significant amount of work remains with respect to modeling the impact on various grid components.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Predicting the propensity for thermally activated β events in metallic glasses via interpretable machine learning

Abstract The elementary excitations in metallic glasses (MGs), i.e., β processes that involve hopping between nearby sub-basins, underlie many unusual properties of the amorphous alloys. A high-efficacy prediction of the propensity for those activated processes from solely the atomic positions, however, has remained a daunting challenge. Recently, employing well-designed site environment descriptors and machine learning (ML), notable progress has been made in predicting the propensity for stress-activated β processes (i.e., shear transformations) from the static structure. However, the complex tensorial stress field and direction-dependent activation could induce non-trivial noises in the data, limiting the accuracy of the structure-property mapping learned. Here, we focus on the thermally activated elementary excitations and generate high-quality data in several Cu-Zr MGs, allowing quantitative mapping of the potential energy landscape. After fingerprinting the atomic environment with short- and medium-range interstice distribution, ML can identify the atoms with strong resistance or high compliance to thermal activation, at a high accuracy over ML models for stress-driven activation events. Interestingly, a quantitative “between-task” transferring test reveals that our learnt model can also generalize to predict the propensity of shear transformation. Our dataset is potentially useful for benchmarking future ML models on structure-property relationships in MGs.

36 MATERIALS SCIENCE↗

AGFormer: Adaptive Spatiotemporal graph informed transformer for multi-reservoir inflow forecasting

Accurate reservoir inflow forecasting is crucial for effective water resource management, yet most machine learning models focus on single-reservoir prediction and overlook spatial dependencies among hydrologically connected reservoirs. Here, we propose AGFormer (Adaptive Graph-Informed Transformer), an end-to-end framework that integrates adaptive graph learning with temporal sequence modeling for multi-reservoir inflow forecasting. A shared encoder and graph attention mechanism generate reservoir-specific embeddings, which are then processed by the Transformer-based encoder–decoder for multi-step inflow forecasting. We also introduce a pretraining paradigm to learn robust temporal embeddings from misaligned historical records. Evaluated on 30 reservoirs in the Upper Colorado River Basin, AGFormer achieves superior seven-day-ahead forecasts, with NSE > 0.75 for 20 reservoirs—outperforming Encoder–Decoder LSTM, GCN+LSTM, and Transformer baselines. Adaptive graph learning captures dynamic inter-reservoir dependencies, and feature attribution aligns with snowmelt-driven hydrology. Incorporating forecasted meteorological inputs further enhances accuracy, demonstrating AGFormer’s potential to support reservoir management under dynamic hydrological conditions.

Adaptive graph learning↗

2040 Vision: A Roadmap for Integrated, Multiscale Materials and System Modeling and Simulation

Over the last few decades, advances in high-performance computing, new materials characterization methods, and, more recently, an emphasis on integrated computational materials engineering (ICME) and additive manufacturing have been a catalyst for multiscale modeling and simulation-based design of materials and structures in the aerospace industry. While these advances have driven significant progress in the development of aerospace components and systems, that progress has been limited by persistent technology and infrastructure challenges that must be overcome to realize the full potential of integrated materials and systems design and simulation modeling throughout the supply chain. As a result, NASA's Transformational Tools and Technology (TTT) Project sponsored a study (performed by a team led by Pratt & Whitney) to define the potential 25-year future state required for integrated multiscale modeling of materials and systems (e.g., load-bearing structures) to accelerate the pace and reduce the expense of innovation in future aerospace and aeronautical systems. This talk will describe the findings of this 2040 Vision study (e.g., the 2040 vision state; the required interdependent core technical work areas, Key Element (KE); associated critical gaps and actions to close those gaps; and major recommendations). This roadmap is a community consensus document and is a result of over 450 professionals input obtain via: i) four society workshops (AIAA, NAFEMS, and two TMS), ii) community-wide survey, and iii) the establishment of 9 expert panels (one per key element) consisting on average 10 non-team members from academia, government and industry to review, update content, and prioritize gaps and actions. It is NASA's hope and desire that this roadmap provides the strategic guidance to both public and private research and development decision makers to make the proposed 2040 vision state a reality and thereby provide a significant advancement in the United States global competitiveness.

Multiscale Modeling↗

NASA’s Safety, Reliability, and Mission Assurance Digital Future

The evolution from “document-centric” to “data-centric” and “model-centric” information leveraging structured data and model-based approaches is at the heart of digital engineering transformational efforts underway across industry and government. It is these approaches that pave the way for data lakes, Authoritative Sources of Truth (ASOTs), and systems- of-systems interoperability and the corresponding transformational benefits thereof. Such benefits include increased data availability, data access equity, data traceability, real-time analytics, batch analytics, and (most importantly) acceleration of the time-to-value and time-to-insights associated with engineering products and analyses. The longer-term benefits of reusability, customization and traceability are even more promising. For Safety and Mission Assurance (SMA), and Mission Success (SMS) activities; realization of such benefits is essential to provide engineers and analysts alike vital information when needed to support critical decision making throughout the entire life cycle. The SMA community often operate in parallel with engineering activities, for which information exchange with relevant context is paramount. Far too often, such information lags key decision points and/or is absent of the robust, integrated, knowledge needed, given inherent barriers associated with traditional document-centric means to data sharing, analysis, and reporting. This paper provides an overview of how NASA’s Office of Safety and Mission Assurance (OSMA) is evolving its policies, standards, guidance, and training to transform to eliminate such barriers, thus realizing the benefits emerging in this new digital era. A roadmap for achieving this digital future is presented along with key building blocks involving use and implementation of concepts such as: Objectives-Hierarchies, Objective-Driven Requirements, Accepted Standards, Safety and Assurance Cases, data digitization (i.e., ontologies, structured data, and model-centric data), FAIR (Findable, Accessible, Interoperable, & Reusable) and/or FAIRUST (Findable, Accessible, Interoperable, Reusable, Understandable, Secure, and Trusted) principles [1]. This paper also describes how OSMA, leveraging the Agency’s overall commitment to Digital Transformation (DT), is using the power of Policy, “Digital” Domain representation, Product Evolution, and Community Outreach and Engagement as part of a strategic vision and roadmap to evolve and transform its SMA organizations to become better able to serve its stakeholders and customers. Future publications will elaborate on these building blocks and deeper concepts.

Authoritative Source of Truth (ASOT),↗