Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

Range and Intensity Image-Based Terrain and Vehicle Relative Pose Estimation System

A navigation system includes an image acquisition device for acquiring a range image of a target vehicle, at least one processor, a memory including a target vehicle model and computer readable program code, where the processor and the computer readable program code are configured to cause the navigation system to convert the range image to a point cloud having three dimensions, compute a transform from the target vehicle model to the point cloud, and use the transform to estimate the target vehicle's attitude and position for capturing the target vehicle.

Gill, Nathaniel↗

When does global attention help: a unified empirical study on atomistic graph learning

Graph neural networks (GNNs) are widely used as surrogates for costly experiments and first-principles simulations to study the behavior of compounds at atomistic scale, and their architectural complexity is constantly increasing to enable the modeling of complex physics. While most recent GNNs combine more traditional message passing neural networks (MPNNs) layers to model short-range interactions with more advanced graph transformers (GTs) with global attention mechanisms to model long-range interactions, it is still unclear when global attention mechanisms provide real benefits over well-tuned MPNN layers due to inconsistent implementations, features, or hyperparameter tuning. We introduce the first unified, reproducible benchmarking framework–built on HydraGNN–that enables seamless switching among four controlled model classes: MPNN, MPNN with chemistry/topology encoders, GPS-style hybrids of MPNN with global attention, and fully fused localglobal models with encoders. Using seven diverse open-source datasets for benchmarking across regression and classification tasks, we systematically isolate the contributions of message passing, global attention, and encoder-based feature augmentation. Our study shows that encoder-augmented MPNNs form a robust baseline, while fused localglobal models yield the clearest benefits for properties governed by long-range interaction effects. We further quantify the accuracycompute trade-offs of attention, reporting its overhead in memory. Together, these results establish the first controlled evaluation of global attention in atomistic graph learning and provide a reproducible testbed for future model development.

Equivariant graph neural networks↗

Probabilistic Diffusion Models Advance Extreme Flood Forecasting

Extreme floods pose escalating risks in a changing climate, yet forecasting remains challenging due to peak flow underestimation and high uncertainty. We introduce diffusion-based runoff model (DRUM), a probabilistic deep learning (DL) approach that advances extreme flood forecasting across representative basins in the contiguous United States. DRUM outperforms state-of-the-art benchmarks, enhancing nowcasting skill for the top 1‰ of flows in 72.3% of studied basins. Under operational scenarios, DRUM extends reliable lead times by nearly a full day for 20- and 50-year floods. When evaluated with measured precipitation, an ideal condition, recall improves by 0.3–0.4 and the early warning window extends by 2.3 days for 50-year floods. The enhancement potential varies regionally, with precipitation-driven flood zones in the eastern and northwestern US benefiting most, gaining 3–7 days in lead time. These findings highlight the transformative potential of diffusion models as a cutting-edge generative AI technique for advancing hydrology and broader Earth system sciences.

54 ENVIRONMENTAL SCIENCES↗

Building Initial Dynamic System Models for Digital Twins of the Cryogenic Moderator System at the ORNL Spallation Neutron Source

This work describes the initial development of dynamic system models of the cryogenic moderator system (CMS) of the Spallation Neutron Source (SNS) at ORNL as a part of the ORNL LDRD funded project Building TRANSFORM to Accelerate Digital Twin Applications for Nuclear Systems, LOIS 10563. The goal of the work is to start the dynamic system modeling effort with the end goal of using them for real-time applications as digital twins. The CMS is a cryogenic liquid hydrogen flow loop that provides moderation of the neutrons that are generated by the SNS. For optimal neutron production, the CMS needs to maintain a steady and controlled density of cryogenic hydrogen in the moderator section thus requiring precise temperature and pressure control. Due to the varied time scales and system characteristics, control of the system is complex, and diagnostics are also difficult. Difficulty in accessing the flow loop during operations, limited instrumentation and unknown design details of the equipment combine to make the case for having sophisticated digital twin models of the system. Operationally the CMS also provides a strong use case for digital twins due to the constant need of optimization and for troubleshooting/diagnostics. The large amount of data collected which are freely available for using in building the model and verifying and validating the model also makes it a great candidate for a proof-of-concept for digital twins. The project extends ORNL's capacity of development and implementation of the open-source dynamic system modeling tool TRANSFORM for engineering design and digital twin/real-time applications. Specific system configuration data for the CMS have been gathered and an initial dynamic model was created in the TRANSFORM library using Dymola as the solution platform. Models of increasing complexity are created to demonstrate the need for a multi-layered approach in digital twin modeling depending on the scale and phenomena being focused on. The dynamic modeling is shown to bring the dynamic operational aspects to the design process for systems as well as serve as a digital twin to the hardware and allow for models to be tuned and compared against real time operational data. These aims should help to push forward strategic goals of application of digital twins and increase the impact of ORNL systems modeling capabilities with TRANSFORM/Modelica for various advanced energy systems.

42 ENGINEERING↗

Model Deformation Measurements of Sonic Boom Models in the NASA Ames 9- by 7-Ft Supersonic Wind Tunnel

The deformations of two sonic-boom models were measured by stereo photogrammetry during tests in the 9- by 7-Ft Supersonic Wind Tunnel at NASA Ames Research Center. The models were geometrically similar but one was 2.75 times as large as the other. Deformation measurements were made by simultaneously imaging the upper surfaces of the models from two directions by calibrated cameras that were mounted behind windows of the test section. Bending and twist were measured at discrete points using conventional circular targets that had been marked along the leading and trailing edges of the wings and tails. In addition, continuous distributions of bending and twist were measured from ink speckles that had been applied to the upper surfaces of the model. Measurements were made at wind-on (M = 1.6) and wind-off conditions over a range of angles of attack between 2.5 deg. and 5.0 deg. At each condition, model deformation was determined by comparing the wind-off and wind-on coordinates of each measurement point after transforming the coordinates to reference coordinates tied to the model. The necessary transformations were determined by measuring the positions of a set of targets on the rigid center-body of the models whose model-axes coordinates were known. Smoothly varying bending and twist measurements were obtained at all conditions. Bending displacements increased in proportion to the square of the distance to the centerline. Maximum deflection of the wingtip of the larger model was about 5 mm (2% of the semispan) and that of the smaller model was 0.9 mm (1% of the semispan). The change in wing twist due to bending increased in direct proportion to distance from the centerline and reached a (absolute) maximum of about -1 at the highest angle of attack for both models. The measurements easily resolved bending displacements as small as 0.05 mm and bending-induced changes in twist as small as 0.05 deg.

model deformation↗

Intrinsic activation energies for ring contraction of allylic cations in zeolites

Cyclic carbocations are important intermediates in zeolite-catalyzed chemistries such as methanol-to-hydrocarbons conversion, naphthenes ring opening, or coke formation. While information about their thermodynamic stability exists, little is known about the kinetics of formation and transformation of cyclic carbocations in zeolites. To fill this knowledge gap, ring contraction of the 1,3,5,5-tetramethylcyclohexenyl cation (C 10 H 17 + ), a representative of 6-membered ring allylic cations, was investigated by in situ UV–vis and IR spectroscopy. Protonic forms of zeolites served as catalysts, at temperatures from 80 °C to 135 °C. Significant oligomerization and hydride transfer in BEA and FAU hampered kinetics analysis, whereas ring contraction dominated in the channels of MOR. The reactant cation, characterized by an electronic absorption at 314 nm and an allylic stretch at 1549 cm −1 , contracted to both a 1,3-alkyl-substituted cyclopentenyl cation (287 nm and 1506 cm −1 ) and a 1,2,3-alkyl-substituted cyclopentenyl cation (297 nm and 1489 cm −1 ). Collection of time-resolved IR spectra and fitting of the intensities with various kinetic models revealed a third transformation, which is expected from thermodynamics: the 1,3-alkyl-substituted cyclopentenyl cation isomerizes to the 1,2,3-alkyl-substituted cyclopentenyl cation. Series of IR spectra recorded at different temperatures delivered intrinsic activation enthalpies (entropies) in MOR of 67 ± 2 kJ mol −1 (−130 ± 6 J mol −1 K −1 ) and 90 ± 3 kJ mol −1 (−70 ± 8 J mol −1 K −1 ) for the contraction to 1,3- and 1,2,3-substituted species, and of 88 ± 5 kJ mol −1 (−84 ± 12 J mol −1 K −1 ) for the isomerization. The findings characterize one path – via contraction of larger rings – to different cyclopentenyl species in zeolites; and the associated, moderate activation energies suggest such transformations contribute to many complex hydrocarbon reaction networks.

Acid catalysis↗

Data-Driven Study of Shape Memory Behavior of Multi-component Ni-Ti Alloys

Ni-Ti based shape memory alloys (SMAs) have found wide-spread use in aerospace, automotive, biomedical, and commercial applications owing to their favorable properties and ease of operation. Especially important for many NASA applications is the ability to tune the martensitic transformation temperature of Ni-Ti alloys by varying the composition and processing conditions. Recently, researchers at NASA have compiled an extensive database of shape memory properties of materials, including over 8,000 multi-component Ni-Ti alloys containing 37 different alloying elements. Using this dataset, machine learning models are trained to predict transformation temperatures, hysteresis, and transformation strain with extremely small errors. These models are used to learn relationships between shape memory behavior and input parameters in the composition and processing space. ML predictions are validated through new experiments. The combination of an extensive dataset and accurate learning models, together, make our approach highly suitable for the rapid discovery of novel SMAs with targeted properties.

Shape Memory Alloys↗

Application of machine learning to estimate fireball characteristics and their uncertainty from infrared spectral data

Experiments or events involving high explosives (HE) can be monitored remotely by infrared (IR) sensors to gather information about the configuration or materials involved in the device. Researchers at the Air Force Institute of Technology (AFIT) developed a phenomenological model for HE fireball spectra in the IR range that allows for parameters to be extracted from Fourier transform infrared (FTIR) data. This model includes parameters tied to physical characteristics of the fireball: temperature, size, soot, and gas concentrations. Previous works have sought to recover these parameters by the fitting of either whole spectra or select wavenumber bands to this phenomenological model. Difficulties arise due to the complex relationships between the parameters to be fit. Uncertainty quantification of the estimated fireball parameters is also problematic since HE experiments do not have any ground truth information on the parameters. It is suggested that artificial neural network (ANN) based approaches may be well suited to this problem, because of their ability to capture complex and highly nonlinear relationships. As such, this work seeks to explore the efficacy of deep artificial neural networks (DNNs) for this problem of parameter recovery from spectra and to also investigate the uncertainty of recovering the fireball parameters from FTIR data. Networks are designed using the hyperparameter optimization tool Hyperopt and trained/tested on artificial data generated using the phenomenological model developed by AFIT. The results of applying the network to the artificial data set are compared to a physics-based band approach that uses a selected number of bands based on their physical properties. Information on the uncertainty of estimating parameters from remotely sensed experimental data is obtained by treating the accuracy of the DNN model on artificial data as an upper bound and by examining the impact of emissivity due to soot on parameter estimation error; the results for artificial data are likely to be optimistic as compared to recovering parameters from experimental data.

42 ENGINEERING↗

Data-Driven Study of Shape Memory Behavior of Multi-Component Ni-Ti Alloys

Ni-Ti based shape memory alloys (SMAs) have found wide-spread use in aerospace, automotive, biomedical, and commercial applications owing to their favorable properties and ease of operation. Especially important for many NASA applications is the ability to tune the martensitic transformation temperature of Ni-Ti alloys by varying the alloy composition and processing conditions. Recently, researchers at NASA have compiled an extensive database of shape memory properties of materials, including over 8,000 multi-component Ni-Ti alloys containing 37 different alloying elements. Using this dataset, machine learning models are trained to predict transformation temperatures, hysteresis, and transformation strain with extremely low mean absolute errors. These models are used to learn relationships between shape memory behavior and input parameters in the composition and processing space. ML predictions are validated through new experiments. The combination of an extensive experimental dataset and accurate learning models, together, make our approach highly suitable for the rapid discovery and design of novel SMAs with targeted properties. We are not aware of any current approaches capable of predicting SMA transformation behavior over such a wide range of compositions and processing conditions.

Shape memory alloys↗

PSCF+: An Extended and Improved Open-Source Software Package for Polymer Self-Consistent Field Calculations

This work introduces PSCF+, a freely available open‑source software tool for modeling how block copolymers self‑assemble into ordered nanoscale structures. The software allows researchers to study a wide range of polymer models and interaction types, making it possible to directly compare theoretical predictions with molecular and simulation‑based studies in a consistent way. PSCF+ incorporates several recently developed numerical methods that significantly reduce computer memory requirements and speed up calculations, enabling efficient use of modern GPU hardware. Here, it also provides automated tools to calculate free‑energy landscapes and phase boundaries, which are essential for understanding and predicting material behavior. Together, these advances make PSCF+ a powerful and flexible platform for studying polymer self‑assembly and support the design of new polymeric materials for scientific and technological applications.

Chemical calculations↗

Scaling transformations and the origins of light relics constraints from cosmic microwave background observations

Here, we use here a family of scaling transformations, that scale key rates in the evolution equations, to analytically understand constraints on light relics from cosmic microwave background (CMB) maps, given cosmological models of varying degrees of complexity. We describe the causes of physical effects that are fundamentally important to the constraining power of the data, with a focus on the two that are most novel. We use as a reference model a cosmological model that admits a scaling transformation that increases light relic energy density while avoiding all of these causes. Constraints on light relics in a given model can then be understood as due to the differences between the given model and the reference model, as long as the additional light relics only interact gravitationally with the Standard Model components. This understanding supports the development of cosmological models that can evade light relics constraints from CMB maps.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep learning-enabled natural language processing to identify directional pharmacokinetic drug–drug interactions

Background. During drug development, it is essential to gather information about the change of clinical exposure of a drug (object) due to the pharmacokinetic (PK) drug-drug interactions (DDIs) with another drug (precipitant). While many natural language processing (NLP) methods for DDI have been published, most were designed to evaluate if (and what kind of) DDI relationships exist in the text, without identifying the direction of DDI (object vs. precipitant drug). Here we present a method for the automatic identification of the directionality of a PK DDI from literature or drug labels. Methods. We reannotated the Text Analysis Conference (TAC) DDI track 2019 corpus for identifying the direction of a PK DDI and evaluated the performance of a fine-tuned BioBERT model on this task by following the training and validation steps prespecified by TAC. Results. This initial attempt showed the model achieved an F-score of 0.82 in identifying sentences as containing PK DDI and an F-score of 0.97 in identifying object versus precipitant drugs in those sentences. Discussion and conclusion. Despite a growing list of NLP methods for DDI extraction, most of them use a common set of corpora to perform general purpose tasks (e.g., classifying a sentence into one of several fixed DDI categories). There is a lack of coordination between the drug development and biomedical informatics method development community to develop corpora and methods to perform specific tasks (e.g., extract clinical exposure changes due to PK DDI). We hope that our effort can encourage such a coordination so that more “fit for purpose” NLP methods could be developed and used to facilitate the drug development process.

59 BASIC BIOLOGICAL SCIENCES↗

Operating advanced scientific instruments with AI agents that learn on the job

Advanced scientific user facilities, such as next generation X-ray light sources and self-driving laboratories, are revolutionizing scientific discovery by automating routine tasks and enabling rapid experimentation and characterizations. However, these facilities must continuously evolve to support new experimental workflows, adapt to diverse user projects, and meet growing demands for more intricate instruments and experiments. This continuous development introduces significant operational complexity, necessitating a focus on usability, reproducibility, and intuitive human-instrument interaction. In this work, we explore the integration of agentic AI, powered by Large Language Models (LLMs), as a transformative tool to achieve this goal. We present our approach to developing a human-in-the-loop pipeline for operating advanced instruments including an X-ray nanoprobe beamline and an autonomous robotic station dedicated to the design and characterization of materials. Specifically, we evaluate the potential of various LLMs as trainable scientific assistants for orchestrating complex, multi-task workflows, which also include multimodal data, optimizing their performance through optional human input and iterative learning. We demonstrate the ability of AI agents to bridge the gap between advanced automation and user-friendly operation, paving the way for more adaptable and intelligent scientific facilities.

Large Language Models↗

Privacy-Aware RAG-Enabled LLMs for Collaborative AI in Organizations

Recent advancements in Large Language Models (LLMs) based on Transformer architectures have significantly improved capabilities in natural language processing and generation. However, deploying LLMs for inter-organizational communication poses challenges, in ensuring privacy and facilitating effective collaboration. This paper introduces a novel decentralized inference meta-agent chatbot that leverages privacy-aware Retrieval-Augmented Generation (RAG)-enabled LLMs for collaborative AI communication across organizations. Built on Microsoft’s Autogen, the platform enables LLMs to autonomously refine responses, enhancing accuracy and relevance. It incorporates advanced hallucination mitigation techniques using Uptrain and a privacy-focused RAG framework that employs synthetic document generation to protect sensitive information. Comprehensive evaluations demonstrate the platform’s effectiveness in maintaining contextual relevance and stringent privacy standards, effectively addressing critical challenges in LLM-enhanced collaborative AI communication. This work represents a significant step toward secure and efficient inter-organizational collaboration using advanced generative AI technologies.

97 - MATHEMATICS AND COMPUTING↗

A System of Conservative Regridding for Ice-Atmosphere Coupling in a General Circulation Model (GCM)

The method of elevation classes, in which the ice surface model is run at multiple elevations within each grid cell, has proven to be a useful way for a low-resolution atmosphere inside a general circulation model (GCM) to produce high-resolution downscaled surface mass balance fields for use in one-way studies coupling atmospheres and ice flow models. Past uses of elevation classes have failed to conserve mass and energy because the transformation used to regrid to the atmosphere was inconsistent with the transformation used to downscale to the ice model. This would cause problems for two-way coupling. A strategy that resolves this conservation issue has been designed and is presented here. The approach identifies three grids between which data must be regridded and five transformations between those grids required by a typical coupled atmosphere-ice flow model. This paper develops a theoretical framework for the problem and shows how each of these transformations may be achieved in a consistent, conservative manner. These transformations are implemented in Glint2, a library used to couple atmosphere models with ice models. Source code and documentation are available for download. Confounding real-world issues are discussed, including the use of projections for ice modeling, how to handle dynamically changing ice geometry, and modifications required for finite element ice models.

Ice-atmosphere↗

Topographic hydro-conditioning to resolve surface depression storage and ponding in a fully distributed hydrologic model

Land surface depressions play a central role in the transformation of rainfall to ponding, infiltration and runoff, yet digital elevation models (DEMs) used by spatially distributed hydrologic models that resolve land surface processes rarely capture land surface depressions at spatial scales relevant to this transformation. Methods to generate DEMs through processing of remote sensing data, such as optical and light detection and ranging (LiDAR) have favored surfaces without depressions to avoid adverse slopes that are problematic for many hydrologic routing methods. Here, in this study, we present a new topographic conditioning workflow, Depression-Preserved DEM Processing (D2P) algorithm, which is designed to preserve physically meaningful surface depressions for depression-integrated and efficient hydrologic modeling. D2P includes several features: (1) an adaptive screening interval for delineation of depressions, (2) the ability to filter out anthropogenic land surface features (e.g., bridges), (3) the ability to blend river smoothing (e.g., a general downslope profile) and depression resolving functionality. From a case study in the Goodwin Creek Experimental Watershed, D2P successfully resolved 86% of the ponds at a DEM resolution of 10 m. Topographic conditioning was achieved with minimum impact as D2P reduced the number of modified cells from the original DEM by 51% compared to a conventional algorithm. Furthermore, hydrologic simulation using a D2P processed DEM resulted in a more robust characterization on surface water dynamics based on higher surface water storage as well as an attenuated and delayed peak streamflow.

54 ENVIRONMENTAL SCIENCES↗

Deep Multimodal Networks for M-type Star Classification with Paired Spectrum and Photometric Image

Abstract Traditional stellar classification methods include spectral and photometric classification separately. Although satisfactory results can be achieved, the accuracy could be improved. In this paper, we pioneer a novel approach to deeply fuse the spectra and photometric images of the sources in an advanced multimodal network to enhance the model’s discriminatory ability. We use Transformer as the fusion module and apply a spectrum–image contrastive loss function to enhance the consistency of the spectrum and photometric image of the same source in two different feature spaces. We perform M-type stellar subtype classification on two data sets with high and low signal-to-noise ratio (S/N) spectra and corresponding photometric images, and the F1-score achieves 95.65% and 90.84%, respectively. In our experiments, we prove that our model effectively utilizes the information from photometric images and is more accurate than advanced spectrum and photometric image classifiers. Our contributions can be summarized as follows: (1) We propose an innovative idea for stellar classification that allows the model to simultaneously consider information from spectra and photometric images. (2) We discover the challenge of fusing low-S/N spectra and photometric images in the Transformer and provide a solution. (3) The effectiveness of Transformer for spectral classification is discussed for the first time and will inspire more Transformer-based spectral classification models.

Astronomy & Astrophysics↗

The five-point bootstrap

We study five-point correlation functions of scalar operators in d-dimensional conformal field theories. We develop a new approach to computing the five-point conformal blocks for exchanged primary operators of arbitrary spin by introducing a generalization of radial coordinates, using an appropriate ansatz, and perturbatively solving two quadratic Casimir differential equations. We then study five-point correlators 〈σσϵσσ〉 in the critical 3d Ising model. We truncate the operator product expansions (OPEs) in the correlator by including a finite number of primary operators with conformal dimension below a cutoff ∆ ⩽ ∆ cutoff . We then compute several OPE coefficients involving ϵ and two spinning operators by demanding that the truncated correlator approximately satisfies the crossing relation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗