Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48

A test of the Johnson-Mehl-Avrami equation

The accuracy of the Johnson-Mehl-Avrami (JMA) equation is evaluated for the special case of two-dimensional crystallization. The volume fraction transformed is determined directly by computer modeling, and the evaluation of the secondary phase obtained is compared with predictions of the JMA equation. The JMA equation if found to be highly acccurate over virtually the entire range of the transformation process.

Weinberg, Michael C.↗

Single-phase power distribution system power flow and fault analysis

Alternative methods for power flow and fault analysis of single-phase distribution systems are presented. The algorithms for both power flow and fault analysis utilize a generalized approach to network modeling. The generalized admittance matrix, formed using elements of linear graph theory, is an accurate network model for all possible single-phase network configurations. Unlike the standard nodal admittance matrix formulation algorithms, the generalized approach uses generalized component models for the transmission line and transformer. The standard assumption of a common node voltage reference point is not required to construct the generalized admittance matrix. Therefore, truly accurate simulation results can be obtained for networks that cannot be modeled using traditional techniques.

Halpin, S. M.↗

SBML Level 3: an extensible format for the exchange and reuse of biological models

Systems biology has experienced dramatic growth in the number, size, and complexity of computational models. To reproduce simulation results and reuse models, researchers must exchange unambiguous model descriptions. We review the latest edition of the Systems Biology Markup Language (SBML), a format designed for this purpose. A community of modelers and software authors developed SBML Level 3 over the past decade. Its modular form consists of a core suited to representing reaction-based models and packages that extend the core with features suited to other model types including constraint-based models, reaction-diffusion models, logical network models, and rule-based models. The format leverages two decades of SBML and a rich software ecosystem that transformed how systems biologists build and interact with models. More recently, the rise of multiscale models of whole cells and organs, and new data sources such as single-cell measurements and live imaging, has precipitated new ways of integrating data with models. We provide our perspectives on the challenges presented by these developments and how SBML Level 3 provides the foundation needed to support this evolution.

59 BASIC BIOLOGICAL SCIENCES↗

Masked Symbol Modeling for Demodulation of Oversampled Baseband Communication Signals in Impulsive Noise-Dominated Channels

Recent breakthroughs in natural language processing show that attention mech- anism in Transformer networks, trained via masked-token prediction, enables models to capture the semantic context of the tokens and internalize the grammar of language. While the application of Transformers to communication systems is a burgeoning field, the notion of context within physical waveforms remains under-explored. This paper addresses that gap by re-examining inter-symbol con- tribution (ISC) caused by pulse-shaping overlap. Rather than treating ISC as a nuisance, we view it as a deterministic source of contextual information embedded in oversampled complex baseband signals. We propose Masked Symbol Model- ing (MSM), a framework for the physical (PHY) layer inspired by Bidirectional Encoder Representations from Transformers methodology. In MSM, a subset of symbol-aligned samples is randomly masked, and a Transformer predicts the missing symbol identifiers using the surrounding “in-between” samples. Through this objective, the model learns the latent syntax of complex baseband waveforms. We illustrate MSM’s potential by applying it to the task of demodulating sig- nals corrupted by impulsive noise, where the model infers corrupted segments by leveraging the learned context. Our results suggest a path toward receivers that interpret, rather than merely detect communication signals, opening new avenues for context-aware PHY layer design.

Bedir, Oguz↗

Generation and Calibration of Linear Models of Aircraft with Highly Coupled Aeroelastic and Flight Dynamics

The lightweight structures and unconventional configurations being considered for the next generation of aircraft mean that any effort to predict or control the flight dynamics is impacted by the structural dynamics. One of the most severe forms of coupling between aeroelasticity and flight dynamics is an instability called body freedom flutter. The existing tools often assume a relatively weak effect of structural dynamics on the flight dynamics, and are therefore incapable of modeling strong interactions like body freedom flutter. A method of combining different sources of data traditionally used for aeroelasticity and flight dynamics is described by reconciling many of the differences between these models. By building upon past modeling efforts, a level of familiarity in the approach is achieved. Generally the differences from the traditional approaches are subtle but significant. The traditional frequency domain flutter model in a modal coordinate system is converted to a form consistent with a time domain flight dynamics model. The time domain rational function approximation about a non-inertial coordinate system and the unique constraints for the conversion between the inertial and non-inertial coordinate systems are discussed. A consistent transformation of the states of aeroelastic models to flight dynamics models is derived, which enables the integration of data from higher fidelity computational fluid dynamics models or wind-tunnel testing. The present method of integrating multidisciplinary data was used to create models that compare well with X-56A flight-test data, including conditions past the flutter speed.

Ouellette, Jeffrey↗

Interpreting and Accelerating Transformers for Jet Tagging

Attention-based transformers are ubiquitous in machine learning applications from natural language processing to computer vision. In high energy physics, one central application is to classify collimated particle showers in colliders based on the particle of origin, known as jet tagging. In this work, we study the interpretatbility and prospects for acceleration of Particle Transformer (ParT), a state-of-the-art model, leverages particle-level attention to improve jet-tagging performance. We analyzing ParT's attention maps and particle-pair correlations in the eta-phi plane, revealing intriguing features, such as a binary attention pattern that identifies critical substructure in jets. These insights enhance our understanding of the model's internal workings and learning process and hint at ways to improve its efficiency. Along these lines, we also explore low-rank attention, attention alternatives, and dynamic quantization to accelerate transformers for jet tagging. With quantization, we achieve a 50% reduction in model size and a 10% increase in inference speed without compromising accuracy. These combined efforts enhance both the performance and the interpretability of transformers in high-energy physics, opening avenues for more efficient and physics-driven model designs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Compressing Vision Transformers in Geospatial Transfer Learning with Manifold-Constrained Optimization

Deploying geospatial foundation models on resource-constrained edge devices demands compact architectures that maintain high downstream performance. However, their large parameter counts and the accuracy loss often induced by compression limit practical adoption.In this work, we leverage manifold-constrained optimization framework DLRT to compress large vision transformer–based geospatial foundation models during transfer learning. By enforcing structured low-dimensional parameterizations aligned with downstream objectives, this approach achieves strong compression while preserving task-specific accuracy. We show that the method outperforms of-the-shelf low-rank methods as LoRA. Experiments on diverse geospatial benchmarks confirm substantial parameter reduction with minimal accuracy loss, enabling high-performing, on-device geospatial models.

Snyder, Thomas [Yale University]↗

An Overview of Nano Multilayers as Model Systems for Developing Nanoscale Microstructures

The microstructural transformations of binary nanometallic multilayers (NMMs) to equiaxed nanostructured materials were explored by characterizing a variety of nanoscale multilayer films. Four material systems of multilayer films, Hf-Ti, Ta-Hf, W-Cr, and Mo-Au, were synthesized by magnetron sputtering, heat treated at 1000 °C, and subsequently characterized by transmission electron microscopy. Binary systems were selected based on thermodynamic models predicting stable nanograin formation with similar global compositions around 20–30 at.%. All NMMs maintained nanocrystalline grain sizes after evolution into an equiaxed structure, where the systems with highly mobile incoherent interfaces or higher energy interfaces showed a more significant increase in grain size. Furthermore, varying segregation behaviors were observed, including grain boundary (GB) segregation, precipitation, and intermetallic formation depending on the material system selected. The pathway to tailored microstructures was found to be governed by key mechanisms and factors as determined by a film’s initial characteristics, including global and local composition, interface energy, layer structure, and material selection. This work presents a global evaluation of NMM systems and demonstrates their utility as foundation materials to promote tailored nanomaterials.

Appleget, Chelsea D.↗

FWP FEAA149: “Next Generation Environmental Barrier Coatings”

Environmental barrier coatings (EBCs) are required coatings for utilization of SiC/SiC ceramic matrix composite (CMC) components in gas turbines, where the EBC represents the life-limiting factor for such components. EBC/CMC systems have shown success in aero-engine applications with increased turbine inlet temperatures and improved efficiencies, which are achieved through higher temperature stability, lower density, and decreased reliance on cooling air compared to traditional superalloys. While industrial gas turbines (IGTs) do not currently utilize SiC/SiC CMCs, the current shift towards low-carbon or carbon-free fuel sources for power generation could result in a need for EBC/CMC components with higher temperature capabilities. In this work, three tasks were outlined to improve understanding of EBC lifetimes to encourage use in IGTs with carbon-free fuel such as hydrogen: 1. Define the bond coating oxidation kinetics and EBC failure criteria, 2. Measure thermal expansion coefficients of each layered material, and 3. Perform advanced characterization and modeling to assess EBC lifetimes. Cyclic steam oxidation tests were conducted on various EBC/Si/SiC chemistries and EBC/SiC architectures to define substrate oxidation kinetics and EBC failure modes. An open-source code was developed to quantify the undulating thermally grown oxide thickness with thousands of measurements from specimen cross-section images. Bond coating oxidation kinetics were determined and used to develop a kinetic and thermodynamic model for predicting EBC lifetimes. High-temperature Raman spectroscopy was utilized for determining the SiO 2 thermally grown oxide phase transformation as the life-limiting feature for EBCs. Model efforts supported the claim that the SiO 2 phase transformation causes elevated stress during thermal cycling with associated cracking that decreases the adhesion strength of the EBC, eventually resulting in coating spallation. The finite element model subroutine will be made publicly available upon internal review. Further development of an EBC lifetime model for IGTs involves definition of a critical SiO 2 thickness for EBC spallation and must also consider both environmental (gas velocity, pressure, etc.) and specimen (EBC dopants, layer architectures, etc.) effects into predicted bond coating oxidation kinetics for long-lifetime components.

36 MATERIALS SCIENCE↗

Transformations For Atmospheric-Radiation Calculations

Atmospheric radiation calculations essential part of climate models and inversion techniques for analyzing remote sensing observations. Two spectral mapping transformation methods developed to reduce computing time and storage requirements greatly while preserving accuracy achievable with rigorous methods. Evolved from "correlated-k" (c-k) method.

West, Robert A.↗

A global multilevel atmospheric model using a vector semi-Lagrangian finite-difference scheme. I - Adiabatic formulation

An adiabatic global multilevel primitive equation model using a two time-level, semi-Lagrangian semi-implicit finite-difference integration scheme is presented. A Lorenz grid is used for vertical discretization and a C grid for the horizontal discretization. The momentum equation is discretized in vector form, thus avoiding problems near the poles. The 3D model equations are reduced by a linear transformation to a set of 2D elliptic equations, whose solution is found by means of an efficient direct solver. The model (with minimal physics) is integrated for 10 days starting from an initialized state derived from real data. A resolution of 16 levels in the vertical is used, with various horizontal resolutions. The model is found to be stable and efficient, and to give realistic output fields. Integrations with time steps of 10 min, 30 min, and 1 h are compared, and the differences are found to be acceptable.

Bates, J. R.↗

Oracle-Preserving Latent Flows

A fundamental task in data science is the discovery, description, and identification of any symmetries present in the data. We developed a deep learning methodology for the simultaneous discovery of multiple non-trivial continuous symmetries across an entire labeled dataset. The symmetry transformations and the corresponding generators are modeled with fully connected neural networks trained with a specially constructed loss function, ensuring the desired symmetry properties. The two new elements in this work are the use of a reduced-dimensionality latent space and the generalization to invariant transformations with respect to high-dimensional oracles. The method is demonstrated with several examples on the MNIST digit dataset, where the oracle is provided by the 10-dimensional vector of logits of a trained classifier. We find classes of symmetries that transform each image from the dataset into new synthetic images while conserving the values of the logits. We illustrate these transformations as lines of equal probability (“flows”) in the reduced latent space. These results show that symmetries in the data can be successfully searched for and identified as interpretable non-trivial transformations in the equivalent latent space.

97 MATHEMATICS AND COMPUTING↗

The development of interior noise and vibration criteria

A generalized model was developed for estimating passenger discomfort response to combined noise and vibration. This model accounts for broadband noise and vibration spectra and multiple axes of vibration as well as the interactive effects of combined noise and vibration. The model has the unique capability of transforming individual components of noise/vibration environment into subjective comfort units and then combining these comfort units to produce a total index of passenger discomfort and useful sub-indices that typify passenger comfort within the environment. An overview of the model development is presented including the methodology employed, major elements of the model, model applications, and a brief description of a commercially available portable ride comfort meter based directly upon the model algorithms. Also discussed are potential criteria formats that account for the interactive effects of noise and vibration on human discomfort response.

Leatherwood, J. D.↗

Developing Multi-Gene CRISPRa/I Programs to Accelerate DBTL Cycles in ABF Hosts Engineered for Chemical Production (CRADA 468)

Bacterial metabolism is comprised of large and complex gene networks that can produce valuable chemical products. Sophisticated organism engineering efforts are required to optimize production of high-value compounds from these networks. In principle, synthetic multi-gene transcriptional programs could be constructed to reengineer these networks for efficient industrial chemical production. In practice, however, our incomplete ability to understand and model the underlying networks, combined with our limited ability to predictably control the expression of multiple genes makes achieving this goal difficult. To overcome these challenges, we will combine new CRISPR-Cas multi-gene expression programs with computational modeling, machine learning, and multi-omics data to enhance the efficacy of design-build-test-learn (DBTL) cycles. For industrially promising microorganisms in early stages of development, creating technologies for rapidly engineering complex multi-gene programs could be transformative for accelerating data- and model-driven strain design. New CRISPR-Cas tools allow programmable gene activation (CRISPRa) or repression (CRISPRi) at multiple genes simultaneously, using the catalytically inactive Cas9 protein (dCas9) with guide RNAs that recognize DNA targets through predictable Watson-Crick base pairing. To enable accelerated DBTL cycles, we will combine these technologies with advanced Agile BioFoundry (ABF) capabilities for multi-omics data collection and machine learning. We will demonstrate the immediate applicability of these tools by rapidly improving the production of an industrial aromatic in multiple ABF organisms. We recently identified and optimized new transcriptional activators that can be linked to programmable CRISPR-Cas DNA binding domains to activate gene expression in E. coli. We can now use these CRISPRa tools as generalizable trans-acting regulators for combinatorial multi-gene expression tuning that can be easily transferred to new pathways and networks without additional genome engineering. We anticipate these tools will also transfer to new hosts. We have recently found that CRISPRa systems developed in E. coli can be readily ported to Pseudomonas putida, suggesting that multi-gene CRISPRa/i programs for diverse ABF organisms may be within reach.

59 BASIC BIOLOGICAL SCIENCES↗

The use of frequency methods in rotorcraft system identification

A new approach to model structure determination is examined. Flight data from the Rotor Systems Research Aircraft (RSRA) are transformed into the frequency domain and truncated to provide band limiting. The stepwise regression technique is then used to identify a quasistatic state-space model from the transformed data. The data processing requirements for both time domain and frequency domain identification are discussed and the results of the two techniques are compared.

Duval, R. W.↗

Low-dimensional physics of clay particle size distribution and layer ordering

Abstract Clays are known for their small particle sizes and complex layer stacking. We show here that the limited dimension of clay particles arises from the lack of long-range order in low-dimensional systems. Because of its weak interlayer interaction, a clay mineral can be treated as two separate low-dimensional systems: a 2D system for individual phyllosilicate layers and a quasi-1D system for layer stacking. The layer stacking or ordering in an interstratified clay can be described by a 1D Ising model while the limited extension of individual phyllosilicate layers can be related to a 2D Berezinskii–Kosterlitz–Thouless transition. This treatment allows for a systematic prediction of clay particle size distributions and layer stacking as controlled by the physical and chemical conditions for mineral growth and transformation. Clay minerals provide a useful model system for studying a transition from a 1D to 3D system in crystal growth and for a nanoscale structural manipulation of a general type of layered materials.

42 ENGINEERING↗

AI Applications to Physics Experiments at Jefferson Lab

We survey how AI/ML is being deployed across Jefferson Lab's experimental and accelerator programs. In EPSCI, Hydra applies computer vision to automate real-time data-quality monitoring across all four experimental halls, replacing manual inspection of hundreds to thousands of histograms per shift. AIEC (AI Experiment Controls) uses ML to stabilize drift chamber gains and is now part of standard CEBAF production running, while AI Optimized Polarization (AIOP) targets autonomous control of polarized targets and photon beam angular alignment. In CASA, cavity fault classification models identify faulted cavities and trip types from waveform data with ~85% and ~78% agreement to labeled data, respectively, and are deployed in production; a separate effort applies LLMs and hybrid search to make the CEBAF operations logbook AI-ready. QCD-focused work includes transformer- and GAN-based generative models for particle-level event simulation, with distributed GAN training scaling studies on Polaris. Additional efforts span ML-on-FPGA for the EIC and a new Data Science Department coordinating anomaly detection, uncertainty quantification, and HPC-scalable ML lab-wide. Collectively, these projects illustrate AI's growing role in improving efficiency across JLab's nuclear physics mission.

Mei, Xinxin [Thomas Jefferson National Accelerator↗