Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

A generalizable machine learning-assisted fast Fourier transform algorithm to simulate the large strain phenomena in polycrystalline materials

Machine learning methods have shown initial promise in constitutive modeling for single crystals or homogenized polycrystals, delivering notable computational efficiency. However, existing machine learning-based constitutive models often lack generalizability, limiting their application across diverse boundary value problems. This study introduces a thermodynamics-informed artificial neural network model to accelerate rate-tangent crystal plasticity fast Fourier transform simulations for cross-scale deformation behaviors of polycrystals under complex loading. Our model integrates microstructural variability and local interactions effectively. To address local effects in each grain, we employ K-means clustering to group Gauss points within the microstructure into clusters assumed to be in similar mechanical states. This approach, based on self-clustering analysis, extends model scope from macroscopic stress response to the granular level, capturing mechanical responses and orientation evolution across grains. This reduces the number of nonlinear problems to solve, with cluster responses propagated throughout each group. The thermodynamics-based artificial neural network-extracted features are further processed using local material state clusters to account for history-dependent deformation and evolving microstructures. Additionally, representative volume element simulations with rate-tangent crystal plasticity fast Fourier transform provide reliable datasets for model training. The proposed model demonstrates high efficiency, accuracy, self-consistency, and enhanced generalizability in predicting strain–stress responses and orientation evolution at both individual grain and aggregate scales under complex loading conditions, such as biaxial tension and arbitrary loading scenarios.

36 MATERIALS SCIENCE↗

Advancing Building Energy Modeling with Large Language Models: Exploration and Case Studies

The rapid progression in artificial intelligence has facilitated the emergence of large language models like ChatGPT, offering potential applications extending into specialized engineering modeling, especially physics-based building energy modeling. This paper investigates the innovative integration of large language models with building energy modeling software, focusing specifically on the fusion of ChatGPT with EnergyPlus. A literature review is first conducted to reveal a growing trend of incorporating large language models in engineering modeling, albeit limited research on their application in building energy modeling. We underscore the potential of large language models in addressing building energy modeling challenges and outline potential applications including simulation input generation, simulation output analysis and visualization, conducting error analysis, co-simulation, simulation knowledge extraction and training, and simulation optimization. Three case studies reveal the transformative potential of large language models in automating and optimizing building energy modeling tasks, underscoring the pivotal role of artificial intelligence in advancing sustainable building practices and energy efficiency. The case studies demonstrate that selecting the right large language model techniques is essential to enhance performance and reduce engineering efforts. The findings advocate a multidisciplinary approach in future artificial intelligence research, with implications extending beyond building energy modeling to other specialized engineering modeling.

building energy modeling↗

Point cloud-based diffusion models for the Electron-Ion Collider

At high-energy collider experiments, generative models can be used for a wide range of tasks, including fast detector simulations, unfolding, searches of physics beyond the Standard Model, and inference tasks. In particular, it has been demonstrated that score-based diffusion models can generate high-fidelity and accurate samples of jets or collider events. This work expands on previous generative models in three distinct ways. First, our model is trained to generate entire collider events, including all particle species with complete kinematic information. We quantify how well the model learns event-wide constraints such as the conservation of momentum and discrete quantum numbers. We focus on the events at the future Electron-Ion Collider, but we expect that our results can be extended to proton-proton and heavy-ion collisions. Second, previous generative models often relied on image-based techniques. The sparsity of the data can negatively affect the fidelity and sampling time of the model. We address these issues using point clouds and a novel architecture combining edge creation with transformer modules called Point Edge Transformers. Third, we adapt the foundation model OmniLearn, to generate full collider events. This approach may indicate a transition toward adapting and fine-tuning foundation models for downstream tasks instead of training new models from scratch.

Araz, Jack Y. [Stony Brook Univ., NY (United State↗

Envisioning U.S. Climate Predictions and Projections to Meet New Challenges

In the face of a changing climate, the understanding, predictions, and projections of natural and human systems are increasingly crucial to prepare and cope with extremes and cascading hazards, determine unexpected feedbacks and potential tipping points, inform long-term adaptation strategies, and guide mitigation approaches. Increasingly complex socio-economic systems require enhanced predictive information to support advanced practices. Such new predictive challenges drive the need to fully capitalize on ambitious scientific and technological opportunities. These include the unrealized potential for very high-resolution modeling of global-to-local Earth system processes across timescales, reduction of model biases, enhanced integration of human systems and the Earth Systems, better quantification of predictability and uncertainties; expedited science-to-service pathways, and co-production of actionable information with stakeholders. Enabling technological opportunities include exascale computing, advanced data storage, novel observations and powerful data analytics, including artificial intelligence and machine learning. Looking to generate community discussions on how to accelerate progress on U.S. climate predictions and projections, representatives of Federally-funded U.S. modeling groups outline here perspectives on a six-pillar national approach grounded in climate science that builds on the strengths of the U.S. modeling community and agency goals. This calls for an unprecedented level of coordination to capitalize on transformative opportunities, augmenting and complementing current modeling center capabilities and plans to support agency missions. Tangible outcomes include projections with horizontal spatial resolutions finer than 10 km, representing extremes and associated risks in greater detail, reduced model errors, better predictability estimates, and more customized projections to support next generation climate services.

54 ENVIRONMENTAL SCIENCES↗

Fabrication of Arrays of Topological Solitons in Patterned Chiral Liquid Crystals for Real-Time Observation of Morphogenesis

Topological solitons have knotted continuous field configurations embedded in a uniform background, and occur in cosmology, biology, and electromagnetism. However, real-time observation of their morphogenesis and dynamics is still challenging because their size and timescale are enormously large or tiny. Liquid crystal (LC) structures are promising candidates for a model-system to study the morphogenesis of topological solitons, enabling direct visualization due to the proper size and timescale. We report a new way is found to rationalize the real-time observation of the generation and transformation of topological solitons using cholesteric LCs confined in patterned substrates. The experimental demonstration shows the topologically protected structures arise via the transformation of topological defects. Numerical modeling based on minimization of free energy closely reconstructs the experimental findings. The fundamental insights obtained from the direct observations pose new theoretical challenges in understanding the morphogenesis of different types of topological solitons within a broad range of scales.

36 MATERIALS SCIENCE↗

Equity and modeling in sustainability science: Examples and opportunities throughout the process

Equity is core to sustainability, but current interventions to enhance sustainability often fall short in adequately addressing this linkage. Models are important tools for informing action, and their development and use present opportunities to center equity in process and outcomes. This Perspective highlights progress in integrating equity into systems modeling in sustainability science, as well as key challenges, tensions, and future directions. We present a conceptual framework for equity in systems modeling, focused on its distributional, procedural, and recognitional dimensions. We discuss examples of how modelers engage with these different dimensions throughout the modeling process and from across a range of modeling approaches and topics, including water resources, energy systems, air quality, and conservation. Synthesizing across these examples, we identify significant advances in enhancing procedural and recognitional equity by reframing models as tools to explore pluralism in worldviews and knowledge systems; enabling models to better represent distributional inequity through new computational techniques and data sources; investigating the dynamics that can drive inequities by linking different modeling approaches; and developing more nuanced metrics for assessing equity outcomes. We also identify important future directions, such as an increased focus on using models to identify pathways to transform underlying conditions that lead to inequities and move toward desired futures. By looking at examples across the diverse fields within sustainability science, we argue that there are valuable opportunities for mutual learning on how to use models more effectively as tools to support sustainable and equitable futures.

54 ENVIRONMENTAL SCIENCES↗

A Markov model reduction technique for fault tolerant processor reliability analysis

A fault tolerant processor (FTP) plays a key role in many high performance, safety-critical control system applications. Realistic modeling of an FTP is crucial to gaining a high degree of confidence in the reliability and safety analysis of such a system. While fidelity is clearly a major consideration, a practical model must also be kept to a moderate size to allow its incorporation into the overall system model. This paper presents a systematic reduction technique that starts from a complex, detailed model of a triple, redundant FTP and produces a low order approximation of very high accuracy. The existence of two distinct time scales represents the key to the success of the technique. No eigenvalue solution or coordinate transformation are needed. The reduced model captures all the important features of the detailed model, is amenable to an analytical solution and provides insight into the reconfiguration behavior of an FTP.

Schor, Andrei L.↗

Bridging the Gap Between Astronomical Datasets: From Proof-of-Concept to AI Model Deployment with Domain Adaptation

Artificial Intelligence is transforming astrophysics, from studying stars and galaxies to analyzing cosmic large-scale structures. However, a critical challenge arises when AI models trained on simulations or past observational data are applied to new observation— leading to domain shifts, reduced robustness, and increased uncertainty of model predictions. This talk will explore these issues, highlighting examples such as galaxy morphology classification and cosmological parameter inference, where AI struggles to adapt across different datasets. We will discuss domain adaptation as a strategy to improve model generalization and mitigate biases—essential for making AI-driven discoveries reliable. Notably, these challenges extend beyond astrophysics, affecting AI applications across physics and other scientific domains. Addressing them is essential for maximizing AI’s impact in advancing scientific research.

Ćiprijanović, Aleksandra [Fermilab]↗

Validating Greater Sage-Grouse Individual-based Model (IBM) Tool (Final Report)

The project focused on validating the previously developed Greater Sage-Grouse Individual-based Model (GrSG IBM; LaGory et al. 2012, 2021). The objective was to transform this predictive, spatially and temporally explicit model into a portable resource to assist siting/resource managers in proactively assessing the cumulative impacts of wind energy development on the greater sage-grouse. Utilizing a bottom-up, individual-based approach, the GrSG IBM accounts for landscape context and species behavior, aiming to reduce uncertainty in estimating development impacts and support ecologically mindful land-based wind energy development. The validation effort covered approximately 6,540 km 2 near the Seven Mile Hill Wind Project in Wyoming. The GrSG IBM tool, built on the NetLogo platform (Tisue and Wilensky 2004), was executed over a 50-year period, with the analysis focusing on years following a 10-year initialization phase. Key results demonstrated the tool’s biological soundness across five key biological metrics: non-chick age class distribution (older than 10 weeks), adult sex ratio, life expectancy, population size, and overall population growth. For instance, the tool estimated that 58.6% of the non-chick population was reproductively immature, while the reference ranges from 51.4% to 57.8% (Patterson 1952, Rogers 1964). Experts confirmed the tool’s estimate was within a reasonable range for the species. The tool estimated average life expectancy of 1.43 years, while the reference ranges from 0.9 years to 1.1 years (Ammann 1957, Hamerstrom 1949). Experts also supported the model’s life-expectancy estimate as ecologically sound for the species in the study area. In terms of population change, the model estimated an annual shift between a 0.6% decline and a 1.0% increase over 50 years. While the reference suggests 2.9% annual decline in range-wide populations (Cortes et al. 2023), that includes many at-risk populations in South Dakota and Washington, for example. Our study area—in the northeastern part of Carbon County and western-edge of Albany County, Wyoming—is one of the remaining greater sage-grouse habitats supporting some of the most stable populations. Experts confirmed that the range of the annual population change spanning from a 0.6% decline to a 1.0% increase estimated by the tool was reasonable for our study area for this reason and confirmed that aligned with population estimates from existing studies on the greater sage-grouse and wind energy development in the study area (LeBeau et al. 2017a, Smith et al. 2024). Furthermore, the project showed that temporally explicit biological metrics generated by the GrSG IBM tool can complement the USGS’ Prioritizing Restoration of Sagebrush Ecosystems Tool (PReSET; Duchardt et al. 2021) by incorporating habitat restoration strategies into seasonal habitat suitability models to visualize population responses over time.

17 WIND ENERGY↗

Defect-Driven Oxide Transformations and the Electrochemical Interphase

The redox reaction pathway is crucial to the sustainable production of the fuels and chemicals required for a carbon-neutral society. Our society is becoming increasingly dependent on devices using batteries and electrolyzers, all of which rely on a series of redox reactions. The overall properties of oxide materials make them very well suited for such electrochemical and catalytic applications due to their associated cationic redox properties and the static site-adsorbate interactions. As these technologies have matured, it has become apparent that defect-driven redox reactions, defect-coupled diffusion, and structural transformations that are both time- and rate-dependent are also critical materials processes. This change in focus, considering not only redox properties but also more complex, dynamic behaviors, represents a new research frontier in the molecular sciences as they are strongly linked to device operation and degradation and lie at the heart of various phenomena that take place at electrochemical interfaces. Fundamental studies of the structural, electronic, and chemical transformation mechanisms are key to the advancement of materials and technological innovations that could be implemented in various electrochemical systems. In this Account, we focus on recent studies and advances in characterizing and understanding the dynamic redox evolution and structural transformations that take place in model perovskites and layered oxides under reactive conditions and correlate them with degradation mechanisms and operations in electrolyzers and batteries. We show that the dynamic evolution of oxygen vacancies and cationic migration in the surface or bulk occurs at the solid-liquid interface, using a combination of different synchrotron-based Xray spectroscopies and scattering probes. Detailed redox-structure-reactivity correlation studies show how defects and diffusion processes can be tailored to drive various physical and chemical transformations in electrolyzers and batteries. We also highlight a strong correlation between oxygen redox reactivity and structural reorganization in both model thin films and particles, helping to bridge the gap between fundamental studies of the reaction mechanism and device applications. Finally, on the basis of these findings, we discuss strategies to probe and tune the redox reactivity and structural stability of the redox-active oxide interphase toward devising efficient pathways for energy and chemical harvesting.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Symmetry of Cosmological Observables, a Mirror World Dark Sector, and the Hubble Constant

We find that a uniform scaling of the gravitational free-fall rates and photon-electron scattering rate leaves most dimensionless cosmological observables nearly invariant. This result opens up a new approach to reconciling cosmic microwave background and large-scale structure observations with high values of the Hubble constant H0: Find a cosmological model in which the scaling transformation can be realized without violating any measurements of quantities not protected by the symmetry. A “mirror world” dark sector allows for effective scaling of the gravitational free-fall rates while respecting the measured mean photon density today. Further model building might bring consistency with the two constraints not yet satisfied: the inferred primordial abundances of deuterium and helium.

79 ASTRONOMY AND ASTROPHYSICS↗

Accounting For Gains And Orientations In Polarimetric SAR

Calibration method accounts for characteristics of real radar equipment invalidating standard 2 X 2 complex-amplitude R (receiving) and T (transmitting) matrices. Overall gain in each combination of transmitting and receiving channels assumed different even when only one transmitter and one receiver used. One characterizes departure of polarimetric Synthetic Aperture Radar (SAR) system from simple 2 X 2 model in terms of single parameter used to transform measurements into format compatible with simple 2 X 2 model. Data processed by applicable one of several prior methods based on simple model.

Freeman, Anthony↗

Southeast Atmosphere Studies: Learning from Model-Observation Syntheses

Concentrations of atmospheric trace species in the United States have changed dramatically over the past several decades in response to pollution control strategies, shifts in domestic energy policy and economics, and economic development (and resulting emission changes) elsewhere in the world. Reliable projections of the future atmosphere require models to not only accurately describe current atmospheric concentrations, but to do so by representing chemical, physical and biological processes with conceptual and quantitative fidelity. Only through incorporation of the processes controlling emissions and chemical mechanisms that represent the key transformations among reactive molecules can models reliably project the impacts of future policy, energy and climate scenarios. Efforts to properly identify and implement the fundamental and controlling mechanisms in atmospheric models benefit from intensive observation periods, during which collocated measurements of diverse, speciated chemicals in both the gas and condensed phases are obtained. The Southeast Atmosphere Studies (SAS, including SENEX, SOAS, NOMADSS and SEAC4RS) conducted during the summer of 2013 provided an unprecedented opportunity for the atmospheric modeling community to come together to evaluate, diagnose and improve the representation of fundamental climate and air quality processes in models of varying temporal and spatial scales. This paper is aimed at discussing progress in evaluating, diagnosing and improving air quality and climate modeling using comparisons to SAS observations as a guide to thinking about improvements to mechanisms and parameterizations in models. The effort focused primarily on model representation of fundamental atmospheric processes that are essential to the formation of ozone, secondary organic aerosol (SOA) and other trace species in the troposphere, with the ultimate goal of understanding the radiative impacts of these species in the southeast and elsewhere. Here we address questions surrounding four key themes: gas-phase chemistry, aerosol chemistry, regional climate and chemistry interactions, and natural and anthropogenic emissions. We expect this review to serve as a guidance for future modeling efforts.

atmospheric chemistry↗

tomoCAM : fast model-based iterative reconstruction via GPU acceleration and non-uniform fast Fourier transforms

X-ray-based computed tomography is a well established technique for determining the three-dimensional structure of an object from its two-dimensional projections. In the past few decades, there have been significant advancements in the brightness and detector technology of tomography instruments at synchrotron sources. These advancements have led to the emergence of new observations and discoveries, with improved capabilities such as faster frame rates, larger fields of view, higher resolution and higher dimensionality. These advancements have enabled the material science community to expand the scope of tomographic measurements towards increasingly in situ and in operando measurements. In these new experiments, samples can be rapidly evolving, have complex geometries and restrictions on the field of view, limiting the number of projections that can be collected. In such cases, standard filtered back-projection often results in poor quality reconstructions. Iterative reconstruction algorithms, such as model-based iterative reconstructions (MBIR), have demonstrated considerable success in producing high-quality reconstructions under such restrictions, but typically require high-performance computing resources with hundreds of compute nodes to solve the problem in a reasonable time. Here, tomoCAM , is introduced, a new GPU-accelerated implementation of model-based iterative reconstruction that leverages non-uniform fast Fourier transforms to efficiently compute Radon and back-projection operators and asynchronous memory transfers to maximize the throughput to the GPU memory. The resulting code is significantly faster than traditional MBIR codes and delivers the reconstructive improvement offered by MBIR with affordable computing time and resources. tomoCAM has a Python front-end, allowing access from Jupyter -based frameworks, providing straightforward integration into existing workflows at synchrotron facilities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Phase Field Modeling of Chemical Reaction Related Damage Evolution in Environmental Barrier Coatings

The advent of next-generation engines necessitates materials capable of withstanding temperatures beyond the reach of current superalloys. SiC-based ceramic matrix composites, augmented with environmental barrier coatings (EBCs), present a promising materials solution. Given the active search for effective and durable EBCs, there is a pressing need for modeling tools to understand and predict damage evolution in these materials to help accelerate their development. This study introduces a phase-field model (PFM) designed to simulate the thermally grown oxides (TGO) and phase transformation in the degradation and failure of EBCs. The model accounts for the severe volume expansion due to oxidation, alongside phase transformations and porosity evolution during thermal cycling, offering a comprehensive view of the damage processes. Simulation results are validated against experimental findings reported in the literature, establishing the model's potential as a significant tool for understanding and improving the resilience of EBCs in cyclic oxidative environments.

fast-diffusion path↗

RingX: Scalable Parallel Attention for Long-Context Learning on HPC

The attention mechanism has become foundational for remarkable AI breakthroughs since the introduction of the Transformer, driving the demand for increasingly longer context to power frontier models such as large-scale reasoning language models and high-resolution image/video generators. However, its quadratic computational and memory complexities present substantial challenges. Current state-of-the-art parallel attention methods, such as ring attention, are widely adopted for long-context training but utilize a point-to-point communication strategy that fails to fully exploit the capabilities of modern HPC network architectures. In this work, we propose ringX, a scalable family of parallel attention methods optimized explicitly for HPC systems. By enhancing workload partitioning, refining communication patterns, and improving load balancing, ringX achieves up to 3.4 × speedup compared to conventional ring attention on the Frontier supercomputer. Optimized for both bi-directional and causal attention mechanisms, ringX demonstrates its effectiveness through training benchmarks of a Vision Transformer (ViT) on a climate dataset and a Generative Pre-Trained Transformer (GPT) model, Llama3 8B. Our method attains an end-to-end training speedup of approximately 1.5 × in both scenarios. To our knowledge, the achieved 38% model FLOPs utilization (MFU) for training Llama3 8B with a 1M-token sequence length on 4,096 GPUs represents one of the highest training efficiencies reported for long-context learning on HPC systems. Our code implementation is available at https://github.com/jqyin/ringX-attention.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗

Towards Neutron Transformation Searches

To probe the origins of the baryon asymmetry, baryon number violation, the last unconfirmed Sakharov condition, must be definitively observed experimentally. Similarly, the nature of dark matter is currently unknown, and calls out for new candidates to be investigated. Each of these issues can be considered through the study of neutron transformations.Some rare baryon number violating processes, such as neutron-antineutron transformations, are expected to probe baryogenesis. Here, I show progress on this discovery target through construction of more accurate Monte Carlo models, the design of future detectors, creation of more complete atmospheric neutrino background simulations, and use of automated analysis techniques within the the NNBAR/HIBEAM experimental program at the European Spallation Source (ESS) and the Deep Underground Neutrino Experiment (DUNE). First simulation-based sensitivities for these experiments will be discussed. Modeling of rare neutron-antineutron transformation and subsequent annihilation will be discussed at length for multiple nuclei useful to these and other collaborations. To go along with this work, more comprehensive lepton-scattering nuclear models must be integrated into neutrino event generators for proper atmospheric neutrino background simulations. I discuss the first furnishing of these backgrounds for DUNE, and I highlight a potential path forward for the community in this vein using precisionmore » electron scattering modeling as a facsimile.Aspects of other potentially related neutron – mirror-neutron oscillations pertinent to dark matter and the neutron lifetime anomaly will also be considered for the ESS HIBEAM experiment. Here, I will present the first experimental sensitivity calculations for a broad range of modular experimental setups which will serve as research and design stepping stones toward NNBAR while producing a multitude of physics results over short time scales.« less

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗