Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Reanalysis of crackle perception data using logistic and logarithmic fits and sound quality metrics

Mathematical models of human auditory perception (sound quality metrics) are explored for the purpose of fitting human subject data from a prior published study. A variety of linearizing transforms and techniques are employed. Ultimately, a model is identified that predicts 98.9% of the variance in average human subject ratings of crackliness of jet noise waveforms.

Swift, Stephen Hales↗

Physics-Infused AI/ML Based Digital-Twin Framework for Flow-Induced-Vibration Damage Prediction in a Nuclear Reactor Heat Exchanger

This report summarizes some of the ongoing work related to the development of an expert-elicitation-digital-twin framework for real time damage state prediction in heat exchanger components of a nuclear reactor. The framework is targeted towards predicting damage associated with coupled low cycle fatigue (associated with regular heat-up, cool-down and power operation transients) and high cycle fatigue (associated with flow induced vibration transients). The overall framework will be based on a NoSQL based database, physics-infused-geometry-dependent virtual-sensor data, different AI/ML techniques-based data-driven-predictive-model applications (Apps) and real-time plant sensor measurements available through few existing sensors. Towards this overall goal, this report updates some of the ongoing work, such as on implementation of a NoSQL Database (such as MongoDB), FE based heat transfer analysis of a heat exchanger (e.g. of a PWR steam generator) for generating geometry-dependent virtual sensor data and evaluation of various AI/ML models such as based on multivariate linear regression, ensembled decision-tree based Random-Forest and Gradient-Boosting regression and high-dimensional-kernel-function-transformation based Support-Vector-Machine regression models. The AI/ML models were evaluated for predicting multi-time-series thermal states at thousands of 3D point-clouds

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Raman spectroscopic characterization of SiO 2 phase transformation and Si substrate stress relevant to EBC performance

To accurately model the long-term durability of environmental barrier coatings (EBCs), a more complete understanding of the phase composition and transformations of the thermally grown oxide SiO 2 (TGO) is desired. For the TGO formed during thermal cycling in steam, cristobalite formation and the subsequent β- to α-cristobalite transformation has been identified as a potentially life-limiting mechanism. In this study, Raman micro-spectroscopy was used to quantify the cristobalite transformation on a polycrystalline Si coupon that was exposed to steam at 1350°C for 100 h. The phase transformation was mapped at 200–260°C on the TGO surface at different ramp rates using a heating stage and a micro-positioning stage. The stress in the Si substrate was also determined using Raman spectroscopy by measuring the stress induced peak shift. The α→β phase transformation produced a 300–500 MPa tensile stress in the Si substrate, which compared well to the stress predicted from the volumetric expansion of the cristobalite. In conclusion, quantifying the phase transformation and residual stress are critical tools in developing the next generation of high performance EBCs.

36 MATERIALS SCIENCE↗

Theory of finite-temperature two-dimensional structural transformations in group-IV monochalcogenide monolayers

An account of two-dimensional (2D) structural transformations in 2D ferroelectrics predicts an evolution from a structure with Pnm 2 1 symmetry into a structure with square P4/nmm symmetry and is consistent with experimental evidence, while another argues for a transformation into a structure with rectangular Pnmm symmetry. An analysis of the assumptions made in these models is provided here, and six fundamental results concerning these transformations are contributed as follows: (i) Softened phonon modes produce rotational modes in these materials. (ii) The transformation to a structure with P4/nmm symmetry occurs at the lowest critical temperature T c . (iii) The hypothesis that one unidirectional optical vibrational mode underpins the 2D transformation is unwarranted. (iv) Being successively more constrained, a succession of critical temperatures ( T c < T c ' < T c " ) occurs in going from molecular dynamics calculations with the NPT and NVT ensembles onto models with unidirectional oscillations. (v) The choice of exchange-correlation functional impacts the estimate of the critical temperature. (vi) Importantly, the correct physical picture of these transformations is one in which rotational modes confer a topological character to the 2D transformation via the proliferation of vortices.

2d ferroelectrics↗

Transforming ESM Physical Parameterization Development Using Machine Learning Trained on Global Cloud-Resolving Models and Process Observations

ESMs robustly predict that 21st century greenhouse warming will slowly increase global mean precipitation, rapidly increase extreme precipitation, and increase subtropical drought. ESMs agree less about precipitation trends and extremes over particular land regions critical to human societies, e. g. in semi-arid regions such as California or the Sahel, or in wetter climates prone to monsoonal rainfall (e. g. southeast Asia) or to tropical cyclones and flooding from mesoscale convective systems (e. g. the southeastern U.S.) Deep convective parameterizations and poor representation of orography and complex vegetated land surfaces contribute to this inter-model spread; clouds, aerosols and sea-surface temperature biases are also key. Reducing regional precipitation projection uncertainty has enormous planning value for water supplies, land use, wildfire, hydropower, flood control, etc. IPCC-class ESMs are making painfully slow progress on this.

54 ENVIRONMENTAL SCIENCES↗

Analytical Modeling of Cache Behavior for Affine Programs

Modern optimizing compiler design integrates program transformation strategies aimed at reducing data movement to/from main memory by exploiting the data cache hierarchy. But in reality, the effect of these transformations on the actual cache miss count is ignored in compilers due to the lack of precise compile-time models of misses for hierarchical caches. The current state of practice for cache miss analysis is based on accurate simulation. However, simulation requires time proportional to the dataset/problem size as well as the number of distinct cache configurations of interest to be evaluated. This paper takes a fundamentally different approach, made possible by focusing on polyhedral programs with static control flow: instead of relying on costly simulation, this paper develops the first closed-form solution for exact modeling of misses in a set-associative cache hierarchy. It enabling simulation-less selection of program transformations at compile-time to optimize cache misses. A push-button tool implementing the approach is developed and used for extensive validation of the complete framework.

affine programs, cache modeling, set associativity↗

Divergent carbon use efficiency-growth rate tradeoff in popular biological growth models

Carbon use efficiency (CUE) is an important trait emerging from processes regulating biological growth. CUE can be computed either based on the growth of structural biomass or total biomass divided by substrate uptake rate. Nonequilibrium thermodynamics and observations suggest that, for an exponentially growing population of cells, structural biomass CUE should first increase, then peak, and finally decrease with specific growth rate; meanwhile, total biomass CUE increases asymptotically with specific growth rate. We compared predictions from six popular models that are often used for plant and microbial growth in existing ecosystem models. We found that, for an exponentially growing population of biological cells, (1) the source-driven Pirt and Compromise models predict that structural biomass CUE increase asymptotically with growth rate; (2) the apparent sink-driven modified Droop model predicts that structural biomass CUE decreases with growth rate; and (3) the sink-driven variable internal storage model and two dynamic energy budget models predict that structural biomass CUE first increases, then peaks, and finally decreases with growth rate. Moreover, the modified Droop model predicts that total biomass CUE is constant with growth rate, while all other five models predict that total biomass CUE increases with growth rate asymptotically. For non-exponential biological growth, we show that there is no static relationship between total biomass CUE or structural biomass CUE with respect to either growth rate or temperature. Therefore, we contend that biological growth models should explicitly represent interactions between substrate acquisition, substate transformation, and maintenance respiration to better capture observed CUE dynamics, and the sink-driven model should be preferred for general ecosystem biogeochemistry modeling.

Tang, Jinyun [Lawrence Berkeley National Laborator↗

Nanocrystal Geometry Governs Phase Transformation Pathways in Palladium Hydride

Pathways and structural dynamics of phase transformations impact performance of materials in energy and information storage technologies. Palladium hydride (PdH x ) nanocrystals are an ideal model system for studying solute-induced phase transformations, where elastic energy from lattice mismatch between α-PdH x and β-PdH x phases is often considered a key to determining the transformation pathways. α/β-PdH x interfacial elastic energy is affected by the confined geometry of a nanocrystal. However, how nanocrystal geometry influences phase transformation pathways is largely unknown. Using in situ liquid phase transmission electron microscopy, we directly visualize hydrogenation in Pd nanocrystals with two geometries, a nanocube and a hexagonal nanoplate. Both follow similar sequences of an initially curved nucleus, interface flattening, and reverse-stage nucleation; however, their evolving α/β-PdH x interfaces exhibit geometry-dependent crystallographic alignments. In nanocubes, {100}-aligned configurations conform to static elastic energy ordering, representing a pathway that maintains a local mechanical equilibrium, whereas nanoplates display both {110}- and {211}-aligned interfaces. Theoretical simulations show that geometry determines the accessibility of alternative phase transformation pathways as the system is driven far from equilibrium during hydrogenation. In conclusion, these findings identify geometry as a fundamental parameter for directing phase transformation pathways, offering design principles for accessing atypical configurations and improving properties of intercalation-based devices.

Lee, Daewon [Lawrence Berkeley National Laboratory↗

End-to-End Differentiable Modeling and Management of the Environment

Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.

54 ENVIRONMENTAL SCIENCES↗

A theory of localized excitations in supercooled liquids

Here, a new connection between the structure and dynamics in glass-forming liquids is presented. We show how the origin of spatially localized excitations, as defined by the dynamical facilitation (DF) theory, can be understood from a structure-based framework. This framework is constructed by associating excitation events in the DF theory to hopping events between energy minima in the potential energy landscape (PEL). By reducing the PEL to an equal energy well picture and applying a harmonic approximation, we develop a field theory to describe elastic fluctuations about inherent states, which are energy minimizing configurations of the PEL. We model an excitation as a shear transformation zone (STZ) inducing a localized pure shear deformation onto an inherent state. We connect STZs to T1 transition events that break the elastic bonds holding the local structure of an inherent state. A formula for the excitation energy barrier, denoted as J σ , is obtained as a function of inherent-state elastic moduli and the radial distribution function. The energy barrier from the current theory is compared to the one predicted by the DF theory where good agreement is found in various two-dimensional continuous poly-disperse atomistic models of glass formers. These results strengthen the role of structure and elasticity in driving glassy dynamics through the creation and relaxation of localized excitations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Effect of bacterial cell addition on Fe(III) reduction and soil organic matter transformation in a farmland soil

The coupled biogeochemical processes of Fe(III) reduction and organic matter transformation profoundly impact terrestrial carbon cycling. However, little is known about the microbial role in soil organic matter (SOM) transformation during Fe(III) bio-reduction. Here we investigated the bio-reduction behavior of a black farmland soil and corresponding SOM transformation under circumneutral and anoxic conditions. A model dissimilatory Fe reducing bacterium, Geobacter sulfurreducens, was added in either live or dead form in order to enhance Fe(III) reduction in soil and to evaluate the accompanying transformation of SOM. The progress of Fe reduction was monitored and SOM transformation was characterized by various spectroscopy methods. Results showed that addition of either dead or live G. sulfurreducens cells increased the Fe(III) reduction rate and extent. Without cell addition, SOM transformation was insignificant within 13 days of incubation, only with some consumption of aliphatic/protein compounds, apparently due to their higher bio-degradability. Addition of dead or live cells resulted in more drastic SOM transformation, but through different mechanisms. With dead cell amendment, cell necromass and debris stimulated the activity of indigenous soil microbes by serving as extra carbon/energy sources. During Fe(III) reduction, the aliphatic/protein compounds were preferentially consumed by indigenous microbial communities, similar to the treatment without cell addition but with a greater extent of consumption. In comparison, addition of live G. sulfurreducens cells stimulated degradation of less bioavailable compounds, including more saturated and higher molecular weight SOM. It is possible that fast depletion of labile SOM by live G. sulfurreducens cells favored utilization of less bioavailable molecules (such as those with more aromatic structures) by the originally dormant species in native microbial community, suggesting an active role of live Fe(III)-reducing bacteria in affecting SOM transformation. In addition, fast assimilation of microbial related carbon, such as aromatic proteins and microbial byproducts, into SOM pools was also observed. Overall, our results suggest that addition of Fe(III)-reducing bacteria to a native soil can enhance Fe(III) reduction and accelerate the turnover of SOM through various mechanisms. The study provides new insights into coupled Fe(III) reduction and SOM transformation, as well as the “priming effect” of SOM using microbial cells as substrate.

54 ENVIRONMENTAL SCIENCES↗

AI-assisted object condensation clustering for calorimeter shower reconstruction at CLAS12

Several nuclear physics studies using the CLAS12 detector rely on the accurate reconstruction of neutrons and photons from its forward angle calorimeter system. These studies often place restrictive cuts when measuring neutral particles due to an overabundance of false clusters created by the existing calorimeter reconstruction software. In this work, we present a new AI approach to clustering CLAS12 calorimeter hits based on the object condensation framework. The model learns a latent representation of the full detector topology using GravNet layers, serving as the positional encoding for an event’s calorimeter hits which are processed by a Transformer encoder. This unique structure allows the model to contextualize local and long range information, improving its performance. Evaluated on one million simulated $e^-$ $+$ $p$ collision events, our method significantly improves cluster trustworthiness: the fraction of reliable neutron clusters, increasing from 8.88% to 30.73%, and photon clusters, increasing from 51.07% to 64.73%. In conclusion, our study also marks the first application of AI clustering techniques for hodoscopic detectors, showing potential for usage in many other experiments.

Calorimeters↗

Performance evaluations of signed and unsigned noisy approximate quantum Fourier arithmetic

The Quantum Fourier Transform (QFT) grants competitive advantages, especially in resource usage and circuit approximation, for performing arithmetic operations on quantum computers, and offers a potential route toward a numerical quantum-computational paradigm. In this paper, we utilize efficient techniques to implement QFT-based integer addition and multiplications. These operations are fundamental to various quantum applications including Shor’s algorithm, weighted-sum optimization problems in data processing and machine learning, and quantum algorithms requiring inner products. We carry out performance evaluations of these implementations based on IBM’s superconducting-qubit architecture using different compatible noise models. We isolate the sensitivity of the component quantum circuits on both one-/two-qubit gate error rates, and the number of the arithmetic operands’ superposed integer states. We analyze performance and identify the most effective approximation depths for unsigned quantum addition and quantum multiplication within the given context. We then perform a similar analysis of signed addition and compare to the unsigned results. We observe significant dependency of the optimal approximation depth on the degree of machine noise and the number of superposed states in certain performance regimes. Finally, we elaborate on the algorithmic challenges—relevant to signed, unsigned, modular and non-modular versions—that could also be applied to current implementations of QFT-based subtraction, division, exponentiation, and their potential tensor extensions. Here, we analyze the performance trends in our results and speculate on possible future developments within this computational paradigm.

Computational models↗

Vision Foundation Models in Remote Sensing: A survey

Artificial intelligence (AI) technologies have profoundly transformed the field of remote sensing (RS), revolutionizing data collection, processing, and analysis. Traditionally reliant on manual interpretation and task-specific models, RS research has been significantly enhanced by the advent of foundation models (FMs)—large-scale pretrained AI models capable of performing a wide array of tasks with unprecedented accuracy and efficiency. This article provides a comprehensive survey of FMs in the RS domain. We categorize these models based on their architectures, pretraining datasets, and methodologies. Through detailed performance comparisons, we highlight emerging trends and the significant advancements achieved by those FMs. Additionally, we discuss technical challenges, practical implications, and future research directions, addressing the need for high-quality data, computational resources, and improved model generalization. Our research also finds that pretraining methods, particularly self-supervised learning (SSL) techniques like contrastive learning (CL) and masked autoencoders (MAEs), remarkably enhance the performance and robustness of FMs. This survey aims to serve as a resource for researchers and practitioners by providing a panorama of advances and promising pathways for the continued development and application of FMs in RS.

data models↗

Bundling measures for food systems transformation: a global, multimodel assessment

Background Current food systems leave one in ten individuals at risk of hunger while driving unsustainable environmental impacts. Inaction risks further exacerbating negative impacts on both human and planetary health. These challenges emerge from complex system interactions, requiring approaches that engage with this complexity and consider how transformation measures interact across food systems. We aimed to quantify the magnitude and uncertainty of the impacts of key food systems transformation measures both individually and in a bundle using an ensemble of global economic models. Methods In this global multimodel assessment, we applied an ensemble of ten state-of-the-art global economic models to evaluate the potential of four key measures in transforming food systems: increasing agricultural productivity, halving food loss and waste, shifting towards healthier diets, and economy-wide climate mitigation policies aligned with limiting warming to 1·5°C. The scenarios used a middle-of-the-road shared socioeconomic pathway for population and gross domestic product growth, climate impact data from Jägermeyr and colleagues, Thornton and colleagues, and Nelson and colleagues, and dietary targets based on the EAT–Lancet healthy reference diet, with model simulations conducted from 2020 to 2050. We then assessed the effect of these measures in isolation and in combination in a bundled scenario. To further understand the interactions between these measures, we conducted a decomposition analysis that distinguishes between the individual effects of a measure (effect when implemented alone), total effects (its contribution within the bundle), and interaction effects (the difference between total and individual effects). This approach aimed to show complementarities and trade-offs that emerge when multiple measures are implemented simultaneously. Findings Our analysis showed that individual measures in isolation are insufficient to achieve high-level environmental objectives and might generate unintended consequences. In contrast, bundling measures produces co-benefits: avoiding 50% of projected agricultural greenhouse gas emissions by 2050 and almost 20% of anticipated land conversion, while moderating food price increases associated with ambitious climate change mitigation policies. Our decomposition analysis further shows that measures can have varying effects across different dimensions. Although dietary shifts and climate mitigation policies are the largest drivers of environmental benefits (each contributing to a median decline of >10 percentage points in non-CO 2 emissions and 5 percentage points in agricultural land use globally), productivity improvements and reducing food loss and waste play essential roles in moderating price increases (each contributing to a median decline of >5 percentage points in average prices). Interpretation This study highlights the importance of implementing coordinated approaches to food system transformation and climate change mitigation rather than relying on isolated interventions. Comprehensive transformation requires understanding how supply-side and demand-side changes can interact with climate mitigation policies, enabling policy makers to design intervention packages that maximise benefits while minimising trade-offs across environmental, economic, and social dimensions.

Sundiang, Marina [Cornell Univ., Ithaca, NY (Unite↗

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗

Nitrate, ammonium, and DON mass time series output for East River stream, vadose zone and groundwater subwatersheds from HAN-SoMo model

There is a growing understanding of the role that bedrock weathering can play as a source of nitrogen (N) to soils, groundwater and river systems. The significance is particularly apparent in mountainous environments where weathering fluxes can be large. However, our understanding of the relative contributions of rock-derived, or geogenic, N to the total N supply of mountainous watersheds remains poorly understood. This dataset contains model output from the High-Altitude Nitrogen Suite of Models (HAN-SoMo), a watershed-scale ensemble of process-based models to quantify the relative sources, transformations, and sinks of geogenic and atmospheric N through a mountain watershed. Our study is based in the East River Watershed (ERW) in the Upper Colorado River Basin. The East River is a near-pristine headwater watershed underlain primarily by an N-rich Mancos Shale bedrock, enabling the timing and magnitude of geogenic and atmospheric contributions to watershed scale dissolved N-exports to be quantified. Six calibration scenarios were developed to explore equifinality using >1600 N concentration measurements from streams, groundwater, and vadose zone samples collected over the course of four years across the watershed, from 2014-2018. The files in this dataset are named according to which calibration scenario they represent (1, 2, 3, NM1, NM2, and NC), and include daily time series model output of nitrate, ammonium and dissolved organic nitrogen concentrations for all six calibration scenarios described in Maavara et al., 2021, "Modeling geogenic and atmospheric nitrogen through the East River Watershed, Colorado Rocky Mountains," PLOS One (in press), for the stream, vadose zone, and groundwater in the five discretized subwatersheds: Rustlers Gulch, Copper Creek, East above Quigley (EAQ), the Middle East River (ME), and Lower Triangle (LT) from Gothic to the Pumphouse. The READ_ME.txt file includes details on units and naming conventions in each of these files. Model files are in .mat format, to be opened in MATLAB.

54 ENVIRONMENTAL SCIENCES↗