Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

Large Language Models (LLMs) for Energy Systems Research

The integration of Large Language Models (LLMs) in energy systems research promises transformative results, as demonstrated in this work, particularly in the realms of information retrieval and legal document analysis. We have developed a chat-based interface, specifically designed to query an extensive corpus of technical reports from the National Renewable Energy Laboratory (NREL). This interface capitalizes on the natural language processing capabilities of LLMs, providing future consumers of NREL research with a user-friendly platform to access and extract valuable information from technical documents, thus enhancing the dissemination of research to the public. In addition to information retrieval, we have employed LLMs to extract renewable energy siting ordinances from a variety of legal documents, a task traditionally driven by significant human labor. This automated extraction not only supports the ongoing development of the high-impact NREL siting ordinance database but also ensures the database's accuracy and comprehensiveness. Crucially, we have augmented the performance of LLMs through the integration of a decision tree framework, resulting in a substantial improvement in extraction accuracy. Comparative analysis with manual efforts has shown that this approach not only rivals but also significantly surpasses human accuracy, heralding increased reliability in legal document analysis for energy systems research. To democratize access to these advancements and foster collaborative research, we introduce the "Energy Language Model" (ELM), an open-source software package. ELM encapsulates the methodologies and tools developed in this work, providing researchers and practitioners with a robust toolkit to conduct similar analyses within their respective domains. Through these contributions, this work underscores the immense potential of LLMs in revolutionizing energy systems research, improving accuracy, efficiency, and accessibility in the field.

automation↗

A Bayesian Multi-fidelity Neural Network to Predict Nonlinear Frequency Backbone Curves

The use of structural mechanics models during the design process often leads to the development of models of varying fidelity. Often low-fidelity models are efficient to simulate but lack accuracy, while the high-fidelity counterparts are accurate with less efficiency. Here, this paper presents a multi-fidelity surrogate modeling approach that combines the accuracy of a high-fidelity finite element model with the efficiency of a low-fidelity model to train an even faster surrogate model that parameterizes the design space of interest. The objective of these models is to predict the nonlinear frequency backbone curves of the Tribomechadynamics Research Challenge benchmark structure which exhibits simultaneous nonlinearities from frictional contact and geometric nonlinearity. The surrogate model consists of an ensemble of neural networks that learn the mapping between low and high-fidelity data through nonlinear transformations. Bayesian neural networks are used to assess the surrogate model's uncertainty. Once trained, the multi-fidelity neural network is used to perform sensitivity analysis to assess the influence of the design parameters on the predicted backbone curves. Additionally, Bayesian calibration is performed to update the input parameter distributions to correlate the model parameters to the collection of experimentally measured backbone curves.

42 ENGINEERING↗

Deep Learning for Subsurface Flow: A Comparative Study of U‐Net, Fourier Neural Operators, and Transformers in Underground Hydrogen Storage

Subsurface flow research is essential for the sustainable management of natural resources and the environment. Deep learning (DL) has significantly advanced this field by developing efficient and accurate surrogate models to replace computationally expensive physics‐based simulations. These surrogate models are commonly used to predict the spatiotemporal evolution of state variables, such as gas saturation and reservoir pressure, in heterogeneous geological formations. Despite the various DL models applied to this task, there is a lack of studies systematically comparing their performance. This absence of comparative analysis leads to somewhat arbitrary DL model selection in subsurface flow research, resulting in suboptimal performance and potentially inaccurate predictions. To bridge this gap, we conduct a systematic comparison study of three popular DL architectures—U‐Net, Fourier Neural Operators (FNO), and Segmentation Transformer (SETR)—in surrogate modeling of underground hydrogen storage (UHS). We focus on UHS due to its promise of enhancing clean energy resilience and its cyclic operational conditions that represent common scenarios in various subsurface applications. We evaluate the models based on accuracy, training cost, and inference speed. The comparison shows that U‐Net achieves the highest accuracy, followed by SETR and FNO. Despite its lower accuracy, FNO has the highest inference speed. SETR offers competitive accuracy with the least training memory usage, demonstrating the potential of transformers in learning subsurface flow. Our results provide guidance for selecting DL models for surrogate modeling in a wide range of subsurface flow problems.

42 ENGINEERING↗

An Integrated Assessment of a G3 GMD Event on Large-Scale Power Grids: From Magnetometer Data to Geomagnetically Induced Current Analysis

Solar activities can cause geomagnetic disturbances (GMDs) that give rise to geomagnetically induced currents (GICs) which may compromise the reliability of the power system. In order to build more reliable models representing GMD interactions with the power grid, the power system’s detailed electrical model must be considered along with fluctuations in the earth’s magnetic and induced surface electric fields. Here, this study investigates the impact of incorporating spatially varying magnetic fields into surface electric field models on GMD risk metrics. A spatially independent magnetic field model and a spatially varying model are compared through simulations. To perform this analysis, the earth’s magnetic field disturbances are transformed into surface electric fields using respective one-dimensional earth conductivity models. Then, the modeling impact of these electric fields is studied using a 2,000-bus grid for Texas and a 25,000-bus grid for the northeast and mid- Atlantic regions of the United States. Simulation results reveal that the inclusion of spatially varying magnetic fields results in considerable differences in GMD risk metrics, highlighting the importance of accounting for spatial variability when assessing GMD risks in the power system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Effects-Based Monitoring of Geomagnetically-Induced Current Using a Convolutional Neural Network

Geomagnetically-induced current (GIC) due to space weather can flow in the power grid causing undesirable effects such as transformer overheating, misoperation of protection devices, and potential blackouts. It is therefore important to monitor GIC in the power grid to improve online situational awareness and decision-making of system operators during a geomagnetic disturbance. To avoid the costly installation of GIC monitors at transformers’ neutrals, it is desirable to find correlations between GIC and already-monitored parameters. Hence, this work proposed the use of a convolutional neural network (CNN) to compute GIC amplitudes from learned patterns in the time-series data of transformer even harmonic currents. Using an electromagnetic transient program, GIC injection simulations were performed for a modeled Dominion Energy Virginia (DEV) substation with two 504 MVA, 500/230 kV transformers. Data collected from these offline simulations were used to train the CNN to provide online GIC monitoring. Testing the CNN performance involved using real GIC measurements from published literature and from a physical GIC monitor in the DEV area. Finally, the results showed that the proposed method was able to provide GIC readings with a root mean squared error of 1.56 A/phase (equivalent to an average accuracy of 94%) for these real GIC waveforms.

42 ENGINEERING↗

Incorporating concentration-dependent sediment microbial activity into methylmercury production kinetics modeling

We report in anoxic environments, anaerobic microorganisms carrying the hgcAB gene cluster can mediate the transformation of inorganic mercury (Hg(II)) to monomethylmercury (MMHg). The kinetics of Hg(II) transformation to MMHg in periphyton from East Fork Poplar Creek (EFPC) in Oak Ridge, TN have previously been modeled using a transient availability model (TAM). The TAM for Hg(II) methylation combines methylation/demethylation kinetics with kinetic expressions for processes that decrease Hg(II) and MMHg availability for methylation and demethylation (multisite sorption of Hg(II) and MMHg, Hg(II) reduction/Hg(0) oxidation). In this study, the TAM is used for the first time to describe MMHg production in sediment. We assessed MMHg production in sediment microcosms using two different sediment types from EFPC: a relatively anoxic, carbon-rich sediment with higher microbial activity (higher CO 2 production from sediment) and a relatively oxic, sandy, carbon-poor sediment with lower microbial activity (lower CO 2 production from sediment). Based on 16s rRNA sequencing, the overall microbial community structure in the two sediments was retained during the incubations. However, the hgcA containing methanogenic Euryarchaeota communities differed between sediment types and their growth followed different trajectories over the course of incubations, potentially contributing to the distinct patterns of MMHg production observed. The general TAM paradigm performed well in describing MMHg production in the sediments. However, the MMHg production and ancillary data suggested the need to revise the model structure to incorporate terms for concentration-dependent microbial activity over the course of the incubations. We modified the TAM to include Monod-type kinetics for methylation and demethylation and observed an improved fit for the carbon-rich, microbially active sediment. Overall our work shows that the TAM can be applied to describe Hg(II) methylation in sediments and that including expressions accounting for concentration-dependent microbial activity can improve the accuracy of the model description of the data in some cases.

54 ENVIRONMENTAL SCIENCES↗

Understanding LIB Battery Electrodes Through Classical Electrochemical Interface Theory

Classical models of the electrochemical interface can be applied to complex and dynamic electrodes to understand important mechanisms that are relevant to device-level phenomena like calendar aging or cycle life. Here, we show that negative alloy electrodes with wide electrochemical windows cannot be assumed to have static interfaces throughout all states of charge. Under high states of charge, the interface -including the solid electrolyte interphase - has characteristics that resemble a highly polarized electrode far from the point of zero charge. Under these conditions, the interface can be understood through the Helmholtz model. At lower states of charge, the interface capacitance decreases and the space charge layer length increases. This transformation resembles a metal electrode approaching the point of zero charge which is understood through the Gouy-Chapman-Stern-Grahame model of the electrochemical interface. This transformation happens at both silicon and carbon-coated silicon interfaces. From voltage-limited cycling experiments, in the limit of the diffuse double layer, the silicon electrode impedance rises significantly in the first few cycles. This rise is related to the continuous electrochemical reduction of electrolyte components from an interface that is not electronically screened from the electrolyte. In other words, an electrode interface for batteries should resemble a 'Helmholtz-like' structure.

anode↗

Evaluating pulse-shaping capabilities of next-generation pulsed power architectures

This project evaluated the pulse shaping capabilities of next-generation pulsed power (NGPP) architectures. NGPP architectures share several common attributes including multiple independent pulse-generation lines, a radial water-insulated impedance transformer, and a central vacuum insulated load region. A multi-module circuit model was developed, incorporating independent pulse-generation lines and a 2-D transmission line mesh of the radial impedance transformer to assess the effects of azimuthal asymmetry in pulse-shaped experiments. Circuit model simulations demonstrated that NGPP architectures are able to produce the the desired current pulse shapes for exemplar NGPP experiments. Additionally, the project explored automated methods for experiment design, including derivative -ree optimization and machine learning. Pulse-shaped experiments require designers to determine machine parameters that reliably produce the desired current pulse at the load, a process that typically relies on expert knowledge and iterative adjustments using the Z circuit model. Given the increased complexity of NGPP systems, this manual approach may be impractical. While the evaluated methods do not eliminate the need for manual iteration, they can reduce the time required for experiment design. Derivative-free optimization automates much of the trial-and-error process, providing a close starting point for manual adjustments or making small modifications to near-final designs. Meanwhile, deep neural network methods can generate a good qualitative match to the desired current pulse in under one second without requiring circuit model simulations.

42 ENGINEERING↗

Warm air intrusions reaching the MOSAiC expedition in April 2020—The YOPP targeted observing period (TOP)

In the spring period of the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition, an initiative was in place to increase the radiosounding frequency during warm air intrusions in the Atlantic Arctic sector. Two episodes with increased surface temperatures were captured during April 12–22, 2020, during a targeted observing period (TOP). The large-scale circulation efficiently guided the pulses of warm air into the Arctic and the observed surface temperature increased from -30°C to near melting conditions marking the transition to spring, as the temperatures did not return to values below -20°C. Back-trajectory analysis identifies 3 pathways for the transport. For the first temperature maximum, the circulation guided the airmass over the Atlantic to the northern Norwegian coast and then to the MOSAiC site. The second pathway was from the south, and it passed over the Greenland ice sheet and arrived at the observational site as a warm but dry airmass due to precipitation on the windward side. The third pathway was along the Greenland coast and the arriving airmass was both warm and moist. The back trajectories originating from pressure levels between 700 and 900 hPa line up vertically, which is somewhat surprising in this dynamically active environment. The processes acting along the trajectory originating from 800 hPa at the MOSAIC site are analyzed. Vertical profiles and surface energy exchange are presented to depict the airmass transformation based on ERA5 reanalysis fields. The TOP could be used for model evaluation and Lagrangian model studies to improve the representation of the small-scale physical processes that are important for airmass transformation. A comparison between MOSAiC observations and ERA5 reanalysis demonstrates challenges in the representation of small-scale processes, such as turbulence and the contributions to various terms of the surface energy budget, that are often misrepresented in numerical weather prediction and climate models.

54 ENVIRONMENTAL SCIENCES↗

GWAS supported by computer vision identifies large numbers of candidate regulators of in planta regeneration in Populus trichocarpa

Plant regeneration is an important dimension of plant propagation and a key step in the production of transgenic plants. However, regeneration capacity varies widely among genotypes and species, the molecular basis of which is largely unknown. Association mapping methods such as genome-wide association studies (GWAS) have long demonstrated abilities to help uncover the genetic basis of trait variation in plants; however, the performance of these methods depends on the accuracy and scale of phenotyping. To enable a large-scale GWAS of in planta callus and shoot regeneration in the model tree Populus, we developed a phenomics workflow involving semantic segmentation to quantify regenerating plant tissues over time. We found that the resulting statistics were of highly non-normal distributions, and thus employed transformations or permutations to avoid violating assumptions of linear models used in GWAS. We report over 200 statistically supported quantitative trait loci (QTLs), with genes encompassing or near to top QTLs including regulators of cell adhesion, stress signaling, and hormone signaling pathways, as well as other diverse functions. Our results encourage models of hormonal signaling during plant regeneration to consider keystone roles of stress-related signaling (e.g. involving jasmonates and salicylic acid), in addition to the auxin and cytokinin pathways commonly considered. The putative regulatory genes and biological processes we identified provide new insights into the biological complexity of plant regeneration, and may serve as new reagents for improving regeneration and transformation of recalcitrant genotypes and species.

59 BASIC BIOLOGICAL SCIENCES↗

Data Repository for Power system Open models With Evolving Resources (DR POWER) (Final Scientific/Technical Report)

Pacific Northwest National Laboratory (PNNL) and the National Rural Electric Cooperative Association (NRECA) created a Data Repository for Power system Open models With Evolving Resources (DR POWER) to establish, curate, and evolve open-access power grid models and scenarios, and measurements and observations, collectively referred to as datasets. Existing open-access datasets are inadequate (too small and not representative of the complexity of the modern grid, and scattered all over the world wide web) for the investigation and benchmarking of the advanced optimization methods needed for next-generation grids. Open-access datasets are static and inadequate in other aspects: they do not provide format transformation tools; there is missing information needed for advanced applications; hard to cite; no model evolution history; difficult to update current collections; and little to no community involvement and feedback. Overall, the current landscape is hindering model evolution and benchmarking standards needed for transformational approaches to be developed, tested, and accepted by the power modeling community.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Scenarios for Future Energy Systems

Energy systems in the U.S. and globally have continuously changed and expanded over the past 200 years, from animal and biomass based energy to electricity and petroleum for heat, light, transportation, and industries of all kinds. The most recent two decades have experienced a significant increase in the use of natural gas, solar, and wind energy, as well as energy efficiency, due to both technology breakthroughs and public policy. With history as a guide, we can expect on-going transformation of our energy system in future decades, including increased electrification and advanced energy technologies, ideally helping meet societal goals of reducing pollution, improving quality of human life, and preserving ecosystems. Scenario modeling is an important tool for planning energy system transformations toward specific goals - such as net-zero greenhouse gas emissions, 100% renewable energy, or energy security - particularly when there are multiple options, conflicting objectives, and significant unknowns about a path forward. This presentation will review the history of energy transitions and the future policy and societal objectives on energy. Then it will summarize multiple scenario studies of future energy systems, with a focus on "100% studies" and the challenges to meeting those objectives. It will also outline of a few of the technologies being researched now that could advance the ongoing energy transition.

ENERGY PLANNING, POLICY, AND ECONOMY,SOLAR ENERGY,↗

EvoProtGrad (Directed Evolution for Proteins with Gradients) [SWR-23-48]

A Python package for directed evolution on a protein sequence with gradient-based discrete Markov chain monte carlo (MCMC). Users are able to compose custom models that map sequence to function with pretrained models, including protein language models (PLMs), to guide and constrain search. Our package natively integrates with the HuggingFace platform and supports PLMs from transformers. Our MCMC sampler identifies promising amino acids to mutate via model gradients taken with respect to the input (i.e., sensitivity analysis). We allow users to compose their own custom target function for MCMC by leveraging the Product of Experts MCMC paradigm. Each model is an "expert" that contributes its own knowledge about the protein's fitness landscape to the overall target function. The sampler is designed to be more efficient and effective than brute force and random search while maintaining most of the generality and flexibility. Additional information can be found in the related publication: https://iopscience.iop.org/article/10.1088/2632-2153/accacd

Emami, Patrick↗

Estimating the impact of tariff-driven behind-the-meter storage operation on distribution grid investments

Increasing growth of distributed solar photovoltaics (PV) and electric vehicles (EV) can strain local distribution networks and require costly upgrades. Distributed battery storage, often deployed alongside PV, can be used to mitigate those costs, depending on how batteries are operated. This study evaluates the potential deferral value of distributed battery storage across a range of tariff structures, focusing on the rate structures most commonly available to residential customers today and related variants. Deferrals are evaluated with a least-cost distribution grid expansion optimization model to identify requirements on line reconductoring, transformer upgrades, and voltage regulator installations under each tariff. Results show that TOU rates and net billing tariffs can yield meaningful deferral value, depending on specific tariff structure features. Under the best performing tariff structure tested, storage produced a median annualized deferral value of $7.18 per kW of storage capacity ( kW S ) across all feeders in the sample, though deferral values were considerably larger for feeders with peak loads that coincide with utility system peak, i.e., timing of TOU peak period. In contrast, under an unrestricted TOU design with no restrictions on grid charging or discharging, the median deferral value was $0/ kW S illustrating the critical importance of tariff structure details.

Rodriguez-Garcia, Luis↗

CoSn-type NiIn1–xSbx (0 ≤ x ≤ 0.17): Site-Selective Substitution, Electronic Structure, Chemical Bonding, and Structural Transformation

CoSn-type intermetallic compounds have emerged as a model platform for Kagome-derived flat-band physics, where subtle chemical perturbations can strongly influence electronic structure and phase stability. Here, we present a combined experimental and theoretical study of Sb-substitution in CoSn-type NiIn1–xSbx (0 ≤ x ≤ 0.17) to elucidate the interplay between site selectivity, solubility limit, chemical bonding, and electronic structure. Rietveld refinements on Neutron powder diffraction data confirmed the selective Sb-substitution at the electron-rich In2 (2d) site forming the honeycomb substructure, while the In1 (1a) site within the Kagome layer remains exclusively occupied by In. Density functional theory (DFT) calculations revealed that pristine CoSn-type NiIn hosts Ni 3d-dominated flat bands near the Fermi level (EF), originating from the Kagome-like Ni substructure. Partial replacement of In by Sb within the honeycomb layer alters these flat-band features below EF, reducing the density of states and suppressing the flat-band topology near the Fermi level. Orbital-resolved electronic structure and chemical-bonding analyses show that Sb-substitution enhances Ni-p-block (In/Sb) covalency and optimizes charge compensation, stabilizing the CoSn-type structure up to the solubility limit x ≈ 0.17. Beyond the limit, the higher-Sb compositions show satellite reflections consistent with an incommensurately modulated phase. These results establish a link between site-selective chemical substitution, bonding optimization, and flat-band electronic structure evolution, providing fundamental insights into how chemical substitution influences the electronic properties of Kagome-based intermetallic compounds.

Roy, Nilanjan [National Institute of Technology Si↗

Nonadiabatic Force Matching for Alchemical Free-Energy Estimation

We propose a method to compute free-energy differences from nonadiabatic alchemical transformations by using flow-based generative models. The method, nonadiabatic force matching, hinges on estimating the dissipation along an alchemical switching process in terms of a nonadiabatic force field that can be learned through stochastic flow matching. The learned field can be used in conjunction with short-time trajectory data to evaluate upper and lower bounds on the alchemical free energy that variationally converge to the exact value if the field is optimal. Applying the method to evaluate the alchemical free energy of atomistic models shows that it can substantially reduce the simulation cost of a free-energy estimate at a negligible loss of accuracy when compared with thermodynamic integration.

Computational chemistry↗

Atomic-Scale Tracking of Topological Defect Motion and Incommensurate Charge Order Melting

Charge order pervades the phase diagrams of quantum materials where it competes with superconducting and magnetic phases, hosts electronic phase transitions and topological defects, and couples to the lattice generating intricate structural distortions. Incommensurate charge order is readily stabilized in manganese oxides, where it is associated with anomalous electronic and magnetic properties, but its nanoscale structural inhomogeneity complicates precise characterization and understanding of its relationship with competing phases. Leveraging atomic-resolution variable-temperature cryogenic scanning transmission electron microscopy, we characterize the thermal evolution of charge order as it transforms from its ground state in a model manganite system. We find that mobile networks of discommensurations and dislocations generate phase inhomogeneity and induce global incommensurability in an otherwise lattice-locked modulation. Driving the order to melt at high temperatures, the discommensuration density grows, and regions of order locally decouple from the lattice periodicity. Published by the American Physical Society 2025

Schnitzer, Noah (ORCID:0000000210421146)↗