Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions

We discuss the challenges and propose research directions for using AI to revolutionize the development of high-performance computing (HPC) software. AI technologies, in particular large language models, have transformed every aspect of software development. For its part, HPC software is recognized as a highly specialized scientific field of its own. We discuss the challenges associated with leveraging state-of-the-art AI technologies to develop such a unique and niche class of software and outline our research directions in the two US Department of Energy–funded projects for advancing HPC Software via AI: Ellora and Durban.

Teranishi, Keita [ORNL] (ORCID:0000000166472690)↗
DOE OSTI2024

Multivariate prognosis of battery advanced state of health via transformers

De-risking energy storage investments necessary to meet CO 2 reduction targets requires a deep understanding of the connections between battery health, design, and use. The historical definition of the battery state of health (SOH) as the percentage of current versus initial capacity is inadequate for this purpose, motivating an expanded SOH consisting of an interrelated set of descriptors including capacity, energy, ionic and electronic impedances, open-circuit voltages, and microstructure metrics. In this work, we introduce deep transformer networks for the simultaneous prognosis of 28 battery SOH descriptors using two cycling datasets representing six lithium-ion cathode chemistries, multiple electrolyte/anode compositions, and different charge-discharge scenarios. The accuracy of these predictions for battery life (with an unprecedented mean absolute error of 19 cycles in predicting end of life for a lithium-iron-phosphate fast-charging dataset) illustrates the promise of deep learning toward providing enhanced understanding and control of battery health.

25 ENERGY STORAGE↗
DOE OSTI2026

Photoluminescent Cu(I) HETPHENs Featuring Bulky Alkyl-Substituted Phenanthrolines

Successful excited-state-enhancing substituent effect strategies applied in homoleptic Cu(I) metal-to-ligand charge transfer (MLCT) chromophores have been adapted to the heteroleptic phenanthroline (HETPHEN) platform, leveraging 2,9-mesityl-1,10-phenanthroline (mesPhen) in conjunction with a series of seven distinct 2,9- and 2,3,4,7,8,9-substituted phenanthrolines. The newly conceived CuHETPHEN complexes feature MLCT lifetimes ranging from 62 to 443 ns, all of which exhibit unprecedented room-temperature photoluminescence and demonstrate quantitative adherence to the energy gap law. TD-DFT calculations successfully modeled the systematic variation in electronic transition intensities, accounting for the experimentally observed UV–vis absorption bands across the entire series of molecules while providing detailed structural explanations for the variations in spectral profiles. These heteroleptic Cu(I) diimine chromophores exhibit thermally activated delayed fluorescence (TADF), with delayed fluorescence occurring between closely spaced 1 MLCT and 3 MLCT excited manifolds, having energetic separations of ΔE = 713–1009 cm –1 , as demonstrated here for the first time. Nanosecond and ultrafast transient absorption spectroscopy verified the MLCT nature of these excited states and extracted the time constants for the initial pseudo-Jahn–Teller distortion and intersystem crossing throughout the entire series of molecules. Lastly, the triplet photosensitization properties of these Cu(I) HETPHENs are demonstrated in a number of model photochemical transformations.

Chromophores↗
DOE OSTI2026

Historical and Future Global Irrigation Energy Consumption by Fuel and Region

Irrigation energy use is a significant component of agricultural production costs, contributing directly to the energy and emissions intensity of crop production and ultimately to food prices. Understanding the existing structure of irrigation energy consumption help achieve food-energy-water security and environmental goals. We present a comprehensive global data set detailing country-level irrigation energy consumption, emphasizing the comparative use of electric, diesel, and emerging solar pumps. To our knowledge, no such data set exists. We draw from a literature review to develop a logistic transformed regression model to estimate the shares of fuel sources for irrigation across countries over historical years to construct a global data set of country-level irrigation energy consumption by multiple fuel sources. Additionally, we compare our estimates of irrigation energy use with agricultural energy use as reported by the International Energy Agency and other external sources. We then use this data to project future irrigation energy use with the Global Change Analysis Model, which is a multisector dynamics model, to showcase the usage of this data set. Projections under the reference scenario show a global shift in fuel types for irrigation pumping, while patterns vary across regions, with India and Pakistan leading in solar-powered irrigation growth and countries like the USA and China continuing to rely primarily on grid electricity. This data set provides a resource to understand the role of irrigation fuel choices within the broader energy sector, as well as the connected agricultural, land use, and water sectors under alternative future scenarios, enabling informed decision making toward efficient agricultural practices.

Global Change Analysis Model (GCAM)↗
DOE OSTI2023

Identifying COVID-19 cases and extracting patient reported symptoms from Reddit using natural language processing

We used social media data from “covid19positive” subreddit, from 03/2020 to 03/2022 to identify COVID-19 cases and extract their reported symptoms automatically using natural language processing (NLP). We trained a Bidirectional Encoder Representations from Transformers classification model with chunking to identify COVID-19 cases; also, we developed a novel QuadArm model, which incorporates Question-answering, dual-corpus expansion, Adaptive rotation clustering, and mapping, to extract symptoms. Our classification model achieved a 91.2% accuracy for the early period (03/2020-05/2020) and was applied to the Delta (07/2021–09/2021) and Omicron (12/2021–03/2022) periods for case identification. We identified 310, 8794, and 12,094 COVID-positive authors in the three periods, respectively. The top five common symptoms extracted in the early period were coughing (57%), fever (55%), loss of sense of smell (41%), headache (40%), and sore throat (40%). During the Delta period, these symptoms remained as the top five symptoms with percent authors reporting symptoms reduced to half or fewer than the early period. During the Omicron period, loss of sense of smell was reported less while sore throat was reported more. Our study demonstrated that NLP can be used to identify COVID-19 cases accurately and extracted symptoms efficiently.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

An implementation of the phase-field model based on coupled thermomechanical finite element solvers for large-strain twinning, explicit dynamic fracture and the classical Stefan problem

The implementation of a phase-field model in finite elements usually requires significant expertise and involves the development of a user element with additional degrees of freedom. An alternative implementation of the phase-field model within a thermo-mechanical finite element simulation package was presented in (Cho et al 2012 Int. J. Solids Struct. 49 1973–1992), where the phase-field variable is treated as the temperature degree of freedom. However, this approach has only been used for small strain phase-field modelling of martensitic transformations and quasistatic phase-field modelling of fracture. Here, we present a phase-field finite element implementation via the temperature degree of freedom for several additional cases from the literature: (i) the large-strain phase-field description of deformation twinning presented in (Clayton and Knap 2011 Physica D 240 841–858), (ii) phase-field description of brittle fracture with inertial effects based on the theory from (Molnár and Gravouil 2017, Finite Elem. Anal. Des. 130 27–38) and (Miehe et al 2010 Int. J. Numer. Methods Eng. 83 1273–1311) and (iii) the classical Stefan problem of solidification presented in (Mackenzie and Robertson 2002 J. Comput. Phys. 181 526–544). The last problem involves the temperature and phase-field variables as unknowns.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗
DOE OSTI2022

Propagating Parameter Uncertainty in Power System Nonlinear Dynamic Simulations Using a Koopman Operator-Based Surrogate Model

In this work, we propose a Koopman operator-based surrogate model for propagating parameter uncertainties in power system nonlinear dynamic simulations. First, we augment a priori known state-space model by reformulating parameters deemed uncertain as pseudo-state variables. Then, we apply the Koopman operator theory to the resulting state-space model and obtain a linear dynamical system model. This transformation allows us to analyze the evolution of the system dynamics through its Koopman eigenfunctions, eigenvalues, and modes. Of particular importance for this letter, the obtained linear dynamical system is a surrogate that enables the evaluation of parameter uncertainties by simply perturbing the initial conditions of the Koopman eigenfunctions associated with the pseudo-state variables. Simulations carried out on the New England test system reveal the excellent performance of the proposed method in terms of accuracy and computational efficiency.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A molecular mediator for reductive concerted proton-electron transfers via electrocatalysis

Electrocatalytic approaches to the activation of unsaturated substrates via reductive concerted proton-electron transfer (CPET) must overcome competing, often kinetically dominant hydrogen evolution. We introduce the design of a molecular mediator for electrochemically triggered reductive CPET through the synthetic integration of a Brønsted acid and a redox mediator. Cathodic reduction at the cobaltocenium redox mediator substantially weakens the homolytic nitrogen-hydrogen bond strength of a Brønsted acidic anilinium tethered to one of the cyclopentadienyl rings. The electrochemically generated molecular mediator is demonstrated to transform a model substrate, acetophenone, to its corresponding neutral α-radical via a rate-determining CPET.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

BrzostekEcologyLab/CORPSE-soil-jars

Efforts to manage soils for carbon (C) sequestration remain limited by our understanding of how differences in plant traits and microbial traits mechanistically drive soil organic C (SOC) storage. Addressing this uncertainty is particularly critical in bioenergy agriculture, due to its potential to enhance soil C and provide a C neutral fuel. As such, we examined differences between two contrasting feedstocks, Zea mays (corn) and Miscanthus x giganteus (miscanthus), in the ability of their litter to form new chemically resistant particulate SOC vs. physically protected mineral associated SOC and used this data to improve the parameterization of a microbial SOC model. We tested a hypothesized conceptual model whereby easy to decompose corn litters drive greater microbial carbon use efficiency (CUE) and the formation of more mineral associated SOC over particulate SOC than more complex miscanthus litters. To do this, we performed a soil microcosm experiment where we added 13C enriched aboveground and belowground litters to soils and traced the fate of the 13C into microbial respiration and SOC pools. We found that corn litters promoted higher microbial CUE (0.37) than miscanthus litters (0.24). In turn, corn litter formed approximately 50% more mineral associated SOC than miscanthus litters. Similarly, structurally complex root litters promoted a lower CUE and formed less mineral associated SOC than leaf and shoot litters for both crops. When we used our data to parameterize the SOC model, we found that modeling microbial trait differences uniquely allowed the model to capture the fate of litter C in SOC. Collectively, we found a robust link between litter quality, microbial efficiency, and the formation of SOC. This link bridges the empirical uncertainty in how different crops can form new soil C and provides an empirical basis for modeling SOC transformations.

Ridgeway, Joanna↗
DOE OSTI2023

Refining Processing Engines from SAPHIRE: Initialization of Fault Tree/Event Tree Solver

SAPHIRE has been extensively employed for over 35 years to model risk-important systems and quantify risk models. As a well-established and thoroughly documented Probabilistic Risk Assessment (PRA) tool, SAPHIRE has continuously tracked computational trends and received regular updates. Despite its ongoing evolution, there remains a need for further enhancements, particularly in dealing with the quantification of exceptionally large models. These improvements could take the form of algorithmic advancements, harnessing the power of parallel computing, and exploring the potential benefits of cloud computing solutions. Considering these aspirations, the notion of a remote solve option was introduced and subsequently evolved into a dedicated project within the SAPHIRE development team. A significant outcome of this initiative is SAPHSOLVE, an engine extracted from the SageRisk API designed specifically for remote solving capabilities. The ongoing project is nearing its culmination, marked by a series of discoveries that have brought undocumented aspects to light. Among these revelations is the intricacy of the input and output format for the SAPHSOLVE engine. This document serves the crucial purpose of meticulously delineating the precise formats for both input and output, as they form an indispensable foundation. The importance of documenting these formats cannot be overstated, as it is a pivotal step in facilitating rigorous testing and comparison. Whether it involves scrutinizing SAPHSOLVE results against those of the internal integrated solver or other external solvers, the ability to construct models or transform existing ones into a compatible SAPHSOLVE format is imperative. Chapter 1 offers a succinct introduction to both SAPHIRE and SAPHSOLVE, followed by Chapter 2 which outlines the roadmap for enhancing SAPHSOLVE. The core of this report is Chapter 3, which intricately elucidates the intricacies of the input and output file formats. To provide a tangible illustration of these formats, a rudimentary example has been compiled and is available in Appendix. SAPHSOLVE represents a novel external solving mechanism developed by the SAPHIRE team, although it has not yet reached the full spectrum of capabilities possessed by SAPHIRE's internal solver. However, the SAPHIRE team has set a comprehensive course for incorporating the functionalities of SAPHSOLVE. A comprehensive outlook on the future of SAPHSOLVE is expounded upon in Chapter 4.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Arbitrary Polynomial Separations in Trainable Quantum Machine Learning

Recent theoretical results in quantum machine learning have demonstrated a general trade-off between the expressive power of quantum neural networks (QNNs) and their trainability; as a corollary of these results, practical exponential separations in expressive power over classical machine learning models are believed to be infeasible as such QNNs take a time to train that is exponential in the model size. We here circumvent these negative results by constructing a hierarchy of efficiently trainable QNNs that exhibit unconditionally provable, polynomial memory separations of arbitrary constant degree over classical neural networks—including state-of-the-art models, such as Transformers—in performing a classical sequence modeling task. This construction is also computationally efficient, as each unit cell of the introduced class of QNNs only has constant gate complexity. We show that contextuality—informally, a quantitative notion of semantic ambiguity—is the source of the expressivity separation, suggesting that other learning tasks with this property may be a natural setting for the use of quantum learning algorithms.

Anschuetz, Eric R. [California Institute of Techno↗
DOE OSTI2024

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

42 ENGINEERING↗
DOE OSTI2026

Advancing Industry 4.0: Multimodal Sensor Fusion for AI-Based Fault Detection in 3D Printing

Additive manufacturing, particularly fused deposition modeling, is transforming modern production by enabling rapid prototyping and complex part fabrication. However, its layer-by-layer process remains vulnerable to faults such as nozzle clogging, filament runout, and layer misalignment, which compromise print quality and reliability. Traditional inspection methods are costly, time-intensive, and often limited to post-process analysis, making them unsuitable for real-time intervention. In this current study, the authors developed a novel, low-cost, and portable faultdetection system that leverages multimodal sensor fusion and artificial intelligence for real-time monitoring in FDM-based 3D printing. The system integrates acoustic, vibration, and thermal sensing into a non-intrusive architecture, capturing complementary data streams that reflect both mechanical and process-related anomalies. Acoustic and thermal sensors operate in a fully contactless manner, while the vibration sensor requires minimal attachment such that it will not interfere with printer hardware, thereby preserving portability and ease of deployment. The multimodal signals are processed into spectrograms and time-frequency features, which are classified using convolutional neural networks for intelligent fault detection. The proposed system advances Industry 4.0 objectives by offering an affordable, scalable, and practical monitoring solution that improves faultdetection accuracy, reduces waste, and supports sustainable, adaptive manufacturing.

42 ENGINEERING↗
DOE OSTI2023

Computed axial lithography optimization system

A system for determining a light intensity field for use in manufacturing a 3D object from a volume of material. The system receives a 3D specification of a 3D geometry for the 3D object that specifies voxels within the volume that contain material that is to be part of the 3D object. The system employs a cost function for effectiveness of a light intensity field in manufacturing the 3D object. The cost function may be an adjoint of an Attenuated Radon Transform that models an energy dose that each voxel would receive during manufacture of the 3D object using the light intensity field. The system applies an optimization technique that employs the cost function to generate a measure of the effectiveness of possible light intensity fields and outputs an indication of a light intensity field that will be effective in manufacturing the 3D object.

Shusteff, Maxim↗