Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Historical and Future Global Irrigation Energy Consumption by Fuel and Region

Irrigation energy use is a significant component of agricultural production costs, contributing directly to the energy and emissions intensity of crop production and ultimately to food prices. Understanding the existing structure of irrigation energy consumption help achieve food-energy-water security and environmental goals. We present a comprehensive global data set detailing country-level irrigation energy consumption, emphasizing the comparative use of electric, diesel, and emerging solar pumps. To our knowledge, no such data set exists. We draw from a literature review to develop a logistic transformed regression model to estimate the shares of fuel sources for irrigation across countries over historical years to construct a global data set of country-level irrigation energy consumption by multiple fuel sources. Additionally, we compare our estimates of irrigation energy use with agricultural energy use as reported by the International Energy Agency and other external sources. We then use this data to project future irrigation energy use with the Global Change Analysis Model, which is a multisector dynamics model, to showcase the usage of this data set. Projections under the reference scenario show a global shift in fuel types for irrigation pumping, while patterns vary across regions, with India and Pakistan leading in solar-powered irrigation growth and countries like the USA and China continuing to rely primarily on grid electricity. This data set provides a resource to understand the role of irrigation fuel choices within the broader energy sector, as well as the connected agricultural, land use, and water sectors under alternative future scenarios, enabling informed decision making toward efficient agricultural practices.

Global Change Analysis Model (GCAM)↗
DOE OSTI2023

Identifying COVID-19 cases and extracting patient reported symptoms from Reddit using natural language processing

We used social media data from “covid19positive” subreddit, from 03/2020 to 03/2022 to identify COVID-19 cases and extract their reported symptoms automatically using natural language processing (NLP). We trained a Bidirectional Encoder Representations from Transformers classification model with chunking to identify COVID-19 cases; also, we developed a novel QuadArm model, which incorporates Question-answering, dual-corpus expansion, Adaptive rotation clustering, and mapping, to extract symptoms. Our classification model achieved a 91.2% accuracy for the early period (03/2020-05/2020) and was applied to the Delta (07/2021–09/2021) and Omicron (12/2021–03/2022) periods for case identification. We identified 310, 8794, and 12,094 COVID-positive authors in the three periods, respectively. The top five common symptoms extracted in the early period were coughing (57%), fever (55%), loss of sense of smell (41%), headache (40%), and sore throat (40%). During the Delta period, these symptoms remained as the top five symptoms with percent authors reporting symptoms reduced to half or fewer than the early period. During the Omicron period, loss of sense of smell was reported less while sore throat was reported more. Our study demonstrated that NLP can be used to identify COVID-19 cases accurately and extracted symptoms efficiently.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

An implementation of the phase-field model based on coupled thermomechanical finite element solvers for large-strain twinning, explicit dynamic fracture and the classical Stefan problem

The implementation of a phase-field model in finite elements usually requires significant expertise and involves the development of a user element with additional degrees of freedom. An alternative implementation of the phase-field model within a thermo-mechanical finite element simulation package was presented in (Cho et al 2012 Int. J. Solids Struct. 49 1973–1992), where the phase-field variable is treated as the temperature degree of freedom. However, this approach has only been used for small strain phase-field modelling of martensitic transformations and quasistatic phase-field modelling of fracture. Here, we present a phase-field finite element implementation via the temperature degree of freedom for several additional cases from the literature: (i) the large-strain phase-field description of deformation twinning presented in (Clayton and Knap 2011 Physica D 240 841–858), (ii) phase-field description of brittle fracture with inertial effects based on the theory from (Molnár and Gravouil 2017, Finite Elem. Anal. Des. 130 27–38) and (Miehe et al 2010 Int. J. Numer. Methods Eng. 83 1273–1311) and (iii) the classical Stefan problem of solidification presented in (Mackenzie and Robertson 2002 J. Comput. Phys. 181 526–544). The last problem involves the temperature and phase-field variables as unknowns.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗
DOE OSTI2022

Propagating Parameter Uncertainty in Power System Nonlinear Dynamic Simulations Using a Koopman Operator-Based Surrogate Model

In this work, we propose a Koopman operator-based surrogate model for propagating parameter uncertainties in power system nonlinear dynamic simulations. First, we augment a priori known state-space model by reformulating parameters deemed uncertain as pseudo-state variables. Then, we apply the Koopman operator theory to the resulting state-space model and obtain a linear dynamical system model. This transformation allows us to analyze the evolution of the system dynamics through its Koopman eigenfunctions, eigenvalues, and modes. Of particular importance for this letter, the obtained linear dynamical system is a surrogate that enables the evaluation of parameter uncertainties by simply perturbing the initial conditions of the Koopman eigenfunctions associated with the pseudo-state variables. Simulations carried out on the New England test system reveal the excellent performance of the proposed method in terms of accuracy and computational efficiency.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A molecular mediator for reductive concerted proton-electron transfers via electrocatalysis

Electrocatalytic approaches to the activation of unsaturated substrates via reductive concerted proton-electron transfer (CPET) must overcome competing, often kinetically dominant hydrogen evolution. We introduce the design of a molecular mediator for electrochemically triggered reductive CPET through the synthetic integration of a Brønsted acid and a redox mediator. Cathodic reduction at the cobaltocenium redox mediator substantially weakens the homolytic nitrogen-hydrogen bond strength of a Brønsted acidic anilinium tethered to one of the cyclopentadienyl rings. The electrochemically generated molecular mediator is demonstrated to transform a model substrate, acetophenone, to its corresponding neutral α-radical via a rate-determining CPET.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

BrzostekEcologyLab/CORPSE-soil-jars

Efforts to manage soils for carbon (C) sequestration remain limited by our understanding of how differences in plant traits and microbial traits mechanistically drive soil organic C (SOC) storage. Addressing this uncertainty is particularly critical in bioenergy agriculture, due to its potential to enhance soil C and provide a C neutral fuel. As such, we examined differences between two contrasting feedstocks, Zea mays (corn) and Miscanthus x giganteus (miscanthus), in the ability of their litter to form new chemically resistant particulate SOC vs. physically protected mineral associated SOC and used this data to improve the parameterization of a microbial SOC model. We tested a hypothesized conceptual model whereby easy to decompose corn litters drive greater microbial carbon use efficiency (CUE) and the formation of more mineral associated SOC over particulate SOC than more complex miscanthus litters. To do this, we performed a soil microcosm experiment where we added 13C enriched aboveground and belowground litters to soils and traced the fate of the 13C into microbial respiration and SOC pools. We found that corn litters promoted higher microbial CUE (0.37) than miscanthus litters (0.24). In turn, corn litter formed approximately 50% more mineral associated SOC than miscanthus litters. Similarly, structurally complex root litters promoted a lower CUE and formed less mineral associated SOC than leaf and shoot litters for both crops. When we used our data to parameterize the SOC model, we found that modeling microbial trait differences uniquely allowed the model to capture the fate of litter C in SOC. Collectively, we found a robust link between litter quality, microbial efficiency, and the formation of SOC. This link bridges the empirical uncertainty in how different crops can form new soil C and provides an empirical basis for modeling SOC transformations.

Ridgeway, Joanna↗
DOE OSTI2023

Refining Processing Engines from SAPHIRE: Initialization of Fault Tree/Event Tree Solver

SAPHIRE has been extensively employed for over 35 years to model risk-important systems and quantify risk models. As a well-established and thoroughly documented Probabilistic Risk Assessment (PRA) tool, SAPHIRE has continuously tracked computational trends and received regular updates. Despite its ongoing evolution, there remains a need for further enhancements, particularly in dealing with the quantification of exceptionally large models. These improvements could take the form of algorithmic advancements, harnessing the power of parallel computing, and exploring the potential benefits of cloud computing solutions. Considering these aspirations, the notion of a remote solve option was introduced and subsequently evolved into a dedicated project within the SAPHIRE development team. A significant outcome of this initiative is SAPHSOLVE, an engine extracted from the SageRisk API designed specifically for remote solving capabilities. The ongoing project is nearing its culmination, marked by a series of discoveries that have brought undocumented aspects to light. Among these revelations is the intricacy of the input and output format for the SAPHSOLVE engine. This document serves the crucial purpose of meticulously delineating the precise formats for both input and output, as they form an indispensable foundation. The importance of documenting these formats cannot be overstated, as it is a pivotal step in facilitating rigorous testing and comparison. Whether it involves scrutinizing SAPHSOLVE results against those of the internal integrated solver or other external solvers, the ability to construct models or transform existing ones into a compatible SAPHSOLVE format is imperative. Chapter 1 offers a succinct introduction to both SAPHIRE and SAPHSOLVE, followed by Chapter 2 which outlines the roadmap for enhancing SAPHSOLVE. The core of this report is Chapter 3, which intricately elucidates the intricacies of the input and output file formats. To provide a tangible illustration of these formats, a rudimentary example has been compiled and is available in Appendix. SAPHSOLVE represents a novel external solving mechanism developed by the SAPHIRE team, although it has not yet reached the full spectrum of capabilities possessed by SAPHIRE's internal solver. However, the SAPHIRE team has set a comprehensive course for incorporating the functionalities of SAPHSOLVE. A comprehensive outlook on the future of SAPHSOLVE is expounded upon in Chapter 4.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Arbitrary Polynomial Separations in Trainable Quantum Machine Learning

Recent theoretical results in quantum machine learning have demonstrated a general trade-off between the expressive power of quantum neural networks (QNNs) and their trainability; as a corollary of these results, practical exponential separations in expressive power over classical machine learning models are believed to be infeasible as such QNNs take a time to train that is exponential in the model size. We here circumvent these negative results by constructing a hierarchy of efficiently trainable QNNs that exhibit unconditionally provable, polynomial memory separations of arbitrary constant degree over classical neural networks—including state-of-the-art models, such as Transformers—in performing a classical sequence modeling task. This construction is also computationally efficient, as each unit cell of the introduced class of QNNs only has constant gate complexity. We show that contextuality—informally, a quantitative notion of semantic ambiguity—is the source of the expressivity separation, suggesting that other learning tasks with this property may be a natural setting for the use of quantum learning algorithms.

Anschuetz, Eric R. [California Institute of Techno↗
DOE OSTI2024

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

42 ENGINEERING↗
DOE OSTI2026

Advancing Industry 4.0: Multimodal Sensor Fusion for AI-Based Fault Detection in 3D Printing

Additive manufacturing, particularly fused deposition modeling, is transforming modern production by enabling rapid prototyping and complex part fabrication. However, its layer-by-layer process remains vulnerable to faults such as nozzle clogging, filament runout, and layer misalignment, which compromise print quality and reliability. Traditional inspection methods are costly, time-intensive, and often limited to post-process analysis, making them unsuitable for real-time intervention. In this current study, the authors developed a novel, low-cost, and portable faultdetection system that leverages multimodal sensor fusion and artificial intelligence for real-time monitoring in FDM-based 3D printing. The system integrates acoustic, vibration, and thermal sensing into a non-intrusive architecture, capturing complementary data streams that reflect both mechanical and process-related anomalies. Acoustic and thermal sensors operate in a fully contactless manner, while the vibration sensor requires minimal attachment such that it will not interfere with printer hardware, thereby preserving portability and ease of deployment. The multimodal signals are processed into spectrograms and time-frequency features, which are classified using convolutional neural networks for intelligent fault detection. The proposed system advances Industry 4.0 objectives by offering an affordable, scalable, and practical monitoring solution that improves faultdetection accuracy, reduces waste, and supports sustainable, adaptive manufacturing.

42 ENGINEERING↗
NASA NTRS1982

Hormone purification by isoelectric focusing in space

The performance of a ground-prototype of an apparatus for recycling isoelectric focusing was evaluated in an effort to provide technology for large scale purification of peptide hormones, proteins, and other biologicals. Special emphasis was given to the effects of gravity on the function of the apparatus and to the determination of potential advantages deriveable from its use in a microgravity environment. A theoretical model of isoelectric focusing sing chemically defined buffer systems for the establishment of the pH gradients was developed. The model was transformed to a form suitable for computer simulations and was used extensively for the design of experimental buffers.

Bier, M.↗
NASA NTRS1982

Satellite remote sensing - An integral tool in acquiring global crop production information

Since NASA's program of research concerning remote sensing was initiated in the 1960s, one of its major objectives has been to advance the state-of-the-art in machine processing of satellite acquired multispectral data. Possibilities have been studied regarding a use of these data to identify type, to monitor condition, and to estimate the ontogenetic stage of cultural vegetation. The present investigation provides a review of the state-of-the-art of the technology used to make remote sensing crop production estimates in foreign regions. Attention is given to Landsat data acquisition, aspects of registration and preprocessing, questions of data transformation, data modeling, proportion estimation, labeling, development stage models, crop condition models, and an outlook regarding future developments.

Hall, F. G.↗
NASA NTRS1983

A survey of automated remote sensing for agriculture

The state-of-the-art of the technology available to make remote sensing crop production estimates is reviewed with reference to several past and present research projects. In particular, attention is given to Landsat data acquisition, registration and preprocessing, data transformation, data modeling, proportion estimation, and labeling. Development stage models and crop condition models are briefly characterized, and areas where further research is needed are identified.

Hall, F. G.↗
NASA NTRS1984

Use of satellite data in agricultural surveys

The state-of-the-art of crop surveying by satellite is reviewed with an emphasis on the signature extension problem. Registration and preprocessing procedures are discussed with refereence to: normalization of the radiometric values of each scene for scene-to-scene differences; registration techniques, implemented at the NASA Johnson Space Center, capable of 0.5 pixel root-mean-square error; and current research in this direction. Data transformation and modeling techniques applied to the Landsat MSS images and a solution for the field-to-field variations of the greenness and brightness temporal trajectories are included. Finally, a review of the mixture decomposition method of labeling and estimating the areal proportions is given.

Hall, F. G.↗