Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Coevolution of Machine Learning and Process-Based Modelling to Revolutionize Earth and Environmental Sciences: A Perspective

Machine learning (ML) applications in Earth and environmental sciences (EES) have gained incredible momentum in recent years. However, these ML applications have largely evolved in ‘isolation’ from the mechanistic, process-based modelling (PBM) paradigms, which have historically been the cornerstone of scientific discovery and policy support. In this perspective, we assert that the cultural barriers between the ML and PBM communities limit the potential of ML, and even its ‘hybridization’ with PBM, for EES applications. Fundamental, but often ignored, differences between ML and PBM are discussed as well as their strengths and weaknesses in light of three overarching modelling objectives in EES, (1) nowcasting and prediction, (2) scenario analysis, and (3) diagnostic learning. The paper ponders over a ‘coevolutionary’ approach to model building, shifting away from a borrowing to a co-creation culture, to develop a generation of models that leverage the unique strengths of ML such as scalability to big data and high-dimensional mapping, while remaining faithful to process-based knowledge base and principles of model explainability and interpretability, and therefore, falsifiability.

Saman Razavi↗
DOE OSTI2026

HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models

As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.

Herron, Emily [ORNL] (ORCID:0000000273008172)↗
DOE OSTI2025

ESAC (EQ-SANS Assisting Chatbot): Application of large language models and retrieval-augmented generation for enhanced user experience at EQ-SANS

Neutron scattering experiments have played vital roles in exploring materials properties in the past decades. While user interfaces have been improved over time, neutron scattering experiments still require specific knowledge or training by an expert due to the complexity of such advanced instrumentation and the limited number of experiments each person may perform each year. This paper introduces an innovative chatbot application that leverages Large Language Models(LLM) and Retrieval-Augmented Generation (RAG) technologies to significantly enhance the user experience at the EQ-SANS, a small-angle neutron scattering instrument at the Spallation Neutron Source of Oak Ridge National Laboratory. Through a user-centric design approach, the EQ-SANS Assisting Chatbot (ESAC) serves as an interactive reference for users, thereby facilitating the use of the instrument by visiting scientists. By bridging the gap between the users of EQ-SANS and the control systems required to perform their experiments, the ESAC sets a new standard for interactive learning and support for the scientific community using large-scale scientific facilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

AI-Based EMT Dynamic Model of PV Systems

Several electromagnetic transient (EMT) dynamic modeling methods are available to model systems like photovoltaic (PV) plants, wind power plants, variable-speed drives, among others. The methods include: (a) physics-based models and (b) data-driven models. The physics-based dynamic models may include high-fidelity switched system model and average-value model that both require the control algorithms included in the models. However, manufacturers typically prefer to provide black-box models to avoid disclosing proprietary. One of the solutions to prevent disclosing control algorithms is the use of data-driven dynamic EMT models of PV systems. In this paper, data-driven dynamic EMT model based on artificial intelligence (AI) algorithms are presented. The AI algorithms evaluated include convolutional neural networks, recurrent neural networks, and nonlinear auto-regressive exogenous model. Automation in generating data and training these models is also discussed in this paper. The results generated by the best AI algorithms have been observed to be greater than 95 % accurate.

Debnath, Suman↗
NASA NTRS1990

Procedure for generating global atmospheric engine emissions data from future supersonic transport aircraft. The 1990 high speed civil transport studies

The input for global atmospheric chemistry models was generated for baseline High Speed Civil Transport (HSCT) configurations at Mach 1.6, 2.2, and 3.2. The input is supplied in the form of number of molecules of specific exhaust constituents injected into the atmosphere per year by latitude and by altitude (for 2-D codes). Seven exhaust constituents are currently supplied: NO, NO2, CO, CO2, H2O, SO2, and THC (Trace Hydrocarbons). An eighth input is also supplied, NO(x), the sum of NO and NO2. The number of molecules of a given constituent emitted per year is a function of the total fuel burned by a supersonic fleet and the emission index (EI) of the aircraft engine for the constituent in question. The EIs for an engine are supplied directly by the engine manufacturers. The annual fuel burn of a supersonic fleet is calculated from aircraft performance and economic criteria, both of which are strongly dependent on basic design parameters such as speed and range. The altitude and latitude distribution of the emission is determined based on 10 Intern. Air Transport Assoc. (IATA) regions chosen to define the worldwide route structure for future HSCT operations and the mission flight profiles.

Sohn, R. A.↗
NASA NTRS2023

Capitol Reef Ecological Conservation: Mapping Vegetation Functional Groups to Inform Invasive Vegetation Management, Ecological Conservation and Restoration in Capitol Reef National Park

Invasive exotic plant (IEP) species have been found within the park boundaries of Capitol Reef National Park (CARE) in Utah. Currently, remotely sensed datasets such as the Rangeland Analysis Platform (RAP) from the United States Department of Agriculture (USDA) have been used to investigate IEP species within the park, but validation of the national RAP program is necessary for informing decisions at a local scale. CARE seeks a remote monitoring solution that can precisely target managerial efforts within the park’s challenging terrain and hard-to-reach locations. To fulfill this objective, we harnessed Landsat 8 Operational Land Imager (OLI) imagery and leveraged Random Forest (RF) modeling to generate classification maps characterizing vegetation functional groups for 2013 and 2022 within the park. Subsequently, the Land Change Modeler (LCM) in Idrisi TerrSet facilitated the production of a predicted classification map for 2033. The team also devised an annual grass probability map to accentuate areas impacted by exotic grasses. A comparative assessment between the RF classification map and the RAP map for 2022 revealed an overall agreement of 47.41%, with disparities primarily arising from differences in bare soil and shrub areas. Significantly, the 2022 RF-generated classification map showcased an impressive overall accuracy of 92.17%. In short, the probability map, the land cover change detection spanning 2013 to 2022, and the forecasting of observed trends into the future aids in the evaluation of invasive plant impacts and facilitation of CARE’s preparedness for potential ecological disturbances. Notably, in comparison to the RAP, the RF classification method generates functional group maps that are more representative of the study area.

Vanchy Li↗
DOE OSTI2023

High–Resolution Maps of Near–Surface Permafrost for Three Watersheds on the Seward Peninsula, Alaska Derived From Machine Learning

Permafrost soils are a critical component of the global carbon cycle and are locally important because they regulate the hydrologic flux from uplands to rivers. Furthermore, degradation of permafrost soils causes land surface subsidence, damaging infrastructure that is crucial for local communities. Regional and hemispherical maps of permafrost are too coarse to resolve distributions at a scale relevant to assessments of infrastructure stability or to illuminate geomorphic impacts of permafrost thaw. Here we train machine learning models to generate meter–scale maps of near–surface permafrost for three watersheds in the discontinuous permafrost region. The models were trained using ground truth determinations of near–surface permafrost presence from measurements of soil temperature and electrical resistivity. We trained three classifiers: extremely randomized trees (ERTr), support vector machines (SVM), and an artificial neural network (ANN). Model uncertainty was determined using k–fold cross validation, and the modeled extents of near–surface permafrost were compared to the observed extents at each site. At–a–site near–surface permafrost distributions predicted by the ERTr produced the highest accuracy (70%–90%). However, the transferability of the ERTr to the sites outside of the training data set was poor, with accuracies ranging from 50% to 77%. The SVM and ANN models had lower accuracies for at–a–site prediction (70%–83%), yet they had greater accuracy when transferred to the non–training site (62%–78%). These models demonstrate the potential for integrating high–resolution spatial data and machine learning models to develop maps of near–surface permafrost extent at resolutions fine enough to assess infrastructure vulnerability and landscape morphology influenced by permafrost thaw.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Dataset for Top Model Decision Tree: Selecting Segmentation Models for Reliable Quantitative Analysis in Low- and Ultralow-Dose CryoEM

Motivation Multiple deep learning model architectures can be used to segment bacterial membranes in cryoEM images. However, an AI-based tool advancement is often presented with only a single segmentation model for broad use, and this single model may show inconsistent results across datasets from different users. Here, we present the Top Model Decision Tree, a model screening framework to screen for the best model to generate bacterial inner and outer membrane masks based on user priorities. We use pre-trained segmentation models from YOLOv11, YOLO26, U-Net, Detectron2 and SAM3 fine-tuned on bacterial inner and outer membranes imaged with cryoEM. Run the Framework This notebook must be opened in Google Colab. Mount Google Drive and run with a GPU-based runtime. Open the notebook and follow steps to git clone in folders and files within this repository. There will be a repeating top_model_decision_tree.ipynb (notebook clone) that will not be used. Save your .png binary mask files and .csv table outputs within your Google Drive or download before closing the notebook. The models and all analysis/training scripts are available at [GitHub: https://github.com/Lynnicia/CryoEM_membranes_top_model_decision_tree and https://github.com/Sireesiru/Semantic-Segmentation-of-bacterial-cell-envelope-using-U-Nets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Short-wavelength-sensitive 2 (Sws2) visual photopigment models combined with atomistic molecular simulations to predict spectral peaks of absorbance

For many species, vision is one of the most important sensory modalities for mediating essential tasks that include navigation, predation and foraging, predator avoidance, and numerous social behaviors. The vertebrate visual process begins when photons of the light interact with rod and cone photoreceptors that are present in the neural retina. Vertebrate visual photopigments are housed within these photoreceptor cells and are sensitive to a wide range of wavelengths that peak within the light spectrum, the latter of which is a function of the type of chromophore used and how it interacts with specific amino acid residues found within the opsin protein sequence. Minor differences in the amino acid sequences of the opsins are known to lead to large differences in the spectral peak of absorbance (i.e. the λ max value). In our prior studies, we developed a new approach that combined homology modeling and molecular dynamics simulations to gather structural information associated with chromophore conformation, then used it to generate statistical models for the accurate prediction of λ max values for photopigments derived from Rh1 and Rh2 amino acid sequences. In the present study, we test our novel approach to predict the λ max of phylogenetically distant Sws2 cone opsins. To build a model that can predict the λ max using our approach presented in our prior studies, we selected a spectrally-diverse set of 11 teleost Sws2 photopigments for which both amino acid sequence information and experimentally measured λ max values are known. The final first-order regression model, consisting of three terms associated with chromophore conformation, was sufficient to predict the λ max of Sws2 photopigments with high accuracy. This study further highlights the breadth of our approach in reliably predicting λ max values of Sws2 cone photopigments, evolutionary-more distant from template bovine RH1, and provided mechanistic insights into the role of known spectral tuning sites.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Deep Image Prior Enabled Full Waveform Inversion (Final Technical Report)

MS Student Naveen Gupta worked on the problem of full waveform inversion (FWI) using neural networks as shown in Figure 1. Our goal was to learn a neural network to represent the subsurface velocity model, which when fed into the FWI module (implemented using a numerical forward model of wave equations) produces amplitude estimates that match with ground-truth observations of amplitude. We used neural networks to solve the inverse problem of estimating velocity distributions for a given seismic amplitude data such that, once trained, our neural network model can generate a distribution of velocity profiles for different random vectors fed as inputs to the neural network model.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1977

DSN system performance test Doppler noise models; noncoherent configuration

The newer model for variance, the Allan technique, now adopted for testing, is analyzed in the subject mode. A model is generated (including considerable contribution from the station secondary frequency standard), and rationalized with existing data. The variance model is definitely sound; the Allan technique mates theory and measure. The mean-frequency model is an estimate; this problem is yet to be rigorously resolved. The unaltered defining expressions are noncovergent, and the observed mean is quite erratic.

Bunce, R.↗
NASA NTRS1981

Equivalent source modeling of the main field using Magsat data

Magsat dipole solution models were obtained with both 32 deg and 21 deg resolution based on a data set extending over 4 months. Time dependence was modeled using first time derivatives for the dipole magnetization vector components. This doubles the n total number of parameters in the solution. The solutions displayed a very slow convergence in time derivatives although at each iteration the conversion of the dipole parameters to spherical harmonic coefficients showed close agreement with the MGST (12/80) spherical harmonic model to degree 13 in the constant terms and 8 in the secular variation terms. The program error in the option to simultaneously estimate observatory anomaly biases was found and corrected. A 32 deg resolution dipole model was generated using the selected magnetic observatory data from 1960-1977 used in the GSFC (9/80) spherical harmonic model.

Source record↗
NASA NTRS1981

Geological terrain models

The initial phase of a program to determine the best interpretation strategy and sensor configuration for a radar remote sensing system for geologic applications is discussed. In this phase, terrain modeling and radar image simulation were used to perform parametric sensitivity studies. A relatively simple computer-generated terrain model is presented, and the data base, backscatter file, and transfer function for digital image simulation are described. Sets of images are presented that simulate the results obtained with an X-band radar from an altitude of 800 km and at three different terrain-illumination angles. The simulations include power maps, slant-range images, ground-range images, and ground-range images with statistical noise incorporated. It is concluded that digital image simulation and computer modeling provide cost-effective methods for evaluating terrain variations and sensor parameter changes, for predicting results, and for defining optimum sensor parameters.

Kaupp, V. H.↗
NASA NTRS2001

The Origin of Noble Gas Isotopic Heterogeneity in Icelandic Basalts

Two models for generation of heterogeneous He, Ne and Ar isotopic ratios in Icelandic basalts are evaluated using a mixing model and the observed noble gas elemental ratios in Icelandic basalts,Ocean island Basalt (OIBs) and Mid-Ocean Ridge Basalt (MORBs). Additional information is contained in the original extended abstract.

Dixon, E. T.↗
NASA NTRS2011

Earth as an Extrasolar Planet: Earth Model Validation Using EPOXI Earth Observations

The EPOXI Discovery Mission of Opportunity reused the Deep Impact flyby spacecraft to obtain spatially and temporally resolved visible photometric and moderate resolution near-infrared (NIR) spectroscopic observations of Earth. These remote observations provide a rigorous validation of whole disk Earth model simulations used to better under- stand remotely detectable extrasolar planet characteristics. We have used these data to upgrade, correct, and validate the NASA Astrobiology Institute s Virtual Planetary Laboratory three-dimensional line-by-line, multiple-scattering spectral Earth model (Tinetti et al., 2006a,b). This comprehensive model now includes specular reflectance from the ocean and explicitly includes atmospheric effects such as Rayleigh scattering, gas absorption, and temperature structure. We have used this model to generate spatially and temporally resolved synthetic spectra and images of Earth for the dates of EPOXI observation. Model parameters were varied to yield an optimum fit to the data. We found that a minimum spatial resolution of approx.100 pixels on the visible disk, and four categories of water clouds, which were defined using observed cloud positions and optical thicknesses, were needed to yield acceptable fits. The validated model provides a simultaneous fit to the Earth s lightcurve, absolute brightness, and spectral data, with a root-mean-square error of typically less than 3% for the multiwavelength lightcurves, and residuals of approx.10% for the absolute brightness throughout the visible and NIR spectral range. We extend our validation into the mid-infrared by comparing the model to high spectral resolution observations of Earth from the Atmospheric Infrared Sounder, obtaining a fit with residuals of approx.7%, and temperature errors of less than 1K in the atmospheric window. For the purpose of understanding the observable characteristics of the distant Earth at arbitrary viewing geometry and observing cadence, our validated forward model can be used to simulate Earth s time dependent brightness and spectral properties for wavelengths from the far ultraviolet to the far infrared.brightness

Robinson, Tyler D.↗
DOE OSTI2026

PFLOTRAN modeling data and scripts associated with “Refining the Hydrogeologic Framework of a Large River Corridor Model Using Waterborne Transient Electromagnetics”

NOTE: The manuscript associated with this data package is currently in review. The data may be revised based on reviewer feedback. Upon manuscript acceptance, this data package will be updated with the final dataset and additional metadata. This data package is associated with the publication “Refining the Hydrogeologic Framework of a Large River Corridor Model Using Waterborne Transient Electromagnetics” submitted to Water Resources Research (Terry et al. 2025). The data package contains the groundwater modeling dataset from PFLOTRAN software. It includes the python script for mesh generation, boundary condition setting, PFLOTRAN input deck formation and postprocessing. It couples groundwater flow and species transport for Hanford Reach river corridor and pipelines the model generation and processing. This model can be used to easily generate the model and analysis for Hanford site. It can also be adjusted to other hydrologic area with ease. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. The data package consists of 6 folders: (1) “data” contains all necessary data as input and intermediate data for processing; (2) “mesh” contains all mesh related files to generate mesh in Hanford Reach river corridor; (3) “model_run” contains the generated script for PFLOTRAN modeling; (4) “notebooks” contains all the Python script to generate the model; (5) “output” contains all the output from the computation; (6) “postprocessing” contains the Python script to generate scientific figure for manuscript. All files are .csv (comma-separated values), .h5 (HDF5 format), .in (input files), .ipynb (Jupyter notebooks), .p (Python pickle), .png (images), .PNG (images), .py (Python scripts), .pyc (Python bytecode), .r (R scripts), .sh (shell scripts), .txt (text files), .vtu (3D mesh/visualization format), .xz (compressed archive), or .zip (compressed archive).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

TensorFlow Quantum: A Software Framework for Quantum Machine Learning

We introduce TensorFlow Quantum (TFQ), an open source library for the rapid prototyping of hybrid quantum-classical models for classical or quantum data. This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators. We provide an overview of the software architecture and building blocks through several examples and review the theory of hybrid quantum-classical neural networks. We illustrate TFQ functionalities via several basic applications including supervised learning for quantum classification, quantum control, simulating noisy quantum circuits, and quantum approximate optimization. Moreover, we demonstrate how one can apply TFQ to tackle advanced quantum learning tasks including meta-learning, layerwise learning, Hamiltonian learning, sampling thermal states, variational quantum eigensolvers, classification of quantum phase transitions, generative adversarial networks, and reinforcement learning. We hope this framework provides the necessary tools for the quantum computing and machine learning research communities to explore models of both natural and artificial quantum systems, and ultimately discover new quantum algorithms which could potentially yield a quantum advantage.

Broughton, Michael↗
NASA NTRS1986

Generation of fluctuations in cosmological models

The initial production of classical matter fluctuations in a scalar quantum field theory in curved spacetime is discussed. It is shown how quantum fluctuations generate classical matter inhomogeneities, and that the resulting spectrum is scale invariant. The relevant events that occur before the scale leaves the Hubble radius are studied.

Brandenberger, Robert H.↗