Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Maximum Mean Discrepancy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

17 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Using kernel-based statistical distance to study the dynamics of charged particle beams in particle-based simulation codes

Measures of discrepancy between probability distributions (statistical distance) are widely used in the fields of artificial intelligence and machine learning. We describe how certain measures of statistical distance can be implemented as numerical diagnostics for simulations involving charged-particle beams. Related measures of statistical dependence are also described. The resulting diagnostics provide sensitive measures of dynamical processes important for beams in nonlinear or high-intensity systems, which are otherwise difficult to characterize. Here, the focus is on kernel-based methods such as maximum mean discrepancy, which have a well-developed mathematical foundation and reasonable computational complexity. Several benchmark problems and examples involving intense beams are discussed. While the focus is on charged-particle beams, these methods may also be applied to other many-body systems such as plasmas or gravitational systems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

SDYN-GANs: Adversarial learning methods for multistep generative models for general order stochastic dynamics

We introduce adversarial learning methods for data-driven generative modeling of dynamics of nth-order stochastic systems. Our approach builds on Generative Adversarial Networks (GANs) with generative model classes based on stable m-step stochastic numerical integrators. From observations of trajectory samples, we introduce methods for learning long-time predictors and stable representations of the dynamics. Our approaches use discriminators based on Maximum Mean Discrepancy (MMD), training protocols using both conditional and marginal distributions, and methods for learning dynamic responses over different time-scales. We show how our approaches can be used for modeling physical systems to learn force-laws, damping coefficients, and noise-related parameters. Our adversarial learning approaches provide methods for obtaining stable generative models for dynamic tasks including long-time prediction and developing simulations for stochastic systems.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2026

Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy

Training machine learning models for radioisotope identification using gamma spectroscopy remains an elusive challenge for many practical applications, largely stemming from the difficulty of acquiring and labeling large, diverse experimental datasets. Simulations can mitigate this challenge, but the accuracy of models trained on simulated data can deteriorate substantially when deployed to an out-of-distribution operational environment. In this study, we demonstrate that unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain, provided unlabeled data from the target domain are available. Conventional supervised techniques are unable to utilize this data because the absence of isotope labels precludes defining a supervised classification loss. Instead, we first pretrain a spectral classifier using labeled synthetic data and subsequently leverage unlabeled target data to align the learned feature representations between the source and target domains. We compare a range of different UDA techniques, finding that minimizing the maximum mean discrepancy (MMD) between source and target feature vectors yields the most consistent improvement to testing scores. For instance, using a custom transformer-based neural network, we achieved a testing accuracy of $0.904 \pm 0.022$ on an experimental LaBr test set after performing unsupervised feature alignment via MMD minimization, compared to $0.754 \pm 0.014$ before alignment. Overall, our results highlight the potential of using UDA to adapt a radioisotope classifier trained on synthetic data for real-world deployment.

Lalor, Peter W.↗
DOE OSTI2023

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the order of 10\textsuperscript{5}, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but the alternative of training only on simulated data results in poor performance on real data. Domain Adaptation may be able to bridge the gap between simulated and real datasets. We utilize domain adaptation for the estimation of Einstein radius ($\Theta_E$) in simulated galaxy-scale gravitational lensing images with different levels of observational realism. We evaluate two domain adaptation techniques - Domain Adversarial Neural Networks (DANN) and Maximum Mean Discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain of lenses simulated to emulate noise conditions in the Dark Energy Survey (DES). We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain dataset. This work is the first application of domain adaptation for a regression task in strong lensing imaging analysis. Our results show the potential of using domain adaptation to perform analysis of future survey data with a deep neural network trained on simulated data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Domain Adaptive Graph Neural Networks for Constraining Cosmological Parameters Across Multiple Data Sets

Deep learning models have been shown to outperform methods that rely on summary statistics, like the power spectrum, in extracting information from complex cosmological data sets. However, due to differences in the subgrid physics implementation and numerical approximations across different simulation suites, models trained on data from one cosmological simulation show a drop in performance when tested on another. Similarly, models trained on any of the simulations would also likely experience a drop in performance when applied to observational data. Training on data from two different suites of the CAMELS hydrodynamic cosmological simulations, we examine the generalization capabilities of Domain Adaptive Graph Neural Networks (DA-GNNs). By utilizing GNNs, we capitalize on their capacity to capture structured scale-free cosmological information from galaxy distributions. Moreover, by including unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), we enable our models to extract domain-invariant features. We demonstrate that DA-GNN achieves higher accuracy and robustness on cross-dataset tasks. Using data visualizations, we show the effects of domain adaptation on proper latent space data alignment. This shows that DA-GNNs are a promising method for extracting domain-independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Simulation Based Inference with Domain Adaptation for Strong Gravitational Lensing

Simulation based inference leverages machine learning to carry out Bayesian inference for systems with intractable likelihoods. However, transitioning a network trained on simulated data to real data runs the risk of encountering domain shift, leading to performance losses. We attempt to implement domain adaptation into the sbi neural posterior estimation framework using the Maximum Mean Discrepancy as an additional network loss. We test two network architectures and use masked autoregressive flow for density estimation. We test the network on a set of 400,000 simulated strong gravitational lensing images generated using deeplenstronomy. The source domain is defined as low noise whereas the target domain has a noise profile sampled from experimentally derived DES survey conditions. We find that, while DA does lead to performance improvements, they are marginal at ~6% less inference error. We also find a similar marginal improvement in uncertainty calibration at around 8%.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Bridging the Gap between Cosmological Simulations with Graph Neural Networks and Domain Adaptation

Deep learning models have been shown to outperform methods that rely on summary statistics, like the power spectrum, in extracting information from complex cosmological data sets. However, due to differences in the subgrid physics implementation and numerical approximations across different simulation suites, models trained on data from one cosmological simulation show a drop in performance when tested on another. Similarly, models trained on any of the simulations would also likely experience a drop in performance when applied to observational data. Training on data from two different suites of the CAMELS hydrodynamic cosmological simulations, we examine the generalization capabilities of Domain Adaptive Graph Neural Networks (DA-GNNs). By utilizing GNNs, we capitalize on their capacity to capture structured scale-free cosmological information from galaxy distributions. Moreover, by including unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), we enable our models to extract domain-invariant features. We demonstrate that DA-GNN achieves higher accuracy and robustness on cross dataset tasks (up to 28% better relative error and up to almost an order of magnitude better χ 2 ). Using data visualizations, we show the effects of domain adaptation on proper latent space data alignment. This shows that DA-GNNs are a promising method for extracting domain-independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Aerodynamic Sensitivities over Separable Shape Tensors

Here, we present a comprehensive aerodynamic sensitivity analysis of airfoil parameterization informed by separable shape tensors. This parameterization approach uniquely benefits the design process by isolating various well-studied shape characteristics, such as airfoil thickness, and providing a well-regulated low-dimensional parameter domain for aerodynamic designs. Exploring the aerodynamic sensitivities of this novel parameterization can provide valuable insights for more robust designs and future manufacturing efforts. We construct a data-driven parameter space of airfoils using principal geodesic analysis of separable shape tensors informed by a curated database containing almost 20,000 suitable engineering airfoils. Analyzing the shape reconstruction error and the maximum mean discrepancy between joint distributions of aerodynamic quantities, we study the dimensionality of the learned parameter space. This simple numerical experiment demonstrates a dramatic dimension reduction that retains design effectiveness and promotes regularity of the shape representations. Finally, we generate new airfoils and use the HAM2D Reynolds-averaged Navier–Stokes solver to predict lift, drag, and moment coefficients. We compute multiple sensitivity metrics to quantify and assert the consistency of parameter influence on the aerodynamic quantities. We also explore low-dimensional polynomial ridge approximations to motivate physical intuitions and offer explanations of the approximated sensitivities.

17 WIND ENERGY↗
DOE OSTI2022

Machine Learning-based Prediction of Departure from Nucleate Boiling Power for the PSBT Benchmark

Machine Learning (ML) has seen an exponential growth in its applications due to its advanced data driven prediction capabilities. The study presents a data-driven approach as a preliminary attempt to predict the power at which departure from nucleate boiling (DNB) occurs in pressurized water reactors (PWRs) by constructing an advanced ML algorithm that takes outlet pressure, inlet temperature and inlet mass flux as the input features. DNB is a critical heat flux (CHF) phenomenon seen in PWRs. The experimental data from the PWR subchannel and bundle tests (PSBT) benchmark is first used to train an artificial neural network (ANN) to predict the DNB power, which produces a root mean square error (RMSE) of 6.89 kW/m when tested on a blind subset of the PSBT data. Since the PSBT dataset is relatively small to train an accurate ANN, a data augmentation methodology based on generative adversarial networks (GANs) is used to expand the training dataset. By assuming that the real data follows a certain distribution, GANs try to learn that underlying distribution to generate similar synthetic data to augment the database and to improve the predictive capabilities of the ANN. The data generated from GANs are validated using 1-nearest neighbor and kernel maximum mean discrepancy. To further ensure data from GAN is similar to PSBT, the data is tested and filtered out using the sub-channel thermal-hydraulic code CTF. The results indicate that with the addition of 120 data points from GAN the RMSE reduces to 4.84 kW/m showing promising results for future developments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Domain Adaptive Graph Neural Networks for Constraining Cosmological Parameters Across Multiple Data Sets

Deep learning models have been shown to outperform methods that rely on summary statistics, like the power spectrum, in extracting information from complex cosmological data sets. However, due to differences in the subgrid physics implementation and numerical approximations across different simulation suites, models trained on data from one cosmological simulation show a drop in performance when tested on another. Similarly, models trained on any of the simulations would also likely experience a drop in performance when applied to observational data. Training on data from two different suites of the CAMELS hydrodynamic cosmological simulations, we examine the generalization capabilities of Domain Adaptive Graph Neural Networks (DA-GNNs). By utilizing GNNs, we capitalize on their capacity to capture structured scale-free cosmological information from galaxy distributions. Moreover, by including unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), we enable our models to extract domain-invariant features. We demonstrate that DA-GNN achieves higher accuracy and robustness on cross-dataset tasks (up to $28\%$ better relative error and up to almost an order of magnitude better $\chi^2$). Using data visualizations, we show the effects of domain adaptation on proper latent space data alignment. This shows that DA-GNNs are a promising method for extracting domain-independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the order of $10^5$, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but the alternative of training only on simulated data results in poor performance on real data. Domain Adaptation may be able to bridge the gap between simulated and real datasets. We utilize domain adaptation for the estimation of Einstein radius ($\Theta_E$) in simulated galaxy-scale gravitational lensing images with different levels of observational realism. We evaluate two domain adaptation techniques - Domain Adversarial Neural Networks (DANN) and Maximum Mean Discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain of lenses simulated to emulate noise conditions in the Dark Energy Survey (DES). We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain dataset. This work is the first application of domain adaptation for a regression task in strong lensing imaging analysis. Our results show the potential of using domain adaptation to perform analysis of future survey data with a deep neural network trained on simulated data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Simulation based inference with domain adaptation for strong gravitational lensing

Simulation based inference leverages machine learning to carry out Bayesian inference in systems with intractable likelihoods. However, transitioning a network trained on simulated data to real data runs the risk of encountering domain shift, leading to performance losses. We attempt to implement domain adaptation into the sbi neural posterior estimation framework using the Maximum Mean Discrepancy as an additional network loss, using masked autoregressive fow (MAF) as our density estimator. We test the network on a set of 400,000 simulated strong gravitational lensing images generated using deeplenstronomy. The source domain is defined as low noise whereas the target domain has a noise profile sampled from experimentally derived DES survey conditions. We find that SBI appears robust against small changes in the data with similar performance on source and target. Moreover, while DA does lead to performance improvements, they are marginal at 6% less inference error.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Domain Adaptive Graph Neural Networks for Constraining Cosmological Parameters Across Multiple Data Sets

State of the art astronomical simulations have provided datasets which enabled the training of novel deep learning techniques for constraining cosmological parameters. However, differences in subgrid physics implementation and numerical approximations among simulation suites lead to differences in simulated datasets, which pose a hard challenge when trying to generalize across diverse data domains and ultimately when applying models to observational data. Recent work reveals deep learning algorithms are able to extract more information from complex cosmological simulations than summary statistics like power spectra. We introduce Domain Adaptive Graph Neural Networks (DA-GNNs), trained on CAMELS data, inspired by CosmoGraphNet (Villanueva-Domingo et al 2023). By utilizing GNNs, we can capitalize on their capacity to capture both astrophysical and topological features of galaxy distributions. Mixing these capabilities with domain adaptation techniques such as Maximum Mean Discrepancy (MMD), which enable extraction of domain-invariant features, our framework demonstrates enhanced accuracy and robustness. We present experimental results, including the alignment of distributions across domains through data visualization. These findings suggest that DA-GNNs are an efficient way of extracting domain independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the magnitude of 105, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but training only on simulated data results in poor performance on real data. Domain adaptation can bridge the gap between simulated and real datasets. We adopt domain adaptation on the estimation of Einstein radius in simulated galaxy-scale gravitational lensing images. We evaluate two domain adaptation techniques - domain adversarial neural networks (DANN) and maximum mean discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain with emulation of DES survey conditions. We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain datasets. Our results show the potential of using domain adaptation to perform analysis on future survey data with a deep neural network trained on simulated data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Recalibration of missing low-frequency variability and trends in the North Atlantic Oscillation

Abstract Multi-decadal trends in the wintertime North Atlantic Oscillation (NAO) are under-represented by coupled general circulation models (CGCMs), consistent with a lack of autocorrelation in their NAO index series. This study proposes and tests two simple “reddening” approaches for correcting this problem in simulated indices based on simple one parameter short-term (AR; Auto-Regressive order 1) and long-term (FD; Fractional-Difference) time series filters. Using CGCMs from the Coupled Model Intercomparison Project Phase 6 (CMIP6), the FD filter successfully improves the autocorrelation structure of the NAO, and in turn the simulation of extreme trends, while the AR filter is less successful. The 1963–1993 NAO trend is the maximum 31-year trend in the historical period. Raw CGCMs underestimate the likelihood of this trend by a factor of ten but this discrepancy is corrected after reddening. CMIP6 future projections show that long-term (2024–2094) NAO ensemble mean trends systematically increase with the magnitude of radiative forcing: -2.4 to 3.5 hPa/century for low-to-high forcing after reddening (more than double the range using raw output). The related likelihood of future maximum 31year trends comparable to 1963–1993 ranges from 3 to 7% whereas none of these CMIP6 projections simulate this without reddening. Near-term projections of the next 31 years (2024–2054) are less sensitive than long term trends to the future scenario, showing weak-to-no forced trend. However, reddening increases the ensemble range by 74% (to +/-1 standard deviation/decade), which could increase/decrease regional climate change signals in the Northern Hemisphere by magnitudes that are underestimated when using raw CGCM output.

Meteorology & Atmospheric Sciences↗
DOE OSTI2025

Preliminary Kinetic Analysis of Non-Equilibrium Plasma- Assisted Methanol Pyrolysis and Oxidation Experiments

Efforts to enhance power generation efficiency and reduce emissions have driven interest in novel combustion techniques, including non-equilibrium plasma (NEP) ignitors. NEP ignitors show promise in improving energy conversion efficiency, fuel reforming, emission control, and lean-flammability limits. However, their adoption is hindered by a limited understanding of the interplay between plasma-enhanced combustion and thermal chemistry, particularly for complex fuels under engine-relevant conditions. Developing experimentally validated kinetic mechanisms is therefore critical. Additionally, the increasing interest in renewable biofuels like ethanol and methanol, coupled with the desirable qualities of NEP ignitors, presents a compelling opportunity for study. Therefore, this work acts as an extension of a previous work (Bopaiah et al., 2023) pertaining to the experimental results of NEP-assisted methanol pyrolysis and oxidation. Experiments were performed with a custom-built plasma flow reactor at 0.5 atm and temperatures from 523-1203 K. All reactive mixtures are extremely diluted to minimize exothermicity due to reactivity, allowing isothermal assumptions and the isolation of plasma chemistry from thermal chemistry. A dielectric barrier discharge plasma, at 14 kV and 15 ns full-width half maximum, was applied to the reactive mixture at varying frequencies to maintain the number of pulses with increasing temperature. Steady-state product speciation was performed downstream of the reactor with ex-situ GC/MS diagnostics. The attained experimental results were examined through in-depth analysis performed by means of an in-development plasma-coupled kinetic mechanism. As discussed in the previous work, the plasma significantly accelerates methanol pyrolysis, increasing stable intermediate production, including oxygenated and nitrile species. Plasma-assisted oxidation shows even faster fuel consumption compared to pyrolysis and a 200 K ignition shift compared to thermal oxidation. For plasma-assisted pyrolysis, the model demonstrates that accelerated fuel consumption stems from dissociative quenching of excited N2 states with fuel and H2, generating H radicals that react to rapidly form CH3 and CH2OH radicals. At low temperatures, these radicals recombine to produce oxygenates, while CH3 drives nitrile and hydrocarbon formation at higher temperatures. While the model captures pyrolysis trends well, discrepancies in methane, ethylene, and ethanol predictions are present. Similarly, the model faces challenges in accurately representing plasma-assisted oxidation, predicting a much steeper fuel gradient and ignition 100 K earlier than the experiment. While a similar scheme to pyrolysis is nested in the reaction pathway, the enhancement of the O and H radical fluxes and their initiation of the OH and HO2 radical pools dominate fuel and intermediate oxidation. The overestimation of these processes is shown to be responsible for the divergence of model from experiment. While the modelling predictions of this preliminary mechanism are not perfect, they serve as a valuable starting point. Primarily, they elicited new reaction pathways that are not otherwise possible in thermal chemistry induced reaction kinetics. The results also provide a basis for the future work that should be performed. For example, theoretical and experimental studies on excited nitrogen species and fuel/fuel radical interactions, quantification of the NOx production, and the kinetics behind the slow ignition observed in oxidation should be emphasized.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗