Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗
NASA NTRSDate not supplied

Deep Learning-Based Negotiation Strategy Selection for Cooperative Conflict Resolution in Urban Air Mobility

This paper presents a collaborative conflict resolution technique using deep neural network-based intelligent search of the solution space. This approach offers a rapid convergence to a mutually acceptable solution for real-time conflict resolution, suitable for urban air mobility operations. Furthermore, the presented technique allows operational flexibility to the urban air mobility agents where these agents can collaboratively devise the solution via integrative negotiation, based on their local utility functions, as long as such a solution does not violate the global safety thresholds. The presented machine-to-machine negotiation method is built on our prior work on holistic assessment of the airspace and potential conflict detection implemented at-the-edge, onboard the unmanned aircraft systems. This paper extends the prior work to augment decision-making at-the-edge, thereby, promising a true distributed control architecture for urban air mobility. In this approach, each agent (a) builds a potential in-flight conflict map, (b) identifies the conflicting agents, (c) dynamically prepares a list of alternatives based on its current utility functions, (d) negotiates with the conflicting agents to pick one of these alternatives, and (e) implements the negotiated alternative to mutually resolve the conflict. Note that such an approach does not require a contingency plan to be made pre-flight, as the conflict resolution strategies are decided and negotiated in real time based on the present state of the agent. The contingency plan, if available, can serve as an input to the real-time conflict resolution strategy formulation, and also can be used as a fallback plan in case the negotiation fails and the impacted agents need to switch to a rule-based/supervisory resolution mode from the discussed distributed resolution mode. The presented collaborative negotiation-based conflict resolution technique incorporates a time-dependent reward function to catalyze collaborative resolution by incentivizing the agents with local and global rewards beneficial to their business operations.

Advanced Air Mobility↗
DOE OSTI2022

New insights on the prevalence of drizzle in marine stratocumulus clouds based on a machine learning algorithm applied to radar Doppler spectra

Abstract. The detection of the early growth of drizzle particles in marine stratocumulus clouds is important for studying the transition from cloud water to rainwater. Radar reflectivity is commonly used to detect drizzle; however, its utility is limited to larger drizzle particles. Alternatively, radar Doppler spectrum skewness has proven to be a more sensitive quantity for the detection of drizzle embryos. Here, a machine learning (ML)-based technique that uses radar reflectivity and skewness for detecting small drizzle particles is presented. Aircraft in situ measurements are used to develop and validate the ML algorithm. The drizzle detection algorithm is applied to three Atmospheric Radiation Measurement (ARM) observational campaigns to investigate the drizzle occurrence in marine boundary layer clouds. It is found that drizzle is far more ubiquitous than previously thought; the traditional radar-reflectivity-based approach significantly underestimates the drizzle occurrence, especially in thin clouds with liquid water paths lower than 50 g m−2. Furthermore, the drizzle occurrence in marine boundary layer clouds differs among the three ARM campaigns, indicating that the drizzle formation, which is controlled by the microphysical process, is regime dependent. A complete understanding of the drizzle distribution climatology in marine stratocumulus clouds calls for more observational campaigns and continuing investigations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

In-depth analysis on parallel processing patterns for high-performance Dataframes

The Data Science domain has expanded monumentally in both research and industry communities during the past decade, predominantly owing to the Big Data revolution. Artificial Intelligence (AI) and Machine Learning (ML) are bringing more complexities to data engineering applications, which are now integrated into data processing pipelines to process terabytes of data. Typically, a significant amount of time is spent on data preprocessing in these pipelines, and hence improving its efficiency directly impacts the overall pipeline performance. The community has recently embraced the concept of Dataframes as the de-facto data structure for data representation and manipulation. However, the most widely used serial Dataframes today (R, pandas) experience performance limitations while working on even moderately large data sets. We believe that there is plenty of room for improvement by taking a look at this problem from a high-performance computing point of view. In a prior publication, we presented a set of parallel processing patterns for distributed dataframe operators and the reference runtime implementation, Cylon. In this paper, we are expanding on the initial concept by introducing a cost model for evaluating the said patterns. Furthermore, we evaluate the performance of Cylon on the ORNL Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

One-Step Ahead Prediction of Thermal Mixing Tee Sensors with Long Short Term Memory (LSTM) Neural Networks

High-temperature advanced reactors under development, such as sodium fast reactors (SFR) and molten salt cooled reactors (MSCR), are expected to offer lower levelized cost of energy (LCOE) compared to existing light water reactor (LWR’s). In the existing light water reactors (LWR’s), operation and maintenance (O&M) expenses constitute the largest fraction of the total operating cost. Some of the O&M costs are related maintenance of sensors which can fail due to exposure to harsh environment in a reactor. The O&M costs of Advanced Reactor (AR)’s are expected to constitute a significant fraction of the total cost as well, because of high temperature and radiation level in AR are likely to cause material fatigue and premature failure of sensors and components. The O&M costs in AR’s could be reduced through integration of advanced informatics of performance-related sensors into a digital twin designed for reactor monitoring. For example, machine learning (ML) could be employed for real-time validation and correction of performance-related sensors, and reducing the number of performance-related physical sensor units through virtual sensing. As part of the effort, we investigate real-time validation of thermal hydraulic sensors through one-step ahead forecasting of sensor values using long short-term memory (LSTM) recurrent neural networks (RNN). The sensors are installed in a flow loop containing a thermal mixing tee, which is a common experimental model to study thermal fatigue in a thermal hydraulic loop. In addition, nonlinear transients generated in a thermal mixing tee constitute a good challenge data set for training and validation of ML algorithms. Sensors in this study include thermocouples, flow meters, and optical fibers for distributed temperature sensing. In one experiment, measurement data sets were obtained for a loop was filled with water, and in another experiment, measurements were performed on a loop filled with liquid metal Galinstan. We have also conducted preliminary investigation of one-step ahead prediction of fiber optics-based distributed temperature sensing with LSTM networks. In predicting fiber-based temperature measurements, we treated each gauge pitch of the fiber as an independent sensor. Accuracy of one-step ahead forecasting was estimated by calculating root mean square error (RMSE) for the test segment of time series of each sensor. RMSE’s for temperature sensors in water loop were, for the most part, lower than for the same sensors in Galinstan loop. The RMSE’s for flow meters were similar for both loops. The RMSE’s for distributed temperature measured with the fiber optic sensor were similar to those of the point sensors. Results of this study demonstrated the capability of LSTM one-step ahead forecasting with RMSE comparable to uncertainty in sensor measurements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Sequence-to-sequence neural networks for short-term electrical load forecasting in commercial office buildings

The U.S. power grid is transforming to become smarter, cleaner, and more effi- cient. This is leading to the addition of significant distributed variable renew- able generation. Due to the variable nature of renewable generation, the short- and long-term supply-demand imbalances are less predictable, and conventional approaches to mitigating the imbalance will not be efficient or cost-effective. To address this challenge, transactive control technologies have been proposed which balance energy generation and consumption with market activity and in- frastructural limitations. Transactive control requires the ability of individual end-use loads to express flexibility as a function of a transactive signal (e.g., price). Empirical gray- and black-box models have been widely used to express flexibility, and although these approaches are generally easy to construct and simple to use, they do not capture the non-linear behavior that some end-use loads represent . Machine learning approaches have been proposed to address this limitation. Although deep learning approaches for forecasting end-use loads have been explored, certain aspects of the application of deep models to load forecasting are not well understood. These aspects include how much training data is required, and how models should be structured and trained. To that end, this work explores how to approach applying deep recurrent neural networks to short-term electrical load forecasting with a case study of four commercial office buildings. We identify data requirements for training accurate models of whole building electricity use conditioned on outdoor temperature, provide insight into model hyperparameter sensitivity, and demonstrate how readily models can be generalized to unseen buildings.

Skomski, Elliott↗
DOE OSTI2020

A Computational Theory for the Emergence of Grammatical Categories in Cortical Dynamics

A general agreement in psycholinguistics claims that syntax and meaning are unified precisely and very quickly during online sentence processing. Although several theories have advanced arguments regarding the neurocomputational bases of this phenomenon, we argue that these theories could potentially benefit by including neurophysiological data concerning cortical dynamics constraints in brain tissue. In addition, some theories promote the integration of complex optimization methods in neural tissue. In this paper we attempt to fill these gaps introducing a computational model inspired in the dynamics of cortical tissue. In our modeling approach, proximal afferent dendrites produce stochastic cellular activations, while distal dendritic branches–on the other hand–contribute independently to somatic depolarization by means of dendritic spikes, and finally, prediction failures produce massive firing events preventing formation of sparse distributed representations. The model presented in this paper combines semantic and coarse-grained syntactic constraints for each word in a sentence context until grammatically related word function discrimination emerges spontaneously by the sole correlation of lexical information from different sources without applying complex optimization methods. By means of support vector machine techniques, we show that the sparse activation features returned by our approach are well suited—bootstrapping from the features returned by Word Embedding mechanisms—to accomplish grammatical function classification of individual words in a sentence. In this way we develop a biologically guided computational explanation for linguistically relevant unification processes in cortex which connects psycholinguistics to neurobiological accounts of language. We also claim that the computational hypotheses established in this research could foster future work on biologically-inspired learning algorithms for natural language processing applications.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1989

Sparse distributed memory

Sparse distributed memory was proposed be Pentti Kanerva as a realizable architecture that could store large patterns and retrieve them based on partial matches with patterns representing current sensory inputs. This memory exhibits behaviors, both in theory and in experiment, that resemble those previously unapproached by machines - e.g., rapid recognition of faces or odors, discovery of new connections between seemingly unrelated ideas, continuation of a sequence of events when given a cue from the middle, knowing that one doesn't know, or getting stuck with an answer on the tip of one's tongue. These behaviors are now within reach of machines that can be incorporated into the computing systems of robots capable of seeing, talking, and manipulating. Kanerva's theory is a break with the Western rationalistic tradition, allowing a new interpretation of learning and cognition that respects biology and the mysteries of individual human beings.

Denning, Peter J.↗
DOE OSTI2025

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

This work presents a data-driven method for learning low-dimensional time-dependent physics-based surrogate models whose predictions are endowed with uncertainty estimates. We use the operator inference approach to model reduction that poses the problem of learning low-dimensional model terms as a regression of state space data and corresponding time derivatives by minimizing the residual of reduced system equations. Standard operator inference models perform well with accurate training data that are dense in time, but producing stable and accurate models when the state data are noisy and/or sparse in time remains a challenge. Another challenge is the lack of uncertainty estimation for the predictions from the operator inference models. Our approach addresses these challenges by incorporating Gaussian process surrogates into the operator inference framework to (1) probabilistically describe uncertainties in the state predictions and (2) procure analytical time derivative estimates with quantified uncertainties. The formulation leads to a generalized least-squares regression and, ultimately, reduced-order models that are described probabilistically with a closed-form expression for the posterior distribution of the operators. The resulting probabilistic surrogate model propagates uncertainties from the observed state data to reduced-order predictions. Furthermore, we demonstrate the method is effective for constructing low-dimensional models of two nonlinear partial differential equations representing a compressible flow and a nonlinear diffusion–reaction process, as well as for estimating the parameters of a low-dimensional system of nonlinear ordinary differential equations representing compartmental models in epidemiology.

Data-driven model reduction↗
DOE OSTI2022

Drugsniffer: An Open Source Workflow for Virtually Screening Billions of Molecules for Binding Affinity to Protein Targets

The SARS-CoV2 pandemic has highlighted the importance of efficient and effective methods for identification of therapeutic drugs, and in particular has laid bare the need for methods that allow exploration of the full diversity of synthesizable small molecules. While classical high-throughput screening methods may consider up to millions of molecules, virtual screening methods hold the promise of enabling appraisal of billions of candidate molecules, thus expanding the search space while concurrently reducing costs and speeding discovery. Here, we describe a new screening pipeline, called drugsniffer, that is capable of rapidly exploring drug candidates from a library of billions of molecules, and is designed to support distributed computation on cluster and cloud resources. As an example of performance, our pipeline required ~40,000 total compute hours to screen for potential drugs targeting three SARS-CoV2 proteins among a library of ~3.7 billion candidate molecules.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Recent increases in annual, seasonal, and extreme methane fluxes driven by changes in climate and vegetation in boreal and temperate wetland ecosystems

Climate warming is expected to increase global methane (CH 4 ) emissions from wetland ecosystems. Although in situ eddy covariance (EC) measurements at ecosystem scales can potentially detect CH 4 flux changes, most EC systems have only a few years of data collected, so temporal trends in CH 4 remain uncertain. Here, we use established drivers to hindcast changes in CH 4 fluxes (FCH 4 ) since the early 1980s. We trained a machine learning (ML) model on CH 4 flux measurements from 22 [methane-producing sites] in wetland, upland, and lake sites of the FLUXNET-CH 4 database with at least two full years of measurements across temperate and boreal biomes. The gradient boosting decision tree ML model then hindcasted daily FCH 4 over 1981-2018 using meteorological reanalysis data. We found that, mainly driven by rising temperature, half of the sites (n = 11) showed significant increases in annual, seasonal, and extreme FCH 4 , with increases in FCH 4 of ca. 10% or higher found in the fall from 1981–1989 to 2010–2018. The annual trends were driven by increases during summer and fall, particularly at high-CH 4 -emitting fen sites dominated by aerenchymatous plants. We also found that the distribution of days of extremely high FCH 4 (defined according to the 95th percentile of the daily FCH 4 values over a reference period) have become more frequent during the last four decades and currently account for 10–40% of the total seasonal fluxes. The share of extreme FCH 4 days in the total seasonal fluxes was greatest in winter for boreal/taiga sites and in spring for temperate sites, which highlights the increasing importance of the non-growing seasons in annual budgets. Our results shed light on the effects of climate warming on wetlands, which appears to be extending the CH 4 emission seasons and boosting extreme emissions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A kinetic-based regularization method for data science applications

We propose a physics-based regularization technique for function learning, inspired by statistical mechanics. By drawing an analogy between optimizing the parameters of an interpolator and minimizing the energy of a system, we introduce corrections that impose constraints on the lower-order moments of the data distribution. This minimizes the discrepancy between the discrete and continuum representations of the data, in turn allowing to access more favorable energy landscapes, thus improving the accuracy of the interpolator. Our approach improves performance in both interpolation and regression tasks, even in high-dimensional spaces. Unlike traditional methods, it does not require empirical parameter tuning, making it particularly effective for handling noisy data. We also show that thanks to its local nature, the method offers computational and memory efficiency advantages over Radial Basis Function interpolators, especially for large datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Predicting U 3 O 8 powder processing conditions: An AI/ML approach analyzing deep learning embeddings of SEM micrographs

High-resolution SEM images of uranium-oxide powders encode micro- and nanoscale clues to their synthesis route and calcination temperature. We trained a ResNet-50 model on 11 commercial-scale U₃O₈ classes, ammonium diuranate (ADU) or uranyl peroxide (H₂O₂) precursors calcined at temperatures ranging from 400 to 750 °C and added a 256-D projection head before the classifier to analyze the learned representation. The best of eight seeds reached 92.4 % accuracy on reserved testing data, but our focus is the structure of the embedding space rather than the accuracy and labels. We quantify class relatedness in the original 256-D space using centroid similarity and distributional distances, and we use Uniform Manifold Approximation Projection (UMAP) for visualization. ‘Unknown’ images from different preparation methods, SEM operators, and from the literature localized near the expected classes under a nearest-centroid analysis without retraining, as well as clustered in similar UMAP space. In conclusion, this embedding-centered workflow complements black-box classification by providing quantitative, similarity-based comparisons of U₃O₈ morphologies and reduces storage space by up to 98 % for image data used in millisecond vector search comparisons.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Predicting Large‐Scale Systematic Missing Pipe Attributes in Water Distribution Networks

Water distribution network (WDN) models are an essential tool used by water utilities for hydraulic analysis. Unfortunately, missing data and insufficient resources often make creating and maintaining these models unfeasible. Existing methods to address missing pipe properties, like sequential imputation for missing values and reconstruction using graph metrics, are designed to accommodate random patterns of missing information and require a significant percentage of the system's attributes to be known. However, these data completeness assumptions do not always align with real‐world scenarios where large sections of the WDN model have missing data. To address this challenge, this study proposes a data‐driven approach for estimating pipe diameter when considering different spatial patterns and degrees of data completeness (i.e., 0%–90%). Using data from 16 WDNs in Kentucky, this study compares the use of machine learning (ML) using topological and geospatial features against an existing deterministic approach. Results demonstrate that WDN models with pipe diameters predicted by the proposed ML method had comparable hydraulic performance to the ground truth models. Moreover, results showed that ML method performance varies between WDNs of differing topological classification. Insights from this study help advance the ability to leverage partial data to create and maintain WDN models amid uncertainty and inadequate resources.

Poff, Jason W. [Oregon State Univ., Corvallis, OR ↗
DOE OSTI2022

Detecting new physics as novelty — Complementarity matters

Novelty detection is a task of machine learning that aims at detecting novel events without a prior knowledge. In particular, its techniques can be applied to detect unexpected signals from new phenomena at colliders. In this paper, we develop an analysis scheme that exploits the complementarity, originally studied in ref. [1], between isolation-based and clustering-based novelty evaluators. This approach can significantly improve the performance and overall applicability of novelty detection at colliders, which we demonstrate using a variety of two dimensional Gaussian samples mimicking collider events. As a further proof of principle, we subsequently apply this scheme to the detection of two significantly different signals at the LHC featuring a $t\bar{t}γγ$ final state: $t\bar{t}h$, giving a narrow resonance in the diphoton mass spectrum, and gravity-mediated supersymmetry, resulting in broad distributions at high transverse momentum. Compared to existing dedicated searches at the LHC, the sensitivities for detecting both signals are found to be encouraging.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Atom Identification in Bilayer Moiré Materials with Gomb-Net

Moiré patterns in van der Waals bilayer materials complicate the analysis of atomic-resolution images, hindering the atomic-scale insight typically attainable with scanning transmission electron microscopy. Here, we report a method to detect the positions and identities of atoms in each of the individual layers that compose twisted bilayer heterostructures. We developed a deep learning model, Gomb-Net, which identifies the coordinates and atomic species in each layer, deconvoluting the moiré pattern. This enables layer-specific mapping of atomic positions and dopant distributions, unlike other commonly used segmentation models which struggle with moiré-induced complexity. Using this approach, we explored the Se atom substitutional site distribution in a twisted fractional Janus WS 2 -WS 2(1–x) Se 2x heterostructure and found that layer-specific implantation sites are unaffected by the moiré pattern’s local energetic or electronic modulation. In conclusion, this advancement enables atom identification within material regimes where it was not possible before, opening new insights into previously inaccessible material physics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Optimizing long-term monitoring of radiation air-dose rates after the Fukushima Daiichi Nuclear Power Plant

Radiation air dose rates near the Fukushima Daiichi Nuclear Power Plant (FDNPP) have been steadily decreasing over the past eight years since the release of radioactive elements in March 2011. Currently, the radiation monitoring program is expected to transition to long-term monitoring after most of the remediation activities are completed. The main long-term monitoring objectives are to (1) confirm the continuing reduction of contaminant and hazard levels, (2) provide assurance for the public, (3) accumulate the basic datasets for scientific knowledge and future preparation, and (4) detect changes or anomalies in contaminant mobility (if they occur), or any unexpected processes or events. In this work, we have developed a methodology for optimizing the monitoring locations of radiation air dose-rate monitoring. Our approach consists of three steps in order to determine monitoring locations in a systematic manner: (1) prioritizing the critical locations, such as schools or regulatory requirement locations, (2) diversifying locations that cover the key environmental controls that are known to influence contaminant mobility and distributions, and (3) capturing the heterogeneity of radiation air-dose rates across the domain. Therefore, for the second step, we use a Gaussian mixture model to identify the representative locations among multiple environmental variables, such as elevation and land-cover types. For the third step, we use a Gaussian process model to capture and estimate the heterogeneity of air-dose rates across the domain. Employing an integrated dose-rate map derived from Bayesian geostatistical methods as a reference map, we distribute the monitoring locations in such a way as to capture the heterogeneity of the reference map. Our results have shown that this approach allows us to select monitoring locations in a systematic manner such that the heterogeneity of air dose rates is captured by the minimal number of monitoring locations.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗