Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed computing, machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Toward a machine-guided approach to energetic material discovery

In this article, we trained a machine learning (ML) model to connect microstructural details of an energetic material formulation to its performance for the purpose of guiding the discovery of new explosive formulations. Our hypothesis was that the algorithm would robustly learn the training data and produce an accurate surrogate model. Specifically, the algorithm learned the relationship between details of the void size distribution (VSD), initiating shock pressure, and the energetic material performance. We used realistic constraints on the VSD and a range of cases were ingested by a physically informed reactive flow model working within a hydrodynamic solver running on high-performance computing resources. The ML algorithm produced a surrogate model that accurately predicted known test points around the parameter space. In addition to the utility of the model and the process used for its development, we noted interesting comparisons between what we, the authors—subject matter experts, would heuristically conclude from the training data and the surrogate model predictions. We detected nuanced details that were missed by the surrogate model; however, these details are not important to an energetic material formulator. We concluded that the algorithm did indeed robustly learn the training data and produce an accurate surrogate model. We further concluded that the surrogate model is a powerful tool to guide the formulator in the absence of subject matter experts and limited-access computing resources.

42 ENGINEERING↗

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗

Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data With Spatial Information

Efficient processing of large-scale time series data is an intricate problem in machine learning. Conventional sensor signal processing pipelines with hand engineered feature extraction often involve huge computational cost with high dimensional data. Deep recurrent neural networks have shown promise in automated feature learning for improved time-series processing. However, generic deep recurrent models grow in scale and depth with increased complexity of the data. This is particularly challenging in presence of high dimensional data with temporal and spatial characteristics. Consequently, this work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to efficiently process complex multi-dimensional time series data with spatial information. Here, the cellular recurrent architecture in the proposed model allows for location-aware synchronous processing of time series data from spatially distributed sensor signal sources. Extensive trainable parameter sharing due to cellularity in the proposed architecture ensures efficiency in the use of recurrent processing units with high-dimensional inputs. This study also investigates the versatility of the proposed DCRNN model for classification of multi-class time series data from different application domains. Consequently, the proposed DCRNN architecture is evaluated using two time-series datasets: a multichannel scalp EEG dataset for seizure detection, and a machine fault detection dataset obtained in-house. The results suggest that the proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.

60 APPLIED LIFE SCIENCES↗

Machine Learning Augmented Predictive and Generative Model for Rupture Life in Ferritic and Austenitic Steels

The Larson-Miller parameter (LMP) offers an efficient and fast scheme to estimate the creep rupture life of alloy materials for high temperature applications. However, owing to poor generalizability and dependence on the constant C, which is typically not known a-priori, estimations using the Larson-Miller parameter often result in suboptimal performance for a wide range of materials. At best it is useful in comparing alloys of similar composition. In this work, three machine learning (ML) schemes were developed for rupture life prediction for 9-12% Cr ferritic-martensitic steels and austenitic stainless steels, i.e., a hierarchical model to parameterize LMP using the LMP constant C to compute rupture life, a hierarchical model to parameterize both C and LMP to compute rupture life, and a direct prediction of rupture life. Specifically, we show that the third scheme, using a gradient boosting algorithm, can be used to train ML models for very accurate prediction of rupture life in a variety of alloys (Pear-son Correlation Coefficient > 0.9 for 9-12% Cr and > 0.8 for austenitic stainless steels). In addition, the Shapley value was used to quantify feature importance, making the model interpretable by identifying the effect of various features on the model performance. Furthermore, a variational autoencoder-based generative model was built by conditioning on the experimental dataset to sample hypothetical synthetic candidate alloys from the learnt joint distribution not existing in both 9-12 % Cr ferritic-martensitic steel and austenitic stainless steel datasets. Finally, based on the predictive and generative model, a reinforcement learning strategy has been proposed to guide experimentalists into designing better heat resistant alloys.

Mamun, Md Osman G.↗

A High-Throughput Computing Infrastructure to Generate Custom, Open Community Geothermal Datasets

The most significant challenge facing geothermal research, development, and deployment is a lack of comprehensive datasets describing the geological and economical properties of North America. Automated knowledge base construction, the process of designing algorithms to analyze text and images to programmatically build new datasets, is one possible solution to this problem. The xDD library of full-text scientific articles (https://xdd.wisc.edu) is one of the largest collections of open and controlled-access scientific documents available for knowledge base construction in the world, but it has been underutilized by experts in geothermal research. The xDD development team attributed the lack of engagement by software developers and geothermal researchers to two perceived shortcomings of the system. First, the workflow for obtaining data from xDD for local development and testing of data mining applications was unnecessarily abstruse and required significant manual intervention by xDD systems administrators. Second, although xDD already held articles from a broad cross-section of scientific literature with an emphasis on the geosciences, it did not have an explicit set of geothermal research documents that could serve as the nucleus of a geothermal data mining application. To address these issues, the Automated Data Extraction PlaTform (ADEPT) was proposed to extend the data distribution capabilities of the xDD system. The ADEPT extension added the following four key features to xDD: 1) integration of National Geothermal Data System (NGDS) documents into the xDD library to provide an explicitly geothermally-themed collection; 2) improved RESTful (i.e., https-protocol driven) web services for external partners to access xDD data for machine learning application development; 3) a web platform for end-users and xDD administrators to coordinate the development of data mining applications from the initial step of browsing available documents to the final stage of deploying a production-quality machine learning application on high-throughput computing infrastructure; and 4) the development of demonstration data mining applications to illustrate the new workflow to potential collaborators. A total of 21,674 geothermal documents from NGDS were fully ingested into the xDD library and the associated metadata is publicly available through the xDD web services; furthermore, the ADEPT web platform is now publicly accessible and fully live at https://xdd.wisc.edu/adept/.

15 GEOTHERMAL ENERGY↗

A Hybrid-Learning Algorithm for Online Dynamic State Estimation in Multimachine Power Systems

With the increasing penetration of distributed generators in the smart grids, having knowledge of rapid real-time electromechanical dynamic states has become crucial to system stability control. Conventional Supervisory Control and Data Acquisition (SCADA)-based dynamic state estimation (DSE) techniques are limited by the slow sampling rates, while the emerging phasor measurement units (PMUs) technology enables rapid real-time measurements at network nodes. Using generator bus terminal voltages, we propose a hybrid-learning DSE (HL-DSE) algorithm to estimate the synchronous machine rotor angle and speed in real time. The HL-DSE takes the power system model into account and trains neuroestimators with real-time data in an online manner. Compared with traditional DSE methods, the HL-DSE overcomes limitations by using a data-driven approach in conjunction with the physical power system model. The time efficiency, accuracy, convergence, and robustness of the proposed algorithm are tested under noises and fault conditions in both small- and large-scale test systems. Simulation results show that the proposed HL-DSE is much more computationally efficient than widely used Kalman filter (KF)-based methods while maintaining comparable accuracy and robustness. In particular, HL-DSE is over 100 times faster than square-root unscented KF (SR-UKF) and 80 times faster than extended KF (EKF). The advantages and challenges of the HL-DSE are also discussed.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Image-to-Image Wildfire Detection via Quantum-Compatible Variational Segmentation from Remotely-sensed Data

Over the last decade, the incidence of wildfires has surged, causing widespread destruction globally. To better comprehend and manage these incidents, remote sensing and aerial missions have been implemented in recent efforts. However, this has resulted in an exponential rise in the amount of remote sensing data utilization, leading to a need for intelligent automation of data extraction in wildfire studies. Machine learning provides an accurate automated approach for detecting these natural anomalies and facilitates decision-makers to take prompt actions. To make insightful decisions in wildfire management, it is imperative to move beyond simple detection and explore the potential of probabilistic generative machine learning for creating "what-if" scenarios for various wildfire conditions. Such models offer improved representation of the stochastic nature of wildfire events. However, the optimization of these models can be computationally expensive, especially when using classical computers. Quantum computers have recently emerged as a promising solution to reduce the computational cost of training such models and improve their performance. In this study, we aim to utilize quantum-compatible machine learning techniques to implement our probabilistic generative approach. To that end, we propose a supervised probabilistic variational model consisting of a U-NET-based image-to-image component along with encoder and decoder networks which work as a variational autoencoder (VAE) component. Additionally, we explore the type of latent distribution type in the VAE component and implement different means for modeling the prior distribution. We further investigate the quantum-compatible versions of the model compared to the classical counterpart and benchmark potential benefits of quantum compatibility over the classical model.

quantum machine learning↗

Satellite Embedding-Based Population Imputation for Areas with Missing Building Footprint Data: A Computer Vision-Based Approach

High-resolution population modeling is important for supporting effective decision-making across diverse sectors. LandScan Mosaic generates population estimates at the level of individual buildings and aggregates them to 3 arc-second grids, and this approach performs well in regions where building footprint data are comprehensive and reliable. However, large portions of the globe still suffer from incomplete, sparse, or entirely missing building stock datasets, creating a structural limitation for strictly building-based population models. To address this research gap, this study proposes a computer vision-based framework that employs Google Earth Engine satellite embeddings and UNet, which allows us to directly impute grid-level population estimates in building-data-deficient areas. Applied to Taiwan as a case study, the framework achieved strong predictive performance with R$^{2}$ of 0.89, RMSE of 18.70, and MAE of 8.41, outperforming traditional machine learning approaches. Notably, the proposed framework effectively addressed building false-positive errors inherent in Global Human Settlement Layer (GHSL) data, correctly identifying uninhabited areas that were erroneously classified as populated. The framework also offers significant advantages for global population mapping, particularly in terms of scalability and temporal consistency, thereby extending the coverage and accuracy of high-resolution population products in data-scarce regions worldwide. Urban planners, decision makers, and related stakeholders can obtain granular population distributions to support more accurate and targeted infrastructure investment, service delivery, resource allocation, and risk assessment decisions.

97 MATHEMATICS AND COMPUTING↗

Recurrent Convolutional Deep Neural Networks for Modeling Time-Resolved Wildfire Spread Behavior

The increasing incidence and severity of wildfires underscores the necessity of accurately predicting their behavior. While high-fidelity models derived from first principles offer physical accuracy, they are too computationally expensive for use in real-time fire response. Low-fidelity models sacrifice some physical accuracy and generalizability via the integration of empirical measurements, but enable real-time simulations for operational use in fire response. Machine learning techniques have demonstrated the ability to bridge these objectives by learning first-principles physics while achieving computational speedups. While deep learning approaches have demonstrated the ability to predict wildfire propagation over large time periods, time-resolved fire-spread predictions are needed for active fire management. Here, in this work, we evaluate the ability of deep learning approaches in accurately modeling the time-resolved dynamics of wildfires. We use an autoregressive process in which a convolutional recurrent deep learning model makes predictions that propagate a wildfire over 15 min increments. We apply the model to four simulated datasets of increasing complexity, containing both field fires with homogeneous fuel distribution as well as real-world topologies sampled from the California region of the United States. We show that even after 100 autoregressive predictions representing more than 24 h of simulated fire spread, the resulting models generate stable and realistic propagation dynamics, achieving a Jaccard score between 0.89 and 0.94 when predicting the resulting fire scar. The inference time of the deep learning models are examined and compared, and directions for future work are discussed.

54 ENVIRONMENTAL SCIENCES↗

DLHub: Simplifying publication, discovery, and use of machine learning models in science

Machine Learning (ML) has become a critical tool enabling new methods of analysis and driving deeper understanding of phenomena across scientific disciplines. There is a growing need for "learning systems" to support various phases in the ML lifecycle. While others have focused on supporting model development, training, and inference, few have focused on the unique challenges inherent in science, such as the need to publish and share models and to serve them on a range of available computing resources. In this paper, we present the Data and Learning Hub for science (DLHub), a learning system designed to support these use cases. Specifically, DLHub enables publication of models, with descriptive metadata, persistent identifiers, and flexible access control. It packages arbitrary models into portable servable containers, and enables low-latency, distributed serving of these models on heterogeneous compute resources. In this work, we show that DLHub supports low-latency model inference comparable to other model serving systems including TensorFlow Serving, SageMaker, and Clipper, and improved performance, by up to 95%, with batching and memoization enabled. We also show that DLHub can scale to concurrently serve models on 500 containers. Finally, we describe five case studies that highlight the use of DLHub for scientific applications.

97 MATHEMATICS AND COMPUTING↗

An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Wang, Tianle [Brookhaven National Laboratory (BNL)↗

Privacy-preserving federated learning: Application to behind-the-meter solar photovoltaic generation forecasting

Here, the growing usage of decentralized renewable energy sources has made accurate estimation of their aggregated generation crucial for maintaining grid flexibility and reliability. However, the majority of distributed photovoltaic (PV) systems are behind-the-meter (BTM) and invisible to utilities, leading to three challenges in obtaining an accurate forecast of their aggregated output. Firstly, traditional centralized prediction algorithms used in previous studies may not be appropriate due to privacy concerns. There is therefore a need for decentralized forecasting methods, such as federated learning (FL), to protect privacy. Secondly, there has been no comparison between localized, centralized, and decentralized forecasting methods for BTM PV production, and the trade-off between prediction accuracy and privacy has not been explored. Lastly, the computational time of data-driven prediction algorithms has not been examined. This article presents a FL power forecasting method for PVs, which uses federated learning as a decentralized collaborative modeling approach to train a single model on data from multiple BTM sites. The machine learning network used to design this FL-based BTM PV forecasting model is a multi-layered perceptron, which ensures privacy and security of the data. Comparing the suggested FL forecasting model to non-private centralized and entirely private localized models revealed that it has a high level of accuracy, with an RMSE that is 18.17% lower than localized models and 9.9% higher than centralized models.

14 SOLAR ENERGY↗

Ensemble learning-iterative training machine learning for uncertainty quantification and automated experiment in atom-resolved microscopy

Deep learning has emerged as a technique of choice for rapid feature extraction across imaging disciplines, allowing rapid conversion of the data streams to spatial or spatiotemporal arrays of features of interest. However, applications of deep learning in experimental domains are often limited by the out-of-distribution drift between the experiments, where the network trained for one set of imaging conditions becomes sub-optimal for different ones. This limitation is particularly stringent in the quest to have an automated experiment setting, where retraining or transfer learning becomes impractical due to the need for human intervention and associated latencies. Here we explore the reproducibility of deep learning for feature extraction in atom-resolved electron microscopy and introduce workflows based on ensemble learning and iterative training to greatly improve feature detection. This approach allows incorporating uncertainty quantification into the deep learning analysis and also enables rapid automated experimental workflows where retraining of the network to compensate for out-of-distribution drift due to subtle change in imaging conditions is substituted for human operator or programmatic selection of networks from the ensemble. This methodology can be further applied to machine learning workflows in other imaging areas including optical and chemical imaging.

36 MATERIALS SCIENCE↗

LTAU-FF: Loss Trajectory Analysis for Uncertainty in atomistic Force Fields

Model ensembles are effective tools for estimating prediction uncertainty in deep learning atomistic force fields. However, their widespread adoption is hindered by high computational costs and overconfident error estimates. In this work, we address these challenges by leveraging distributions of per-sample errors obtained during training and employing a distance-based similarity search in the model latent space. Our method, which we call LTAU (Loss Trajectory Analysis for Uncertainty), efficiently estimates the full probability distribution function of errors for any test point using the logged training errors, achieving speeds that are 2–3 orders of magnitudes faster than typical ensemble methods and allowing it to be used for tasks where training or evaluating multiple models would be infeasible. We apply LTAU towards estimating parametric uncertainty in atomistic force fields (LTAU-FF), demonstrating that it produces well-calibrated confidence intervals and predicts errors that correlate strongly with the true errors for data near the training domain. Furthermore, we show that the errors predicted by LTAU-FF can be used in practical applications for detecting out-of-domain data, tuning model performance, and predicting failure during simulations. We believe that LTAU will be a valuable tool for uncertainty quantification in atomistic force fields and is a promising method that should be further explored in other domains of machine learning.

97 MATHEMATICS AND COMPUTING↗

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗

Building near-real-time processing pipelines with the spark-MPI platform

Advances in detectors and computational technologies provide new opportunities for applied research and the fundamental sciences. Concurrently, dramatic increases in the three V's (Volume, Velocity, and Variety) of experimental data and the scale of computational tasks produced the demand for new real-time processing systems at experimental facilities. Recently, this demand was addressed by the Spark-MPI approach connecting the Spark data-intensive platform with the MPI high-performance framework. In contrast with existing data management and analytics systems, Spark introduced a new middleware based on resilient distributed datasets (RDDs), which decoupled various data sources from high-level processing algorithms. The RDD middleware significantly advanced the scope of data-intensive applications, spreading from SQL queries to machine learning to graph processing. Spark-MPI further extended the Spark ecosystem with the MPI applications using the Process Management Interface. The paper explores this integrated platform within the context of online ptychographic and tomographic reconstruction pipelines.

97 MATHEMATICS AND COMPUTING↗

Laboratory earthquake forecasting: A machine learning competition

Earthquake prediction, the long-sought holy grail of earthquake science, continues to confound Earth scientists. Could we make advances by crowdsourcing, drawing from the vast knowledge and creativity of the machine learning (ML) community? We used Google’s ML competition platform, Kaggle, to engage the worldwide ML community with a competition to develop and improve data analysis approaches on a forecasting problem that uses laboratory earthquake data. The competitors were tasked with predicting the time remaining before the next earthquake of successive laboratory quake events, based on only a small portion of the laboratory seismic data. The more than 4,500 participating teams created and shared more than 400 computer programs in openly accessible notebooks. Complementing the now well-known features of seismic data that map to fault criticality in the laboratory, the winning teams employed unexpected strategies based on rescaling failure times as a fraction of the seismic cycle and comparing input distribution of training and testing data. In addition to yielding scientific insights into fault processes in the laboratory and their relation with the evolution of the statistical properties of the associated seismic data, the competition serves as a pedagogical tool for teaching ML in geophysics. The approach may provide a model for other competitions in geosciences or other domains of study to help engage the ML community on problems of significance.

58 GEOSCIENCES↗