Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Designing a parallel Feel-the-Way clustering algorithm on HPC systems

This paper introduces a new parallel clustering algorithm, named Feel-the-Way clustering algorithm, that provides better or equivalent convergence rate than the traditional clustering methods by optimizing the synchronization and communication costs. Our algorithm design centers on how to optimize three factors simultaneously: reduced synchronizations, improved convergence rate, and retained same or comparable optimization cost. To compare the optimization cost, we use the Sum of Square Error (SSE) cost as the metric, which is the sum of the square distance between each data point and its assigned clusters. Compared with the traditional MPI k-means algorithm, the new Feel-the-Way algorithm requires less communications among participating processes. As for the convergence rate, the new algorithm requires fewer number of iterations to converge. As for the optimization cost, it obtains the SSE costs that are close to the k-means algorithm. In the paper, we first design the full-step Feel-the-Way k-means clustering algorithm that can significantly reduce the number of iterations that are required by the original k-means clustering method. Next, we improve the performance of the full-step algorithm by adopting an optimized sampling-based approach, named reassignment-history-aware sampling. Our experimental results show that the optimized sampling-based Feel-the-Way method is significantly faster than the widely used k-means clustering method, and can provide comparable optimization costs. More extensive experiments with several synthetic datasets and real-world datasets (e.g., MNIST, CIFAR-10, ENRON, and PLACES-2) show that the new parallel algorithm can outperform the open source MPI k-means library by up to 110% on a high-performance computing system using 4,096 CPU cores. In addition, the new algorithm can take up to 51% fewer iterations to converge than the k-means clustering algorithm.

97 MATHEMATICS AND COMPUTING↗

A cross-dimensional analysis of data-driven short-term load forecasting methods with large-scale smart meter data

Electricity load forecasting is essential to utility operation and power grid stability. A wide spectrum of data-driven methods, ranging from linear regression models to more recent deep learning models have been adopted to forecast electric load over the years. However, there still lacks a holistic evaluation of the applicability of conventional statistical and machine learning based algorithms with respect to different temporal and spatial scopes, computational requirements, and sensitivity of model-tuning. Enabled by a large-scale electricity load profile dataset of over 40,000 residential customers in a utility region, we conducted a cross-dimensional analysis of data-driven load forecasting methods. Three regression-based and seven deep learning algorithms with different model configurations were evaluated in terms of their overall and peak load prediction accuracy, and training burdens, across spatial aggregation levels ranging from the transformer, feeder, substation, to neighborhood. We found, first, the load forecasting accuracy is constrained by a predictability boundary, influenced by the forecasting horizon and spatial aggregation level. Specifically, RandomForest, XGBoost, TFT, TSMixer, and TiDE models achieved less than 10 % prediction error for up to 96-h ahead forecasting for district, substation, and feeder levels, while other models struggle at long-horizon predictions; Second, for winter and summer peak load dates, most models were able to predict the peak demand timing within ± 1 h, but the prediction percentage error varied by models, with TFT and TiDE models being the top performers; Third, models with similar prediction accuracy can differ in training burden by an order of magnitude. Therefore, choosing model configurations that balance prediction performance and computational resource is an important practical consideration for large-scale deployment of the machine learning based load forecasting. The outcome of this study can guide researchers and practitioners to choose the proper load forecasting algorithms based on their problem scope, required accuracy, and available resources. The predictability boundary can serve as a benchmark for electricity load forecasting problems with new algorithms and datasets.

Li, Han↗

Heuristic Evaluation Methods Applied to a Predictive Maintenance Chatbot

The need for an accessible iterative approach for evaluating prospective artificial intelligence (AI)/ML based technologies in the nuclear industry is needed, given the nature of algorithms and rapid advancements. This paper explores existing heuristic design principles for user-centered design and evaluates them based on their relevancy and usefulness for evaluating AI/ ML based technologies. Researchers at the Idaho National Laboratory (INL) have developed a machine learning software application called VIsualization for PrEdictive maintenance Recommendation (VIPER), which is used to help users understand and engage with the tool to learn more about work orders, data used, predictive maintenance, and machine learning (ML) algorithms. Early user research studies used to access VIPER’s technology readiness level have occurred; however, there is room for further improvement of the software through heuristic evaluations along with other methods and user testing. This work describes the applicability of heuristic evaluation methods and cognitive walkthroughs to help ensure human readiness for prospective AI/ ML based applications, using VIPER as a candidate use case. This work supports industry in ensuring that prospective AI/ML based technologies are usable and useful for plant personnel at nuclear power plants, ultimately leading to their safe, reliable, and efficient use.

99 - GENERAL AND MISCELLANEOUS↗

HIGH-LOW FIDELITY THERMAL HYDRAULIC COUPLING USING AI/MACHINE LEARNING ALGORITHMS

The primary goal of the US Department of Energy (DOE) office of Nuclear Energy Integrated Energy Systems (IES) program is to develop the tools and framework for coupling multi-scale and multi-physical thermal and electrical energy usage and storage systems. High- and low-fidelity (high–low) coupling is a key feature of multi-scale, multi-component systems and has been an important focus of research in the nuclear energy community for the past two decades. An essential feature of demonstrating the capability to couple high-fidelity and low-fidelity systems for real-time applications are surrogate/reduced order models (ROM). For the purposes of this study, surrogate models are essentially Blackbox models, typically developed using supervised Machine learning (ML) algorithms. The surrogate models can be used to mimic the response of high-fidelity models to represent large historical datasets and coupled with more general low-fidelity system models distributed as Functional Mock-up Interface (FMI) or Functional Mock-up Units (FMU) modules. The example is demonstrated with Spallation Neutron Source (SNS) First Target Station flow loop data. The flow loop is a liquid mercury loop with a pump, piping, heat exchange, and internal heat generation in the target window. This work elucidates some of the potential benefits and future needs of developing tools for high–low system coupling of energy systems.

Williams, Wesley↗

Anomaly Detection and Mitigation in FACTS-based Wide-Area Voltage Control Systems using Machine Learning

With the increasing deployment of Flexible AC Transmission System (FACTS) devices in wide-area voltage control systems (WAVCS) for achieving improved voltage stability of bulk power systems, the possibility for cyber attacks on these systems is also increasing. Successful stealthy cyber attacks that are difficult to detect by traditional informational technology (IT)-based cybersecurity solutions or threshold-based bad data detectors can lead to a voltage collapse in power grid. This paper presents the testbed-based attacks implementation and real-time evaluation of machine learning (ML) algorithm for detecting and mitigating stealthy cyber attacks on FACTS-based WAVCS on a hardware-in-the-loop (HIL) testbed. Initially, we discuss the implementation of a fuzzy logic controller (FLC) that controls a Static VAR Compensator (SVC) device deployed in a two-area four-machine Kundur power system for improving transient voltage stability. Later, the ML-based Anomaly Detection and Mitigation (ADM) system is implemented on the cyber-physical HIL testbed to detect and mitigate various stealthy cyber attacks, which are injected in real-time over the wide-area network (WAN). The experimental results show accurate and effective performance of ADM system in detecting and mitigating anomalies while keeping the grid stable and within the system operating limits, as defined by the North America Electric Reliability Corporation (NERC).

24 POWER TRANSMISSION AND DISTRIBUTION↗

Multi-Kernel Support Vector Machine based Predictive Maintenance of Circulating Water Pumps in Nuclear Power Plants

Multi-Kernel Support Vector Machine (MK-SVM) is a machine learning classification algorithm that can assist in the development of predictive maintenance strategies for nuclear power plant systems. Predictive maintenance can alleviate maintenance costs and enhance reliability of plant systems. In this work, MK-SVM is utilized for determining the health of the circulating water system (CWS) in a nuclear power plant.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Enhanced Surgical Decision-Making Tools in Breast Cancer: Predicting 2-Year Postoperative Physical, Sexual, and Psychosocial Well-Being following Mastectomy and Breast Reconstruction (INSPiRED 004)

Background: We sought to predict clinically meaningful changes in physical, sexual, and psychosocial well-being for women undergoing cancer-related mastectomy and breast reconstruction 2 years after surgery using machine learning (ML) algorithms trained on clinical and patient-reported outcomes data. Patients and Methods: We used data from women undergoing mastectomy and reconstruction at 11 study sites in North America to develop three distinct ML models. We used data of ten sites to predict clinically meaningful improvement or worsening by comparing pre-surgical scores with 2 year follow-up data measured by validated Breast-Q domains. We employed ten-fold cross-validation to train and test the algorithms, and then externally validated them using the 11th site’s data. We considered area-under-the-receiver-operating-characteristics-curve (AUC) as the primary metric to evaluate performance. Results: Overall, between 1454 and 1538 patients completed 2 year follow-up with data for physical, sexual, and psychosocial well-being. In the hold-out validation set, our ML algorithms were able to predict clinically significant changes in physical well-being (chest and upper body) (worsened: AUC range 0.69–0.70; improved: AUC range 0.81–0.82), sexual well-being (worsened: AUC range 0.76–0.77; improved: AUC range 0.74–0.76), and psychosocial well-being (worsened: AUC range 0.64–0.66; improved: AUC range 0.66–0.66). Baseline patient-reported outcome (PRO) variables showed the largest influence on model predictions. Conclusions: Machine learning can predict long-term individual PROs of patients undergoing postmastectomy breast reconstruction with acceptable accuracy. This may better help patients and clinicians make informed decisions regarding expected long-term effect of treatment, facilitate patient-centered care, and ultimately improve postoperative health-related quality of life.

60 APPLIED LIFE SCIENCES↗

From Machine Learning to Nuclear Digital Twins

This presentation provides multiple case studies of Machine Learning, Uncertainty Quantification, Reduced Order Modeling, to achieve digital transformation and provide the components for nuclear digital twinning. The presentation is for an international virtual event titled: Consultancy Meeting on Applications of AI and Pattern Recognition Techniques for Uncertainty Quantification in Nuclear Power Modelling and Simulation held Oct 21-22, 2021.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

An Efficient Bayesian Approach to Learning Droplet Collision Kernels: Proof of Concept Using “Cloudy,” a New n -Moment Bulk Microphysics Scheme

The small-scale microphysical processes governing the formation of precipitation particles cannot be resolved explicitly by cloud resolving and climate models. Instead, they are represented by microphysics schemes that are based on a combination of theoretical knowledge, statistical assumptions, and fitting to data (“tuning”). Historically, tuning was done in an ad hoc fashion, leading to parameter choices that are not explainable or repeatable. Recent work has treated it as an inverse problem that can be solved by Bayesian inference. The posterior distribution of the parameters given the data—the solution of Bayesian inference—is found through computationally expensive sampling methods, which require over $\mathcal{O}$(10 5 ) evaluations of the forward model; this is prohibitive for many models. We present a proof of concept of Bayesian learning applied to a new bulk microphysics scheme named “Cloudy,” using the recently developed Calibrate-Emulate-Sample (CES) algorithm. Cloudy models collision-coalescence and collisional breakup of cloud droplets with an adjustable number of prognostic moments and with easily modifiable assumptions for the cloud droplet mass distribution and the collision kernel. The CES algorithm uses machine learning tools to accelerate Bayesian inference by reducing the number of forward evaluations needed to $\mathcal{O}$(10 2 ). It also exhibits a smoothing effect when forward evaluations are polluted by noise. In a suite of perfect-model experiments, we show that CES enables computationally efficient Bayesian inference of parameters in Cloudy from noisy observations of moments of the droplet mass distribution. In an additional imperfect-model experiment, a collision kernel parameter is successfully learned from output generated by a Lagrangian particle-based microphysics model.

54 ENVIRONMENTAL SCIENCES↗

Prediction of gas hydrate saturation using machine learning and optimal set of well-logs

We report resistivity and acoustic logs are widely used to estimate gas hydrate saturation in various sedimentary systems using one of the two popular methods ((1) acoustic velocity and (2) electrical resistivity), but the limitations of these two methods are often overlooked, which include (i) well-specific calibration of empirical exponents in the electrical resistivity method, (ii) assumption of known pore morphology for gas hydrates in the acoustic velocity method, and (iii) presence of unknown mineralogy and bulk modulus terms in the acoustic velocity method. NMR-density porosity-derived gas hydrate saturation based on the analysis of the transverse magnetization relaxation time (T2) is considered the most precise method, but acquisition of NMR-based logs is limited at relatively recent drilled sites; additionally, its use in conventional oil and gas reservoirs is not that common due to higher cost and operational deployment limitations associated with acquiring NMR well-logs. This study proposes a new method that predicts gas hydrate saturation (S h ) for any well using porosity, bulk density, and compressional wave (P wave) velocity well-logs with neural network (or stochastic gradient descent regression) without any well-specific calibration and/or other aforementioned shortcomings of the existing methods. The method is developed by examining the underlying dependency between S h and different combinations of well-logs, chosen from 6 routine logs, with 12 different machine learning (ML) algorithms. The accuracy of the proposed method in predicting S h is ~ 84%, which is better than the accuracy of seismic and electrical resistivity methods (≤ 75%) per the results reported by three different studies. The robustness of the method in the specific case of permafrost-associated gas hydrates is demonstrated with well-log data from two wells drilled on the Alaska North Slope.

58 GEOSCIENCES↗

Predicting boron coordination in multicomponent borate and borosilicate glasses using analytical models and machine learning

Accurate prediction of boron coordination in multicomponent glasses is critical in glass science and technology as it strongly affects the properties of borate and borosilicate glasses. We have collected a dataset containing 657 glasses from literature with boron coordination values and developed models using analytical functions based on the well accepted Dell, Xiao and Bray model. Good prediction of boron coordination with a R 2 value higher than 0.8 was obtained. The large variation of boron coordination from experiments, originated from sample preparations and characterizations, led to difficulties in obtaining models with better prediction performance. Various machine learning (ML) algorithms were evaluated and slightly better prediction performance was observed; however, interpretation of the ML models is less straight forward. In conclusion, this study developed various models capable of providing quantitative boron coordination predictions, providing insights into its structural roles in multi-component glasses, and suggesting fruitful areas for future research.

36 MATERIALS SCIENCE↗

Emerging materials intelligence ecosystems propelled by machine learning

We report that the age of cognitive computing and artificial intelligence (AI) is just dawning. Inspired by its successes and promises, several AI ecosystems are blossoming, many of them within the domain of materials science and engineering. These materials intelligence ecosystems are being shaped by several independent developments. Machine learning (ML) algorithms and extant materials data are utilized to create surrogate models of materials properties and performance predictions. Materials data repositories, which fuel such surrogate model development, are mushrooming. Automated data and knowledge capture from the literature (to populate data repositories) using natural language processing approaches is being explored. The design of materials that meet target property requirements and of synthesis steps to create target materials appear to be within reach, either by closed-loop active-learning strategies or by inverting the prediction pipeline using advanced generative algorithms. AI and ML concepts are also transforming the computational and physical laboratory infrastructural landscapes used to create materials data in the first place. Surrogate models that can outstrip physics-based simulations (on which they are trained) by several orders of magnitude in speed while preserving accuracy are being actively developed. Automation, autonomy and guided high-throughput techniques are imparting enormous efficiencies and eliminating redundancies in materials synthesis and characterization. The integration of the various parts of the burgeoning ML landscape may lead to materials-savvy digital assistants and to a human-machine partnership that could enable dramatic efficiencies, accelerated discoveries and increased productivity. Here, we review these emergent materials intelligence ecosystems and discuss the imminent challenges and opportunities. The materials research landscape is being transformed by the infusion of approaches based on machine learning. This Review discusses the emerging materials intelligence ecosystems and the potential of human-machine partnerships for fast and efficient virtual materials screening, development and discovery.

36 MATERIALS SCIENCE↗

Machine Learning Driven Sensitivity Analysis of E3SM Land Model Parameters for Wetland Methane Emissions

Methane (CH 4 ) is globally the second most critical greenhouse gas after carbon dioxide, contributing to 16%–25% of the observed atmospheric warming. Wetlands are the primary natural source of methane emissions globally. However, wetland methane emission estimates from biogeochemistry models contain considerable uncertainty. One of the main sources of this uncertainty arises from the numerous uncertain model parameters within various physical, biological, and chemical processes that influence methane production, oxidation, and transport. Sensitivity Analysis (SA) can help identify critical parameters for methane emission and achieve reduced biases and uncertainties in future projections. This study performs SA for 19 selected parameters responsible for critical biogeochemical processes in the methane module of the Energy Exascale Earth System Model (E3SM) land model (ELM). The impact of these parameters on various CH 4 fluxes is examined at 14 FLUXNET- CH 4 sites with diverse vegetation types. Given the extensive number of model simulations needed for global variance-based SA, we employ a machine learning (ML) algorithm to emulate the complex behavior of ELM methane biogeochemistry. We found that parameters linked to CH 4 production and diffusion generally present the highest sensitivities despite apparent seasonal variation. Comparing simulated emissions from perturbed parameter sets against FLUXNET-CH 4 observations revealed that better performances can be achieved at each site compared to the default parameter values. This presents a scope for further improving simulated emissions using parameter calibration with advanced optimization techniques.

54 ENVIRONMENTAL SCIENCES↗

Towards Automatic and Agile AI/ML Accelerator Design with End-to-End Synthesis

Domain-specific designs offer greater energy efficiency and performance gain than general-purpose processors. For this reason, modern system-on-chips have a significant portion of their silicon area with custom accelerators. However, designing hardware by hand is laborious and time-consuming, given the large design space and the performance, power, and area constraints that are not realized in the software. Moreover, domain-specific algorithms (e.g., machine learning models) are evolving quickly, challenging the accelerator design further. To address these issues, this paper presents SODA Synthesizer, an automated open-source high-level ML framework to Verilog modular compiler targeting AI/ML Application-Specific Integrated Circuits (ASICs) accelerators. SODA tightly couples the Multi- Level Intermediate Representation (MLIR) compiler infrastructure [24] and open-source HLS approaches. Thus, SODA can support various ML frameworks and algorithms and can perform optimizations that combine specialized architecture templates and conventional HLS to generate the hardware modules. In addition, SODA’s closed-loop design space exploration (DSE) engine allows developers to perform end-to-end design space explorations on different metrics and technology nodes.

Zhang, Jeff↗

Machine Learning-Enabled Quantitative Analysis of Optically Obscure Scratches on Nickel-Plated Additively Manufactured (AM) Samples

Additively manufactured metal components often have rough and uneven surfaces, necessitating post-processing and surface polishing. Hardness is a critical characteristic that affects overall component properties, including wear. This study employed K-means unsupervised machine learning to explore the relationship between the relative surface hardness and scratch width of electroless nickel plating on additively manufactured composite components. The Taguchi design of experiment (TDOE) L9 orthogonal array facilitated experimentation with various factors and levels. Initially, a digital light microscope was used for 3D surface mapping and scratch width quantification. However, the microscope struggled with the reflections from the shiny Ni-plating and scatter from small scratches. To overcome this, a scanning electron microscope (SEM) generated grayscale images and 3D height maps of the scratched Ni-plating, thus enabling the precise characterization of scratch widths. Optical identification of the scratch regions and quantification were accomplished using Python code with a K-means machine-learning clustering algorithm. The TDOE yielded distinct Ni-plating hardness levels for the nine samples, while an increased scratch force showed a non-linear impact on scratch widths. The enhanced surface quality resulting from Ni coatings will have significant implications in various industrial applications, and it will play a pivotal role in future metal and alloy surface engineering.

36 MATERIALS SCIENCE↗

Informing nuclear physics via machine learning methods with differential and integral experiments

Information from differential nuclear-physics experiments and theory is often too uncertain to accurately define nuclear-physics observables such as cross sections or energy spectra. Integral experimental data, representing the applications of these observables, are often more precise but depend simultaneously on too many of them to unambiguously identify issues in the observable with human expert analysis alone. Here, we explore how we can leverage physics knowledge gained from differential experimental data, nuclear theory, integral experiments, and neutron-transport calculations to better understand nuclear-physics observables in the context of the application area represented by integral experiments. We support this task with machine-learning methods to discern trends in a large amount of convoluted data. Differential and integral information was used in an analysis augmented by the random forest and the Shapley additive explanations metric. We chose as an application area one that is represented by criticality measurements and pulsed-sphere neutron-leakage spectra. We show one representative example ( 241 Pu fission observables) where the combination of differential and integral information allowed to resolve issues in data representing these observables. As a starting point, the machine learning (ML) algorithms highlighted several observables as leading potentially to bias in simulating integral experiments. Differential information, paired with sensitivity to integral quantities, allowed us then to pinpoint one specific observable ( 241 Pu fission cross section) as the main driver of bias. The comparison to integral experiments, on the other hand, allowed us to indicate a likely reliable experiment among several discrepant ones for this observables. In other cases (e.g., 239 Pu observables), we were not able to resolve the confounding introduced by integral experiments but instead highlighted the need for targeted new experiments and theory developments to better constrain the nuclear-physics space for the application area represented by integral experiments. We were able to combine information from differential experimental data, nuclear-physics theory, integral experiments, and neutron-transport simulations of the latter experiments with the help of the random forest algorithm and expert judgment. This combination of knowledge allows to improve our description of nuclear-physics observables as applied to a particular application area.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Physics-informed machine learning to predict solvatochromic parameters of designer solvents with case studies in CO 2 and lignin dissolution

The polarity of solvents plays a critical role in various research applications, particularly in their solubilities. Polarity is conveniently characterized by the Kamlet-Taft parameters that is, the hydrogen bonding acidity (α), the basicity (β), and the polarizability (π*). Obtaining Kamlet-Taft parameters is very important for designer solvents, namely ionic liquids (ILs) and deep eutectic solvents (DESs). However, given the unlimited theoretical number of combinations of ionic pairs in ILs and hydrogen-bond donor/acceptor pairs in DESs, experimental determination of their Kamlet-Taft parameters is impractical. To address this, the present study developed two different machine learning (ML) algorithms to predict Kamlet-Taft parameters for designer solvents using quantum chemically derived input features. The ML models developed in the present study showed accurate predictions with high R 2 and low RMSE values. Further, in the context of present interest in the circular bioeconomy, the relationship between the basicities and acidities of designer solvents and their ability to dissolve lignin and carbon dioxide (CO 2 ) is discussed. Our method thus guides the design of effective solvents with optimal Kamlet-Taft parameter values dissolving and converting biomass and CO 2 into valuable chemicals.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Disentangling Electronic Transport and Hysteresis at Individual Grain Boundaries in Hybrid Perovskites via Automated Scanning Probe Microscopy

Underlying the rapidly increasing photovoltaic efficiency and stability of metal halide perovskites (MHPs) is the advancement in the understanding of the microstructure of polycrystalline MHP thin film. Over the past decade, intense efforts have been aimed at understanding the effect of microstructures on MHP properties, including chemical heterogeneity, strain disorder, phase impurity, etc. It has been found that grain and grain boundary (GB) are tightly related to lots of microscale and nanoscale behavior in MHP thin films. Atomic force microscopy (AFM) is widely used to observe grain and boundary structures in topography and subsequently to study the correlative surface potential and conductivity of these structures. For now, most AFM measurements have been performed in imaging mode to study the static behavior; in contrast, AFM spectroscopy mode allows us to investigate the dynamic behavior of materials, e.g., conductivity under sweeping voltage. However, a major limitation of AFM spectroscopy measurements is that they require manual operation by human operators, and as such only limited data can be obtained, hindering systematic investigations of these microstructures. Here, we designed a workflow combining the conductive AFM measurement with a machine learning (ML) algorithm to systematically investigate grain boundaries in MHPs. The trained ML model can extract GBs locations from the topography image, and the workflow drives the AFM probe to each GB location to perform a current–voltage (IV) curve automatically. Then, we are able to have IV curves at all GB locations, allowing us to systematically understand the property of GBs. Using this method, we discovered that the GB junction points are less conductive, potentially more photoactive, and can play critical roles in MHP stability, while most previous works only focused on the difference between GB and grains.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗