Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed computing, machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Harmonized Automatic Relay Mitigation of Nefarious Intentional Events (HARMONIE) - Special Protection Scheme (SPS)

The harmonized automatic relay mitigation of nefarious intentional events (HARMONIE) special protection scheme (SPS) was developed to provide adaptive, cyber-physical response to unpredictable disturbances in the electric grid. The HARMONIE-SPS methodology includes a machine learning classification framework that analyzes real time cyber-physical data and determines if the system is in normal conditions, cyber disturbance, physical disturbance, or cyber-physical disturbance. This classification then informs response, if needed and/or suitable, and included cyber-physical corrective actions. Beyond standard power system mitigations, a few novel approaches were developed that included a consensus algorithm-based relay voting scheme, an automated power system triggering condition and corrective action pairing algorithm, and a cyber traffic routing optimization algorithm. Both the classification and response techniques were tested within a newly integrated emulation environment composed of a real-time digital simulator (RTDS) and SCEPTRE™. This report details the HARMONIE-SPS methodology, highlighting both the classification and response techniques, and the subsequent testing results from the emulation environment.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Object detection with deep learning for rare event search in the GADGET II TPC

In the pursuit of identifying rare two-particle events within the GADGET II Time Projection Chamber (TPC), this paper presents a comprehensive approach for leveraging Convolutional Neural Networks (CNNs) and various data processing methods. To address the inherent complexities of 3D TPC track reconstructions, the data is expressed in 2D projections and 1D quantities. This approach capitalizes on the diverse data modalities of the TPC, allowing for the efficient representation of the distinct features of the 3D events, with no loss in topology uniqueness. Additionally, it leverages the computational efficiency of 2D CNNs and benefits from the extensive availability of pre-trained models. Given the scarcity of real training data for the rare events of interest, simulated events are used to train the models to detect real events. To account for potential distribution shifts when predominantly depending on simulations, significant perturbations are embedded within the simulations. This produces a broad parameter space that works to account for potential physics parameter and detector response variations and uncertainties. These parameter-varied simulations are used to train sensitive 2D CNN object detectors. When combined with 1D histogram peak detection algorithms, this multi-modal detection framework is highly adept at identifying rare, two-particle events in data taken during experiment 21072 at the Facility for Rare Isotope Beams (FRIB), demonstrating a 100% recall for events of interest. Here, we present the methods and outcomes of our investigation and discuss the potential future applications of these techniques.

Convolutional neural network↗

Generalized representative structures for atomistic systems

A new method is presented to generate atomic structures that reproduce the essential characteristics of arbitrary material systems, phases, or ensembles. Previous methods allow one to reproduce the essential characteristics (e.g. the chemical disorder) of a large random alloy within a small crystal structure. The ability to generate small representations of random alloys, along with the restriction to crystal systems, results from using the fixed-lattice cluster correlations to describe structural characteristics. A more general description of the structural characteristics of atomic systems is obtained using complete sets of atomic environment descriptors. These are used within for generating representative atomic structures without restriction to fixed lattices. A general data-driven approach is provided here utilizing the atomic cluster expansion (ACE) basis. The N-body ACE descriptors are a complete set of atomic environment descriptors that span both chemical and spatial degrees of freedom and are used within for describing atomic structures. The generalized representative structure (GRS) method presented within generates small atomic structures that reproduce ACE descriptor distributions corresponding to arbitrary structural and chemical complexity. It is shown that systematically improvable representations of crystalline systems on fixed parent lattices, amorphous materials, liquids, and ensembles of atomic structures may be produced efficiently through optimization algorithms. With the GRS method, we highlight reduced representations of atomistic machine-learning training datasets that contain similar amounts of information and small 40–72 atom representations of liquid phases. The ability to use GRS methodology as a driver for informed novel structure generation is also demonstrated. The advantages over other data-driven methods and state-of-the-art methods restricted to high-symmetry systems are highlighted.

atomic cluster expansion↗

LandScan Mosaic

The LandScan program at Oak Ridge National Laboratory (ORNL), in collaboration with the National Geospatial-Intelligence Agency (NGA), continues to deliver the most accurate and up to date global, high resolution gridded population data. Additionally, the latest advancements in the LandScan HD methodology led to reduced latency in development of rapid updates for geopolitical events. With momentum towards reporting more up to date population estimates, feedback from the user community expressed interest in reporting population estimates in ranges - whether to express a level of uncertainty or confirm to leadership and stakeholders the modeled data are estimates. Building upon the need to understand uncertainty or confidence in the modeled data and report ranges at the global scale, LandScan Mosaic was developed. LandScan Mosaic represents the next generation of high-resolution population modeling, building upon the established success of previous LandScan HD iterations. While LandScan HD employed a deterministic big data fusion approach, LandScan Mosaic enhances this methodology by integrating advanced machine learning techniques to impute missing, yet crucial, population model parameters. This advancement allows for probabilistic modeling of building occupancy and population distribution, incorporating uncertainty quantification through Monte Carlo sampling methods. By combining big data fusion with machine learning-driven imputation and stochastic modeling, LandScan Mosaic provides a more comprehensive and robust representation of population dynamics. LandScan Mosaic will be following the in the footsteps of its longstanding counterpart LandScan Global and releasing a global gridded population raster, at the 3-arcsecond resolution. This technical report documents the current stage of development of LandScan Mosaic, detailing the methodologies and data sources behind the modeling. Stakeholders are encouraged to use this document as an authoritative reference for insight into Mosaic’s data development processes. However, readers should note that LandScan Mosaic remains in a late-stage research and development phase, and methodologies and data presented here are subject to refinements ahead of the anticipated global release in Summer 2025. Feedback and inquiries from users and stakeholders are welcomed as we continue to refine and enhance this important population resource.

97 MATHEMATICS AND COMPUTING↗

Data-Centric Approach to Capture Non-Polynomial Nonlinear Dynamics

We propose an analytical construction of observable functions in the extended dynamic mode decomposition (EDMD) algorithm. EDMD is a numerical method for approximating the spectral properties of the Koopman operator. The choice of observable functions is fundamental for applying EDMD to nonlinear problems arising in systems and control. Existing methods either start from a set of dictionary functions and look for the subset that best fits the underlying nonlinear dynamics or rely on machine learning algorithms to “learn” observable functions. Conversely, in this paper, we start from the dynamical system model and lift it through the Lie derivatives, rendering it into a polynomial form. This proposed transformation into a polynomial form is exact and provides an adequate set of observable functions. The strength of the proposed approach is its applicability to a broader class of nonlinear dynamical systems, particularly those with nonpolynomial functions and compositions thereof. Moreover, it retains the physical interpretability of the underlying dynamical system and can be readily integrated into existing numerical libraries. We demonstrate the proposed approach with an application to electric power systems. The modeled system consists of a single generator connected to an infinite bus, where nonlinear terms include sine and cosine functions. The results demonstrate the effectiveness of the proposed procedure in off-attractor nonlinear dynamics for estimation and prediction; the observable functions obtained from the proposed construction outperform methods that use dictionary functions comprising monomials or radial basis functions.

extended dynamic mode decomposition↗

Topological Machine Learning Methods for Power System Responses to Contingencies

While deep learning tools, coupled with the emerging machinery of topological data analysis, are proven to deliver various performance gains in a broad range of applications, from image classification to biosurveillance to blockchain fraud detection, their utility in areas of high societal importance such as power system modeling and, particularly, resilience quantification in the energy sector yet remains untapped. To provide fast acting synthetic regulation and contingency reserve services to the grid while having minimal disruptions on customer quality of service, we propose a new topology-based system that depends on a neural network architecture for impact metric classification and prediction in power systems. This novel topology-based system allows one to evaluate the impact of three power system contingency types, in conjunction with transmission lines, transformers, and transmission lines combined with transformers. We show that the proposed new neural network architecture equipped with local topological measures facilitates more accurate classification of unserved load as well as the amount of unserved load. In addition, we are able to learn more about the complex relationships between electrical properties and local topological measurements on their simulated response to contingencies for the NREL-SIIP power system.

contingency analysis↗

A Comparison of Machine Learning Methods of Association Tested on Dense Nodal Arrays

The association of phase picks to form events is one of the fundamental components of seismology. Large and dense sensor networks, such as >1000 geophone arrays (and distributed acoustic sensing), offer unique challenges in association due to the vast numbers of observations and high likelihood of errant picks. In addition, the large number of stations can greatly increase the time it takes to perform the association. For this reason, machine learning (ML) methods might provide a more optimal method of association for such networks. In this work, we examine how well ML methods (e.g., Gaussian mixture model association, PhaseLink, and Graph Earthquake Neural Interpretation Engine) can incorporate dense seismic arrays into regional networks and how well they handle the increasing numbers of stations. Here, we test their capabilities on two dense seismic deployments, one within Rock Valley Nevada (52 nodes and a 9-station sparse local network), and the LArge-n Seismic Survey in Oklahoma dense nodal array (>1800 vertical-component geophones). Processing data from these two different styles of dense seismic deployments allows testing of how the ML algorithms can merge array data with a broader regional network, how they deal with poorly picked phases, and how they handle anthropogenic noise. We compare the ML-associated bulletins to those obtained using the Rapid Earthquake Association and Location algorithm, a more traditional method of association. We find that there are very small differences in results between the methods for small networks (<100 stations) with low pick rates. For large networks (>1000), there are enough errant picks that some of the ML methods start to create false events out of noise. We also find that the ML methods vary in computation time significantly but are all faster than the traditional method tested here.

58 GEOSCIENCES↗

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (β) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

97 MATHEMATICS AND COMPUTING↗

Decentralised Reinforcement Learning for Dynamic Cyberattack Response in Microgrid Networks

Microgrids rely on communication networks for reliable operation, which makes them inherently vulnerable to cyberattacks. Such attacks can destabilise system dynamics and drive states away from their nominal operating trajectories. Although several physics-informed and machine learning-based strategies have been developed to counter these threats, the rapidly evolving cyber landscape enables adversaries to bypass static defences or rules-based mitigation approaches. This paper proposes a dynamic, online-trained and fully decentralised reinforcement learning (RL)-based cyberattack response framework to protect microgrids from evolving cyberattacks. The proposed framework deploys multiple deep Q-networks (DQNs), each associated with a distributed energy resource (DER), to enable localised and adaptive attack mitigation. In this framework, each DQN processes local voltage and frequency measurements—combined with intrusion detection system (IDS) alerts—as observations and rewards to guide decision-making. Extensive simulation studies demonstrate the robustness of the proposed framework under diverse attack scenarios and varying IDS-induced detection delays. Comparative analysis highlights its superiority over existing static or preexisting rules-based mitigation approaches. Finally, we present an analysis that shows the framework's scalability to real-life microgrids with more interacting agents.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Cyberattack Detection and Mitigation on Central Volt‐VAr Using Circuit Law and Machine Learning

ABSTRACT In a distribution grid, voltage is maintained within a nominal range through a Volt‐VAr function that controls capacitor banks, reactive power of distributed energy resources (DER), and on‐load tap changers (OLTC). Availability of communications helps with the implementation of central Volt‐VAr control; however, it also opens the system to cyberattacks, causing voltage disturbances. Previous work has shown the adverse impacts of false data injection (FDI) on the central Volt‐VAr control; however, very few works have studied methods to detect and mitigate FDI on Volt‐VAr control. This paper addresses gaps in the detection and mitigation of FDI on the measurement packets of a central Volt‐VAr control. This work uses a two‐stage algorithm for cyberattack detection since the accuracy of a single‐stage machine learning (ML)–based detection method decreases while dealing with unseen data. The first stage is based on the verification of measurements against circuit laws, and the second stage utilizes a tree search algorithm and an ML method to detect the falsified data. This paper compares long short‐term memory (LSTM) and bidirectional LSTM (BiLSTM) as the employed ML algorithms. Finally, the mitigation algorithm replaces the falsified data with the estimated output of the ML algorithm. The effectiveness of the proposed method is tested for several cases using the IEEE 13‐bus test system in PSCAD software.

Beikbabaei, Milad [Bradley Department of Electrica↗

Plant Single-Cell Solutions for Energy and the Environment (Second Workshop Report)

Plants are important sources of energy and materials, and they collectively represent a critical component of Earth’s ecosystem. With increasing environmental stresses due to climate change and intensive agricultural practices, the need for resilient plants is greater than ever before. To secure plant resources for bioenergy, biomaterials, food, and ecosystem adaptation, a deeper understanding of the fundamental biology of plants at a cellular level is urgently needed. Plants contain a multitude of specialized cell types that compose tissues and organs. Pathogens often target specific cell types within plants, and the response of one cell to a particular stimulus is likely to be distinct from its neighbor because of underlying molecular and contextual differences. Understanding how these responses are distributed among cells, the main goal of single-cell approaches, will substantially enhance our ability to use targeted engineering for improving plant productivity and resilience. Furthermore, single-cell approaches are necessary to understand the interactions between plants and other ecosystem members such as fungi, bacteria, and archaea. Unlocking these gene-response mechanisms at a cellular level can improve our ability to adapt plants to environmental stresses, increasing their utility as feedstocks for biomaterials and bioenergy. Recent advances in high-throughput sequencing, mass spectrometry, microfluidics and miniaturization, artificial intelligence and machine learning, and bioinformatics have greatly improved our ability to detect and understand processes at a cellular level. In mammalian systems, single-cell transcriptomics has already led to many advances, such as newly identified cell types and cell-targeted treatment of diseases, and mass spectrometry-based single-cell proteomics has recently been demonstrated as a promising emerging technology. However, plant single-cell omics has lagged behind mammalian approaches due to the high cost of the technologies relative to available resources and to the innate biological features of plants, including the complexity of the cell wall and polyploidy. To better understand how single-cell methods could enable plant science, Lawrence Berkeley National Laboratory (Berkeley Lab) hosted a workshop on April 29, 2021, that brought together a diverse group of leaders in plant and/or single-cell biology. Attendees represented federal research programs and domestic and international academic institutions. During the workshop, three presenters described the current state of research in both experimental and computational approaches. While the focus of the workshop was on factors preventing plant biology researchers from fully adopting single-cell methodologies, workshop participants agreed that most barriers could be overcome with focused, strategic investment and coordinated efforts among institutions leading to significant scientific discoveries that would be difficult to obtain using more conventional technologies.

59 BASIC BIOLOGICAL SCIENCES↗

Physics consistent machine learning framework for inverse modeling with applications to ICF capsule implosions

In high energy density physics (HEDP) and inertial confinement fusion (ICF), predictive modeling is complicated by uncertainty in parameters that characterize various aspects of the modeled system, such as those characterizing material properties, equation of state (EOS), opacities, and initial conditions. Typically, however, these parameters are not directly observable. What is observed instead is a time sequence of radiographic projections using X-rays. In this work, we define a set of sparse hydrodynamic features derived from the outgoing shock profile and outer material edge, which can be obtained from radiographic measurements, to directly infer such parameters. Our machine learning (ML)-based methodology involves a pipeline of two architectures, a radiograph-to-features network (R2FNet) and a features-to-parameters network (F2PNet), that are trained independently and later combined to approximate a posterior distribution for the parameters from radiographs. We show that the machine learning architectures are able to accurately infer initial conditions and EOS parameters, and that the estimated parameters can be used in a hydrodynamics code to obtain density fields, shocks, and material interfaces that satisfy thermodynamic and hydrodynamic consistency. Finally, we demonstrate that features resulting from an unknown EOS model can be successfully mapped onto parameters of a chosen analytical EOS model, implying that network predictions are learning physics, with a degree of invariance to the underlying choice of EOS model. To the best of our knowledge, our framework is the first demonstration of recovering both thermodynamic and hydrodynamic consistent density fields from noisy radiographs.

97 MATHEMATICS AND COMPUTING↗

Domain-Adapted Convolutional Networks for Satellite Image Classification: A Large-Scale Interactive Learning Workflow

Satellite imagery often exhibits large spatial extent areas that encompass object classes with considerable variability. This often limits large-scale model generalization with machine learning algorithms. Notably, acquisition conditions, including dates, sensor position, lighting condition, and sensor types, often translate into class distribution shifts introducing complex nonlinear factors and hamper the potential impact of machine learning classifiers. Here, this article investigates the challenge of exploiting satellite images using convolutional neural networks (CNN) for settlement classification where the class distribution shifts are significant. We present a large-scale human settlement mapping workflow based-off multiple modules to adapt a pretrained CNN to address the negative impact of distribution shift on classification performance. To extend a locally trained classifier onto large spatial extents areas we introduce several submodules: First, a human-in-the-loop element for relabeling of misclassified target domain samples to generate representative examples for model adaptation; second, an efficient hashing module to minimize redundancy and noisy samples from the mass-selected examples; and third, a novel relevance ranking module to minimize the dominance of source example on the target domain. The workflow presents a novel and practical approach to achieve large-scale domain adaptation with binary classifiers that are based-off CNN features. Experimental evaluations are conducted on areas of interest that encompass various image characteristics, including multisensors, multitemporal, and multiangular conditions. Domain adaptation is assessed on source–target pairs through the transfer loss and transfer ratio metrics to illustrate the utility of the workflow.

47 OTHER INSTRUMENTATION↗

Using convolutional neural networks to accelerate three-dimensional coherent synchrotron radiation computations

Calculating the effects of coherent synchrotron radiation (CSR) is one of the most computationally expensive tasks in accelerator physics. Here, we use convolutional neural networks (CNNs), along with a latent conditional diffusion (LCD) model, trained on physics-based simulations to speed up calculations. Specifically, we produce the 3D CSR wakefields generated by electron bunches in circular orbit in the steady-state condition. Two datasets are used for training and testing the models: wakefields generated by three-dimensional Gaussian electron distributions and wakefields from a sum of up to 25 three-dimensional Gaussian distributions. The CNNs are able to accurately produce the 3D wakefields ∼250–1000 times faster than the numerical calculations, while the LCD achieves a gain of a factor of ∼34. We also test the extrapolation and out-of-distribution generalization ability of the models. They generalize well on distributions with larger spreads than what they were trained on but struggle with smaller spreads.

43 PARTICLE ACCELERATORS↗

Scalable and Energy-Efficient Methods for Interactive Exploration of Scientific Data

The main scientific contributions of this project are the following novel concepts for multidimensional arrays: shape-based similarity join (SIGMOD 2016), incremental view maintenance (SIGMOD 2017), user-defined stencil functions (HPDC 2017), and distributed caching for in-situ processing (SSDBM 2018). Building on our collaboration with the astrophysics group at LBNL, we applied these techniques to the data generated in the Palomar Transient Factory (PTF) astronomical survey. They played a pivotal role in the first-ever observation of a neutron star merger, which produces gravitational waves and turns out to be the origin of heavy elements, including gold. This has lead to a Science magazine article that has received extensive media coverage on ACM TechNews, Slashdot, FiveThirtyEight, and Quanta Magazine, among others. Additionally, two other articles detailing related aspects of the same discovery have been published in the Astrophysical Journal Letters journal. These publications have more than 3,000 citations according to Google Scholar (as of February 2022). This cross-disciplinary collaboration provided very good opportunities to apply database techniques to real-life scientific problems. The fact that they facilitated major discoveries in astrophysics proves the importance of our research. In addition to the work on multidimensional array databases, this project has also developed stochastic gradient descent (SGD) optimization algorithms for training large scale machine learning models, methods for querying in-situ data, and a database query optimizer based on sketch synopses.

79 ASTRONOMY AND ASTROPHYSICS↗

Neural Image Compression: Generalization, Robustness, and Spectral Biases

Recent advances in neural image compression (NIC) have resulted in models which are starting to outperform traditional codecs. While this has led to growing excitement about using these methods in real-world applications, the successful adoption of any machine learning system (including NIC) in the wild requires it to generalize (and be robust) to unseen distribution shifts at deployment time. Unfortunately, current research lacks comprehensive datasets and informative tools to evaluate and understand compression performance in real-world settings. To bridge this crucial gap, first, this paper presents a comprehensive benchmark suite to evaluate the out-of-distribution (OOD) performance of image compression methods. Specifically, we design CLIC-C and Kodak-C by introducing 15 common corruptions to popular CLIC and Kodak benchmarks. Next, we propose spectrally inspired introspection tools to gain a deeper understanding of errors introduced by image compression methods as well as their OOD performance. To this end, we carry out a detailed performance comparison of the classical codec with various variants of NIC (e.g., original, variable rate, pruned), revealing intriguing findings that challenge our current understanding of the strengths and limitations of NIC. Finally, we corroborate our empirical findings with theoretical analysis, providing an in-depth view of the OOD performance of NIC. Our benchmarks, spectral introspection tools, and findings provide a crucial bridge to the real-world adoption of NIC. We hope that our work will propel future efforts in designing more robust and generalizable NIC methods.

neural networks, Variational Autoencoder, robustne↗

Smart Methane Emission Detection System Development Final Report

Working with the Department of Energy's National Energy Technology Laboratory, Southwest Research Institute® (SwRI®) developed a system to identify methane leaks reliably, accurately, and autonomously at critical midstream sections of the natural gas distribution network in real- time for the purpose of mitigating methane emissions using Optical Gas Imaging (OGI) cameras. SwRI's Smart Leak Detection – Methane (SLED/M) adds a high degree of automation to the process of methane leak detection to minimize sources of human error, minimize response time to a leak event, and maximize midstream visibility. Furthermore, SwRI has been working towards integrating Quantitative OGI (QOGI) capabilities into this existing technology. By leveraging Deep Learning, SwRI now has the capability to estimate fugitive emission leak rates quickly and reliably, which allows operators to detect emissions, quantify leak rate, prioritize repairs, and validate the repairs in a single instrument. The next generation QOGI technology leverages the same cameras used in Leak Detection and Repair (LDAR) programs, with improvements in safety and speed for traditional quantification-based repairs, ultimately leading to less overhead cost for the operators.

03 NATURAL GAS↗

Rapid subsurface analysis of frequency-domain thermoreflectance images with K-means clustering

K-means clustering analysis is applied to frequency-domain thermoreflectance (FDTR) hyperspectral image data to rapidly screen the spatial distribution of thermophysical properties at material interfaces. Performing FDTR while raster scanning a sample consisting of 8.6 μm of doped-silicon (Si) bonded to a doped-Si substrate identifies spatial variation in the subsurface bond quality. Routine thermal analysis at select pixels quantifies this variation in bond quality and allows assignment of bonded, partially bonded, and unbonded regions. Performing this same routine thermal analysis across the entire map, however, becomes too computationally demanding for rapid screening of bond quality. To address this, K-means clustering was used to reduce the dimensionality of the dataset from more than 20 000 pixel spectra to just K = 3 component spectra. The three component spectra were then used to express every pixel in the image through a least-squares minimized linear combination providing continuous interpolation between the components across spatially varying features, e.g., bonded to unbonded transition regions. Fitting the component spectra to the thermal model, thermal properties for each K cluster are extracted and then distributed according to the weighting established by the regressed linear combination. Thermophysical property maps are then constructed and capture significant variation in bond quality over 25 μm length scales. The use of K-means clustering to achieve these thermal property maps results in a 74-fold speed improvement over explicit fitting of every pixel.

36 MATERIALS SCIENCE↗