Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “random networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Filament-Free Bulk Resistive Memory Enables Deterministic Analogue Switching

Digital computing is nearing its physical limits as computing needs and energy consumption rapidly increase. Analogue-memory-based neuromorphic computing can be orders of magnitude more energy efficient at data-intensive tasks like deep neural networks, but has been limited by the inaccurate and unpredictable switching of analogue resistive memory. Filamentary resistive random access memory (RRAM) suffers from stochastic switching due to the random kinetic motion of discrete defects in the nanometer-sized filament. Here, this stochasticity is overcome by incorporating a solid electrolyte interlayer, in this case, yttria-stabilized zirconia (YSZ), toward eliminating filaments. Filament-free, bulk-RRAM cells instead store analogue states using the bulk point defect concentration, yielding predictable switching because the statistical ensemble behavior of oxygen vacancy defects is deterministic even when individual defects are stochastic. Both experiments and modeling show bulk-RRAM devices using TiO2-X switching layers and YSZ electrolytes yield deterministic and linear analogue switching for efficient inference and training. Bulk-RRAM solves many outstanding issues with memristor unpredictability that have inhibited commercialization, and can, therefore, enable unprecedented new applications for energy-efficient neuromorphic computing. Beyond RRAM, this work shows how harnessing bulk point defects in ionic materials can be used to engineer deterministic nanoelectronic materials and devices.

36 MATERIALS SCIENCE↗

Tailor : Altering Skip Connections for Resource-Efficient Inference

Deep neural networks use skip connections to improve training convergence. However, these skip connections are costly in hardware, requiring extra buffers and increasing on- and off-chip memory utilization and bandwidth requirements. In this article, we show that skip connections can be optimized for hardware when tackled with a hardware-software codesign approach. We argue that while a network’s skip connections are needed for the network to learn, they can later be removed or shortened to provide a more hardware-efficient implementation with minimal to no accuracy loss. We introduceTailor, a codesign tool whose hardware-aware training algorithm gradually removes or shortens a fully trained network’s skip connections to lower the hardware cost.Tailorimproves resource utilization by up to 34% for block random access memories (BRAMs), 13% for flip-flops (FFs), and 16% for look-up tables (LUTs) for on-chip, dataflow-style architectures.Tailorincreases performance by 30% and reduces memory bandwidth by 45% for a two-dimensional processing element array architecture.

Computer Science↗

Model Exploration of an Information-Based Healthcare Intervention Using Parallelization and Active Learning

This paper describes the application of a large-scale active learning method to characterize the parameter space of a computational agent-based model developed to investigate the impact of CommunityRx, a clinical information-based health intervention that provides patients with personalized information about local community resources to meet basic and self-care needs. Additionally, the diffusion of information about community resources and their use is modeled via networked interactions and their subsequent effect on agents' use of community resources across an urban population. A random forest model is iteratively fitted to model evaluations to characterize the model parameter space with respect to observed empirical data. We demonstrate the feasibility of using high-performance computing and active learning model exploration techniques to characterize large parameter spaces; by partitioning the parameter space into potentially viable and non-viable regions, we rule out regions of space where simulation output is implausible to observed empirical data. We argue that such methods are necessary to enable model exploration in complex computational models that incorporate increasingly available micro-level behavior data. We provide public access to the model and high-performance computing experimentation code.

97 MATHEMATICS AND COMPUTING↗

Nanopore networks in colloidal silica assemblies characterized by XCT for confined fluid flow modeling

Fluid flow through nanopore structures has exhibited different behavior than that described by Darcy's law, derived from pore networks of micrometer scale. Here, pressure assemblies of colloidal silica spheres of specified diameters from 94 to 620 nanometers were prepared as model nano-porous media. The smaller silica spheres tended to form random packing, but some substantial ordering was found in the packing of the larger silica spheres. X-ray computed tomography (XCT), with a voxel resolution of 16 nm, was used to characterize the nanopore networks. The porosity of the Nano-XCT pore network (34%) was less than the total porosity of the assemblies as determined by X-ray attenuation (52%), possibly due to the limits of voxel resolution during segmentation. Fluid flow in the nanopore networks was simulated using the single-phase Lattice Boltzmann Method, and the simulated permeability was compared with the empirical and experimental values. Based on our characterization, although a well-ordered packing of silica spheres was not achieved, it was found that the nanopore networks in the colloidal silica assemblies had pore size distributions corresponding to the particle sizes. The simulated permeability was less than the experimental measurement for water flow, but the complex packing of silica spheres and surface chemistry issues need to be considered in future research.

02 PETROLEUM↗

Finding diverse ways to improve algebraic connectivity through multi-start optimization

The algebraic connectivity, also known as the Fiedler value, is a spectral measure of network connectivity that can be increased through edge addition. We present an algorithm for producing many diverse ways to add a fixed number of edges to a network to achieve a near optimal Fiedler value. Previous Fielder value optimization algorithms (i.e. the greedy algorithm) output only one solution. Obtaining a single solution is rarely good enough for real-world network redesign problems, as practical constraints (political, physical or financial) may prevent implementation. Our algorithm takes a multi-start optimization approach, adding a random initial edge and then applies a greedy heuristic to improve the Fiedler value. The random choice moves us to a new region of the search space, enabling discovery of diverse solutions. Additionally, we present a Determinantal Point Process framework for quantifying diversity. We then apply a Markov chain Monte Carlo technique to sift through the large number of output solutions and locate a smaller, more manageable collection of highly diverse solutions that can be presented to network redesign engineers. We demonstrate the effectiveness of our algorithm on real-world graphs with varied structures.

97 MATHEMATICS AND COMPUTING↗

Lithium-Ion Battery Diagnostics Using Electrochemical Impedance via Machine-Learning

Diagnosing battery states such as health, state-of-charge, or temperature is crucial for ensuring the safety and reliability of electrochemical energy storage systems. While some states, such as temperature, may be measured using cheap sensors, accurate diagnosis of battery health metrics usually requires time-consuming performance measurements, making them infeasible for use in real-world operation. These health metrics can be measured during lab-testing and then estimated on-line using predictive life models or via state observer algorithms such as Kalman filters, but these predictive methods should be supplemented by actual measurement of battery health whenever possible to ensure reliability. Rapid measurement of battery health may be done by various types of fast diagnostic techniques such as electrochemical impedance spectroscopy (EIS), which can be performed in only a few minutes and require only a fraction of the energy and power needed for a full charge and discharge measurement. But there is a substantial challenge for estimating battery health using EIS data, as EIS is sensitive to cell temperature, state-of-charge, current, and resting time in addition to health. Thus, utilizing EIS data to predict battery capacity requires correcting for all these additional variables, a task that is extremely difficult to handle analytically. This talk utilizes machine-learning methods to estimate the effectiveness of battery capacity prediction from EIS data, leveraging a data set of hundreds of EIS measurements recorded at varying temperature and state-of-charge throughout a 500-day aging study of 32 commercial, large-format NMC-Graphite lithium-ion batteries. Using EIS as input to machine-learning models is complicated by the nonlinear response of impedance to battery health, temperature, and state-of-charge, as well as the collinearity between the impedance response at neighboring frequencies, which can easily lead to overfit models. To train robust models, features from EIS data need to be extracted from the data or some subset of critical frequencies selected. Many approaches for extracting and selecting features from EIS data from electrochemical analysis and machine-learning fields were identified for analysis: using the entire raw spectra; selection of one, two, or many frequencies from the entire spectra; selecting interesting points from the EIS measurement using domain knowledge; fitting EIS with an equivalent-circuit model; calculating statistics on the raw impedance values; and reducing the dimensionality of the data using unsupervised linear (principal component analysis) and non-linear (uniform manifold approximation and projection) methods. These approaches were rigorously compared using a machine-learning pipeline approach, training linear, Gaussian process, and random forest regression models and quantifying performance using cross-validation as well as a held-out test set. An artificial neural network model trained on the raw spectra was also tested. Promising pipelines were fine-tuned via Bayesian hyperparameter optimization using cross-validation loss and training with class-specific weights to counter data set imbalance. The most reliable method for utilizing impedance in this work was the selection of two optimal frequencies through an exhaustive search, resulting in about 2% mean absolute error on test data for both Gaussian process and random forest model architectures. Interrogation of a variety of models reveals critical frequencies of 100 Hz and 103 Hz for this data set, though the optimal set of frequencies is not necessarily intuitive, i.e., the best performing models are not simply those that use impedance at frequencies that have the highest correlation to the relative discharge capacity. The best performing model is an ensemble model, which is able to predict battery capacity with 1.9% mean absolute error for unseen cells using impedance recorded at a variety of temperatures and states-of-charge.

battery↗

Bayesian tomography of high-dimensional on-chip biphoton frequency combs with randomized measurements

Abstract Owing in large part to the advent of integrated biphoton frequency combs, recent years have witnessed increased attention to quantum information processing in the frequency domain for its inherent high dimensionality and entanglement compatible with fiber-optic networks. Quantum state tomography of such states, however, has required complex and precise engineering of active frequency mixing operations, which are difficult to scale. To address these limitations, we propose a solution that employs a pulse shaper and electro-optic phase modulator to perform random operations instead of mixing in a prescribed manner. We successfully verify the entanglement and reconstruct the full density matrix of biphoton frequency combs generated from an on-chip Si 3 N 4 microring resonator in up to an 8 × 8-dimensional two-qudit Hilbert space, the highest dimension to date for frequency bins. More generally, our employed Bayesian statistical model can be tailored to a variety of quantum systems with restricted measurement capabilities, forming an opportunistic tomographic framework that utilizes all available data in an optimal way.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Upscaling Wetland Methane Emissions From the FLUXNET–CH4 Eddy Covariance Network (UpCH4 v1.0): Model Development, Network Assessment, and Budget Comparison

Wetlands are responsible for 20%–31% of global methane (CH 4 ) emissions and account for a large source of uncertainty in the global CH 4 budget. Data-driven upscaling of CH 4 fluxes from eddy covariance measurements can provide new and independent bottom-up estimates of wetland CH 4 emissions. Here, we develop a six-predictor random forest upscaling model (UpCH4), trained on 119 site-years of eddy covariance CH 4 flux data from 43 freshwater wetland sites in the FLUXNET-CH4 Community Product. Network patterns in site-level annual means and mean seasonal cycles of CH 4 fluxes were reproduced accurately in tundra, boreal, and temperate regions (Nash-Sutcliffe Efficiency ~0.52–0.63 and 0.53). UpCH4 estimated annual global wetland CH 4 emissions of 146 ± 43 TgCH 4 y –1 for 2001–2018 which agrees closely with current bottom-up land surface models (102–181 TgCH 4 y –1 ) and overlaps with top-down atmospheric inversion models (155–200 TgCH 4 y –1 ). However, UpCH4 diverged from both types of models in the spatial pattern and seasonal dynamics of tropical wetland emissions. We conclude that upscaling of eddy covariance CH 4 fluxes has the potential to produce realistic extra-tropical wetland CH 4 emissions estimates which will improve with more flux data. To reduce uncertainty in upscaled estimates, researchers could prioritize new wetland flux sites along humid-to-arid tropical climate gradients, from major rainforest basins (Congo, Amazon, and SE Asia), into monsoon (Bangladesh and India) and savannah regions (African Sahel) and be paired with improved knowledge of wetland extent seasonal dynamics in these regions.

54 ENVIRONMENTAL SCIENCES↗

Energy-efficient Mott activation neuron for full-hardware implementation of neural networks

To circumvent the von Neumann bottleneck, substantial progress has been made towards in-memory computing with synaptic devices. However, compact nanodevices implementing non-linear activation functions are required for efficient full-hardware implementation of deep neural networks. Here, in this work, we present an energy-efficient and compact Mott activation neuron based on vanadium dioxide and its successful integration with a conductive bridge random access memory (CBRAM) crossbar array in hardware. The Mott activation neuron implements the rectified linear unit function in the analogue domain. The neuron devices consume substantially less energy and occupy two orders of magnitude smaller area than those of analogue complementary metal–oxide semiconductor implementations. The LeNet-5 network with Mott activation neurons achieves 98.38% accuracy on the MNIST dataset, close to the ideal software accuracy. We perform large-scale image edge detection using the Mott activation neurons integrated with a CBRAM crossbar array. Our findings provide a solution towards large-scale, highly parallel and energy-efficient in-memory computing systems for neural networks.

electrical and electronic engineering↗

Predictive machine learning approaches for the microstructural behavior of multiphase zirconium alloys

Abstract Zirconium alloys are widely used in harsh environments characterized by high temperatures, corrosivity, and radiation exposure. These alloys, which have a hexagonal closed packed (h.c.p.) structure thermo-mechanically degrade, when exposed to severe operating environments due to hydride formation. These hydrides have a different crystalline structure, than the matrix, which results in a multiphase alloy. To accurately model these materials at the relevant physical scale, it is necessary to fully characterize them based on a microstructural fingerprint, which is defined here as a combination of features that include hydride geometry, parent and hydride texture and crystalline structure of these multiphase alloys. Hence, this investigation will develop a reduced order modeling approach, where this microstructural fingerprint is used to predict critical fracture stress levels that are physically consistent with microstructural deformation and fracture modes. Machine Learning (ML) methodologies based on Gaussian Process Regression, random forests, and multilayer perceptrons (MLP) were used to predict material fracture critical stress states. MLPs, or neural networks, had the highest accuracy on held-out test sets across three predetermined strain levels of interest. Hydride orientation, grain orientation or texture, and hydride volume fraction had the greatest effect on critical fracture stress levels and had partial dependencies that were highly significant, and in comparison hydride length and hydride spacing have less effects on fracture stresses. Furthermore, these models were also used accurately predicted material response to nominal applied strains as a function of the microstructural fingerprint.

36 MATERIALS SCIENCE↗

Risk-Aware Measurement Synchronization and Recovery for DSSE With Heterogeneous Data Sources

Power distribution systems are increasingly integrating heterogeneous sensors with varying data reporting rates and types, which pose challenges to achieving observability at the desired temporal resolution of distribution system state estimation (DSSE). Multisensor failures caused by extreme events exacerbate these issues, introducing substantial uncertainties into DSSE. This article proposes a novel solution to these challenges by ensuring high-resolution system observability despite heterogeneous data sources and multisensor failures. First, a deep learning architecture combining long short-term memory (LSTM) and graph convolutional network (GCN) is employed to synchronize meters with different reporting rates, aiming to achieve system observability. A random-walk-model-based approach is introduced to generate pseudo-measurements while properly characterizing their uncertainties under multisensor failures. Finally, a disaster-risk-informed observability metric (RiOM) is defined to quantify the uncertainty associated with state estimation results. The proposed framework offers deeper insights into the system observability on the fly compared with conventional analysis. The effectiveness of the framework is demonstrated on an IEEE standard test case and a large-scale real-world distribution feeder in mid-Minnesota in the U.S.

97 MATHEMATICS AND COMPUTING↗

Wavelet and Deep-Learning-Based Approach for Generation System Problematic Parameters Identification and Calibration

Accurate models of generation systems are critical for maintaining reliable and secure grid operations. In this paper, a novel and systematic approach is proposed to identify and calibrate the generation system problematic parameters using continuous wavelet transform (CWT) and advanced deep-learning technology. The phasor measurement unit (PMU) data are used through “event playback” to check whether the parameter calibration is required, and if yes, a group of suspicious parameters will be identified as the primary problematic parameter candidates (PPCs). These primary PPCs are randomly perturbed to generate the event playback simulation data, which are used by the CWT and convolutional neural networks (CNNs) to further narrow down the primary PPCs into a smaller set of candidates. Then, the identified candidates are perturbed again to generate massive event playback simulation data for training a parameter calibration neural network. Here, we designed a multi-output neural network structure to find the mappings between the perturbed parameters and the simulation data using both CNN and long short-term memory (LSTM) models. Finally, the well-trained and tested CNN-LSTM model is used to estimate the accurate value of the suspicious parameters with actual PMU measurements. The proposed CNN-LSTM network can accurately and reliably estimate the generation-system problematic parameters, and has better performance when compared to other machine-learning methods, such as the multilayer perceptron network and the conditional variational autoencoder method. The accuracy and effectiveness of the proposed approach have been validated through simulation and real-world data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

HYPPO v1.0.0

HYPPO is a hyperparameter optimization (HPO) software that uses surrogate modeling and uncertainty quantification to provide reliable neural network architectures. Similar state-of-the-art HPO technologies do not use such features and mostly perform random searches to explore the parameter space, which can easily become computationally expensive. Using HYPPO, the user is able to reduce by an order of magnitude the number of evaluations necessary to identify the most optimal region in the hyperparameter space. Finally, using asynchronous nested parallelism, we are able to decrease by two orders of magnitude the amount of time required to complete the HPO process.

Dumont, Vincent↗

Zero-Power Analog Optical Processing

The motivation behind this research is the growing challenge of handling the massive amounts of data generated by modern imaging systems. Conventional digital image processing techniques are struggling to keep pace with the demands of high-resolution and high-speed imaging systems for remote sensing due to their high-power consumption and data storage requirements. We present a novel approach based on analog photonics to address this challenge. The proposed system utilizes a silicon-photonics-based image encoder positioned after image formation and initial optical-to-electrical conversion. The photonic encoder compresses image data using a passive disordered photonic structure to perform kernel-type random projections of the raw data. The compressed data is then processed by a back-end neural network, which reconstructs the original image with high fidelity (structural similarity exceeding 90%). Our proposed approach has the potential to compress images with ~ 1000X lower power consumption compared to digital approaches with data rates exceeding 1 terapixel/second.

97 MATHEMATICS AND COMPUTING↗

OPF-Learn: An Open-Source Framework for Creating Representative AC Optimal Power Flow Datasets: Preprint

Increasing levels of renewable generation motivate a growing interest in data-driven approaches for AC optimal power flow (AC OPF) to manage uncertainty. However, a lack of disciplined dataset creation and benchmarking prohibits useful comparison between approaches in the literature. To instigate confidence, models must be able to reliably predict solutions across a wide range of operating conditions. This paper develops the OPF-Learn package for Julia and Python which uses a computationally efficient approach to create representative datasets that span a wide spectrum of the AC OPF feasible region. Load profiles are uniformly sampled from a convex set that contains the AC OPF feasible set. For each infeasible point found, the convex set is reduced using infeasibility certificates, found by utilizing properties of a relaxed formulation. The framework is shown to generate datasets which are more representative of the entire feasible space versus traditional techniques seen in the literature, improving machine learning model performance.

dataset↗

Nonlinear power flow control for networked AC/DC microgrids

A method for designing feedforward and feedback controllers for integration of stochastic sources and loads into a nonlinear networked AC/DC microgrid system is provided. A reduced order model for general networked AC/DC microgrid systems is suitable for HSSPFC control design. A simple feedforward steady state solution is utilized for the feedforward controls block. Feedback control laws are provided for the energy storage systems. A HSSPFC controller design is implemented that incorporates energy storage systems that provides static and dynamic stability conditions for both the DC random stochastic input side and the AC random stochastic load side. Transient performance was investigated for the feedforward/feedback control case. Numerical simulations were performed and provided power and energy storage profile requirements for the networked AC/DC microgrid system overall performance. The HSSPFC design can be implemented in the Matlab/Simulink environment that is compatible with real time simulation/controllers.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗

Deep Learning Approaches to Surrogates for Solving the Diffusion Equation for Mechanistic Real-World Simulations

In many mechanistic medical, biological, physical, and engineered spatiotemporal dynamic models the numerical solution of partial differential equations (PDEs), especially for diffusion, fluid flow and mechanical relaxation, can make simulations impractically slow. Biological models of tissues and organs often require the simultaneous calculation of the spatial variation of concentration of dozens of diffusing chemical species. One clinical example where rapid calculation of a diffusing field is of use is the estimation of oxygen gradients in the retina, based on imaging of the retinal vasculature, to guide surgical interventions in diabetic retinopathy. Furthermore, the ability to predict blood perfusion and oxygenation may one day guide clinical interventions in diverse settings, i.e., from stent placement in treating heart disease to BOLD fMRI interpretation in evaluating cognitive function (Xie et al., 2019; Lee et al., 2020). Since the quasi-steady-state solutions required for fast-diffusing chemical species like oxygen are particularly computationally costly, we consider the use of a neural network to provide an approximate solution to the steady-state diffusion equation. Machine learning surrogates, neural networks trained to provide approximate solutions to such complicated numerical problems, can often provide speed-ups of several orders of magnitude compared to direct calculation. Surrogates of PDEs could enable use of larger and more detailed models than are possible with direct calculation and can make including such simulations in real-time or near-real time workflows practical. Creating a surrogate requires running the direct calculation tens of thousands of times to generate training data and then training the neural network, both of which are computationally expensive. Often the practical applications of such models require thousands to millions of replica simulations, for example for parameter identification and uncertainty quantification, each of which gains speed from surrogate use and rapidly recovers the up-front costs of surrogate generation. We use a Convolutional Neural Network to approximate the stationary solution to the diffusion equation in the case of two equal-diameter, circular, constant-value sources located at random positions in a two-dimensional square domain with absorbing boundary conditions. Such a configuration caricatures the chemical concentration field of a fast-diffusing species like oxygen in a tissue with two parallel blood vessels in a cross section perpendicular to the two blood vessels. To improve convergence during training, we apply a training approach that uses roll-back to reject stochastic changes to the network that increase the loss function. The trained neural network approximation is about 1000 times faster than the direct calculation for individual replicas. Because different applications will have different criteria for acceptable approximation accuracy, we discuss a variety of loss functions and accuracy estimators that can help select the best network for a particular application. We briefly discuss some of the issues we encountered with overfitting, mismapping of the field values and the geometrical conditions that lead to large absolute and relative errors in the approximate solution.

60 APPLIED LIFE SCIENCES↗