Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network convergence”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Distributed Finite-Time Termination for Consensus Algorithm in Switching Topologies

Here, in this article, we present a finite-time stopping criterion for consensus algorithms in networks with dynamic communication topology. Prior state of the art has established convergence to the consensus value; however, the asymptotic convergence of these algorithms poses a challenge in practical settings where the response from agents is required in finite time. To this end, we propose a maximum-minimum protocol that propagates the global maximum and minimum values of agent states (while running the consensus algorithm) in the network. This article focuses on establishing that the global maximum and minimum values are strictly monotonic even for a dynamic topology, and they can be used to distributively ascertain the closeness to convergence in finite time. We rigorously show that each node can have access to the global maximum and minimum by running the proposed maximum-minimum protocol to realize a finite-time stopping criterion for the otherwise asymptotic consensus algorithm. The practical utility of the algorithm is illustrated through experiments where each agent is instantiated by a NodeJS socket.io server.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1995

Locomotion training of legged robots using hybrid machine learning techniques

In this study artificial neural networks and fuzzy logic are used to control the jumping behavior of a three-link uniped robot. The biped locomotion control problem is an increment of the uniped locomotion control. Study of legged locomotion dynamics indicates that a hierarchical controller is required to control the behavior of a legged robot. A structured control strategy is suggested which includes navigator, motion planner, biped coordinator and uniped controllers. A three-link uniped robot simulation is developed to be used as the plant. Neurocontrollers were trained both online and offline. In the case of on-line training, a reinforcement learning technique was used to train the neurocontroller to make the robot jump to a specified height. After several hundred iterations of training, the plant output achieved an accuracy of 7.4%. However, when jump distance and body angular momentum were also included in the control objectives, training time became impractically long. In the case of off-line training, a three-layered backpropagation (BP) network was first used with three inputs, three outputs and 15 to 40 hidden nodes. Pre-generated data were presented to the network with a learning rate as low as 0.003 in order to reach convergence. The low learning rate required for convergence resulted in a very slow training process which took weeks to learn 460 examples. After training, performance of the neurocontroller was rather poor. Consequently, the BP network was replaced by a Cerebeller Model Articulation Controller (CMAC) network. Subsequent experiments described in this document show that the CMAC network is more suitable to the solution of uniped locomotion control problems in terms of both learning efficiency and performance. A new approach is introduced in this report, viz., a self-organizing multiagent cerebeller model for fuzzy-neural control of uniped locomotion is suggested to improve training efficiency. This is currently being evaluated for a possible patent by NASA, Johnson Space Center. An alternative modular approach is also developed which uses separate controllers for each stage of the running stride. A self-organizing fuzzy-neural controller controls the height, distance and angular momentum of the stride. A CMAC-based controller controls the movement of the leg from the time the foot leaves the ground to the time of landing. Because the leg joints are controlled at each time step during flight, movement is smooth and obstacles can be avoided. Initial results indicate that this approach can yield fast, accurate results.

Simon, William E.↗
DOE OSTI2022

Modeling design and control problems involving neural network surrogates

Here, we consider nonlinear optimization problems that involve surrogate models represented by neural networks. We demonstrate first how to directly embed neural network evaluation into optimization models, highlight a difficulty with this approach that can prevent convergence, and then characterize stationarity of such models. We then present two alternative formulations of these problems in the specific case of feedforward neural networks with ReLU activation: as a mixed-integer optimization problem and as a mathematical program with complementarity constraints. For the latter formulation we prove that stationarity at a point for this problem corresponds to stationarity of the embedded formulation. Each of these formulations may be solved with state-of-the-art optimization methods, and we show how to obtain good initial feasible solutions for these methods. We compare our formulations on three practical applications arising in the design and control of combustion engines, in the generation of adversarial attacks on classifier networks, and in the determination of optimal flows in an oil well network.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1996

Unipolar Terminal-Attractor Based Neural Associative Memory with Adaptive Threshold

A unipolar terminal-attractor based neural associative memory (TABAM) system with adaptive threshold for perfect convergence is presented. By adaptively setting the threshold values for the dynamic iteration for the unipolar binary neuron states with terminal-attractors for the purpose of reducing the spurious states in a Hopfield neural network for associative memory and using the inner-product approach, perfect convergence and correct retrieval is achieved. Simulation is completed with a small number of stored states (M) and a small number of neurons (N) but a large M/N ratio. An experiment with optical exclusive-OR logic operation using LCTV SLMs shows the feasibility of optoelectronic implementation of the models. A complete inner-product TABAM is implemented using a PC for calculation of adaptive threshold values to achieve a unipolar TABAM (UIT) in the case where there is no crosstalk, and a crosstalk model (CRIT) in the case where crosstalk corrupts the desired state.

Liu, Hua-Kuang↗
DOE OSTI2021

Deep learning simulations of the microwave sky

Here we present 500 high-resolution, full-sky millimeter-wave deep learning (DL) simulations that include lensed CMB maps and correlated foreground components. We find that these MillimeterDL simulations can reproduce a wide range of non-Gaussian summary statistics matching the input training simulations, while only being optimized to match the power spectra. The procedure we develop in this work enables the capability to mass produce independent full-sky realizations from a single expensive full-sky simulation, when ordinarily the latter would not provide enough training data. We also circumvent a common limitation of high-resolution DL simulations that they be confined to small sky areas, often due to memory or GPU issues; we do this by developing a “stitching” procedure that can recover the large-scale, high-order statistics and avoid discontinuities or repeated features in the maps. In addition, since our network takes as input a full-sky lensing convergence map, it can in principle take a full-sky lensing convergence map from any large-scale structure (LSS) simulation and generate the corresponding lensed CMB and correlated foreground components at millimeter wavelengths; this is especially useful in the current era of combining results from both CMB and LSS surveys, which require a common set of simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Phenotypically anchored transcriptomics across diverse agrichemicals reveals conserved pathways and unique gene expression signatures in zebrafish

Agrichemicals such as herbicides, fungicides, insecticides, and biocides are widely used in agriculture, yet some are associated with adverse effects in humans and the environment. While many of these chemicals have been extensively studied in vitro and are included in the EPA’s ToxCast program, comprehensive in vivo comparisons using RNA sequencing across structurally diverse agrichemicals, in a single screening platform, are lacking. In this study, we examined structurally diverse agrichemicals found in the U.S. Environmental Protection Agency’s (EPA) Toxcast Phase I and II library by statically exposing early life stage zebrafish at 6 h post fertilization (hpf) until 120 hpf at concentrations ranging from 0.25 to 100 µM. Morphological outcomes were assessed at 120 hpf across 10 endpoints, including yolk sac edema, craniofacial malformations, and axis abnormalities. Chemicals that produced robust concentration-response relationships were selected for transcriptomic profiling. For transcriptomic analysis, zebrafish were statically exposed to each chemical and sampled at 48 hpf, prior to the onset of morphological effects observed at 120 hpf. Differential expression analysis identified between 0 and 4,538 differentially expressed genes (DEGs) per chemical, with no clear correlation to morphological severity. Both DEG and co-expression network analyses revealed chemical-specific expression patterns that converged on shared biological pathways, including neurodevelopment and cytoskeletal organization. Key regulatory genes such as mylpfa and krt4 were identified within co-expression modules, suggesting their potential role in conserved toxicity mechanisms. Semantic similarity analysis of enriched gene ontology (GO) terms, when compared to existing datasets, highlighted gaps in the annotation of neurodevelopmental processes, indicating that some in vivo effects may not be fully captured by current curated resources. The results provide new insights into the modes of action of diverse agrichemicals and establish a framework for understanding how agrichemical structure relates to biological function in a vertebrate model.

agrichemical↗
DOE OSTI2021

Stepwise Dynamic Calibration of a Hydromechanical Simulation Using Time-Lapse Vertical Seismic Profile

This study aims to develop a methodology for calibrating subsurface stress changes through time-lapse Vertical Seismic Profiling (VSP) integration. The selected study site is the 13-10A injector well within the ongoing CO2-EOR operation of the Farnsworth Field Unit. The Time-lapse VSP dataset carries the combined effects of fluid substitution and mean effective stress changes, thereby providing a dataset amenable for the calibration of production and injection-induced stress changes. The concept is similar to calibrating a reservoir simulation model in that the process honor real field data to set up an inverse problem. The solution optimizes the independent and impactful geomechanical parameters that replicate the observed time-lapse seismic velocity changes. This stress calibration is enabled by 4D geomechanical modeling and the VSP Integration workflow. This calibration benefits from extensive geological, geophysical and geomechanical characterization through 3D seismic data, geophysical well logs, and core assessed as part of the 1D MEM conducted on the 13-10A subject well. These data are used to develop a site-specific rock physics model. The Biot Gassmann workflow combines rock physics and reservoir simulation outputs to determine the fluid substitution contribution to seismic velocity change. Additionally, modeled seismic velocity attributed to mean effective stress are determined from the geomechanical simulation outputs, and the stress-velocity relationship developed from the ultrasonic seismic velocity measurements on the extracted Morrow B core. A penalty function is then formed between the modeled seismic velocities and the observed time-lapse VSP dataset. Four independent and impactful geomechanical parameters have been determined. These are the bulk modulus and shear modulus for zero porosity and the shear and compressional seismic velocity to mean effective stress derivatives. The dataset of numerous coupled hydromechanical- geomechanical simulation realizations is built by combining variations of the four stated geomechanical parameters. A machine learning-assisted workflow comprised of an artificial neural network and a particle swarm optimizer are used to converge on the optimal geomechanical parameters. The successful execution of this workflow has affirmed the suitability of acoustic time-lapse measurements for 4D-VSP geomechanical stress calibration pending measurable stress sensitivities within the anticipated effective stress changes and the availability of suitable and reliable datasets for petroelastic modeling.

02 PETROLEUM↗
DOE OSTI2024

Optimization strategies for produced water networks with integrated desalination facilities

Optimal management and desalination of produced water is a major challenge for U.S. oil and gas development. Integrating rigorous desalination models into multi-period produced water network optimization problems presents several hurdles, which need to be tackled using advanced optimization strategies. Here, in this work, a novel multi-period produced water network formulation with separate solid and liquid flows is introduced to avoid singularities at zero flows. Rigorous steady state desalination models based on mechanical vapor recompression are embedded at the desalination sites in the network model. An integrated optimization formulation is developed to co-optimize the design of desalination units along with the operation of the network. Furthermore, a more robust approach based on the trust region filter method is developed to efficiently integrate complex desalination models into the multi-period planning problem. Both optimization approaches are demonstrated on a produced water network from the PARETO library (Drouven et al., 2022) using thermal desalination units. Our results show that while the TRF and integrated approaches have comparable solve times, the TRF approach has better performance reliability in terms of solver convergence. Furthermore, the optimal solution obtained by embedding rigorous models into the network is significantly different than when desalination costs are approximated using simple cost models, which motivates further research in this field.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1988

Orthogonal patterns in binary neural networks

A binary neural network that stores only mutually orthogonal patterns is shown to converge, when probed by any pattern, to a pattern in the memory space, i.e., the space spanned by the stored patterns. The latter are shown to be the only members of the memory space under a certain coding condition, which allows maximum storage of M=(2N) sup 0.5 patterns, where N is the number of neurons. The stored patterns are shown to have basins of attraction of radius N/(2M), within which errors are corrected with probability 1 in a single update cycle. When the probe falls outside these regions, the error correction capability can still be increased to 1 by repeatedly running the network with the same probe.

Baram, Yoram↗
NASA NTRS1994

A neural network controller for automated composite manufacturing

At McDonnell Douglas Aerospace (MDA), an artificial neural network based control system has been developed and implemented to control laser heating for the fiber placement composite manufacturing process. This neurocontroller learns an approximate inverse model of the process on-line to provide performance that improves with experience and exceeds that of conventional feedback control techniques. When untrained, the control system behaves as a proportional plus integral (PI) controller. However after learning from experience, the neural network feedforward control module provides control signals that greatly improve temperature tracking performance. Faster convergence to new temperature set points and reduced temperature deviation due to changing feed rate have been demonstrated on the machine. A Cerebellar Model Articulation Controller (CMAC) network is used for inverse modeling because of its rapid learning performance. This control system is implemented in an IBM compatible 386 PC with an A/D board interface to the machine.

Lichtenwalner, Peter F.↗
DOE OSTI2022

A backward SDE method for uncertainty quantification in deep learning

Here, we develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Stochastic Gradient-Based Distributed Bayesian Estimation in Cooperative Sensor Networks

Distributed Bayesian inference provides a full quantification of uncertainty offering numerous advantages over point estimates that autonomous sensor networks are able to exploit. However, fully-decentralized Bayesian inference often requires large communication overheads and low network latency, resources that are not typically available in practical applications. In this paper, we propose a decentralized Bayesian inference approach based on stochastic gradient Langevin dynamics, which produces full posterior distributions at each of the nodes with significantly lower communication overhead. We provide analytical results on convergence of the proposed distributed algorithm to the centralized posterior, under typical network constraints. Finally, we also provide extensive simulation results to demonstrate the validity of the proposed approach.

42 ENGINEERING↗
DOE OSTI2023

Automated Adversary-in-the-Loop Cyber-Physical Defense Planning

Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this article presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion—by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP) that determines the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero-sum game between the adversary and defender. We use a model-free reinforcement learning method to solve the adversary’s problem as a function of the defense strategy. We then employ Bayesian optimization (BO) to find an approximate best-response for the defender to harden the network against the resulting adversary policy. This process is iterated multiple times to improve the strategy for both players. We demonstrate the effectiveness of our approach on a ransomware-inspired graph with a smart building system as the physical process. Numerical studies show that our method converges to a Nash equilibrium for various defender-specific costs of network hardening.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

AADL: Anderson Accelerated Deep Learning

We propose a stable, distributed approach to perform AA that accelerates the convergence rate of stochastic first-order optimizers to train neural networks. Differently from previous works, we do not alter neither the scheme to perform AA nor the loss function minimized during the training. To improve robustness against stagnation, we customize general guidelines that suggest to relax the frequency of AA corrections by performing AA only at the end of an entire training epoch. To improve robustness of AA against the stochastic oscillations of first-order optimizers, we average the gradients computed on consecutive stochastic optimization updates. The improved regularity of the converging sequence and the reduced amplitude of stochastic oscillations across consecutive optimization steps allows AA to efficiently extrapolate an improved converging sequence, thereby overcoming limitations of existing approaches to perform AA on stochastic optimization.

Lupo Pasini, Massimiliano [Oak Ridge National Lab.↗
DOE OSTI2024

Fast and converged classical simulations of evidence for the utility of quantum computing before fault tolerance

A recent quantum simulation of observables of the kicked Ising model on 127 qubits implemented circuits that exceed the capabilities of exact classical simulation. We show that several approximate classical methods, based on sparse Pauli dynamics and tensor network algorithms, can simulate these observables orders of magnitude faster than the quantum experiment and can also be systematically converged beyond the experimental accuracy. Our most accurate technique combines a mixed Schrödinger and Heisenberg tensor network representation with the Bethe free entropy relation of belief propagation to compute expectation values with an effective wave function–operator sandwich bond dimension >16,000,000, achieving an absolute accuracy, without extrapolation, in the observables of <0.01, which is converged for many practical purposes. We thereby identify inaccuracies in the experimental extrapolations and suggest how future experiments can be implemented to increase the classical hardness.

Science & Technology - Other Topics↗
NASA NTRS1991

Surface wind convergence as a short-term predictor of cloud-to-ground lightning at Kennedy Space Center: A four-year summary and evaluation

Since 1986, USAF forecasters at NASA-Kennedy have had available a surface wind convergence technique for use during periods of convective development. In Florida during the summer, most of the thunderstorm development is forced by boundary layer processes. The basic premise is that the life cycle of convection is reflected in the surface wind field beneath these storms. Therefore the monitoring of the local surface divergence and/or convergence fields can be used to determine timing, location, longevity, and the lightning hazards which accompany these thunderstorms. This study evaluates four years of monitoring thunderstorm development using surface wind convergence, particularly the average over the area. Cloud-to-ground (CG) lightning is related in time and space with surface convergence for 346 days during the summers of 1987 through 1990 over the expanded wind network at KSC. The relationships are subdivided according to low level wind flow and midlevel moisture patterns. Results show a one in three chance of CG lightning when a convergence event is identified. However, when there is no convergence, the chance of CG lightning is negligible.

Watson, Andrew I.↗
DOE OSTI2025

Synergistic learning with multi-task DeepONet for efficient PDE problem solving

Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Finally, our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.

42 ENGINEERING↗
DOE OSTI2024

Deep unsupervised learning using spike-timing-dependent plasticity

Abstract Spike-timing-dependent plasticity (STDP) is an unsupervised learning mechanism for spiking neural networks that has received significant attention from the neuromorphic hardware community. However, scaling such local learning techniques to deeper networks and large-scale tasks has remained elusive. In this work, we investigate a Deep-STDP framework where a rate-based convolutional network, that can be deployed in a neuromorphic setting, is trained in tandem with pseudo-labels generated by the STDP clustering process on the network outputs. We achieve 24.56% higher accuracy and 3.5 × faster convergence speed at iso-accuracy on a 10-class subset of the Tiny ImageNet dataset in contrast to a k -means clustering approach.

Lu, Sen↗