Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “networking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Multilevel-in-width training for deep neural network regression

A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. Here, we draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Modeling design and control problems involving neural network surrogates

Here, we consider nonlinear optimization problems that involve surrogate models represented by neural networks. We demonstrate first how to directly embed neural network evaluation into optimization models, highlight a difficulty with this approach that can prevent convergence, and then characterize stationarity of such models. We then present two alternative formulations of these problems in the specific case of feedforward neural networks with ReLU activation: as a mixed-integer optimization problem and as a mathematical program with complementarity constraints. For the latter formulation we prove that stationarity at a point for this problem corresponds to stationarity of the embedded formulation. Each of these formulations may be solved with state-of-the-art optimization methods, and we show how to obtain good initial feasible solutions for these methods. We compare our formulations on three practical applications arising in the design and control of combustion engines, in the generation of adversarial attacks on classifier networks, and in the determination of optimal flows in an oil well network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nonconvex regularization for sparse neural networks

Convex ℓ 1 regularization using an infinite dictionary of neurons has been suggested for constructing neural networks with desired approximation guarantees, but can be affected by an arbitrary amount of over-parametrization. This can lead to a loss of sparsity and result in networks with too many active neurons for the given data, in particular if the number of data samples is large. As a remedy, in this paper, a nonconvex regularization method is investigated in the context of shallow ReLU networks: We prove that in contrast to the convex approach, any resulting (locally optimal) network is finite even in the presence of infinite data (i.e., if the data distribution is known and the limiting case of infinite samples is considered). Moreover, here we show that approximation guarantees and existing bounds on the network size for finite data are maintained.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Prediction of local concentration fields in porous media with chemical reaction using a multi scale convolutional neural network

The study of solute transport in porous media is of interest in many chemical engineering systems. Some example applications include packed bed catalytic reactors, filtration devices, and batteries. The pore scale modeling of these systems is time consuming and may require large computing resources, for this reason computational fluid dynamics (CFD) simulations are not practical if a large number of simulations is required, like in multiscale modeling, where a model at a large scale calls for pore scale simulations. It has been shown that neural networks can be trained with a dataset of flow simulations and then predict fields orders of magnitude faster, and with less computational resources, in new domains. However, it is crucial to provide the neural network with an effective description of the domain and the undergoing operating conditions to be able to train models that generalize accurately in unseen samples. Therefore, research is needed to employ neural networks in new complex systems. The appropriate training of a network for predicting coupled flow and solute transport processes is an outstanding problem due to the complex interplay between geometry and operating conditions. In this work, we train a multi scale convolutional neural network (MSNet) with a diverse dataset of simulations of transport and chemical reaction in porous media to predict the local concentration fields in images of porous media. Our dataset contains a wide diversity of sphere pack arrangements under different operating conditions (Péclet and Reynolds numbers). Further, we train a robust model by employing different input descriptors that represent the medium and the different operating conditions of each system. Our trained model is able to provide nearly instantaneous predictions, compared to around twenty hours of the CFD workflow, with less than 3.5% error on new geometries and transport conditions. Thus the model could be easily integrated in a multiscale workflow where fast response is needed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Physics-guided neural networks with engineering domain knowledge for hybrid process modeling

As neural networks are more frequently used to solve problems in science and engineering, the methods used to incorporate scientific knowledge into these networks are becoming increasingly complex. Here, this work breaks down these complicated techniques into a set of basic strategies which can easily be applied to diverse situations. Several novel neural networks are built using the categories laid out in this work. These networks are tested on simulated data from a continuous stirred tank reactor (CSTR) model to evaluate the advantages provided by each network. The three points demonstrated in this work are: (1) architectural hybrid models can speed up convergence and reduce the amount of data necessary to train a model; (2) adding a physics-guided loss function can improve model generalization and make models more physically consistent; (3) using physics-guided initialization and transfer learning improves accuracy and speeds up convergence, but can harm generalizability if used incorrectly.

42 ENGINEERING↗
DOE OSTI2025

Optimizing the design and operation of water networks: Two decomposition approaches

We consider the design and operation of water networks simultaneously. Water network problems can be divided into two categories: the design problem and the operation problem. The design problem involves determining the appropriate pipe sizing and placements of pump stations, while the operation problem involves scheduling pump stations over multiple time periods to account for changes in supply and demand. Our focus is on networks that involve water co-produced with oil and gas. While solving the optimization formulation for such networks, we found that obtaining a primal (feasible) solution is more challenging than obtaining dual bounds using off-the-shelf mixed-integer nonlinear programming solvers. Therefore, we propose two methods to obtain good primal solutions. One method involves a decomposition framework that utilizes a convex reformulation, while the other is based on time decomposition. To test our proposed methods, we conduct computational experiments on a network derived from the PARETO case study.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Ensemble cure kinetics network (ECK-Net): A method to derive cure kinetics of thermosetting resin

This paper introduces an Ensemble Cure Kinetics Network (ECK-Net), a neural network (NN)–based framework for modeling the cure kinetics of thermosetting resins within a phenomenological context. ECK-Net replaces traditional analytic models, which require extensive chemical insight and multiple isothermal/non-isothermal experiments, with a data-driven surrogate that maps nonlinear relationships between temperature, degree of cure, and reaction rate from differential scanning calorimetry data. The proposed approach predicts input-dependent kinetic coefficients of a generalized nth-order reaction equation rather than reaction rates directly, enabling a single unified model to represent various epoxy systems without relying on iso-conversional analysis or predefined functional forms. To ensure robustness, multiple independently trained networks under different random initializations are blended through an ensemble strategy, effectively mitigating the stochastic variability inherent to neural networks. The framework is validated using experimental datasets from multiple resin systems, including aerospace-grade materials (Toray 3900-2, Cycom 5320-1, and Hexcel 8552) and a windmill-grade resin (RIMR 035c). The model accurately reproduces the temporal evolution of the degree of cure under manufacturers’ recommended cure cycles across all tested resins systems, yielding Pearson’s correlation coefficients of 0.992, 0.994, 0.993, 0.997, respectively. To demonstrate process-level applicability, the trained network was implemented within the Abaqus environment to simulate out-of-autoclave (OOA) curing process of the CFRP panel composed of Toray T830H-6K/3900-2D prepreg. The simulation results showed excellent agreement with experimental temperature response (maximum peak temperature, simulation: 189.6 °C, experiment: 188.5 °C) and the final degree of cure (simulation: 0.948, experiment: 0.960 ± 0.013), confirming ECK-Net’s capability as a reliable alternative to conventional cure kinetics modeling methods.

Composite curing↗
DOE OSTI2025

Networked Microgrid Topology Reconfiguration to Promote Fairness in Proactive Load Shedding

Increasing occurrences of natural disasters and grid emergency events consistently challenge the safe and reliable operations of power systems. During such emergency situations, system operators may proactively shed load to mitigate risks. However, uncoordinated implementation of load shedding may disrupt electricity supply and even lead to cascading failures. Meanwhile, it is crucial to address potential biases affecting different customers when executing load shedding. This paper addresses the dynamic topology reconfiguration problem for networked microgrids with distributed energy resources under emergency conditions. Specifically, we propose a novel rolling-horizon optimization model that integrates fairness-aware constraints into the networked microgrid topology reconfiguration. Unlike existing approaches that focus solely on efficiency or apply fairness considerations in static settings, our method explicitly incorporates temporal fairness constraints to restrict repeated or excessive load curtailment for load blocks. Moreover, the fairness-aware constraints are specifically developed for the context of dynamic networked microgrid topology reconfiguration, and are designed to be convex or amenable to linear reformulations, which offers a more tractable alternative to traditional models with non-convex formulations. Numerical studies on a modified IEEE 13-bus system and a larger-sized SMART-DS networked microgrid system demonstrate the performance of the proposed algorithm towards more fairness-aware networked microgrid topology reconfiguration decision-making.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Adaptive Activation Functions Accelerate Convergence in Deep and Physics-informed Neural Networks

We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima.

machine leaning, Bad minima, Inverse problems, Phy↗
DOE OSTI2022

Explicitly antisymmetrized neural network layers for variational Monte Carlo simulation

The combination of neural networks and quantum Monte Carlo methods has arisen as a promising path forward for highly accurate electronic structure calculations. Previous proposals have combined equivariant neural network layers with a final antisymmetric layer in order to satisfy the antisymmetry requirements of the electronic wavefunction. However, to date it is unclear if one can represent antisymmetric functions of physical interest, and it is difficult to precisely measure the expressiveness of the antisymmetric layer. Here, this work attempts to address this problem by introducing explicitly antisymmetrized universal neural network layers. This approach has a computational cost which increases factorially with respect to the system size, but we are nonetheless able to apply it to small systems to better understand how the structure of the antisymmetric layer affects its performance. We first introduce a generic antisymmetric (GA) neural network layer, which we use to replace the entire antisymmetric layer of the highly accurate ansatz known as the FermiNet. We demonstrate that the resulting FermiNet-GA architecture can yield effectively the exact ground state energy for small atoms and molecules. We then consider a factorized antisymmetric (FA) layer which more directly generalizes the FermiNet by replacing the products of determinants with products of antisymmetrized neural networks. We find, interestingly, that the resulting FermiNet-FA architecture does not significantly outperform the FermiNet. This strongly suggests that the sum of products of antisymmetries is a key limiting aspect of the FermiNet architecture. To explore this further, we investigate a slight modification of the FermiNet, called the full determinant mode, which replaces each product of determinants with a single combined determinant. We find that the full single-determinant FermiNet closes a large part of the gap between the standard single-determinant FermiNet and FermiNet-GA on small atomic and molecular problems. Surprisingly, on the nitrogen molecule at a dissociating bond length of 4.0 Bohr, the full single-determinant FermiNet can outperform the largest standard FermiNet calculation with 64 determinants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A differentiable approach to the maximum independent set problem using dataless neural networks

The success of machine learning solutions for reasoning about discrete structures has brought attention to its adoption within combinatorial optimization algorithms. Such approaches generally rely on supervised learning by leveraging datasets of the combinatorial structures of interest drawn from some distribution of problem instances. Reinforcement learning has also been employed to find such structures. Here, in this paper, we propose a different approach in that no data is required for training the neural networks that produce the solution. In this sense, what we present is not a machine learning solution, but rather one that is dependent on neural networks and where backpropagation is applied to a loss function defined by the structure of the neural network architecture as opposed to a training dataset. In particular, we reduce the popular combinatorial optimization problem of finding a maximum independent set to a neural network and employ a dataless training scheme to refine the parameters of the network such that those parameters yield the structure of interest. Additionally, we propose a universal graph reduction procedure to handle large-scale graphs. The reduction exploits community detection for graph partitioning and is applicable to any graph type and/or density. Experimental results on both real and synthetic graphs demonstrate that our proposed method performs on par or outperforms state-of-the-art learning-based methods in terms of the size of the found set without requiring any training data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A scalable algorithm for the optimization of neural network architectures

In this work, we propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified by other hyperparameter search algorithms in terms of accuracy and computational cost. Numerical results performed on benchmark datasets show that, for these datasets, our method outperforms state-of-the-art hyperparameter optimization algorithms in terms of attainable predictive performance by the selected neural network architecture, and time-to-solution for the hyperparameter optimization to complete.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

With the increasing penetration of distributed energy resources in distribution networks, Volt-VAR control and optimization (VVC/VVO) have become very important to ensure an acceptable quality of service to all customers. System operators can rely on slow-responding utility devices, including capacitor banks and on-load tap changing transformers, along with fast-responding battery and photovoltaic (PV) inverters for the VVC/VVO implementation. Because of variations in response time of these two classes of devices, and different control actions (discrete versus continuous), coordinated and optimal scheduling and operation have become of utmost importance. Here, this paper develops a look-ahead deep reinforcement learning (DRL)-based multi-objective VVO technique to improve the voltage profile of active distribution networks, decrease network and inverter power loss, and save the operational cost of the grid. It proposes a deep deterministic policy gradient (DDPG)-based approach to schedule the optimal reactive and/or active power set-points of fast-responding inverters, and a deep Q-network (DQN)-based DRL agent to schedule the discrete decisions variables of slow-responding assets. The reactive power output of PV and battery smart inverters are scheduled at 30-minute intervals and the capacitors’ commitment status is scheduled with several hour intervals. The proposed framework is validated on the modified IEEE 34-bus and 123-bus test cases with embedded PV and PV-plus-storage. To validate the efficacy of the proposed VVO, it is compared with several scenarios, including the base case without VVO, localized droop control of DERs, DDPG-only, and twin delayed DDPG (TD3) agent-based DRL techniques. The results justify the superior performance of the proposed method to improve the voltage profile, reduce network power loss, and minimize the look-ahead grid operational cost while minimizing the undesirable power losses in inverters as a result of power factor adjustments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Development of Steady-State and Dynamic Mass and Energy Constrained Neural Networks for Distributed Chemical Systems Using Noisy Transient Data

The paper presents the development of algorithms for mass and energy constrained neural network models that can exactly conserve the overall mass and energy of distributed chemical process systems, even though the noisy transient data used for optimal model training violate the same. In contrast to approximately satisfying mass and energy balance constraints of a system by soft penalization of objective function, algorithms have been developed for solving equality-constrained nonlinear optimization problems, thus providing the guarantee of exactly satisfying the system mass and energy conservation laws. For developing dynamic mass-energy constrained network models for distributed systems, hybrid series and parallel dynamic-static neural networks have been leveraged. The developed algorithms for solving both the training and forward problems are validated using both steady-state and dynamic data in the presence of various noise characteristics. The developed data-driven algorithms are flexible to exactly satisfy mass and energy balance constraints for dynamic chemical processes if the system holdup information is available. The proposed network structures and algorithms are applied to the development of data-driven lumped and distributed models of an adiabatic superheater/reheater system, a nonisothermal continuous stirred tank reactor, as well as an electrically heated plug-flow reactor system where one form of energy gets transformed to another. It has been observed that the mass-energy constrained neural networks yield a root mean squared error of <1% with respect to the system truth for the case studies evaluated in this work.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Creation of Self-Semi-Interpenetrating Network Structures in PIM-1 Membranes for Enhanced Physical Aging Resistance

A series of self-semi-interpenetrating network (ssIPN) thin films based on PIM-1 structure were developed by end-cross-linking telechelic PIM-1 oligomers end-capped with curable carboxylic acid groups to form model networks, which are penetrated by linear high-molecular-weight PIM-1 chains. ssIPN films with systematically varied network content ranging from 10 to 30 wt % were comprehensively examined on their microstructure and gas permeation properties. Fresh PIM-1 ssIPN films exhibited gas separation performances close to those of as-cast linear PIM-1, where the films closely followed the upper bound trade-off line, gaining as much selectivity as they lose gas permeability as the network content increases. This indicates that gas permeability is largely preserved despite cross-linking. Wide-angle X-ray scattering supported this observation, with peaks shifting toward lower d-spacing as oligomer content increased, suggesting tighter chain packing with higher cross-linkable oligomer loading. In physical aging studies over two months, all ssIPN films outperform linear PIM-1 for H 2 /CH 4 and O 2 /N 2 separations. Notably, the PIM-1 ssIPN with 10 wt % network content showed the best physical aging resistance, with negligible permeability loss even after one month. Finally, these findings highlight a promising macromolecular strategy for enhancing the physical aging resistance of microporous polymer membranes for gas separation.

PIM-1↗
DOE OSTI2025

Degrees of Rate Control in Interconnected Reaction Networks

Overall reactions in interconnected networks exhibit net, forward, and reverse rates that are governed by both constitutive elementary steps in the pathway of interest and branching elementary steps that lead to alternative products. Accordingly, steps in branching pathways exhibit negative net, forward, and reverse degrees of rate control, as they reduce reaction flux to the desired product. We here contextualize the forward and reverse degrees of rate control in terms of kinetic resistances (inverse of rates) and leverage the additive nature of kinetic resistance to decouple kinetic driving forces contributed by constitutive elementary steps and branching points (nodal species) in interconnected networks. Regardless of the network connectivity, forward and reverse degrees of rate control are shown to converge at equilibrium. Away from equilibrium, we identify two critical features of interconnected networks: stoichiometric regularity─condition where all stoichiometric numbers are unity─and pathway symmetry around nodal species─condition where branching pathways share the same rate constants, stoichiometry, and species concentrations/activities─that result in (i) equal forward, reverse, and consequently net degrees of rate control and (ii) forward and reverse degrees of rate control that exhibit constant offsets, respectively, across all extents of reaction. Furthermore, our discourse further provides a mathematical description for the influence of stoichiometric irregularity and pathway asymmetry on forward and reverse degrees of rate control. Altogether, the presented work details the effects of network (inter)connectivity and stoichiometry on reaction kinetics and, in doing so, establishes general protocols for capturing these effects as additive terms in the formulation of forward and reverse degrees of rate control.

10 SYNTHETIC FUELS↗
DOE OSTI2022

Interplay of River and Tidal Forcings Promotes Loops in Coastal Channel Networks

We report the dense populations that inhabit global coastlines have an uncertain future due to increased flooding, storms, and human modification. The channel networks of deltas and marshes that plumb these coastlines present diverse architectures, including well-studied dendritic topologies. However, the quasi-stable loops that exist in nearly all coastal networks have not yet been explained. We present a model for self-organizing networks inspired by vascular biophysics to show that loops emerge when the relative forcings between rivers and tides are comparable, resulting in interplay between hydrodynamic forcings at short time scales relative to network evolution. Using field data and satellite imaging, we confirm this control on 21 field networks. Our comparison provides compelling evidence that hydrodynamic fluctuations are capable of stabilizing loops in geophysical systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Reactive Transport Modeling of Mineral Precipitation and Carbon Trapping in Discrete Fracture Networks

In this study we use numerical experiments to analyze reactive flow and transport behavior in discrete intersecting fracture networks, focusing on (a) how reaction-induced changes in physical and chemical properties affect flow connectivity and (b) how fracture networks developed in the Earth's critical zone contribute to carbon sequestration via mineral weathering reactions. In the first part of the study, we used two-dimensional reactive flow and transport simulations to analyze the impacts of mixing in a natural discrete fracture network. We concluded that reaction-induced changes can substantially alter the flow connectivity, especially at fracture intersections. The second set of simulations considered the problem of natural weathering of fractured mafic and ultramafic rocks in the partially saturated Earth's critical zone as a function of infiltration rates, fracture permeability, and partially saturated flow parameters. As a model system, we considered an incongruent reaction network with dissolution of forsterite and precipitation of magnesite. The behavior is complex in terms of the rate-controlling processes because of the multicomponent nature of the system as shown by the grid Peclet number: the CO 2 behavior is gas diffusion-controlled in the partially saturated zone, while the rate of water flow via the Damkӧhler number controls Mg 2+ transport through the fracture network. The amounts of carbon that can be trapped are modest, but the naturally fractured domain considered here provides a useful “base case” against which various engineered solutions can be compared.

58 GEOSCIENCES↗