Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

DeepClone: Lightweight State Replication of Deep Learning Models for Data Parallel Training

Training modern deep neural network (DNN) models involves complex workflows triggered by model exploration, sensitivity analysis, explainability, etc. A key primitive in this context is the ability to clone a model training instance, i.e. “fork” the training process in a potentially different direction, which enables comparisons of different evolution paths using variations of training data and model parameters. However, in a quest improve the training throughput, a mix of data parallel, model parallel, pipeline parallel and layer-wise parallel approaches are making the problem of cloning highly complex. In this paper, we explore the problem of efficient cloning under such circumstances. To this end, we leverage several properties of data-parallel training and layer-wise parallelism to design DeepClone, a cloning approach based on augmenting the execution graph to gain direct access to tensors, which are then sharded and reconstructed asynchronously in order to minimize runtime overhead, standby duration, readiness duration. Compared with state-of-art approaches, DeepClone shows orders of magnitude improvement for several classes of DNN models.

data-parallel training↗
DOE OSTI2023

Development of a hybrid neural network and transfer learning model for optimized ICP-MS/MS operation

Correct function and calibration of instrumentation is a crucial assumption for any scientific experiment. One such instrument, tandem inductively coupled plasma mass spectrometer (ICP-MS/MS), has in-depth calibration settings that range across 30+ different parameters, making it difficult to determine optimal conditions without expertise and some degree of trial and error. Often, these settings are hand-tuned, a time-intensive process prone to local maxima and human error. While some automation is available, the automation also may favor local optimizations over a global optimum. In addition to these difficulties, day to day instrument variability can further complicate the calibration process. We propose a solution to this problem as a machine learning (ML) algorithm that learns how each parameter helps determine the calibration sensitivity across several elements, and re-weights parameters over time as instrument variability changes (e.g., a global neural network (NN) with a time-dependent transfer learning (TL) component). This model would be able to generate a surface of predicted calibration sensitivities and their respective parameters, and a simple multivariate algorithm would be able to pull out the optimum results with the settings associated with them. Here-in, we describe our initial findings in working towards this goal, including data extraction from historical files, exploratory data analysis, and some initial model building to better describe the data and the feasibility of our goal.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1990

Networks for image acquisition, processing and display

The human visual system comprises layers of networks which sample, process, and code images. Understanding these networks is a valuable means of understanding human vision and of designing autonomous vision systems based on network processing. Ames Research Center has an ongoing program to develop computational models of such networks. The models predict human performance in detection of targets and in discrimination of displayed information. In addition, the models are artificial vision systems sharing properties with biological vision that has been tuned by evolution for high performance. Properties include variable density sampling, noise immunity, multi-resolution coding, and fault-tolerance. The research stresses analysis of noise in visual networks, including sampling, photon, and processing unit noises. Specific accomplishments include: models of sampling array growth with variable density and irregularity comparable to that of the retinal cone mosaic; noise models of networks with signal-dependent and independent noise; models of network connection development for preserving spatial registration and interpolation; multi-resolution encoding models based on hexagonal arrays (HOP transform); and mathematical procedures for simplifying analysis of large networks.

Ahumada, Albert J., Jr.↗
DOE OSTI2023

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Advancements on Multi-Fidelity Random Fourier Neural Networks: Application to Hurricane Modeling for Wind Energy

Multi-fidelity approaches are emerging as effective strategies in computational science to handle otherwise intractable tasks like Uncertainty Quantification (UQ), training of Machine Learning (ML) models, and optimization, for expensive high-fidelity applications in which the amount of available simulations or data is limited. The main idea is simple: large datasets generated for low-fidelity approximations of the problem at hand are fused with a much sparser dataset for the target (high-fidelity) system. In this paper, we build on our recent success in designing random Fourier Neural Networks (rFNNs) [1] to target problems arising in wind energy applications and in particular problems of interest for hurricane modeling. In this context, data for the high-fidelity models are limited and lower fidelity alternatives are needed. In this work, we introduce a novel multi-fidelity training approach for our rFNNs and demonstrate its use on a simple verification problem and on a hurricane modeling problem in which high-fidelity data are generated via Large-Eddy Simulations (LES), while low-fidelity data are given by a mesoscale model. Initial results demonstrate how the multi-fidelity training approach can improve the quality of the resulting surrogate.

Fourier Neural Networks↗
DOE OSTI2022

Physics-informed machine learning modeling for predictive control using noisy data

Due to the occurrence of over-fitting at the learning phase, the modeling of chemical processes via artificial neural networks (ANN) by using corrupted data (i.e., noisy data) is an ongoing challenge. Therefore, this work investigates the effect of both Gaussian and non-Gaussian noise on the performance of process-structure based recurrent neural networks (RNN) models, which take the form of partially-connected RNN models in this work, that are used to approximate a class of multi-input-multi-outputs nonlinear systems. Furthermore, two different techniques, specifically Monte Carlo dropout and co-teaching, are utilized in the development of partially-connected RNN models. Here, these two techniques are employed to reduce the over-fitting in ANNs when noisy data is used in the training process and, hence, to improve the open-loop accuracy as well as the closed-loop performance under a Lyapunov-based model predictive controller (MPC). Aspen Plus Dynamics, a well-known high-fidelity process simulator, is used to simulate a large-scale chemical process application in order to demonstrate the anticipated improvements in both open-loop approximation and closed-loop controller performance in the presence of Gaussian and non-Gaussian noise in the data set using physics-informed RNNs.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2010

Using Agent Base Models to Optimize Large Scale Network for Large System Inventories

The aim of this paper is to use Agent Base Models (ABM) to optimize large scale network handling capabilities for large system inventories and to implement strategies for the purpose of reducing capital expenses. The models used in this paper either use computational algorithms or procedure implementations developed by Matlab to simulate agent based models in a principal programming language and mathematical theory using clusters, these clusters work as a high performance computational performance to run the program in parallel computational. In both cases, a model is defined as compilation of a set of structures and processes assumed to underlie the behavior of a network system.

Shameldin, Ramez Ahmed↗
DOE OSTI2021

AtomSets as a hierarchical transfer learning framework for small and large materials datasets

Abstract Predicting properties from a material’s composition or structure is of great interest for materials design. Deep learning has recently garnered considerable interest in materials predictive tasks with low model errors when dealing with large materials data. However, deep learning models suffer in the small data regime that is common in materials science. Here we develop the AtomSets framework, which utilizes universal compositional and structural descriptors extracted from pre-trained graph network deep learning models with standard multi-layer perceptrons to achieve consistently high model accuracy for both small compositional data (<400) and large structural data (>130,000). The AtomSets models show lower errors than the graph network models at small data limits and other non-deep-learning models at large data limits. They also transfer better in a simulated materials discovery process where the targeted materials have property values out of the training data limits. The models require minimal domain knowledge inputs and are free from feature engineering. The presented AtomSets model framework can potentially accelerate machine learning-assisted materials design and discovery with less data restriction.

Chen, Chi (ORCID:0000000180087043)↗
DOE OSTI2021

Predicting Metabolic Reaction Networks with Perturbation-Theory Machine Learning (PTML) Models

Background: Checking the connectivity (structure) of complex Metabolic Reaction Networks(MRNs) models proposed for new microorganisms with promising properties is an importantgoal for chemical biology. Objective: In principle, we can perform a hand-on checking (Manual Curation). However, this is achallenging task due to the high number of combinations of pairs of nodes (possible metabolic reactions). Results: The CPTML linear model obtained using the LDA algorithm is able to discriminate nodes(metabolites) with the correct assignation of reactions from incorrect nodes with values of accuracy,specificity, and sensitivity in the range of 85-100% in both training and external validation dataseries. Methods: In this work, we used Combinatorial Perturbation Theory and Machine Learning techniquesto seek a CPTML model for MRNs >40 organisms compiled by Barabasis’ group. First, wequantified the local structure of a very large set of nodes in each MRN using a new class of node indexcalled Markov linear indices fk. Next, we calculated CPT operators for 150000 combinationsof query and reference nodes of MRNs. Last, we used these CPT operators as inputs of differentML algorithms. Conclusion: Meanwhile, PTML models based on Bayesian network, J48-Decision Tree and RandomForest algorithms were identified as the three best non-linear models with accuracy greaterthan 97.5%. The present work opens the door to the study of MRNs of multiple organisms usingPTML models.

Pharmacology & Pharmacy↗
DOE OSTI2021

Prediction of electron density and pressure profile shapes on NSTX-U using neural networks

A new model for prediction of electron density and pressure profile shapes on NSTX and NSTX-U has been developed using neural networks. The model has been trained and tested on measured profiles from experimental discharges during the first operational campaign of NSTX-U. By projecting profiles onto empirically derived basis functions, the model is able to efficiently and accurately reproduce profile shapes. In order to project the performance of the model to upcoming NSTX-U operations, a large database of profiles from the operation of NSTX is used to test performance as a function of available data. The rapid execution time of the model is well suited to the planned applications, including optimization during scenario development activities, and real-time plasma control. Finally, a potential application of the model to real-time profile estimation is demonstrated.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Prediction of electron density and pressure profile shapes on NSTX-U using neural networks

A new model for prediction of electron density and pressure profile shapes on NSTX and NSTX-U has been developed using neural networks. The model has been trained and tested on measured profiles from experimental discharges during the first operational campaign of NSTX-U. By projecting profiles onto empirically derived basis functions, the model is able to efficiently and accurately reproduce profile shapes. In order to project the performance of the model to upcoming NSTX-U operations, a large database of profiles from the operation of NSTX is used to test performance as a function of available data. The rapid execution time of the model is well suited to the planned applications, including optimization during scenario development activities, and real-time plasma control. A potential application of the model to real-time profile estimation is demonstrated.

Boyer, Mark↗
DOE OSTI2020

Predictive Skill of Deep Learning Models Trained on Limited Sequence Data

In this report we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, especially variants of recurrent neural networks (RNNs) have been studied for influenza forecasting, and have achieved higher forecasting skill compared to conventional models such as ARIMA models. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block temporal convolutional networks and simple neural attentive meta-learner for epidemiological forecasting and test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control

Topological variations in power system is a common phenomenon and can impose significant challenges to traditional controllers of power system. Recent study revealed the strength of deep reinforcement learning (DRL) based approaches in power system preventive and corrective control. But topological variations are difficult to capture using classical fully connected neural network (FCN) model and has not been explicitly modeled in previous work. Hence, we develop a Graph Convolutional Network (GCN) based DRL framework to tackle topology changes in control design of power system. The GCN model exploits the graph structure of the power network and helps the DRL agent to embed the topology information during learning process. Our GCN based approach is evaluated using the IEEE-39 bus system and it outperforms the FCN-based DRL scheme in terms of training convergence and control performance considering grid topology changes.

Hossain, Ramij Raja↗
DOE OSTI2020

Truck platooning in the U.S. national road network: A system-level modeling approach

Truck platooning enables a group of trucks to move close together, which helps reduce truck fuel use and increase effective road capacity. In this paper, a system-level equilibrium model is developed to characterize spontaneous truck platooning with coexistence of non-platooning vehicles in a network, by explicitly accounting for the interlocking relationship among platoon formation time, truck fuel saving, and increase in effective road capacity. To equilibrate the relationships, an algorithm is proposed which involves a diagonalization approach and a bush based algorithm to solve decomposed subproblems. The condition of proportionality is imposed to obtain unique traffic flows for each class of vehicles on road links. In addition, a spatially constrained multivariate clustering technique is employed to construct origin/destination zones that are smaller than the coarse Freight Analysis Framework (FAF) zones, while maintaining reasonable computational burden for network traffic assignment. Model implementation in the U.S. shows that platooning could lead to 7.9% fuel saving among platoonable trucks in 2025 and a comparable increase in effective capacity of platoonable road links, which would account for 60% of rural interstate roads. The fuel saving and road capacity improvement translate into an annual cost reduction of $\$$868 million for the U.S. intercity trucking sector and reduced road infrastructure investment needs worth $\$$4.8 billion. Extensive sensitivity analysis further reveals that fuel saving of platoonable trucks increases with platoon size but decreases with inter-truck distance in a platoon. Fuel saving potential suggests that priority should be given to rural rather than urban roads in deploying platooning technologies. As we expected, greater market penetration of platooning technologies means higher fuel saving and greater increase in effective road capacity.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

A large-scale, agent-based simulation of metropolitan freight movements with passenger and freight market interactions

This study summarizes the first stage in the implementation of an agent-based freight modeling system that has a global representation of agents and detailed modeling of a large-scale transportation network. The model is used to evaluate the transportation and energy impacts of goods movement across urban and national scales. The framework is implemented within POLARIS, a C++-based Planning and Operations Language for Agent-based Regional Integrated Simulation, which consists of an activity-based modeling (ABM) and dynamic traffic assignment (DTA) system that has robust features for passenger travel. This platform provides a tool to model interactions among consumers, producers, and the transportation system. The main objective of this initial implementation is to implement a freight model within POLARIS following an agent-based paradigm with behavioral and simulation methods. This paper presents the initial framework and illustrates the application of the model. Building upon earlier works, a parcel location assignment algorithm for business establishments in the population is documented, along with a method for estimating establishment production and consumption volumes. In addition to population generation, other features of the model include push-pull supply chains, multimodal path choice, choice of transportation logistics node, and dynamic traffic assignment. A module with e-commerce supply and demand was also developed to analyze the effects of e-commerce delivery on last-mile energy use and congestion.

agent-based freight↗
DOE OSTI2024

Investigating the effects of precise mass measurements of Ru and Pd isotopes on machine learning mass modeling

Atomic masses are a foundational quantity in our understanding of nuclear structure, astrophysics, and fundamental symmetries. The longstanding goal of creating a predictive global model for the binding energy of a nucleus remains a significant challenge, however, and prompts the need for precise measurements of atomic masses to serve as anchor points for model developments. We present precise mass measurements of neutron-rich Ru and Pd isotopes performed at the Californium Rare Isotope Breeder Upgrade facility at Argonne National Laboratory using the Canadian Penning Trap mass spectrometer. The masses of 108 Ru, 110 Ru, and 116 Pd were measured to a relative mass precision $\delta$⁢$m/m$ ≈ 10 -8 via the phase-imaging ion-cyclotron-resonance technique, and represent an improvement of approximately an order of magnitude over previous measurements. Further, these mass data were used in conjunction with the physically interpretable machine learning (PIML) model, which uses a mixture density neural network to model mass excesses via a mixture of Gaussian distributions. The effects of our new mass data on a Bayesian-updating of a PIML model are presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Non-Idealities in Lab-Scale Kinetic Testing: A Theoretical Study of a Modular Temkin Reactor

The Temkin reactor can be applied for industrial relevant catalyst testing with unmodified catalyst particles. It was assumed in the literature that this reactor behaves as a cascade of continuously stirred tank reactors (CSTR). However, this assumption was based only on outlet gas composition or inert residence time distribution measurements. The present work theoretically investigates the catalytic CO2 methanation as a test case on different catalyst geometries, a sphere, and a ring, inside a single Temkin reaction chamber under isothermal conditions. Axial gas-phase species profiles from detailed computational fluid dynamics (CFD) are compared with a CSTR and 1D plug-flow reactor (PFR) model using a sophisticated microkinetic model. In addition, a 1D chemical reactor network (CRN) model was developed, and model parameters were adjusted based on the CFD simulations. Whereas the ideal reactor models overpredict the axial product concentrations, the CRN model results agree well with the CFD simulations, especially under low to medium flow rates. This study shows that complex flow patterns greatly influence species fields inside the Temkin reactor. Although residence time measurements suggest CSTR-like behavior, the reactive flow cannot be described by either a CSTR or PFR model but with the developed CRN model.

Wehinger, Gregor D. (ORCID:0000000217743391)↗
NASA NTRS2024

Nodal Modeling of Liquid Propellant Feed and Pressurization System

Nodal or network flow modeling plays an important role in the design and operation of the feed and pressurization system of a liquid rocket engine. Model development and execution time is relatively short for nodal codes in comparison to the Navier-Stokes based CFD codes. Nodal models also allow the inclusion of several components into one model to predict the behavior of a larger system. Unlike CFD models, the nodal models do not need very fine discretization of the flow field because they use empirical correlations to model fluid friction and heat transfer. This paper presents several applications of nodal modeling of liquid propellant feed and pressurization systems using the Generalized Fluid System Simulation Program (GFSSP), a nodal code developed at NASA/ Marshall Space Flight Center. GFSSP discretizes the flow field into nodes which are connected by branches. The mass and energy conservation equations and the equation of state are solved to calculate pressure, temperature, and resident mass at the nodes. The momentum equations are solved at the branches to calculate flow rates. Applications include a) tank pressurization by inert gas as well as autogenous pressurization by gaseous propellant, b) submerged pressurization by helium, c) self-pressurization due to boil-off of cryogenic propellant, d) chilldown of a transfer line of a cryogenic propellant feed system, and e) chilldown and filling of a cryogenic tank. Each of the above-mentioned models was verified and validated by comparing with test data.

Nodal Model↗