Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

Kinetics-Informed Neural Networks

Chemical kinetics and reaction engineering consists of the phenomenological framework for the disentanglement of reaction mechanisms, optimization of reaction performance and the rational design of chemical processes. Here, we utilize feed-forward artificial neural networks as basis functions to solve ordinary differential equations (ODEs) constrained by differential algebraic equations (DAEs) that describe microkinetic models (MKMs). We present an algebraic framework for the mathematical description and classification of reaction networks, types of elementary reaction, and chemical species. Under this framework, we demonstrate that the simultaneous training of neural nets and kinetic model parameters in a regularized multi-objective optimization setting leads to the solution of the inverse problem through the estimation of kinetic parameters from synthetic experimental data. We analyze a set of scenarios to establish the extent to which kinetic parameters can be retrieved from transient kinetic data, and assess the robustness of the methodology with respect to statistical noise. Furthermore, this approach to inverse kinetic ODEs can assist in the elucidation of reaction mechanisms based on transient data.

36 MATERIALS SCIENCE↗

A gradient-based deep neural network model for simulating multiphase flow in porous media

We report simulation of multiphase flow in porous media is crucial for the effective management of subsurface energy and environment-related activities. The numerical simulators used for modeling such processes rely on spatial and temporal discretization of the governing mass and energy balance partial-differential equations (PDEs) into algebraic systems via finite-difference/volume/element methods. These simulators usually require dedicated software development and maintenance, and suffer low efficiency from a runtime and memory standpoint for problems with multi-scale heterogeneity, coupled-physics processes or fluids with complex phase behavior. Therefore, developing cost-effective, data-driven models can become a practical choice, and in this work, we choose deep learning approaches as they can handle high dimensional data and accurately predict state variables with strong nonlinearity. In this paper, we describe a gradient-based deep neural network (GDNN) constrained by the physics related to multiphase flow in porous media. We tackle the nonlinearity of flow in porous media induced by rock heterogeneity, fluid properties, and fluid-rock interactions by decomposing the nonlinear PDEs into a dictionary of elementary differential operators. We use a combination of operators to handle rock spatial heterogeneity and fluid flow by advection. Since the augmented differential operators are inherently related to the physics of fluid flow, we treat them as first principles prior knowledge to regularize the GDNN training. We use the example of pressure management at geologic CO 2 storage sites, where CO 2 is injected in saline aquifers and brine is produced, and apply GDNN to construct a predictive model that is trained with physics-based simulation data and emulates the physics process. We demonstrate that GDNN can effectively predict the nonlinear patterns of subsurface responses, including the temporal and spatial evolution of the pressure and saturation plumes. We also successfully extend the GDNN to convolutional neural network (CNN), namely gradient-based CNN (GCNN), and validate its capability to improve the prediction accuracy. GDNN has great potential to tackle challenging problems that are governed by highly nonlinear physics and enable the development of data-driven models with higher fidelity.

42 ENGINEERING↗

Optical neural network system for pose determination of spinning satellites

An optical neural network architecture and algorithm based on a Hopfield optimization network are presented for multitarget tracking. This tracker utilizes a neuron for every possible target track, and a quadratic energy function of neural activities which is minimized using gradient descent neural evolution. The neural net tracker is demonstrated as part of a system for determining position and orientation (pose) of spinning satellites with respect to a robotic spacecraft. The input to the system is time sequence video from a single camera. Novelty detection and filtering are utilized to locate and segment novel regions from the input images. The neural net multitarget tracker determines the correspondences (or tracks) of the novel regions as a function of time, and hence the paths of object (satellite) parts. The path traced out by a given part or region is approximately elliptical in image space, and the position, shape and orientation of the ellipse are functions of the satellite geometry and its pose. Having a geometric model of the satellite, and the elliptical path of a part in image space, the three-dimensional pose of the satellite is determined. Digital simulation results using this algorithm are presented for various satellite poses and lighting conditions.

Lee, Andrew↗

HOLISMOKES IX. Neural network inference of strong-lens parameters and uncertainties from ground-based images

Modeling of strong gravitational lenses is a necessity for further applications in astrophysics and cosmology. With the large number of detections in current and upcoming surveys, such as the Rubin Legacy Survey of Space and Time (LSST), it is pertinent to investigate automated and fast analysis techniques beyond the traditional and time-consuming Markov chain Monte Carlo sampling methods. Building upon our (simple) convolutional neural network (CNN), we present here another CNN, specifically a residual neural network (ResNet), that predicts the five mass parameters of a singular isothermal ellipsoid (SIE) profile (lens center x and y, ellipticity e x and e y , Einstein radius θ E ) and the external shear (γ ext, 1 , γ ext, 2 ) from ground-based imaging data. In contrast to our previous CNN, this ResNet further predicts the 1σ uncertainty for each parameter. To train our network, we use our improved pipeline to simulate lens images using real images of galaxies from the Hyper Suprime-Cam Survey (HSC) and from the Hubble Ultra Deep Field as lens galaxies and background sources, respectively. We find very good recoveries overall for the SIE parameters, especially for the lens center in comparison to our previous CNN, while significant differences remain in predicting the external shear. From our multiple tests, it appears that most likely the low ground-based image resolution is the limiting factor in predicting the external shear. Given the run time of milli-seconds per system, our network is perfectly suited to quickly predict the next appearing image and time delays of lensed transients. Therefore, we use the network-predicted mass model to estimate these quantities and compare to those values obtained from our simulations. Unfortunately, the achieved precision allows only a first-order estimate of time delays on real lens systems and requires further refinement through follow-up modeling. Nonetheless, our ResNet is able to predict the SIE and shear parameter values in fractions of a second on a single CPU, meaning that we are able to efficiently process the huge amount of galaxy-scale lenses expected in the near future.

79 ASTRONOMY AND ASTROPHYSICS↗

Including Physics-Informed Atomization Constraints in Neural Networks for Reactive Chemistry

Machine learning interatomic potentials (MLIPs) have emerged as powerful tools for investigating atomistic systems with high accuracy and a relatively low computational cost. However, a common and unaddressed challenge with many current neural network (NN) MLIP models is their limited ability to accurately predict the relative energies of systems containing isolated or nearly isolated atoms, which appear in various reactive processes. To address this limitation, we present a mathematical technique for modifying any existing atom-centered NN architecture to account for the energies of isolated atoms. The result produces a consistent prediction of the atomization energy (AE) of a system using minimal constraints on the model. Using this technique, we build a model architecture that we call hierarchically interacting particle neural network (HIP-NN)-AE, an AE-constrained version of the HIP-NN, as well as ANI-AE, the AE-constrained version of the accurate NN engine for molecular energies (ANI). Our results demonstrate AE consistency of AE-constrained models, which drastically improves the AE predictions for the models. We compare the AE-constrained approach to unconstrained models as well as models from the literature in other scenarios, such as bond dissociation energies, bond dissociation pathways, and extensibility tests. These results show that the constraints improve the model performance in some of these tasks and do not negatively affect the performance on any tasks. The AE constraint approach thus offers a robust solution to the challenges posed by isolated atoms in energy prediction tasks.

74 ATOMIC AND MOLECULAR PHYSICS↗

Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks

One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs), especially with arbitrary initial data. We address this problem by taking advantage of recent advances in scientific machine learning and the spectral dynamically orthogonal (DO) and borthogonal (BO) methods for representing stochastic processes. The recently introduced DO/BO methods reduce the SPDE to solving a system of deterministic PDEs and a system of stochastic ordinary differential equations. Specifically, we propose two new physics-informed neural networks (PINNs) for solving time-dependent SPDEs, namely the neural network (NN)-DO/BO methods. The proposed methods incorporate the DO/BO constraints into the loss function (along with the modal decomposition of the SPDE) with an implicit form instead of generating explicit expressions for the temporal derivatives of the DO/BO modes. Hence, the NN-DO/BO methods can overcome some of the drawbacks of the original DO/BO methods. For example, we do not need the assumption that the covariance matrix of the random coefficients is invertible as in the original DO method, and we can remove the assumption of no eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO methods can be used to solve time-dependent stochastic inverse problems with the same formulation and same computational complexity as for forward problems. Furthermore, we demonstrate the capability of the proposed methods via several numerical examples, namely: (1) A linear stochastic advection equation with deterministic initial condition: we obtain good results with the proposed methods, while the original DO/BO methods cannot be applied directly in this case. (2) Long-time integration of the stochastic Burgers' equation: we show the good performance of NN-DO/BO methods, especially the effectiveness of the NN-BO approach for such problems with many eigenvalue crossings during the whole time evolution, while the original BO method fails. (3) Nonlinear reaction diffusion equation: we consider both the forward problem and the inverse problems, including very noisy initial point values, to investigate the flexibility of the NN-DO/BO methods in handling inverse and mixed type problems. Taken together, these simulation results demonstrate that the NN-DO/BO methods can be employed to effectively quantify uncertainty propagation in a wide range of physical problems, but future work should address the efficiency issue of PINNs for forward problems.

97 MATHEMATICS AND COMPUTING↗

Development of a Convolutional Neural Network Classifier for Data Starved Spectra - 20199

The Institute for Clean Energy Technology (ICET) at Mississippi State University is exploring the utility of machine learning in augmenting its mobile radiation surveying platforms, which are currently being developed as means to survey depleted uranium contaminated areas in support of remediation and decommissioning efforts. Mobile survey platforms provide a means to efficiently scan large areas of interest while reducing human exposure to radiation and other hazards. The survey platforms can also be used for scanning for any gamma emitting isotope in addition to depleted uranium. The spectral data that the platforms collect may be data starved with relatively low counts and poorly defined spectral features depending on the speed of the platforms and scintillation detector selection. Such data-starved spectra are difficult to use for isotope identification, requiring advanced knowledge of the possible radionuclides that could be present and environmental factors that could attenuate signals or introduce background noise. These factors in combination with the volume of survey data increases the time it takes to perform analysis of survey data when the source type is unknown. There are a number of algorithms in the field of machine learning that can be used to classify data that would be challenging and time-consuming for a human to identify. Supervised machine learning algorithms train models based on extensive amounts of human-labeled training data. Once sufficiently trained, these models can be used to quickly make high-fidelity predictions on new data. Convolutional neural networks are machine learning algorithms that excel in learning representations of 'shapes'. They do this by taking numerical input data and convolving them with spatial feature detectors referred to as filters. These filters are incrementally adjusted to reduce the prediction error on the data during the backpropagation step of training. Discussed in this paper is the development of a convolutional neural network classifier (CNNC) that can utilize spectral survey data for source discrimination and isotope identification. Bench-top laboratory experiments data using LaBr{sub 3}(Ce) scintillation detectors were used to train and evaluate the performance of the developed CNNC. The CNNC is capable of discriminating a variety of gamma emitting source types, differentiating different forms of uranium (depleted vs. natural), and estimating the amount of uranium for a known geometry. The discussed CNNC may be useful in scenarios where survey systems are deployed in situations where hazardous radioactive material maybe present, but the type is unknown. When used in remediation applications the CNNC can be used to screen-out false positives, helping reduce remediation costs. (authors)

07 ISOTOPE AND RADIATION SOURCES↗

Reduced‐Order Modeling of Energetic Materials Using Physics‐Aware Recurrent Convolutional Neural Networks in a Latent Space (LatentPARC)

Physics-aware deep learning (PADL) has gained popularity for use in spatiotemporal dynamics simulations, such as those in computational modeling of energetic materials (EM). We show that the challenge PADL methods face while learning complex field evolution problems can be simplified and accelerated by decoupling it into two tasks: learning complex geometric features in evolving fields and modeling dynamics over these features in a lower-dimensional feature space. We build upon our previous work on physics-aware recurrent convolutional neural networks (PARC). PARC embeds knowledge of underlying physics into its neural network architecture for more robust and accurate prediction of evolving physical fields. PARC was shown to effectively learn complex nonlinear features such as the formation of hotspots and coupled shock fronts in various initiation scenarios of EMs, as a function of microstructures, serving effectively as a microstructure-aware burn model. Here, we further accelerate PARC and reduce its computational cost by projecting the original dynamics onto a lower-dimensional invariant manifold, or “latent space.” The projected latent representation encodes the complex geometry of evolving fields (e.g., temperature and pressure) in a set of data-driven features. The reduced dimension of this latent space allows us to learn the dynamics during the initiation of EM with a lighter and more efficient model. We observe a significant decrease in training and inference time while maintaining results comparable to PARC at inference. This work takes steps towards enabling rapid prediction of EM thermomechanics at larger scales and characterization of EM structure–property–performance linkages at a full application scale.

Mathematics and Computing↗

A Neural Network Aero Design System for Advanced Turbo-Engines

An inverse design method calculates the blade shape that produces a prescribed input pressure distribution. By controlling this input pressure distribution the aerodynamic design objectives can easily be met. Because of the intrinsic relationship between pressure distribution and airfoil physical properties, a neural network can be trained to choose the optimal pressure distribution that would meet a set of physical requirements. The neural network technique works well not only as an interpolating device but also as an extrapolating device to achieve blade designs from a given database. Two validating test cases are discussed.

Sanz, Jose M.↗

Neural network modelling of thermal stratification in a solar DHW storage

In this study an artificial neural network (ANN) model is introduced for modelling the layer temperatures in a storage tank of a solar thermal system. The model is based on the measured data of a domestic hot water system. The temperatures distribution in the storage tank divided in 8 equal parts in vertical direction were calculated every 5 min using the average 5 min data of solar radiation, ambient temperature, mass flow rate of collector loop, load and the temperature of the layers in previous time steps. The introduced ANN model consists of two parts describing the load periods and the periods between the loads. The identified model gives acceptable results inside the training interval as the average deviation was 0.22 C during the training and 0.24 C during the validation. (author)

14 SOLAR ENERGY↗

Using Deep Neural Networks as Cost-Effective Surrogate Models for Super-Parameterized E3SM Radiative Transfer

Deep neural networks (DNNs) are implemented in Super-Parameterized Energy Exascale Earth System Model (SP-E3SM) to imitate the shortwave and longwave radiative transfer calculations. These DNNs were able to emulate the radiation parameters with an accuracy of 90–95% at a cost of 8–10 times cheaper than the original radiation parameterization. Here, a comparison of time-averaged radiative fluxes and the prognostic variables manifested qualitative and quantitative similarity between the DNN emulation and the original parameterization. It has also been found that the differences between the DNN emulation and the original parameterization are comparable to the internal variability of the original parameterization. Although the DNNs developed in this investigation emulate the radiation parameters for a specific set of initial conditions, the results justify the need of further research to generalize the use of DNNs for the emulations of full model radiation and other parameterization for seasonal predictions and climate simulations.

58 GEOSCIENCES↗

Advances in Neural Network Detection and Retrieval of Multilayer Clouds for CERES Using Multispectral Satellite Data

An artificial neural network (ANN) algorithm, employing several Aqua MODIS infrared channels, the retrieved total cloud visible optical depth, and vertical humidity profiles is trained to detect multilayer (ML) ice-over-water cloud systems as identified by matched CloudSat and CALIPSO (CC) data. The multilayer ANN, or MLANN, algorithm is also trained to retrieve the optical depth and the top and base heights of the upper-layer ice clouds in ML systems. The trained MLANN was applied to independent MODIS data resulting in a combined ML and single layer hit rate of 80% (77%) for nonpolar regions during the day (night). The results are more accurate than currently available methods and the previous version of the MLANN. Upper-layer cloud top and base heights are accurate to ±1.2 km and ±1.6 km, respectively, while the uncertainty in optical depth is ±0.457 and ±0.556 during day and night, respectively. Areas of further improvement and development are identified and will be addressed in future versions of the MLANN.

CERES↗

Development of a data-driven neural network model for electron thermal transport in NSTX

A data-driven electron thermal transport neural network (ETT-NN) model, trained on TRANSP interpretative analysis results of National Spherical Torus Experiment (NSTX), was developed to enable faster and more accurate ETT computation for spherical tokamaks (STs). The model incorporates both convolutional NNs and recurrent NNs, allowing it to simultaneously account for the spatial and temporal non-localities and multi-scale features of turbulent transport, which have been considered only in a limited manner in conventional models. The model was validated through interpretative analysis and predictive simulations using Tokamak Reactor Integrated Automated Suite for Simulation and Computation, demonstrating relatively high accuracy. Additionally, parameter scans were performed on test discharges known to exhibit specific turbulent modes, such as microtearing mode, trapped electron mode, kinetic ballooning mode, and electron temperature gradient mode. The scanning results revealed that the ETT-NN model exhibits the same trends as those observed in conventional gyrokinetic simulations or theories, while also capturing the global nature of turbulent transport, indicating that the data-driven model accurately reflects the underlying physical characteristics. Furthermore, due to the dimensionless nature of the model, we can feasibly expand its applicability by incorporating data from other devices and uncovering the characteristics of ETT in STs in the future.

NSTX↗

An a priori evaluation of a principal component and artificial neural network based combustion model in diesel engine conditions

A principal component analysis (PCA) and artificial neural network (ANN) based chemistry tabulation approach is presented. ANNs are used to map the thermochemical state onto a low-dimensional manifold consisting of five control variables that have been identified using PCA. Three canonical configurations are considered to train the PCA-ANN model: a series of homogeneous reactors, a nonpremixed flamelet, and a two-dimensional lifted flame. The performance of the model in predicting the thermochemical manifold of a spatially-developing turbulent jet flame in diesel engine thermochemical conditions is a priori evaluated using direct numerical simulation (DNS) data. The PCA-ANN approach is compared with a conventional tabulation approach (tabulation using ad hoc defined control variables and linear interpolation). The PCA-ANN model provides higher accuracy and requires several orders of magnitude less memory. Here, these observations indicate that the PCA-ANN model is superior for chemistry tabulation, especially for modelling complex chemistries that present multiple combustion modes as observed in diesel combustion. The performance of the PCA-ANN model is then compared to the optimal estimator, i.e. the conditional mean from the DNS. The results indicate that the PCA-ANN model gives high prediction accuracy, comparable to the optimal estimator, especially for major species and the thermophysical properties. Higher errors are observed for the minor species and reaction rate predictions when compared to the optimal estimator. It is shown that the prediction of minor species and reaction rates can be improved by using training data that exhibits a variation of parameters as observed in the turbulent flame. The output of the ANN is analysed to assess mass conservation. It is observed that the ANN incurs a mean absolute error of 0.05% in mass conservation. Furthermore, it is demonstrated that this error can be reduced by modifying the cost function of the ANN to penalise for deviation from mass conservation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Artificial Neural Networks to Predict Cognitive Impairment of Rodents Subjected to Space Radiation

INTRODUCTION We use artificial neural networks (ANNs) as an example machine learning (ML) tool to predict the cognitive performance impairment of rats induced by irradiation. The experimental data in the analyses is attentional set-shifting (ATSET) test scores from a rodent model exposed to ≤15 cGy of individual galactic cosmic radiation (GCR) ions: 4He, 28Si, or 56Fe, expected for a Lunar or Mars mission [1]. This work investigates rats at a subject-based level and uses applied dose and performance scores taken before irradiation to predict whether a rat will be impaired when irradiated. The results of this study are significant to crewed space missions as they support the potential of predicting an astronaut’s impairment in a specific task before spaceflight through the implementation of appropriately trained ML tools. METHODS Data used in this work are scores from the ATSET, a multi-stage constrained cognitive flexibility test [2]. Our computational model utilizes the number of attempts to reach the criterion to pass a stage as a behavioral performance measure for rats. We use the post-irradiation scores, generate thresholds from cumulative distribution plots of non-irradiated rats, and calculate the percent of irradiated rats whose scores fall below the threshold to infer how each radiation type/dose affects a population. Rats scoring above the threshold are labeled impaired while the others are non-impaired. We then employ ANNs as a typical ML technique, and use each subject’s individual scores taken before radiation along with the applied dose, to predict their personal susceptibility to cognitive impairment due to space radiation exposure. RESULTS AND CONCLUSION A significant finding is the exhibition of a dose-dependent increasing probability of impairment for 1 to 10 cGy of 28Si or 56Fe in the simple discrimination (SD) stage of the ATSET, and for 1 to 10 cGy of 56Fe in the compound discrimination (CD) stage. On a subject-based level, implementing ML classifiers such as ANNs identifies rats that have a higher tendency for impairment after GCR exposure [1]. The receiver operating characteristic (ROC) and the precision-recall (PR) curves of the ML models show a better prediction of impairment when 56Fe is the ion in question in both SD (Figure 1) and CD stages. They, however, do not depict impairment due to 4He in SD (Figure 1) and 28Si in CD, suggesting no dose-dependent impairment response in these cases. In this work, “good” prediction pertains to “better-than-random-chance”, due to the limited sample size and the high inter- and intra-individual variabilities in response to brain stimulation paradigms, as applicable to both animals and humans. More behavioral tests and biomarkers should be investigated on the same subjects, to be fed to the ML models to capture the agents responsible for performance alterations of some individuals versus others.

machine learning↗

Tuning Neural Network Models for Improved Prediction of Boundary Layer Transition

Boundary layer transition can strongly impact flight vehicle performance as it influences surface skin friction and aerodynamic heating, making accurate transition prediction a key to designing next generation aircraft. Artificial neural networks (ANNs) have shown promise toward predicting laminar-turbulent transition based on linear stability correlations. The computational efficiency of ANNs and the substantially reduced user involvement in relation to direct computations based on the linear stability theory (LST) makes them an attractive methodology for integrating the LST based correlations in computational fluid dynamics codes. Tollmien-Schlichting (TS) waves correspond to the dominant transition mechanism in 2D or weakly 3D subsonic boundary layers, such as those encountered in general aviation applications. Improvements to neural network model accuracy in predicting the amplification rates of TS instability waves have been investigated by leveraging recent machine learning developments in conjunction with surrogate optimization techniques and via suitable augmentation of the data used to train the networks. The optimized models trained on the modified dataset reduced the average transition location errors on different airfoils at several flow conditions by 51% of the original manually-tuned network’s errors on the same flow cases. The actual transition locations were derived from the Langley Stability and Transition Analysis Code (LASTRAC).

Machine Learning↗

Embedding hard physical constraints in neural network coarse-graining of three-dimensional turbulence

In recent years, deep learning approaches have shown much promise in modeling complex systems in the physical sciences. A major challenge in deep learning of partial differential equations is enforcing physical constraints and boundary conditions. In this work, we propose a general framework to directly embed the notion of an incompressible fluid into convolutional neural networks, and apply this to coarse-graining of turbulent flow. These physics-embedded neural networks leverage interpretable strategies from numerical methods and computational fluid dynamics to enforce physical laws and boundary conditions by taking advantage the mathematical properties of the underlying equations. Here, we demonstrate results on three-dimensional fully developed turbulence, showing that this technique drastically improves local conservation of mass, without sacrificing performance according to several other metrics characterizing the fluid flow.

97 MATHEMATICS AND COMPUTING↗

Design Sensitivity for a Subsonic Aircraft Predicted by Neural Network and Regression Models

A preliminary methodology was obtained for the design optimization of a subsonic aircraft by coupling NASA Langley Research Center s Flight Optimization System (FLOPS) with NASA Glenn Research Center s design optimization testbed (COMETBOARDS with regression and neural network analysis approximators). The aircraft modeled can carry 200 passengers at a cruise speed of Mach 0.85 over a range of 2500 n mi and can operate on standard 6000-ft takeoff and landing runways. The design simulation was extended to evaluate the optimal airframe and engine parameters for the subsonic aircraft to operate on nonstandard runways. Regression and neural network approximators were used to examine aircraft operation on runways ranging in length from 4500 to 7500 ft.

Hopkins, Dale A.↗