Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORK”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Efficient human activity recognition with spatio-temporal spiking neural networks

In this study, we explore Human Activity Recognition (HAR), a task that aims to predict individuals' daily activities utilizing time series data obtained from wearable sensors for health-related applications. Although recent research has predominantly employed end-to-end Artificial Neural Networks (ANNs) for feature extraction and classification in HAR, these approaches impose a substantial computational load on wearable devices and exhibit limitations in temporal feature extraction due to their activation functions. To address these challenges, we propose the application of Spiking Neural Networks (SNNs), an architecture inspired by the characteristics of biological neurons, to HAR tasks. SNNs accumulate input activation as presynaptic potential charges and generate a binary spike upon surpassing a predetermined threshold. This unique property facilitates spatio-temporal feature extraction and confers the advantage of low-power computation attributable to binary spikes. We conduct rigorous experiments on three distinct HAR datasets using SNNs, demonstrating that our approach attains competitive or superior performance relative to ANNs, while concurrently reducing energy consumption by up to 94%.

60 APPLIED LIFE SCIENCES↗

Ferroelectric Memory Capacitors For Neural Networks

Thin-film ferroelectric capacitors proposed as nonvolatile analog memory devices. Intended primarily for use as synaptic connections in electronic neural networks. Connection strengths (synaptic weights) stored as nonlinear remanent polarizations of ferroelectric films. Ferroelectric memory and interrogation capacitors combined into memory devices in vertical or lateral configurations. Photoconductive layer modulated by light provides variable resistance to alter bias signal applied to memory capacitor. Features include nondestructive readout, simplicity, and resistance to ionizing radiation. Interrogated without destroying stored analog data. Also amenable to very-large-scale integration. Allows use of ac coupling, eliminating errors caused by dc offsets in amplifier circuits of neural networks.

Thakoor, Sarita↗

Allosteric prediction via convolutional neural networks and protein structural and dynamical features

Allostery is the phenomenon whereby a binding event or covalent modification at one site in a protein modulates function at a distal site, thus changing a protein’s functional state. As such, it is a ubiquitous aspect of protein functional regulation. Computationally predicting allosteric states is important as part of the broader challenge of functional annotation, but it also has practical implications for drug development, as targeting an allosteric site often affords greater specificity compared with targeting an orthosteric site. This study introduces a machine learning approach to predict the allosteric functional state using the small G-protein KRas as the model system, due to its implication in many types of cancer and being well studied as a result with many x-ray crystallographic structures of KRas available with different mutations and ligands bound. Using structural and dynamical features that can be cast as images, namely interatomic distances, contact maps, covariance, and mutual information, supervised learning was performed using convolutional neural networks. Two pretrained convolutional neural network architectures, GoogLeNet and ResNet18, were fine-tuned to classify KRas into active or inactive states based on these features. Across training regimes, atomic contact maps emerged as the most effective structural feature, whereas linearized mutual information outperformed covariance in capturing dynamical correlations relevant to allostery. Models achieved significant validation accuracy, with atomic contact maps yielding up to 90% accuracy. In conclusion, the findings suggest that integrating global structural rearrangements and correlated motion patterns with deep learning can reliably predict protein allosteric states, offering a promising framework for understanding allosteric regulation and developing targeted therapeutics.

Rajeshwar T., Rajitha [Oak Ridge National Laborato↗

Emulation of the calculations of final r -process abundance patterns with a neural network

This work explores the construction of a fast emulator for the calculation of the final pattern of nucleosynthesis in the rapid neutron capture process (the r-process). An emulator is built using a feed-forward artificial neural network (ANN). We train the ANN with nuclear data and relative abundance patterns. We take as input the β-decay half-lives and the one-neutron separation energy of the nuclei in the rare-earth region. The output is the final isotopic abundance pattern. In this work, we focus on the nuclear data and abundance patterns in the rare-earth region to reduce the dimension of the input and output space. We show that the ANN can capture the effect of the changes in the nuclear physics inputs on the final r-process abundance pattern in the adopted astrophysical conditions. We employ the deep ensemble method to quantify the prediction uncertainty of the neural network emulator. The emulator achieves a speed-up by a factor of about 20 000 in obtaining a final abundance pattern in the rare-earth region. The emulator may be utilized in statistical analyses such as uncertainty quantification, inverse problems, and sensitivity analysis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble

In this work we explore the capability of physics-informed neural networks (PINNs) to discover multiple solutions. Many real-world phenomena governed by nonlinear differential equations (DEs), such as fluid flow, exhibit multiple solutions under the same conditions, yet capturing this solution multiplicity remains a significant challenge. A key difficulty lies in providing appropriate initial conditions or guesses, as widely used time-marching schemes and Newton’s method are highly sensitive to these choices when solving complex computational problems. While machine learning models, particularly PINNs, have shown promise in solving DEs, their ability to capture multiple solutions remains underexplored. In this work, we propose a simple and practical approach using PINNs to learn and discover multiple solutions. We first demonstrate that PINNs, when combined with random initialization and deep ensemble method—originally developed for uncertainty quantification—can effectively uncover multiple solutions to nonlinear ordinary and partial DEs. Although training large ensembles of PINNs may appear computationally demanding, this can be done efficiently using vectorization techniques supported by modern deep learning frameworks, allowing many networks to be trained simultaneously. Our approach highlights the critical role of initialization in shaping solution diversity, addressing an often-overlooked aspect of machine learning for scientific computing. Furthermore, we propose utilizing PINN-generated solutions as initial conditions or initial guesses for conventional numerical solvers to enhance accuracy and efficiency in capturing multiple solutions. Extensive numerical experiments, including the Allen–Cahn equation and cavity flow, where our approach successfully identifies both stable and unstable solutions, validate the effectiveness of our method. These findings establish a general and efficient framework for addressing solution multiplicity in nonlinear DEs.

97 MATHEMATICS AND COMPUTING↗

AN ARTIFICIAL NEURAL NETWORK EVALUATION OF TUBERCULOSIS USING GENETIC AND PHYSIOLOGICAL PATIENT DATA

When doctors see more cases of patients with tell-tale symptoms of a disease, it is hoped that they will be able to recognize an infection administer treatment appropriately, thereby speeding up recovery for sick patients. We hope that our studies can aid in the detection of tuberculosis by using a computer model called an artificial neural network. Our model looks at patients with and without tuberculosis (TB). The data that the neural network examined came from the following: patient' age, gender, place, of birth, blood type, Rhesus (Rh) factor, and genes of the human Leukocyte Antigens (HLA) system (9q34.1) present in the Major Histocompatibility Complex. With availability in genetic data and good research, we hope to give them an advantage in the detection of tuberculosis. We try to mimic the doctor's experience with a computer test, which will learn from patient data the factors that contribute to TB.

60 APPLIED LIFE SCIENCES↗

Frame invariant neural network closures for Kraichnan turbulence

Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in developing parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we put forth a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work opens up a possibility of connecting physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models.

42 ENGINEERING↗

Enabling complete multichannel nonadiabatic dynamics: A global representation of the two-channel coupled, 1,2 1 A and 1 3 A states of NH 3 using neural networks

Global coupled three-state two-channel potential energy and property/interaction (dipole and spin–orbit coupling) surfaces for the dissociation of NH 3 (Ã) into NH + H 2 and NH 2 + H are reported. The permutational invariant polynomial-neural network approach is used to simultaneously fit and diabatize the electronic Hamiltonian by fitting the energies, energy gradients, and derivative couplings of the two coupled lowest-lying singlet states as well as fitting the energy and energy gradients of the lowest-lying triplet state. The key issue in fitting property matrix elements in the diabatic basis is that the diabatic surfaces must be smooth, that is, the diabatization must remove spikes in the original adiabatic property surfaces attributable to the switch of electronic wavefunctions at the conical intersection seam. Here, we employ the fit potential energy matrix to transform properties in the adiabatic representation to a quasi-diabatic representation and remove the discontinuity near the conical intersection seam. The property matrix elements can then be fit with smooth neural network functions. The coupled potential energy surfaces along with the dipole and spin–orbit coupling surfaces will enable more accurate and complete treatment of optical transitions, as well as nonadiabatic internal conversion and intersystem crossing.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Message-passing neural networks for high-throughput polymer screening

Machine learning methods have shown promise in predicting molecular properties, and given sufficient training data, machine learning approaches can enable rapid high-throughput virtual screening of large libraries of compounds. Graph-based neural network architectures have emerged in recent years as the most successful approach for predictions based on molecular structure and have consistently achieved the best performance on benchmark quantum chemical datasets. However, these models have typically required optimized 3D structural information for the molecule to achieve the highest accuracy. These 3D geometries are costly to compute for high levels of theory, limiting the applicability and practicality of machine learning methods in high-throughput screening applications. In this study, we present a new database of candidate molecules for organic photovoltaic applications, comprising approximately 91,000 unique chemical structures. Compared to existing datasets, this dataset contains substantially larger molecules (up to 200 atoms) as well as extrapolated properties for long polymer chains. We show that message-passing neural networks trained with and without 3D structural information for these molecules achieve similar accuracy, comparable to state-of-the-art methods on existing benchmark datasets. Furthermore, these results therefore emphasize that for larger molecules with practical applications, near-optimal prediction results can be obtained without using optimized 3D geometry as an input. We further show that learned molecular representations can be leveraged to reduce the training data required to transfer predictions to a new density functional theory functional.

14 SOLAR ENERGY↗

“Understanding Robustness Lottery”: A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches

Deep learning approaches have provided state-of-the-art performance in many applications by relying on large and overparameterized neural networks. However, such networks are very brittle and are difficult to deploy on resource-limited platforms. Model pruning, i.e., reducing the size of the network, is a widely adopted strategy that can lead to a more robust and compact model. Many heuristics exist for model pruning, but our understanding of the pruning process remains limited due to the black-box nature of a neural network model. Empirical studies show that some heuristics improve performance whereas others can make models more brittle. Here, this work aims to shed light on how different pruning methods alter the network’s internal feature representation and the corresponding impact on model performance. To facilitate a comprehensive comparison and characterization of the high-dimensional model feature space, we introduce a visual geometric analysis of feature representations. We evaluated a set of critical geometric concepts decomposed from the commonly adopted classification loss and used them to design a visualization system to compare and highlight the impact of pruning on model performance and feature representation. The proposed tool provides an environment for an in-depth comparison of pruning methods and a comprehensive understanding of how the model responds to common data corruption. By leveraging the proposed visualization, machine learning researchers can reveal the similarities between pruning methods and redundancy in robustness evaluation benchmarks, obtain geometric insights about the differences between pruned models that achieve superior robustness performance, and identify samples that are robust or fragile to model pruning and common data corruption.

Li, Zhimin [Univ. of Utah, Salt Lake City, UT (Uni↗

An architecture for designing fuzzy logic controllers using neural networks

Described here is an architecture for designing fuzzy controllers through a hierarchical process of control rule acquisition and by using special classes of neural network learning techniques. A new method for learning to refine a fuzzy logic controller is introduced. A reinforcement learning technique is used in conjunction with a multi-layer neural network model of a fuzzy controller. The model learns by updating its prediction of the plant's behavior and is related to the Sutton's Temporal Difference (TD) method. The method proposed here has the advantage of using the control knowledge of an experienced operator and fine-tuning it through the process of learning. The approach is applied to a cart-pole balancing system.

Berenji, Hamid R.↗

Deep Neural Network Based Unsteady Flamelet Progress Variable Approach in a Supersonic Combustor

Higher dimensional flamelet manifolds are essential in capturing the coupled effects of pressure gradients and unsteady chemical kinetics observed in supersonic combustion applications. Previous studies have validated the feasibility of using deep neural networks as an alternative to computation-ally intensive multidimensional flamelet table storage and lookup. This approach has demonstrated a significant reduction in memory footprint and enabled the use of larger dimensional tabulated manifolds for supersonic combustion in canonical problems. In this study, the Unsteady Flamelet Progress Variable (UFPV)-ANN model implemented in the VULCAN-CFD code is validated by the Burrows-Kurkov supersonic mixing/combustion configuration. The well characterized experimental problem consists of hydrogen injection into a supersonic vitiated crossflow that results in a lifted flame structure. The initial model consists of a 4-dimensional table where the independent variables Z, C, Xst, P are tabulated using an unsteady flamelet code with boundary conditions corresponding to the vitiated air conditions. The results show the development of a lifted flame structure and over-all acceptable agreement with finite-rate chemistry (FRC) simulation and the experimental data. Moreover, direct mapping between the independent variables and the flamelet table is replaced by a deep neural network for significant memory reduction. The results indicate that the UFPV-ANN approach can retrieve the same solution as the memory intensive lookup table approach.

Flamelet↗

Understanding the surface wave characteristics using 2D particle-in-cell simulation and deep neural network

Here, the characteristics of the surface waves along the interface between a plasma and a dielectric material have been investigated using kinetic particle-in-cell simulations. A microwave source of GHz frequency has been used to trigger the surface wave in the system. The outcome indicates that the surface wave gets excited along the interface of plasma and the dielectric tube and appears as light and dark patterns in the electric field profiles. The dependency of radiation pressure on the dielectric permittivity and supplied input frequency has been investigated. Further, we assessed the capabilities of neural networks to predict the radiation pressure for a given system. The proposed deep neural network model is aimed at developing accurate and efficient data-driven plasma surface wave devices.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Respecting causality for training physics-informed neural networks

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date PINNs have not been successful in simulating dynamical systems whose solution exhibits multi-scale, chaotic or turbulent behavior. In this work we attribute this shortcoming to the inability of existing PINNs formulations to respect the spatio-temporal causal structure that is inherent to the evolution of physical systems. We argue that this is a fundamental limitation and a key source of error that can ultimately steer PINN models to converge towards erroneous solutions. We address this pathology by proposing a simple re-formulation of PINNs loss functions that can explicitly account for physical causality during model training. We demonstrate that this simple modification alone is enough to introduce significant accuracy improvements, as well as a practical quantitative mechanism for assessing the convergence of a PINNs model. Here we provide state-of-the-art numerical results across a series of benchmarks for which existing PINNs formulations fail, including the chaotic Lorenz system, the Kuramoto–Sivashinsky equation in the chaotic regime, and the Navier–Stokes equations. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such systems, introducing new opportunities for their applicability to problems of industrial complexity.

97 MATHEMATICS AND COMPUTING↗

Structural search for stable Mg–Ca alloys accelerated with a neural network interatomic model

Here, we have combined a neural network formalism with metaheuristic structural global search algorithms to systematically screen the Mg–Ca binary system for new (meta)stable alloys. The combination of these methods allows for an efficient exploration of the potential energy surface beyond the possibility of the traditional searches based on ab initio energy evaluations. The identified pool of low-enthalpy structures was complemented with special quasirandom structures (SQS) at different stoichiometries. In addition to the only Mg–Ca phase known to form under standard synthesis conditions, C14-Mg 2 Ca, the search has uncovered several candidate materials that could be synthesized under elevated temperatures or pressures. We demonstrate that the vibrational entropy lowers the relative free energy of several phases with magnesium kagome layers: C15 and C36 Laves structures at the 2:1 composition and an orthorhombic oS36 structure at the 7:2 composition. The estimated phase transition temperatures close to the melting point leave open the possibility of synthesizing the predicted materials at high temperatures. At high pressures up to 10 GPa, two new phases at the 1:1 and 3:1 Mg:Ca stoichiometries become thermodynamically stable and should form in multi-anvil experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Predicting turbulent wake flow of marine hydrokinetic turbine arrays in large-scale waterways via physics-enhanced convolutional neural networks

We present a physics-enhanced convolutional neural network (PECNN) algorithm for reconstructing the mean flow and turbulence statistics in the wake of marine hydrokinetic (MHK) turbine arrays installed in large-scale meandering rivers. The algorithm embeds the mass and momentum conservation equations into the loss function of the PECNN algorithm to improve the physical realism of the reconstructed flow fields. The PECNN is trained using large eddy simulation (LES) results of the wake flow of a single row of turbines in a virtual meandering river. Subsequently, the trained PECNN is applied to predict the wake flow of MHK turbines with arrangements and positionings different than those considered during the training process. The PECNN predictions are validated using the results of separately performed LES. The results show that the PECNN algorithm can accurately predict the wake flow of MHK turbine farms at a small fraction of the cost of LES. The PECNN can improve the accuracy by around 1% and reduce the physical constraint indices by around 50% compared to the CNN without physical constraints. This work underscores the potential of PECNN to develop reduced-order models for control co-design and optimization of MHK turbine arrays in natural riverine environments.

Mechanics↗