Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,675 records · Page 93

Computerized method for evaluating diagnostic image quality of calcified plaque images in cardiac CT: Validation on a physical dynamic cardiac phantom

Purpose: In cardiac computed tomography (CT), important clinical indices, such as the coronary calcium score and the percentage of coronary artery stenosis, are often adversely affected by motion artifacts. As a result, the expert observer must decide whether or not to use these indices during image interpretation. Computerized methods potentially can be used to assist in these decisions. In a previous study, an artificial neural network (ANN) regression model provided assessability (image quality) indices of calcified plaque images from the software NCAT phantom that were highly agreeable with those provided by expert observers. The method predicted assessability indices based on computer-extracted features of the plaque. In the current study, the ANN-predicted assessability indices were used to identify calcified plaque images with diagnostic calcium scores (based on mass) from a physical dynamic cardiac phantom. The basic assumption was that better quality images were associated with more accurate calcium scores. Methods: A 64-channel CT scanner was used to obtain 500 calcified plaque images from a physical dynamic cardiac phantom at different heart rates, cardiac phases, and plaque locations. Two expert observers independently provided separate sets of assessability indices for each of these images. Separate sets of ANN-predicted assessability indices tailored to each observer were then generated within the framework of a bootstrap resampling scheme. For each resampling iteration, the absolute calcium score error between the calcium scores of the motion-contaminated plaque image and its corresponding stationary image served as the ground truth in terms of indicating images with diagnostic calcium scores. The performances of the ANN-predicted and observer-assigned indices in identifying images with diagnostic calcium scores were then evaluated using ROC analysis. Results: Assessability indices provided by the first observer and the corresponding ANN performed similarly (AUC{sub OBS1}=0.80 [0.73,0.86] vs AUC{sub ANN1}=0.88 [0.82,0.92]) as that of the second observer and the corresponding ANN (AUC{sub OBS2}=0.87 [0.83,0.91] vs AUC{sub ANN2}=0.90 [0.85,0.94]). Moreover, the ANN-predicted indices were generated in a fraction of the time required to obtain the observer-assigned indices. Conclusions: ANN-predicted assessability indices performed similar to observer-assigned assessability indices in identifying images with diagnostic calcium scores from the physical dynamic cardiac phantom. The results of this study demonstrate the potential of using computerized methods for identifying images with diagnostic clinical indices in cardiac CT images.

60 APPLIED LIFE SCIENCES↗

Intelligent Computer Vision System for Automated Classification

In this paper we investigate an Intelligent Computer Vision System applied for recognition and classification of commercially available cork tiles. The system is capable of acquiring and processing gray images using several feature generation and analysis techniques. Its functionality includes image acquisition, feature extraction and preprocessing, and feature classification with neural networks (NN). We also discuss system test and validation results from the recognition and classification tasks. The system investigation also includes statistical feature processing (features number and dimensionality reduction techniques) and classifier design (NN architecture, target coding, learning complexity and performance, and training with our own metaheuristic optimization method). The NNs trained with our genetic low-discrepancy search method (GLPtauS) for global optimisation demonstrated very good generalisation abilities. In our view, the reported testing success rate of up to 95% is due to several factors: combination of feature generation techniques; application of Analysis of Variance (ANOVA) and Principal Component Analysis (PCA), which appeared to be very efficient for preprocessing the data; and use of suitable NN design and learning method.

97 MATHEMATICS AND COMPUTING↗

Usage-based Lifing of Lithium-Ion Battery with HybridPhysics-Informed Neural Networks

Lithium-ion batteries are commonly used to power unmanned aircraft vehicles (UAVs).The ability to model and forecast the remaining useful life of these batteries enables UAV reliability assurance. Building accurate models for battery state of charge and state of health based on first principles is challenging due to the complex electrochemistry that governs battery operations and computational complexity required to solve them. Therefore, reduced order models are often used due to their ability to capture the overall battery discharge. Un-fortunately, these simplifications lead to residual discrepancy between model predictions and observed data. In this paper, we present a hybrid modeling approach merging reduced-order models and neural networks. In this approach, while most of the input-output relationship is captured by Nernst and Butler-Volmer equations, data-driven kernels reduce the gap between predictions and observations. We validate our approach using data publicly available through the NASA Prognostics Center of Excellence repository. Results showed that our hybrid battery prognosis model can be successfully calibrated, even with a limited number of observations.

Lithium-ion Battery↗

Improved image classification with neural networks by fusing multispectral signatures with topological data

Automated schemes are needed to classify multispectral remotely sensed data. Human intelligence is often required to correctly interpret images from satellites and aircraft. Humans suceed because they use various types of cues about a scene to accurately define the contents of the image. Consequently, it follows that computer techniques that integrate and use different types of information would perform better than single source approaches. This research illustrated that multispectral signatures and topographical information could be used in concert. Significantly, this dual source tactic classified a remotely sensed image better than the multispectral classification alone. These classifications were accomplished by fusing spectral signatures with topographical information using neural network technology. A neural network was trained to classify Landsat mulitspectral signatures. A file of georeferenced ground truth classifications were used as the training criterion. The network was trained to classify urban, agriculture, range, and forest with an accuracy of 65.7 percent. Another neural network was programmed and trained to fuse these multispectral signature results with a file of georeferenced altitude data. This topological file contained 10 levels of elevations. When this nonspectral elevation information was fused with the spectral signatures, the classifications were improved to 73.7 and 75.7 percent.

Harston, Craig↗

Active Learning A Neural Network Model For Gold Clusters & Bulk From Sparse First Principles Training Data

Small metal clusters are of fundamental scientific interest and of tremendous significance in catalysis. These nanoscale clusters display diverse geometries and structural motifs depending on the cluster size; a knowledge of this size-dependent structural motifs and their dynamical evolution has been of longstanding interest. Given the high computational cost of first-principles calculations, molecular modeling and atomistic simulations such as molecular dynamics (MD) has proven to be an important complementary tool to aid this understanding. Classical MD typically employ predefined functional forms which limits their ability to capture such complex size-dependent structural and dynamical transformation. Neural Network (NN) based potentials represent flexible alternatives and in principle, well-trained NN potentials can provide high level of flexibility, transferability and accuracy on-par with the reference model used for training. A major challenge, however, is that NN models are interpolative and requires large quantities (similar to 10 4 or greater) of training data to ensure that the model adequately samples the energy landscape both near and far-from-equilibrium. A highly desirable goal is minimize the number of training data, especially if the underlying reference model is first-principles based and hence expensive. In this work, we introduce an active learning (AL) scheme that trains a NN model on-the-fly with minimal amount of first-principles based training data. Our AL workflow is initiated with a sparse training dataset (similar to 1 to 5 data points) and is updated on-the-fly via a Nested Ensemble Monte Carlo scheme that iteratively queries the energy landscape in regions of failure and updates the training pool to improve the network performance. Using a representative system of gold clusters, we demonstrate that our AL workflow can train a NN with similar to 500 total reference calculations. Using an extensive DFT test set of similar to 1100 configurations, we show that our AL-NN is able to accurately predict both the DFT energies and the forces for clusters of a myriad of different sizes. Our NN predictions are within 30 meV/atom and 40 meV/angstrom of the reference DFT calculations. Moreover, our AL-NN model also adequately captures the various size-dependent structural and dynamical properties of gold clusters in excellent agreement with DFT calculations and available experiments. We finally show that our AL-NN model also captures bulk properties reasonably well, even though they were not included in the training data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Compact Microscope Imaging System with Intelligent Controls

The figure presents selected views of a compact microscope imaging system (CMIS) that includes a miniature video microscope, a Cartesian robot (a computer- controlled three-dimensional translation stage), and machine-vision and control subsystems. The CMIS was built from commercial off-the-shelf instrumentation, computer hardware and software, and custom machine-vision software. The machine-vision and control subsystems include adaptive neural networks that afford a measure of artificial intelligence. The CMIS can perform several automated tasks with accuracy and repeatability . tasks that, heretofore, have required the full attention of human technicians using relatively bulky conventional microscopes. In addition, the automation and control capabilities of the system inherently include a capability for remote control. Unlike human technicians, the CMIS is not at risk of becoming fatigued or distracted: theoretically, it can perform continuously at the level of the best human technicians. In its capabilities for remote control and for relieving human technicians of tedious routine tasks, the CMIS is expected to be especially useful in biomedical research, materials science, inspection of parts on industrial production lines, and space science. The CMIS can automatically focus on and scan a microscope sample, find areas of interest, record the resulting images, and analyze images from multiple samples simultaneously. Automatic focusing is an iterative process: The translation stage is used to move the microscope along its optical axis in a succession of coarse, medium, and fine steps. A fast Fourier transform (FFT) of the image is computed at each step, and the FFT is analyzed for its spatial-frequency content. The microscope position that results in the greatest dispersal of FFT content toward high spatial frequencies (indicating that the image shows the greatest amount of detail) is deemed to be the focal position.

McDowell, Mark↗

An attention-based neural ordinary differential equation framework for modeling inelastic processes

To preserve strictly conservative behavior as well as model the variety of dissipative behavior displayed by solid materials, we propose a significant enhancement to the internal state variable-neural ordinary differential equation (ISV-NODE) framework. In this data-driven, physics-constrained modeling framework internal states are inferred rather than prescribed. The ISV-NODE consists of: (a) a stress model dependent on observable deformation and inferred internal state, and (b) a model of the evolution of the internal states. The enhancements to ISV-NODE proposed in this work are multifold: (a) a partially input convex neural network stress potential provides polyconvexity in terms of observed strain while leaving the inferred state unconstrained, and (b) an internal state flow model uses common latent features to inform novel attention-based gating and drives the flow of internal state only in dissipative regimes. We demonstrated that this architecture can accurately model dissipative and conservative behavior across an isotropic, isothermal elastic-viscoelastic-elastoplastic spectrum with three exemplars, while maintaining fundamental principles by design.

97 MATHEMATICS AND COMPUTING↗

Survey of Deep Learning and Physics-Based Approaches in Computational Wave Imaging

Computational wave imaging (CWI) extracts hidden structure and physical properties of a volume of material by analyzing wave signals that traverse that volume. Applications include seismic exploration of the Earth’s subsurface, acoustic imaging and nondestructive testing (NDT) in material science, and ultrasound computed tomography (USCT) in medicine. Current approaches for solving CWI problems can be divided into two categories: those rooted in traditional physics and those based on deep learning. Physics-based methods stand out for their ability to provide high-resolution and quantitatively accurate estimates of acoustic properties within the medium. However, they can be computationally intensive and are susceptible to ill-posedness and nonconvexity typical of CWI problems. Machine learning (ML)-based computational methods have recently emerged, offering a different perspective to address these challenges. Diverse scientific communities have independently pursued the integration of deep learning in CWI. This review discusses how contemporary scientific ML techniques, and deep neural networks in particular, have been developed to enhance and integrate with traditional physics-based methods for solving CWI problems. We present a structured framework that consolidates existing research spanning multiple domains, including computational imaging, wave physics, and data science. This study concludes with important lessons learned from existing ML-based methods and identifies technical hurdles and emerging trends through a systematic analysis of the extensive literature on this topic.

42 ENGINEERING↗

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗

Learning Physically Interpretable Atmospheric Models From Data With WSINDy

The multiscale and turbulent nature of Earth's atmosphere has historically rendered accurate weather modeling a hard problem. Recently, there has been an explosion of interest surrounding data-driven approaches to weather modeling, which in many cases show improved forecasting accuracy and computational efficiency when compared to traditional methods. However, many of the current data-driven approaches employ highly parameterized neural networks, often resulting in uninterpretable models and limited gains in scientific understanding. In this work, we address the interpretability problem by explicitly discovering partial differential equations governing atmospheric phenomena, identifying symbolic mathematical models with direct physical interpretations. The purpose of this paper is to demonstrate that, in particular, the weak-form sparse identification of nonlinear dynamics (WSINDy) algorithm can learn effective atmospheric models from both simulated and assimilated data. Our approach adapts the standard WSINDy algorithm to work with high-dimensional fluid data of arbitrary spatial dimension.

58 GEOSCIENCES↗

Efficient hybrid explicit-implicit learning for multiscale problems

Splitting method is a powerful method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. Here, in this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. We propose a machine learning assisted splitting scheme which improves the efficiency of the scheme meanwhile preserves the accuracy. We consider a recently introduced multiscale splitting algorithms, where the multiscale problem is solved on a coarse grid. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. This splitting concept allows identifying degrees of freedom that need implicit treatment. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation of the splitting scheme successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection between them. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem which approximates the target. Our machine learning model can incorporate and encode the given information from two different problems and then solve the target problems. We conduct four numerical examples and the results show that our method is stable and accurate.

97 MATHEMATICS AND COMPUTING↗

Multi-View Convolutional Neural Network for Data Spoofing Cyber-Attack Detection in Distribution Synchrophasors

Security of Distribution Synchrophasors Data (DSD) is of paramount importance as the data is used for critical smart grid applications including situational awareness, advanced protection, and dynamic control. Unfortunately, the DSD are attractive targets for malicious attackers aiming to damage grid. Data spoofing is a new class of deceiving attack, where the DSD of one Phasor Measurement Units (PMUs) is tampered by other PMUs thereby spoiling measurement based applications. In order to address this issue, a source authentication based data spoofing attack detection method is proposed using Multi-view Convolutional Neural Network (MCNN). First, common components embedded in raw frequency measurements from DSD are removed by Savitzky-Golay (SG) filter. Second, fast S transform (FST) is utilized to extract representative spatial fingerprints via time frequency analysis. Third, the spatial fingerprint is fed to MCNN, which combines dilated and standard convolutions for automatic feather extraction and source identification. Finally, according to the output of MCNN, spoofing attack detection is performed via threshold criterion. Extensive experiments with actual DSD from multiple locations in FNET/Grideye are conducted to verify the effectiveness of the proposed method.

97 MATHEMATICS AND COMPUTING↗

Deep Learning Emulation of Atmospheric Correction for Geostationary Sensors

New generation geostationary satellites make reflectance observations available at a continental scale with unprecedented spatiotemporal resolution and spectral range. Generating Earth monitoring products from these observations requires retrieval of the basic parameter, surface reflectance (SR), by atmospheric correction (AC). Algorithms for atmospheric correction, including Multi-Angle Implementation of Atmospheric Correction (MAIAC), are adapted for each sensor and are too computationally complex to be run in real time, relying instead on look-up tables with precomputed values. Machine learning methods, including convolutional neural networks, have demonstrated performance in learning complex, nonlinear mappings and extracting insight from high-dimensional remote sensing data. In this work, we present a deep learning emulator of MAIAC to retrieve both SR and cloud products. Using this adaptation of deep learning-based emulation to remote sensing, we demonstrate stable SR retrieval over a variety of land covers and viewing conditions and accurate cloud detection. Further, a comparison of computation time suggests emulation as a compelling alternative for expensive physical simulation, especially for applications benefited by near-real time data, such as agricultural management and disaster response.

Duffy, Kate↗

Uncertainty Quantification for Neutron Shield Using Convolutional Neural Networks

Uncertainty quantification from radiation transport calculations was conducted using a Bayesian inference approach. A surrogate model, using a convolutional neural network, was employed to emulate the neutron fluence, which was simulated with a Monte Carlo radiation transport model. This allowed for a computationally cheap approach to evaluate input parameters and to sample their corresponding posterior probability distributions. Experimental data from the literature were employed to perform uncertainty quantification studies for concrete shields. As a result, the method is a nonintrusive approach that enables studies with multiple input parameters and can be applied to any radiation transport model.

Bayesian inference↗

Development of lean, efficient, and fast physics-framed deep-learning-based proxy models for subsurface carbon storage

In this work, we present deep-learning-based surrogate models for CCUS developed with four different algorithms and a physics-framed two-phase flow problem involving displacement of water by CO 2 . The deep-learning models were trained using 3D datasets describing the pressure plume, CO 2 saturation plume, and water extraction rate generated by numerical simulation. The hyperparameters defining the architecture of the neural networks were optimized to determine the slimmest network size and training parameters that give the most efficient performance at the least training cost. To develop a robust model that closely mimics the governing physical laws, the discretized form of the two-phase fluid transport equation was used to formulate the supervised deep-learning task. The algorithms investigated in this study predicted the data to above 95% accuracy, with the multi-layer perceptron model demonstrating the best performance by balancing training speed, prediction time, and prediction accuracy with lean network capacity. Furthermore, the surrogate models simultaneously predict reservoir pressure and CO 2 saturation in every grid block, including the surface well extraction rate and bottomhole pressure, at all simulation times for a given static model realization in just a few seconds on a standard desktop computer. A key outcome of this study is that limits can be placed on network design parameters to avoid over designing neural networks, with associated efficiencies in training and prediction times. This is very useful because large volumes of data may be generated in CCUS projects and over-design of neural network architectures imposes penalties that are antithetical to the goal of near-real time forecasting.

58 GEOSCIENCES↗

Quantum-Inspired Bayesian Sampling for Uncertainty Quantification and Machine Learning (Final Technical Report)

With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.

97 MATHEMATICS AND COMPUTING↗

Optimization with Neural Network Feasibility Surrogates: Formulations and Application to Security-Constrained Optimal Power Flow

In many areas of constrained optimization, representing all possible constraints that give rise to an accurate feasible region can be difficult and computationally prohibitive for online use. Satisfying feasibility constraints becomes more challenging in high-dimensional, non-convex regimes which are common in engineering applications. A prominent example that is explored in the manuscript is the security-constrained optimal power flow (SCOPF) problem, which minimizes power generation costs, while enforcing system feasibility under contingency failures in the transmission network. In its full form, this problem has been modeled as a nonlinear two-stage stochastic programming problem. In this work, we propose a hybrid structure that incorporates and takes advantage of both a high-fidelity physical model and fast machine learning surrogates. Neural network (NN) models have been shown to classify highly non-linear functions and can be trained offline but require large training sets. In this work, we present how model-guided sampling can efficiently create datasets that are highly informative to a NN classifier for non-convex functions. We show how the resultant NN surrogates can be integrated into a non-linear program as smooth, continuous functions to simultaneously optimize the objective function and enforce feasibility using existing non-linear solvers. Overall, this allows us to optimize instances of the SCOPF problem with an order of magnitude CPU improvement over existing methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Genetic learning in rule-based and neural systems

The design of neural networks and fuzzy systems can involve complex, nonlinear, and ill-conditioned optimization problems. Often, traditional optimization schemes are inadequate or inapplicable for such tasks. Genetic Algorithms (GA's) are a class of optimization procedures whose mechanics are based on those of natural genetics. Mathematical arguments show how GAs bring substantial computational leverage to search problems, without requiring the mathematical characteristics often necessary for traditional optimization schemes (e.g., modality, continuity, availability of derivative information, etc.). GA's have proven effective in a variety of search tasks that arise in neural networks and fuzzy systems. This presentation begins by introducing the mechanism and theoretical underpinnings of GA's. GA's are then related to a class of rule-based machine learning systems called learning classifier systems (LCS's). An LCS implements a low-level production-system that uses a GA as its primary rule discovery mechanism. This presentation illustrates how, despite its rule-based framework, an LCS can be thought of as a competitive neural network. Neural network simulator code for an LCS is presented. In this context, the GA is doing more than optimizing and objective function. It is searching for an ecology of hidden nodes with limited connectivity. The GA attempts to evolve this ecology such that effective neural network performance results. The GA is particularly well adapted to this task, given its naturally-inspired basis. The LCS/neural network analogy extends itself to other, more traditional neural networks. Conclusions to the presentation discuss the implications of using GA's in ecological search problems that arise in neural and fuzzy systems.

Smith, Robert E.↗