Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Design and Challenges of Edge Computing ASICs on Front-End Electronics

In situ or hardware-embedded data processing of raw signals, close to their source, in radiation detectors is expected to provide dramatic improvements in data quality and volumes. However, the implementation of artificial neural networks (ANNs) in the front-end electronics, and the design of custom integrated circuits (ASICs), comes with challenges. In addition, detectors have to operate with limited power budget and implement complex functionalities in a very dense space. They often are exposed to extreme conditions as they work in high-radiation environments and/or cryogenic temperatures. This paper presents examples of applications and design methodologies for in-situ ANNs, along with the challenges of retaining the fidelity of the trained networks. For illustration, we use the problem of estimating the energy deposited by the radiation from digitized waveforms. The proposed implementation starts with an ML algorithm trained in Qkeras and eventually leads to an equivalent ASIC implementation. The associated design challenges in realizing energy and area efficient implementations in CMOS processes are reviewed. Novel approaches that employ hybrid technologies (combination of CMOS with memristors), in-memory computing models, and bio-inspired spiking neural networks are also highlighted

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Denoising diffusion algorithm for inverse design of microstructures with fine-tuned nonlinear material properties

Here we introduce a denoising diffusion algorithm to discover microstructures with nonlinear fine-tuned properties. Denoising diffusion probabilistic models are generative models that use diffusion-based dynamics to gradually denoise images and generate realistic synthetic samples. By learning the reverse of a Markov diffusion process, we design an artificial intelligence to efficiently manipulate the topology of microstructures to generate a massive number of prototypes that exhibit constitutive responses sufficiently close to designated nonlinear constitutive behaviors. To identify the subset of microcstructures with sufficiently precise fine-tuned properties, a convolutional neural network surrogate is trained to replace high-fidelity finite element simulations to filter out prototypes outside the admissible range. Results of this study indicate that the denoising diffusion process is capable of creating microstructures of fine-tuned nonlinear material properties within the latent space of the training data. More importantly, this denoising diffusion algorithm can be easily extended to incorporate additional topological and geometric modifications by introducing high-dimensional structures embedded in the latent space. Numerical experiments are conducted on the open-source mechanical MNIST data set (Lejeune, 2020). Consequently, this algorithm is not only capable of performing inverse design of nonlinear effective media, but also learns the nonlinear structure–property map to quantitatively understand the multiscale interplay among the geometry, topology, and their effective macroscopic properties.

42 ENGINEERING↗
DOE OSTI2021

Inversion of coherent surface scattering images via deep learning network

Coherent surface scattering imaging is a powerful tool for imaging a surface/interface of a thin nanostructure deposited on an opaque substrate. A mathematical conversion of an object image from a scattering pattern is essential for coherent surface scattering imaging to visualize structures of specimens. It has been achieved by using phase retrieval algorithms requiring oversampling in scattering patterns and employing alternating projection approaches. It is a computationally challenging and time-consuming process. In this paper, we demonstrate CSSI-NN, which is a deep learning neural network model to predict images of objects from scattering intensities in coherent surface scattering imaging. This model allowing for the instant outcome from scattering patterns would be tremendously beneficial not only for effective experiments but also for data analysis of phase retrieval.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Training Spiking Neural Networks Using Combined Learning Approaches

Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.

Elbrecht, Daniel↗
DOE OSTI2020

Chasing Accreted Structures within Gaia DR2 Using Deep Learning

In previous work, we developed a deep neural network classifier that only relies on phase-space information to obtain a catalog of accreted stars based on the second data release of Gaia (DR2). In this paper, we apply two clustering algorithms to identify velocity substructure within this catalog. We focus on the subset of stars with line-of-sight velocity measurements that fall in the range of Galactocentric radii $r\in [6.5,9.5]\,{\rm{kpc}}$ and vertical distances $| z| \lt 3\,{\rm{kpc}}$. Known structures such as Gaia Enceladus and the Helmi stream are identified. The largest previously unknown structure, Nyx, is a vast stream consisting of at least 200 stars in the region of interest. This study displays the power of the machine-learning approach by not only successfully identifying known features but also discovering new kinematic structures that may shed light on the merger history of the Milky Way.

Astronomy & Astrophysics↗
NASA NTRS1990

Intensity dependent spread processor and workstation

The Intensity Dependent Spread (IDS) is an adaptive algorithm which is modified according to the local intensity in the scene. (This results in a nonlinear process which cannot take advantage of rather nice linear transform methods.) The computation is similar to a neural net whereby intensity information is moving from each input pixel to a set of surrounding output pixels in a manner described by Cornsweet and Yellott. A prototype of a very large scale integration IDS processor is being developed and implemented in a workstation environment. The workstation consists of a SUN 3/260 and a DATACUBE pipeline processor. The IDS prototype is a board set which operates in the DATA CUBE processor. The SUN 3/260 performs control, background processing, IDS simulation and image display functions.

Westrom, George↗
DOE OSTI2019

Using a machine learning approach to determine the space group of a structure from the atomic pair distribution function

Here a method is presented for predicting the space group of a structure given a calculated or measured atomic pair distribution function (PDF) from that structure. The method utilizes machine learning models trained on more than 100 000 PDFs calculated from structures in the 45 most heavily represented space groups. In particular, a convolutional neural network (CNN) model is presented which yields a promising result in that it correctly identifies the space group among the top-6 estimates 91.9% of the time. The CNN model also successfully identifies space groups for 12 out of 15 experimental PDFs. Interesting aspects of the failed estimates are discussed, which indicate that the CNN is failing in similar ways as conventional indexing algorithms applied to conventional powder diffraction data. This preliminary success of the CNN model shows the possibility of model-independent assessment of PDF data on a wide class of materials.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Iterative multi-task learning and inference from seismic images

Seismic interpretation aims to extract quantitative and interpretable attributes from a seismic image produced using some migration method to inform characteristics of a subsurface reservoir or target of interest. Current paradigms for computing seismic attributes mostly rely on single-task algorithms. We develop an iterative, multi-task machine learning method to learn and infer multiple attributes from a seismic image. This method is composed of two stages: a multi-task inference stage and a multi-modal, multi-task refinement stage. The basic mechanism of this method is that we train a multi-task inference neural network (NN) to estimate a set of attributes, including a relative geological time (RGT), a denoised higher-resolution (DHR) seismic image, and multiple fault attributes (including probability, dip, and strike), from a low-resolution, noisy seismic image; then we input the inferred attributes to a multi-task refinement NN to enhance the raw inference results iteratively. The two multi-task NNs are trained separately based on synthetic seismic images and associated attributes generated by a geological modeling algorithm. The software we intend to release is a PyTorch implementation of this multi-task learning method for both 2D and 3D cases along with scripts to run the training/validation. The algorithm and software can be a useful tool for automatic seismic interpretation.

Gao, Kai↗
DOE OSTI2022

GW-PINN: A deep learning algorithm for solving groundwater flow equations

Machine learning methods provide new perspective for more convenient and efficient prediction of groundwater flow. In this study, a deep learning method “GW-PINN” without labeled data for solving groundwater flow equations with wells was proposed. GW-PINN takes the physics inform neural network (PINN) as the backbone and uses either the hard or soft constraint in the loss function for training. A locally refined sampling strategy (LRS) is adopted to generate the consistent spatial sampling points for problems with strong hydraulic head change, and then combined with an appropriate temporal sampling scheme to obtain the final spatial-temporal sampling points. A snowball-style two-stage training strategy by dividing the temporal domain into two subdomains is designed to decrease the sampling points. Five cases were designed to test the training performance of GW-PINN under different sampling strategies and two constraints. The predicted results of GW-PINN were compared with MODFLOW and the analytical solution. The results demonstrate that GW-PINN possesses strong ability in capturing the hydraulic head change for both confined and un-confined aquifers. The hard constraint owns more robust learning ability than the soft constraint. The LRS strategy can generate more accurate results with much fewer sampling points than traditional sampling strategies, and the snowball-style two-stage training strategy is significantly efficient for problems with the drastic change of hydraulic head. Additionally, the application of GW-PINN as a surrogate model for parameterized groundwater flow equations is illustrated. This study provides an option tool for efficient groundwater flow simulation, especially for those with local refinements are needed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗
DOE OSTI2021

A Hybrid System for Learning Classical Data in Quantum States

Deep neural network-powered artificial intelligence has rapidly changed our daily life with various applications. However, as one of the essential steps of deep neural networks, training a heavily-weighted network requires a tremendous amount of computing resources. Especially in the post Moore’s Law era, the limit of semiconductor fabrication technology has restricted the development of learning algorithms to cope with the increasing high intensity training data. Meanwhile, quantum computing has exhibited its significant potential in terms of speeding up the traditionally compute-intensive workloads. For example, Google illustrates quantum supremacy by completing a sampling calculation task in 200 seconds, which is otherwise impracticable on the world’s largest supercomputers. To this end, quantum-based learning becomes an area of interest, with the promising of a quantum speedup. In this paper, we propose GenQu, a hybrid and general-purpose quantum framework for learning classical data through quantum states. We evaluate GenQu with real datasets and conduct experiments on both simulations and real quantum computer IBM-Q. Our evaluation demonstrates that, comparing with classical solutions, the proposed models running on GenQu framework achieve similar accuracy with a much smaller number of qubits, while significantly reducing the parameter size by up to 95.86% and converging speedup by 66.67% faster.

Stein, Samuel A.↗
DOE OSTI2025

Machine-Learning-Based Multiscale Methods for 3D Modelling of Granular Materials by Incorporating History-Dependent State Variables

Over the past decades, the prevalence of machine learning (ML) methods has made the development of ML-based constitutive models for granular materials undoubtedly a popular subject. Numerous studies have been made to feature the loading path or history-dependent stress-strain response of granular media using neural networks. In this work, a novel finite element method (FEM)–ML multiscale approach was developed by incorporating internal variables to improve the simulation accuracy of 3D history-dependent granular materials for the first time. To this end, a surrogate constitutive model based on the single-step-based multi-layer perceptron (MLP) neural network was used to replace representative volume element (RVE) simulations conducted by the discrete element method (DEM) in the multiscale FEM–DEM approach. Although the prediction principle of the MLP aligns with the FEM algorithm, artificially added internal variables are required to differentiate the loading history. To address this issue, history variables associated with the Frobenius norm are proposed to be fed into the MLP coupled with the strain tensor to extract the history-dependent behaviour of granular assemblies. The developed FEM–ML approach was demonstrated in 3D conventional triaxial compression (CTC) simulations. Compared to the multiscale FEM–DEM approach, the proposed FEM–ML method exhibits a significantly improved computational efficiency.

granular materials↗
DOE OSTI2021

Deep learning of parameterized equations with applications to uncertainty quantification

We propose a learning algorithm for discovering unknown parameterized dynamical systems by using observational data of the state variables. Our method is built upon and extends the recent work of discovering unknown dynamical systems, in particular those using deep neural network (DNN). We propose a DNN structure, largely based upon the residual network (ResNet), to not only learn the unknown form of the governing equation but also take into account the random effect embedded in the system, which is generated by the random parameters. Once the DNN model is successfully constructed, it is able to produce system prediction over longer term and for arbitrary parameter values. For uncertainty quantification, it allows us to conduct uncertainty analysis by evaluating solution statistics over the parameter space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Integrated Cyber/Physical Grid Resiliency Modeling

This project explored coupling modeling and analysis methods from multiple domains to address complex hybrid (cyber and physical) attacks on mission critical infrastructure. Robust methods to integrate these complex systems are necessary to enable large trade-space exploration including dynamic and evolving cyber threats and mitigations. Reinforcement learning employing deep neural networks, as in the AlphaGo Zero solution, was used to identify "best" (or approximately optimal) resilience strategies for operation of a cyber/physical grid model. A prototype platform was developed and the machine learning (ML) algorithm was made to play itself in a game of 'Hurt the Grid'. This proof of concept shows that machine learning optimization can help us understand and control complex, multi-dimensional grid space. A simple, yet high-fidelity model proves that the data have spatial correlation which is necessary for any optimization or control. Our prototype analysis showed that the reinforcement learning successfully improved adversary and defender knowledge to manipulate the grid. When expanded to more representative models, this exact type of machine learning will inform grid operations and defense - supporting mitigation development to defend the grid from complex cyber attacks! This same research can be expanded to similar complex domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Exploration of Quantum Machine Learning and AI Accelerators for Fusion Science

The dawn of the noisy intermediate-scale quantum (NISQ) era sparked rapid development in variational quantum algorithms. These algorithms, utilizing parameterized quantum circuit optimized by classical computers with feedback, are practical under the constraints of the current hardware and can potentially show quantum advantage. In recent years, these variational circuit are applied to neural networks, hoping to boost the triumphant success of deep learning. We simulate various of quantum-classical hybrid neural networks applied to scientific tasks, and seek to understand their capabilities and limitations. Specifically, we devise quantum convolutional layers and apply them to deep neural networks used for predicting plasma disruption in fusion reactors.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Algorithmic and GPU enhancements for molecular dynamics in Cabana and LAMMPS

The focus of this milestone was to implement and test new algorithms and optimizations for improved GPU performance by molecular dynamics (MD) and related particle codes. The work was done in the CoPA Cabana library, the CabanaMD mini-app which uses Cabana to implement and benchmark prototypical MD models, and the LAMMPS MD code which is used by the ECP EXAALT project as well as for materials modeling generally. Efforts included (a) implementation of new or optimized algorithms for fundamental MD kernels like neighbor-list building and FFTs which are also widely used in other particle methods, (b) performance testing of data structure layouts for both simple pairwise and computationally intense neural network potentials, (c) optimizations aimed at improving GPU performance for models with small atom counts, and (d) optimizations specific to the SNAP interatomic potential due to its importance for the EXAALT project.

36 MATERIALS SCIENCE↗
NASA NTRS1986

Contrast-Sensitivity Research

Report presents study of visual effects of frequencies of luminance patterns and particularly how frequency components affect visibilty of spatial phase differences over several octaves. Of interest to researchers engaged in development of algorithms relating to visual processing in robots, incorporation of human factors in design of visual displays, and development of set of rules visual system uses to reconstruct three-dimensional perception from two-dimensional neural representation.

Lawton, T. A.↗
NASA NTRS1996

Advanced Intelligent System Application to Load Forecasting and Control for Hybrid Electric Bus

The primary motivation for this research emanates from providing a decision support system to the electric bus operators in the municipal and urban localities which will guide the operators to maintain an optimal compromise among the noise level, pollution level, fuel usage etc. This study is backed up by our previous studies on study of battery characteristics, permanent magnet DC motor studies and electric traction motor size studies completed in the first year. The operator of the Hybrid Electric Car must determine optimal power management schedule to meet a given load demand for different weather and road conditions. The decision support system for the bus operator comprises three sub-tasks viz. forecast of the electrical load for the route to be traversed divided into specified time periods (few minutes); deriving an optimal 'plan' or 'preschedule' based on the load forecast for the entire time-horizon (i.e., for all time periods) ahead of time; and finally employing corrective control action to monitor and modify the optimal plan in real-time. A fully connected artificial neural network (ANN) model is developed for forecasting the kW requirement for hybrid electric bus based on inputs like climatic conditions, passenger load, road inclination, etc. The ANN model is trained using back-propagation algorithm employing improved optimization techniques like projected Lagrangian technique. The pre-scheduler is based on a Goal-Programming (GP) optimization model with noise, pollution and fuel usage as the three objectives. GP has the capability of analyzing the trade-off among the conflicting objectives and arriving at the optimal activity levels, e.g., throttle settings. The corrective control action or the third sub-task is formulated as an optimal control model with inputs from the real-time data base as well as the GP model to minimize the error (or deviation) from the optimal plan. These three activities linked with the ANN forecaster proving the output to the GP model which in turn produces the pre-schedule of the optimal control model. Some preliminary results based on a hypothetical test case will be presented for the load forecasting module. The computer codes for the three modules will be made available fe adoption by bus operating agencies. Sample results will be provided using these models. The software will be a useful tool for supporting the control systems for the Electric Bus project of NASA.

Momoh, James↗