Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Deep learning methods for obtaining photometric redshift estimations from images

ABSTRACT Knowing the redshift of galaxies is one of the first requirements of many cosmological experiments, and as it is impossible to perform spectroscopy for every galaxy being observed, photometric redshift (photo-z) estimations are still of particular interest. Here, we investigate different deep learning methods for obtaining photo-z estimates directly from images, comparing these with ‘traditional’ machine learning algorithms which make use of magnitudes retrieved through photometry. As well as testing a convolutional neural network (CNN) and inception-module CNN, we introduce a novel mixed-input model that allows for both images and magnitude data to be used in the same model as a way of further improving the estimated redshifts. We also perform benchmarking as a way of demonstrating the performance and scalability of the different algorithms. The data used in the study comes entirely from the Sloan Digital Sky Survey (SDSS) from which 1 million galaxies were used, each having 5-filtre (ugriz) images with complete photometry and a spectroscopic redshift which was taken as the ground truth. The mixed-input inception CNN achieved a mean squared error (MSE) =0.009, which was a significant improvement ($30{{\ \rm per\ cent}}$) over the traditional random forest (RF), and the model performed even better at lower redshifts achieving a MSE = 0.0007 (a $50{{\ \rm per\ cent}}$ improvement over the RF) in the range of z < 0.3. This method could be hugely beneficial to upcoming surveys, such as Euclid and the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST), which will require vast numbers of photo-z estimates produced as quickly and accurately as possible.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

A differentiable approach to the maximum independent set problem using dataless neural networks

The success of machine learning solutions for reasoning about discrete structures has brought attention to its adoption within combinatorial optimization algorithms. Such approaches generally rely on supervised learning by leveraging datasets of the combinatorial structures of interest drawn from some distribution of problem instances. Reinforcement learning has also been employed to find such structures. Here, in this paper, we propose a different approach in that no data is required for training the neural networks that produce the solution. In this sense, what we present is not a machine learning solution, but rather one that is dependent on neural networks and where backpropagation is applied to a loss function defined by the structure of the neural network architecture as opposed to a training dataset. In particular, we reduce the popular combinatorial optimization problem of finding a maximum independent set to a neural network and employ a dataless training scheme to refine the parameters of the network such that those parameters yield the structure of interest. Additionally, we propose a universal graph reduction procedure to handle large-scale graphs. The reduction exploits community detection for graph partitioning and is applicable to any graph type and/or density. Experimental results on both real and synthetic graphs demonstrate that our proposed method performs on par or outperforms state-of-the-art learning-based methods in terms of the size of the found set without requiring any training data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Demonstration of Algorithm for Sensor Placement Optimization using Simulation Data

This deliverable reports FY26 progress in advancing a neural-network-based Green’s-function framework for reconstructing neutron-flux distributions from ex-core measurements and for translating reconstruction requirements into a practical detector-layout strategy. Building on the FY25 formulation, the present work had two main objectives: (1) refine and re-evaluate the reconstruction methodology on an updated Purdue University Reactor Number One (PUR-1) model, and (2) develop a systematic, Green’s-function-guided procedure for boundary detector placement.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Graph Neural Networks for Parameterized Quantum Circuits Expressibility Estimation (Rev.1)

Parameterized quantum circuits (PQCs) are fundamental to quantum machine learning (QML), quantum optimization, and variational quantum algorithms (VQAs). The expressibility of PQCs is a measure that determines their capability to harness the full potential of the quantum state space. It is thus a crucial guidepost to know when selecting a particular PQC ansatz. However, the existing technique for expressibility computation through statistical estimation requires a large number of samples, which poses significant challenges due to time and computational resource constraints. This paper introduces a novel approach for expressibility estimation of PQCs using Graph Neural Networks (GNNs). We demonstrate the predictive power of our GNN model with a dataset consisting of 25,000 samples from the noiseless IBM QASM Simulator and 12,000 samples from three distinct noisy quantum backends. The model accurately estimates expressibility, with root mean square errors (RMSE) of 0.05 and 0.06 for the noiseless and noisy backends, respectively. We compare our model’s predictions with reference circuits from Sim et al. and IBM Qiskit’s hardwareefficient ansatz sets to further evaluate our model’s performance. Our experimental evaluation in noiseless and noisy scenarios reveals a close alignment with ground truth expressibility values, highlighting the model’s efficacy. Moreover, our model exhibits promising extrapolation capabilities, predicting expressibility values with low RMSE for out-of-range qubit circuits trained solely on only up to 5-qubit circuit sets. This work thus provides a reliable means of efficiently evaluating the expressibility of diverse PQCs on noiseless simulators and hardware.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Upscaling Reactive Transport and Clogging in Shale Microcracks by Deep Learning

Fracture networks in shales exhibit multiscale features. A rock system may contain a few main fractures and thousands of microcracks, whose length and aperture are orders of magnitude smaller than the former. It is computationally prohibitive to resolve all the fractures explicitly for such multiscale fracture networks. One traditional approach is to model the small-scale features (e.g., microcracks in shales) as an effective medium. Although this fracture-matrix conceptualization significantly reduces the problem complexity, there are classes of physical processes that cannot be accurately upscaled by effective medium approximations, e.g., microcrack clogging during mineral reactions. Here, we employ deep learning in place of effective medium theory to upscale physical processes in small-scale features. Specifically, we consider reactive transport in a fracture-microcrack network where microcracks can be clogged by precipitation. A deep learning multiscale algorithm is developed, in which the microcracks are upscaled as a wall boundary condition of the main fractures. The wall boundary condition is constructed by recurrent neural networks, which take concentration histories as input and predict the solute transport from main fractures to microcracks. The deep learning multiscale algorithm is firstly employed in specific scenarios, then a general model is developed which can work under various conditions. The new approach is validated against fully resolved simulations and an analytical solution, providing a reliable and efficient solution for problems that cannot be upscaled by effective medium models.

58 GEOSCIENCES↗
NASA NTRSDate not supplied

1D-Convolutional Neural Network Architecture for Generalized Time-Segmentation Tasks

Time segmentation of experimental data is a common and often difficult task. Consequently, it is of interest to automate this type of segmentation to reduce manual inputs, which are labor intensive and less consistent. However, simple thresholding algorithms are often insufficiently robust due either to noise or inconsistent data. This paper proposes a simple 1D convolutional neural net (CNN) architecture as a generalized solution for typical time segmentation tasks. The layer architecture, training methods, and methods for simple customization are described as well as the results of application to three separate arc jet data streams: facility condition segmentation, video highlight segmentation, and calorimeter time-series segmentation.

Machine learning↗
DOE OSTI2023

Semi-supervised graph neural networks for pileup noise removal

Abstract The high instantaneous luminosity of the CERN Large Hadron Collider leads to multiple proton–proton interactions in the same or nearby bunch crossings (pileup). Advanced pileup mitigation algorithms are designed to remove this noise from pileup particles and improve the performance of crucial physics observables. This study implements a semi-supervised graph neural network for particle-level pileup noise removal, by identifying individual particles produced from pileup. The graph neural network is firstly trained on charged particles with known labels, which can be obtained from detector measurements on data or simulation, and then inferred on neutral particles for which such labels are missing. This semi-supervised approach does not depend on the neutral particle pileup label information from simulation, and thus allows us to perform training directly on experimental data. The performance of this approach is found to be consistently better than widely-used domain algorithms and comparable to the fully-supervised training using simulation truth information. The study serves as the first attempt at applying semi-supervised learning techniques to pileup mitigation, and opens up a new direction of fully data-driven machine learning pileup mitigation studies.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

From Latent Dynamics to Meaningful Representations

While representation learning has been central to the rise of machine learning and artificial intelligence, a key problem remains in making the learnt representations meaningful. For this the typical approach is to regularize the learned representation through prior probability distributions. However such priors are usually unavailable or are ad hoc. To deal with this, recent efforts have shifted towards leveraging the insights from physical principles to guide the learning process. In this spirit, we propose a purely dynamics-constrained representation learning framework. Instead of relying on predefined probabilities, we restrict the latent representation to follow overdamped Langevin dynamics with a learnable transition density — a prior driven by statistical mechanics. We show this is a more natural constraint for representation learning in stochastic dynamical systems, with the crucial ability to uniquely identify the ground truth representation. We validate our framework for different systems including a real-world fluorescent DNA movie dataset. Here, we show that our algorithm can uniquely identify orthogonal, isometric and meaningful latent representations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

A review of non-cognitive applications for neuromorphic computing

Abstract Though neuromorphic computers have typically targeted applications in machine learning and neuroscience (‘cognitive’ applications), they have many computational characteristics that are attractive for a wide variety of computational problems. In this work, we review the current state-of-the-art for non-cognitive applications on neuromorphic computers, including simple computational kernels for composition, graph algorithms, constrained optimization, and signal processing. We discuss the advantages of using neuromorphic computers for these different applications, as well as the challenges that still remain. The ultimate goal of this work is to bring awareness to this class of problems for neuromorphic systems to the broader community, particularly to encourage further work in this area and to make sure that these applications are considered in the design of future neuromorphic systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Analytic Theory for the Dynamics of Wide Quantum Neural Networks

Here, parametrized quantum circuits can be used as quantum neural networks and have the potential to outperform their classical counterparts when trained for addressing learning problems. To date, much of the results on their performance on practical problems are heuristic in nature. In particular, the convergence rate for the training of quantum neural networks is not fully understood. Here, we analyze the dynamics of gradient descent for the training error of a class of variational quantum machine learning models. We define wide quantum neural networks as parametrized quantum circuits in the limit of a large number of qubits and variational parameters. Then, we find a simple analytic formula that captures the average behavior of their loss function and discuss the consequences of our findings. For example, for random quantum circuits, we predict and characterize an exponential decay of the residual training error as a function of the parameters of the system. Finally, we validate our analytic results with numerical experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Data-driven modeling and control of an X-ray bimorph adaptive mirror

Adaptive X-ray mirrors are being adopted on high-coherent-flux synchrotron and X-ray free-electron laser beamlines where dynamic phase control and aberration compensation are necessary to preserve wavefront quality from source to sample, yet challenging to achieve. Additional difficulties arise from the inability to continuously probe the wavefront in this context, which demands methods of control that require little to no feedback. In this work, a data-driven approach to the control of adaptive X-ray optics with piezo-bimorph actuators is demonstrated. This approach approximates the non-linear system dynamics with a discrete-time model using random mirror shapes and interferometric measurements as training data. For mirrors of this type, prior states and voltage inputs affect the shape-change trajectory, and therefore must be included in the model. Without the need for assumed physical models of the mirror's behavior, the generality of the neural network structure accommodates drift, creep and hysteresis, and enables a control algorithm that achieves shape control and stability below 2 nm RMS. Using a prototype mirror and ex situ metrology, it is shown that the accuracy of our trained model enables open-loop shape control across a diverse set of states and that the control algorithm achieves shape error magnitudes that fall within diffraction-limited performance.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Neuromorphic Backpropagation Algorithm Software

This software is an implementation of the backpropagation algorithm for Intel's neuromorphic research processor, Loihi. The software implements the neural circuit itself and a demonstration application where the MNIST dataset (a standard test problem) is used to train a neural network.

Sornborger, Andrew↗
DOE OSTI2023

DLIO: A DATA-CENTRIC BENCHMARK FOR DEEP LEARNING APPLICATIONS

SF-22-136 Deep learning has been shown as a successful method for various tasks, and its popularity results in numerous open-source deep learning software tools. Deep learning has been applied to a broad spectrum of scientific domains such as cosmology, particle physics, computer vision, fusion, and astrophysics. Scientists have performed a great deal of work to optimize the computational performance of deep learning frameworks. However, the same cannot be said for I/O performance. As deep learning algorithms rely on big-data volume and variety to effectively train neural networks accurately, I/O is a significant bottleneck on large-scale distributed deep learning training. DLIO, is a novel representative benchmark suite built based on the I/O profiling of the selected workloads. DLIO can be utilized to accurately emulate the I/O behavior of modern deep learning applications. Using DLIO, application developers and system software solution architects can identify potential I/O bottlenecks in their applications and guide optimizations to boost the I/O performance leading to lower training times. The storage vendor can also use DLIO as a guide for designing and optimize the storage and filesystem targeting at deep learning application.

ZHENG, HUIHUO↗
DOE OSTI2023

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗
DOE OSTI2020

Semisupervised Learning for Seismic Monitoring Applications

The impressive performance that deep neural networks demonstrate on a range of seismic monitoring tasks depends largely on the availability of event catalogs that have been manually curated over many years or decades. However, the quality, duration, and availability of seismic event catalogs vary significantly across the range of monitoring operations, regions, and objectives. Semisupervised learning (SSL) enables learning from both labeled and unlabeled data and provides a framework to leverage the abundance of unreviewed seismic data for training deep neural networks on a variety of target tasks. We apply two SSL algorithms (mean-teacher and virtual adversarial training) as well as a novel hybrid technique (exponential average adversarial training) to seismic event classification to examine how unlabeled data with SSL can enhance model performance. In general, we find that SSL can perform as well as supervised learning with fewer labels. We also observe in some scenarios that almost half of the benefits of SSL are the result of the meaningful regularization enforced through SSL techniques and may not be attributable to unlabeled data directly. Lastly, the benefits from unlabeled data scale with the difficulty of the predictive task when we evaluate the use of unlabeled data to characterize sources in new geographic regions. Finally, in geographic areas where supervised model performance is low, SSL significantly increases the accuracy of source-type classification using unlabeled data.

58 GEOSCIENCES↗
NASA NTRS1992

A multilayer perceptron solution to the match phase problem in rule-based artificial intelligence systems

In rule-based AI planning, expert, and learning systems, it is often the case that the left-hand-sides of the rules must be repeatedly compared to the contents of some 'working memory'. The traditional approach to solve such a 'match phase problem' for production systems is to use the Rete Match Algorithm. Here, a new technique using a multilayer perceptron, a particular artificial neural network model, is presented to solve the match phase problem for rule-based AI systems. A syntax for premise formulas (i.e., the left-hand-sides of the rules) is defined, and working memory is specified. From this, it is shown how to construct a multilayer perceptron that finds all of the rules which can be executed for the current situation in working memory. The complexity of the constructed multilayer perceptron is derived in terms of the maximum number of nodes and the required number of layers. A method for reducing the number of layers to at most three is also presented.

Sartori, Michael A.↗
NASA NTRS2000

Speed challenge: a case for hardware implementation in soft-computing

For over a decade, JPL has been actively involved in soft computing research on theory, architecture, applications, and electronics hardware. The driving force in all our research activities, in addition to the potential enabling technology promise, has been creation of a niche that imparts orders of magnitude speed advantage by implementation in parallel processing hardware with algorithms made especially suitable for hardware implementation. We review our work on neural networks, fuzzy logic, and evolvable hardware with selected application examples requiring real time response capabilities.

neural networks fuzzy logic evolvable hardware sof↗