Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Deployment of inference as a service at the US CMS Tier-2 data centers

Coprocessors, especially GPUs, will be a vital ingredient of data production workflows at the HL-LHC. At CMS, the GPU-as-a-service approach for production workflows is implemented by the SONIC project (Services for Optimized Network Inference on Coprocessors). SONIC provides a mechanism for outsourcing computationally demanding algorithms, such as neural network inference, to remote servers, where requests from multiple clients are intelligently distributed across multiple GPUs by a load-balancing service. This talk highlights the recent progress in deploying SONIC at selected U.S. CMS Tier-2 data centers. Using realistic CMS Run3 data processing workflows, such as those containing transformer-based algorithms, we demonstrate how SONIC is integrated into the production-like environment to enable accelerated inference offloading. We will present developments from both the client and server sides, including production job and data center configurations for NVIDIA and AMD GPUs. We will also present performance scaling benchmarks and discuss the challenges of operating SONIC in CMS production, such as server discovery, GPU saturation, fallback server logic, etc.

Holzman, Burt↗

Complexity-calibrated benchmarks for machine learning reveal when prediction algorithms succeed and mislead

Abstract Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano’s inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least $$\sim 60\%$$ ∼ 60 % higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines—specifically, large $$\epsilon$$ ϵ -machines—are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

97 MATHEMATICS AND COMPUTING↗

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation

Deep learning has exhibited remarkable results across diverse areas. To understand its success, substantial research has been directed towards its theoretical foundations. Nev- ertheless, the majority of these studies examine how well deep neural networks can model functions with uniform regularities. In this paper, we explore a different angle: how deep neural networks can adapt to varying degrees of smoothness in functions and nonuni- form data distributions across different locations and scales. More precisely, we focus on a broad class of functions defined by nonlinear tree-based approximation methods. This class encompasses a range of function types, such as functions with uniform regularities and discontinuous functions. We develop nonparametric approximation and estimation theories for this class using deep ReLU networks. Our results show that deep neural networks are adaptive to the nonuniform smoothness of functions and nonuniform data distributions at different locations and scales. We apply our results to several function classes, and derive the corresponding approximation and generalization errors. The validity of our results is demonstrated through numerical experiments.

97 MATHEMATICS AND COMPUTING↗

Light-powered end-to-end neutron detection and imaging with an edge-deployed optical AI chip

Neutron detection is widely used in many applications including nuclear physics, nuclear energy, nuclear technologies and nuclear safeguards. Developing an end-to-end neutron detection and imaging workflow paves way towards fully automated processes for many applications. We implemented an automated workflow for neutron detection experiments which use a solid state image sensor to capture neutron hits as a digital image. We deploy the workflow to an edge-based optical neural network (ONN) to increase the radiation-hardness and lifetime of neutron detection instruments. We present a two-stage neural network framework for detection of neutrons at sub-pixel resolution. The first stage uses a region proposal network to efficiently detect and extract neutron hits from the input camera image. The second stage feeds the extracted hits into a fully connected neural network to predict the sub-pixel hit position. The performance of the two-stage framework is evaluated using the edge-based ONN. The results show that we can achieve above 96% neutron detection accuracy as well as sub-pixel and sub-micron position resolution, while enjoying the advantages of the ONN hardware including radiation-hardness, low energy consumption and high computing speed for integrated edge camera and hardware deployment, when compared with electronic counterparts.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Sparse Binary Matrix-Vector Multiplication on Neuromorphic Computers

Neuromorphic computers offer the opportunity for low-power, efficient computation. Though they have been primarily applied to neural network tasks, there is also the opportunity to leverage the inherent characteristics of neuromorphic computers (low power, massive parallelism, collocated processing and memory) to perform non-neural network tasks. Here, we demonstrate how an approach for performing sparse binary matrix-vector multiplication on neuromorphic computers. We describe the approach, which relies on the connection between binary matrix-vector multiplication and breadth first search, and we introduce the algorithm for performing this calculation in a neuromorphic way. We validate the approach in simulation. Finally, we provide a discussion of the runtime of this algorithm and discuss where neuromorphic computers in the future may have a computational advantage when performing this computation.

Schuman, Catherine↗

Automated isotope identification algorithm using artificial neural networks

There is a need to develop an algorithm that can determine the relative activities of radio-isotopes in a large dataset of low-resolution gamma-ray spectra that contains a mixture of many radio-isotopes. Low-resolution gamma-ray spectra that contain mixtures of radio-isotopes often exhibit feature over-lap, requiring algorithms that can analyze these features when overlap occurs. While machine learning and pattern recognition algorithms have shown promise for the problem of radio-isotope identification, their ability to identify and quantify mixtures of radio-isotopes has not been studied. Because machine learning algorithms use abstract features of the spectrum, such as the shape of overlapping peaks and Compton continuum, they are a natural choice for analyzing radio-isotope mixtures. An artificial neural network (ANN) has be trained to calculate the relative activities of 32 radio-isotopes in a spectrum. Furthermore, the ANN is trained with simulated gamma-ray spectra, allowing easy expansion of the library of target radio-isotopes. In this paper we present our initial algorithms based on an ANN and evaluate them against a series measured and simulated spectra.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Application of machine learning for modeling brønsted-guggenheim-scatchard specific ion interaction theory (SIT) coefficients

Machine learning methodologies can provide insight into Brønsted-Guggenheim-Scatchard specific ion interaction theory (SIT) parameter values where experimental data availability may be limited. This study develops and executes machine learning frameworks to model the SIT interaction coefficient, ε. Key findings include successful estimations of ε via artificial neural networks using clustering and value prediction approaches. Additionally, applicability to other chemical parameters is also assessed briefly. Models developed here provide support for a use-case of machine learning in geologic nuclear waste disposal research applications, namely in predictions of chemical behaviors of high ionic strength solutions (i.e., subsurface brines).

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Digital Modeling on Large Kernel Metamaterial Neural Network

Deep neural networks (DNNs) utilized recently are physically deployed with computational units (e.g., CPUs and GPUs). Such a design might lead to a heavy computational burden, significant latency, and intensive power consumption, which are critical limitations in applications such as Internet of Things (IoT), edge computing, and usage of drones. Recent advances in optical computational units (e.g., metamaterial) have shed light on energy-free and light-speed neural networks. However, the digital design of the metamaterial neural network (MNN) is fundamentally limited by its physical limitations, such as precision, noise, and bandwidth during fabrication. Moreover, the unique advantages of MNN’s (e.g., light-speed computation) are not fully explored via standard 3×3 convolution kernels. In this paper, we propose a novel large kernel metamaterial neural network (LMNN) that maximizes the digital capacity of the state-of-the-art (SOTA) MNN with model re-parametrization and network compression, while also considering the optical limitation explicitly. The new digital learning scheme can maximize the learning capacity of MNN while modeling the physical restrictions of meta-optics. With the proposed LMNN, the computation cost of the convolutional front-end can be offloaded to fabricated optical hardware. The experimental results on two publicly available datasets demonstrate that the optimized hybrid design improved classification accuracy while reducing computational latency. In conclusion, the development of the proposed LMNN is a promising step towards the ultimate goal of energy-free and light-speed AI.

97 MATHEMATICS AND COMPUTING↗

Calibrating Bayesian generative machine learning for Bayesiamplification

Recently, combinations of generative and Bayesian deep learning have been introduced in particle physics for both fast detector simulation and inference tasks. These neural networks aim to quantify the uncertainty on the generated distribution originating from limited training statistics. The interpretation of a distribution-wide uncertainty however remains ill-defined. We show a clear scheme for quantifying the calibration of Bayesian generative machine learning models. For a Continuous Normalizing Flow applied to a low-dimensional toy example, we evaluate the calibration of Bayesian uncertainties from either a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights, to gauge their behaviour on unsteady distribution edges. Well calibrated uncertainties can then be used to roughly estimate the number of uncorrelated truth samples that are equivalent to the generated sample and clearly indicate data amplification for smooth features of the distribution.

97 MATHEMATICS AND COMPUTING↗

Learning Optimal Multigrid Smoothers via Neural Networks

Multigrid methods are one of the most efficient techniques for solving large sparse linear systems arising from partial differential equations (PDEs) and graph Laplacians from machine learning applications. One of the key components of multigrid is smoothing, which aims at reducing high-frequency errors on each grid level. However, finding optimal smoothing algorithms is problem-dependent and can impose challenges for many problems. In this paper, we propose an efficient adaptive framework for learning optimized smoothers from operator stencils in the form of convolutional neural networks (CNNs). Here, the CNNs are trained on small-scale problems from a given type of PDEs based on a supervised loss function derived from multigrid convergence theories and can be applied to large-scale problems of the same class of PDEs. Numerical results on anisotropic rotated Laplacian problems and variable coefficient diffusion problems demonstrate improved convergence rates and solution time compared with classical hand-crafted relaxation methods.

97 MATHEMATICS AND COMPUTING↗

Deep learning of parameterized equations with applications to uncertainty quantification

We propose a learning algorithm for discovering unknown parameterized dynamical systems by using observational data of the state variables. Our method is built upon and extends the recent work of discovering unknown dynamical systems, in particular those using deep neural network (DNN). We propose a DNN structure, largely based upon the residual network (ResNet), to not only learn the unknown form of the governing equation but also take into account the random effect embedded in the system, which is generated by the random parameters. Once the DNN model is successfully constructed, it is able to produce system prediction over longer term and for arbitrary parameter values. For uncertainty quantification, it allows us to conduct uncertainty analysis by evaluating solution statistics over the parameter space.

97 MATHEMATICS AND COMPUTING↗

Application of deep artificial neural networks to multi-dimensional flamelet libraries and spray flames

The “curse of dimensionality” has limited the applicability and expansion of tabulated combustion models. While the tabulated flamelet model and other multi-dimensional manifold approaches have shown predictive capability, the associated tabulation involves the storage of large lookup tables, requiring large memory as well as multi-dimensional interpolation subroutines, all implemented during runtime. As such, this work investigates the use of deep artificial neural networks to replace lookup tables in order to reduce the memory footprint and increase the computational speed of tabulated flamelets and related approaches. Specifically, different strategic approaches to training the artificial neural network models are explored and a grouped multi-target artificial neural network is introduced, which takes advantage of the ability of artificial neural networks to map an input space to multiple targets by classifying the species based on their correlation to one another. The grouped multi-target artificial neural network approach is validated by applying it to an n-dodecane spray flame using conditions of the Spray A flame from the Engine Combustion Network and comparing global flame characteristics for different ambient conditions using a well-established large-eddy simulation framework. The same framework is then extended to the simulations of methyl decanoate combustion in a compression ignition engine. The validation studies show that the grouped multi-target artificial neural networks are able to accurately capture flame liftoff, autoignition, two-stage heat release and other quantitative trends over a range of conditions. The use of neural networks in conjunction with the grouping mechanism as performed in the grouped multi-target artificial neural network produces a significant reduction in the memory footprint and computational costs for the code and, thus, widens the operating envelope for higher fidelity engine simulations with detailed mechanisms.

97 MATHEMATICS AND COMPUTING↗

The TENNLab Exploratory Neuromorphic Computing Framework

Spiking, neuromorphic computing systems are in a period of active exploration by the computing community. While they feature computational expressiveness beyond both von Neumann computing models and feed-forward neural networks, they are also challenging to design and program. The TENNLab exploratory neuromorphic computing framework is a software infrastructure, soon to be open-source, whose goal is to enable potential users of spiking, neuromorphic computing systems to develop applications and evaluate computing architectures, and for architecture researchers to develop and evaluate their architectures with a variety of applications. In this letter, we present the software architecture of the TENNLab framework.

97 MATHEMATICS AND COMPUTING↗

Kolmogorov-Arnold wavefunctions

Here, this work investigates Kolmogorov-Arnold network-based (KAN) wave-function Ansätz as viable representations for quantum Monte Carlo simulations. Through systematic analysis of one-dimensional model systems, we evaluate their computational efficiency and representational power against established methods. Our numerical experiments suggest some efficient training methods and we explore how the computational cost scales with desired precision, particle number, and system parameters. Roughly speaking, KANs seem to be 10 times cheaper computationally than other neural-network-based Ansätz . We also introduce a novel approach for handling strong short-range potentials—a persistent challenge for many numerical techniques—which generalizes efficiently to higher-dimensional, physically relevant systems with short-ranged strong potentials common in atomic and nuclear physics.

1-dimensional systems↗

Sensitive and Specific Peak Detection for SELDI-TOF Mass Spectrometry Using a Wavelet/Neural-Network Based Approach

SELDI-TOF mass spectrometer’s compact size and automated, high throughput design have been attractive to clinical researchers, and the platform has seen steady-use in biomarker studies. Despite new algorithms and preprocessing pipelines that have been developed to address reproducibility issues, visual inspection of the results of SELDI spectra preprocessing by the best algorithms still shows miscalled peaks and systematic sources of error. This suggests that there continues to be problems with SELDI preprocessing. In this work, we study the preprocessing of SELDI in detail and introduce improvements. While many algorithms, including the vendor supplied software, can identify peak clusters of specific mass (or m/z) in groups of spectra with high specificity and low false discover rate (FDR), the algorithms tend to underperform estimating the exact prevalence and intensity of peaks in those clusters. Thus group differences that at first appear very strong are shown, after careful and laborious hand inspection of the spectra, to be less than significant. Here we introduce a wavelet/neural network based algorithm which mimics what a team of expert, human users would call for peaks in each of several hundred spectra in a typical SELDI clinical study. The wavelet denoising part of the algorithm optimally smoothes the signal in each spectrum according to an improved suite of signal processing algorithms previously reported (the LibSELDI toolbox under development). The neural network part of the algorithm combines those results with the raw signal and a training dataset of expertly called peaks, to call peaks in a test set of spectra with approximately 95% accuracy. The new method was applied to data collected from a study of cervical mucus for the early detection of cervical cancer in HPV infected women. The method shows promise in addressing the ongoing SELDI reproducibility issues.

59 BASIC BIOLOGICAL SCIENCES↗

Structural Health Monitoring of Microreactor Safety Systems Using Convolutional Neural Networks

Microreactors, a class of modular reactors with net power output of less than 20 MWth, have innovative applications in nuclear and nonnuclear industries due to their portability, reliability, resilience, and high capacity factors. In order to operate microreactors on a wider scale, it is essential to bring down maintenance life-cycle costs while ensuring the integrity of operating such systems. Autonomous operations in microreactors using augmented digital-twin (DT) technology can serve as a cost-effective solution by increasing awareness about the system’s health. Structural health monitoring (SHM) is a key component of nuclear DT frameworks. Artificial neural networks can be beneficial to detect degradation in the nuclear safety systems, such as piping equipment systems, by monitoring the sensor data obtained from the plant and its corresponding structures, systems and components. In this report, an SHM methodology is presented which uses convolutional neural networks to determine degraded locations and their corresponding degradation-severity levels at various locations of nuclear piping equipment systems. A simple pipe system, subjected to seismic loads, is selected to design the post-hazard SHM framework. The effectiveness of the proposed SHM methodology is demonstrated by obtaining high accuracy in detecting degraded locations as well as the severity levels.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Deep Learning Models Augment Analyst Decisions for Event Discrimination

Long-term seismic monitoring networks are well positioned to leverage advances in machine learning because of the abundance of labeled training data that curated event catalogs provide. In this study, we explore the use of Convolutional and Recurrent Neural Networks to accomplish discrimination of explosive and tectonic sources for local distances. Using a 5-year event catalog generated by the University of Utah Seismograph Stations we train models to produce automated event labels using 90 s event spectrograms from 3-component and single-channel sensors. Both network architectures are able to replicate analyst labels above 98%. Most commonly, model error is the result of label error (70% of cases). Accounting for mislabeled events (~1% of the catalog) model accuracy for both models increases to above 99%. Finally, classification accuracy remains above 98% for shallow tectonic events, indicating that spectral characteristics controlled by event depth do not play a dominant role in event discrimination.

58 GEOSCIENCES↗

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture resulting in high-dimensional, difficult to interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper we apply cutting edge techniques from TDA with the goal of gaining insight towards interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers and discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight as to how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.

topological data analysis, deep learning↗