Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “binary neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Classifying multispectral data by neural networks

Several energy functions for synthesizing neural networks are tested on 2-D synthetic data and on Landsat-4 Thematic Mapper data. These new energy functions, designed specifically for minimizing misclassification error, in some cases yield significant improvements in classification accuracy over the standard least mean squares energy function. In addition to operating on networks with one output unit per class, a new energy function is tested for binary encoded outputs, which result in smaller network sizes. The Thematic Mapper data (four bands were used) is classified on a single pixel basis, to provide a starting benchmark against which further improvements will be measured. Improvements are underway to make use of both subpixel and superpixel (i.e. contextual or neighborhood) information in tile processing. For single pixel classification, the best neural network result is 78.7 percent, compared with 71.7 percent for a classical nearest neighbor classifier. The 78.7 percent result also improves on several earlier neural network results on this data.

Telfer, Brian A.↗
DOE OSTI2022

High performance FPGA embedded system for machine learning based tracking and trigger in sPhenix and EIC

We present a comprehensive end-to-end pipeline to classify triggers versus background events in this paper. This pipeline makes online decisions to select signal data and enables the intelligent trigger system for efficient data collection in the Data Acquisition System (DAQ) of the upcoming sPHENIX and future EIC (Electron-Ion Collider) experiments. Starting from the coordinates of pixel hits that are lightened by passing particles in the detector, the pipeline applies three-stage of event processing (hits clustering, track reconstruction, and trigger detection) and labels all processed events with the binary tag of trigger versus background events. The pipeline consists of deterministic algorithms such as clustering pixels to reduce event size, tracking reconstruction to predict candidate edges, and advanced graph neural network-based models for recognizing the entire jet pattern. In particular, we apply the message-passing graph neural network to predict links between hits and reconstruct tracks and a hierarchical pooling algorithm (DiffPool) to make the graph-level trigger detection. We obtain an impressive performance (≥70% accuracy) for trigger detection with only 3200 neuron weights in the end-to-end pipeline. We deploy the end-to-end pipeline into a field-programmable gate array (FPGA) and accelerate the three stages with speedup factors of 1152, 280, and 21, respectively.

Instruments & Instrumentation↗
NASA NTRS1988

On the stability, storage capacity, and design of nonlinear continuous neural networks

The stability, capacity, and design of a nonlinear continuous neural network are analyzed. Sufficient conditions for existence and asymptotic stability of the network's equilibria are reduced to a set of piecewise-linear inequality relations that can be solved by a feedforward binary network, or by methods such as Fourier elimination. The stability and capacity of the network is characterized by the post synaptic firing rate function. An N-neuron network with sigmoidal firing function is shown to have up to 3N equilibrium points. This offers a higher capacity than the (0.1-0.2)N obtained in the binary Hopfield network. Moreover, it is shown that by a proper selection of the postsynaptic firing rate function, one can significantly extend the capacity storage of the network.

Guez, Allon↗
DOE OSTI2021

Measurement of hybrid rocket solid fuel regression rate for a slab burner using deep learning

This study presents an imaging-based deep learning tool to measure the fuel regression rate in a 2D slab burner experiment for hybrid rocket fuels. The slab burner experiment is designed to verify mechanistic models of reacting boundary layer combustion in hybrid rockets by the measurement of fuel regression rates. A DSLR camera with a high intensity flash is used to capture images throughout the burn and the images are then used to find the fuel boundary to calculate the regression rate. A U-net convolutional neural network architecture is explored to segment the fuel from the experimental images. Here, a Monte-Carlo Dropout process is used to quantify the regression rate uncertainty produced from the network. The U-net computed regression rates are compared with values from other techniques from literature and show error less than 10%. An oxidizer flux dependency study is performed and shows the U-net predictions of regression rates are accurate and independent of the oxidizer flux, when the images in the training set are not over-saturated. Training with monochrome images is explored and is not successful at predicting the fuel regression rate from images with high noise. The network is superior at filtering out noise introduced by soot, pitting, and wax deposition on the chamber glass as well as the flame when compared to traditional image processing techniques, such as threshold binary conversion and spatial filtering. U-net consistently provides low error image segmentations to allow accurate computation of the regression rate of the fuel.

42 ENGINEERING↗
NASA NTRS1991

Encoding unique global minima in nested neural networks

Nested neural networks are constructed from outer products of patterns over (-1,0,1)N, whose nonzero bits define subnetworks and the subcodes stored in them. The set of permissible words, which are network-size binary patterns composed of subcode words that agree in their common bits, is characterized, and their number is derived. It is shown that if the bitwise products of the subcode words are linearly independent, the permissible words are the unique global minima of the Hamiltonian associated with the network.

Baram, Yoram↗
DOE OSTI2022

Artificial Neural Networks for In-Cycle Prediction of Knock Events

Downsized turbocharged engines have been increasingly popular in modern light-duty vehicles due to their fuel efficiency benefits. However, high power density in such engines is achieved thanks to high in-cylinder pressure and temperature conditions that increase knock propensity. Next-cycle control has been studied as a method to reduce the damaging effects of knock by operating the engine in a low knock probability condition. This exploratory study looks at the feasibility of in-cycle knock prediction as a tool for advanced knock control algorithms. A methodology is proposed to 1) choose in-cycle features of the pressure trace that highly correlate with knock events and 2) train artificial neural networks to predict in-cycle knock events before knock onset. The methodology was validated at different operating conditions and different levels of generalization. Precision and recall were used as metrics to evaluate the binary classifier. However, the Fowlkes-Mallows (FM) index was used to compare the result of the clustering algorithm at different operating conditions. The results showed a maximum FM index of 0.7 when the prediction was done at knock onset and a minimum FM index of 0.45 when the prediction was done at spark timing.

42 ENGINEERING↗
DOE OSTI2022

Training a Quantum Annealing Based Restricted Boltzmann Machine on Cybersecurity Data

A restricted Boltzmann machine (RBM) is a generative model that could be used in effectively balancing a cybersecurity dataset because the synthetic data a RBM generates follows the probability distribution of the training data. RBM training can be performed using contrastive divergence (CD) and quantum annealing (QA). QA-based RBM training is fundamentally different from CD and requires samples from a quantum computer. We present a real-world application that uses a quantum computer. Specifically, we train a RBM using QA for cybersecurity applications. The D-Wave 2000Q has been used to implement QA. RBMs are trained on the ISCX data, which is a benchmark dataset for cybersecurity. For comparison, RBMs are also trained using CD. CD is a commonly used method for RBM training. Our analysis of the ISCX data shows that the dataset is imbalanced. We present two different schemes to balance the training dataset before feeding it to a classifier. The first scheme is based on the undersampling of benign instances. The imbalanced training dataset is divided into five sub-datasets that are trained separately. A majority voting is then performed to get the result. Our results show the majority vote increases the classification accuracy up from 90.24% to 95.68%, in the case of CD. For the case of QA, the classification accuracy increases from 74.14% to 80.04%. In the second scheme, a RBM is used to generate synthetic data to balance the training dataset. We show that both QA and CD-trained RBM can be used to generate useful synthetic data. Balanced training data is used to evaluate several classifiers. Among the classifiers investigated, K-Nearest Neighbor (KNN) and Neural Network (NN) perform better than other classifiers. They both show an accuracy of 93%. Our results show a proof-of-concept that a QA-based RBM can be trained on a 64-bit binary dataset. The illustrative example suggests the possibility to migrate many practical classification problems to QA-based techniques. Further, we show that synthetic data generated from a RBM can be used to balance the original dataset.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Data-Driven Framework for Automated Detection of Aircraft-Generated Signals in Seismic Array Data Using Machine Learning

Abstract Ground motions associated with aircraft overflights can cover a significant portion of the seismic data collected by shallowly emplaced seismometers, such as new nodal and Distributed Acoustic Sensing systems. This article describes the first published framework for automated detection of aircraft on single channel and multichannel seismic data. The seismic data are converted to spectrograms in a sliding time window and classified as aircraft or nonaircraft in each window using a deep convolutional neural network trained with analyst-labeled data. A majority voting scheme is used to convert the output from the sequence of sliding time windows onto a decision time sequence for each channel and to combine the binary classifications on the decision time sequences across multiple channels. Precision, recall, and F-score are used to quantify the detection performance of the algorithm on nodal data using fourfold time-series cross validation. By applying our framework to data from the Sage Brush Flats nodal array in Southern California, we provide a benchmark performance and demonstrate the advantage of using an array of sensors.

Geochemistry & Geophysics↗
DOE OSTI2026

Input specific neural networks

Neural networks have emerged as powerful tools for mapping between inputs and outputs. However, their black-box nature limits the ability to encode or impose specific structural relationships between inputs and outputs. Many scientific and engineering problems, such as constitutive modeling in solid mechanics, require networks that can enforce convexity, monotonicity, or other structural constraints to ensure physical consistency. Here, we introduce the Input Specific Neural Network (ISNN), a new architecture that enables multiple, distinct constraints to be imposed on different input subsets for scalar-valued outputs. This framework unifies convex, monotone–convex, monotone, and arbitrary mappings within a single network for the first time. Two ISNN architectures with analytical first- and second-order derivatives are developed. We demonstrate the performance on synthetic toy problems, inverse problems in isotropic hyperelasticity, and finite element simulations. ISNNs achieve improved extrapolation behavior, require fewer invariant inputs than standard input convex networks for polyconvex potentials, and enable significant computational savings via manual differentiation. We also show how ISNNs can be used to learn structural relationships between inputs and outputs via a binary gating mechanism. Particularly, ISNNs are employed to model a homogenized anisotropic free energy potential in a decoupled multiscale setting. The network learns whether or not the potential should be modeled as polyconvex and retains only the relevant layers while using the minimum number of inputs. ISNNs provide a flexible foundation for embedding structural priors into neural networks, enhancing both interpretability and stability. They are broadly applicable across computational mechanics and other scientific domains requiring constrained functional relationships.

Jadoon, Asghar A. [Univ. of Texas, Austin, TX (Uni↗
DOE OSTI2025

Transferable predictions of energetic and structural properties for refractory solid solution alloys across chemical compositions

We present a data-efficient approach to train graph neural networks (GNNs) on density functional theory (DFT) data for accurate and transferable predictions of energetic and structural properties of refractory solid solution alloys in the niobium-tantalum-vanadium (Nb-Ta-V) chemical space. We start by training the GNN model only on DFT data that describes refractory binary alloys niobium-tantalum (Nb-Ta), niobium-vanadium (Nb-V), and tantalum-vanadium (Ta-V) to predict formation enthalpy and root mean squared displacement. Once trained, the GNN predictions are tested on DFT data describing refractory ternary alloys Nb-Ta-V. While, unsurprisingly, direct transferability from binary to ternary is not sufficiently accurate, augmenting the training with only 1% of the available ternary data (uniformly distributed across the entire range of chemical compositions) improves significantly the quality of the GNN predictions. For comparison, we assess the transferability in the opposite direction by training GNN models on ternary Nb-Ta-V data and making predictions on binaries Nb-Ta, Nb-V, and Ta-V, which exhibits notably higher predictive errors. The proposed methodology, which favors transferability from lower-component to higher-component alloys, offers an efficient path towards avoiding the curse of dimensionality incurred when collecting DFT data for discovery and design of multi-component disordered alloys.

Density functional theory calculations↗
DOE OSTI2023

Machine-Guided Design of Oxidation-Resistant Superconductors for Quantum Information Applications

Decoherence in superconducting qubits has long been attributed to two-level systems arising from the surfaces and interfaces present in real devices. A recent significant step in reducing decoherence was the replacement of superconducting niobium by superconducting tantalum, resulting in a tripling of transmon qubit lifetimes (T1). The identity, thickness, and quality of the native surface oxide, is thought to play a major role, as tantalum only has one oxide whereas niobium has several. Here we report the development of a thermodynamic metric to rank materials based on their potential to form a well-defined, thin, surface oxide. We first computed this metric for known binary and ternary metal alloys using data available from the Materials Project and experimentally validated the strengths and limits of this metric through the preparation and controlled oxidation of eight known metal alloys. Then we trained a convolutional neural network to predict the value of this metric from atomic composition and atomic properties. This allowed us to compute the metric for materials that are not present in the Materials Project, including a large selection of known superconductors, and, when combined with Tc, allowed us to identify new candidate superconductors for quantum information science and engineering (QISE) applications. We tested the oxidation resistance of a pair of these predictions experimentally. Our results are expected to lay the foundation for the tailored and rapid selection of improved superconductors for QISE.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Neural architecture search via similarity adaptive guidance

Evolutionary neural network architecture search (ENAS) has attracted the attention of many experts due to its global optimization capabilities to automatically search for convolutional neural network architectures based on the target task. The current search space for ENAS is not to design a fully structured network, but to search for smaller cell architectures to reduce search costs. However, blind search strategies do not effectively utilize the potential experience of the population. In order to utilize the potential experience learned by the current population to guide the evolutionary search of the population, we propose a similarity guided neural network architecture search algorithm based on cell architecture, which utilizes the similarity between pairwise architectures in the population as empirical knowledge learned by the population. Our proposed algorithm provides a novel method for calculating architecture similarity, which calculates architecture similarity separately from the cell and macro-structure. Then we decouple the connections and operations in the cell and calculate connection and operation similarity separately. In addition, we propose adaptive similarity selection and binary tournament selection strategies to enhance the algorithm’s global and local search capabilities and effectively explore the search space. Finally, we design an improved single-point crossover operator to enhance the local search ability of the evolutionary operator. The experimental results show that SAGNAS is a competitive algorithm that achieves 97.44% and 81.60% in CIFAR10 and CIFAR100 with only 1.9 GPU-days spent.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

SwitchX : Gmin-Gmax Switching for Energy-efficient and Robust Implementation of Binarized Neural Networks on ReRAM Xbars

Memristive crossbars can efficiently implement Binarized Neural Networks (BNNs) wherein the weights are stored in high-resistance states (HRS) and low-resistance states (LRS) of the synapses. We propose SwitchX mapping of BNN weights onto ReRAM crossbars such that the impact of crossbar non-idealities, that lead to degradation in computational accuracy, are minimized. Essentially, SwitchX maps the binary weights in such a manner that a crossbar instance comprises of more HRS than LRS synapses. We find BNNs mapped onto crossbars with SwitchX to exhibit better robustness against adversarial attacks than the standard crossbar mapped BNNs, the baseline. Finally, we combine SwitchX with state-aware training (that further increases the feasibility of HRS states during weight mapping) to boost the robustness of a BNN on hardware. We find that this approach yields stronger defense against adversarial attacks than adversarial training, a state-of the-art software defense. We perform experiments on a VGG16 BNN with benchmark datasets (CIFAR-10, CIFAR-100 and TinyImagenet) and use Fast Gradient Sign Method (ϵ = 0.05 to 0.3) and Projected Gradient Descent (ϵ = $\frac{2}{255}$ to $\frac{32}{255}$, α = $\frac{2}{255}$) adversarial attacks. We show that SwitchX combined with state-aware training can yield upto ~35% improvements in clean accuracy and ~6–16% in adversarial accuracies against conventional BNNs. Furthermore, an important by-product of SwitchX mapping is increased crossbar power savings, owing to an increased proportion of HRS synapses, which is furthered with state-aware training. We obtain upto ~21–22% savings in crossbar power consumption for state-aware trained BNN mapped via SwitchX on 16 × 16 and 32 × 32 crossbars using the CIFAR-10 and CIFAR-100 datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

El-CID: a filter for gravitational-wave electromagnetic counterpart identification

ABSTRACT As gravitational-wave (GW) interferometers become more sensitive and probe ever more distant reaches, the number of detected binary neutron star mergers will increase. However, detecting more events farther away with GWs does not guarantee corresponding increase in the number of electromagnetic counterparts of these events. Current and upcoming wide-field surveys that participate in GW follow-up operations will have to contend with distinguishing the kilonova (KN) from the ever increasing number of transients they detect, many of which will be consistent with the GW sky-localization. We have developed a novel tool based on a temporal convolutional neural network architecture, trained on sparse early-time photometry and contextual information for Electromagnetic Counterpart Identification (El-CID). The overarching goal for El-CID is to slice through list of new transient candidates that are consistent with the GW sky localization, and determine which sources are consistent with KNe, allowing limited target-of-opportunity resources to be used judiciously. In addition to verifying the performance of our algorithm on an extensive testing sample, we validate it on AT2017gfo – the only EM counterpart of a binary neutron star merger discovered to date – and AT2019npv – a supernova that was initially suspected as a counterpart of the GW event, GW190814, but was later ruled out after further analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Physics-inspired spatiotemporal-graph AI ensemble for the detection of higher order wave mode signals of spinning binary black hole mergers

We present a new class of AI models for the detection of quasi-circular, spinning, non-precessing binary black hole mergers whose waveforms include the higher order gravitational wave modes ($\ell$, |m|) = {(2,2), (2,1), (3,3), (3,2), (4,4)}, and mode mixing effects in the $\ell$ = 3, |m| = 2 harmonics. These AI models combine hybrid dilated convolution neural networks to accurately model both short- and long-range temporal sequential information of gravitational waves; and graph neural networks to capture spatial correlations among gravitational wave observatories to consistently describe and identify the presence of a signal in a three detector network encompassing the Advanced LIGO and Virgo detectors. We first trained these spatiotemporal-graph AI models using synthetic noise, using 1.2 million modeled waveforms to densely sample this signal manifold, within 1.7 h using 256 NVIDIA A100 GPUs in the Polaris supercomputer at the Argonne Leadership Computing Facility. This distributed training approach exhibited optimal classification performance, and strong scaling up to 512 NVIDIA A100 GPUs. With these AI ensembles we processed data from a three detector network, and found that an ensemble of 4 AI models achieves state-of-the-art performance for signal detection, and reports two misclassifications for every decade of searched data. We distributed AI inference over 128 GPUs in the Polaris supercomputer and 128 nodes in the Theta supercomputer, and completed the processing of a decade of gravitational wave data from a three detector network within 3.5 h. Finally, we fine-tuned these AI ensembles to process the entire month of February 2020, which is part of the O3b LIGO/Virgo observation run, and found 6 gravitational waves, concurrently identified in Advanced LIGO and Advanced Virgo data, and zero false positives. This analysis was completed in one hour using one NVIDIA A100 GPU.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Fast decay of classification error in variational quantum circuits

Variational quantum circuits (VQCs) have shown great potential in near-term applications. However, the discriminative power of a VQC, in connection to its circuit architecture and depth, is not understood. To unleash the genuine discriminative power of a VQC, we propose a VQC system with the optimal classical post-processing—maximum-likelihood estimation on measuring all VQC output qubits. Via extensive numerical simulations, we find that the error of VQC quantum data classification typically decays exponentially with the circuit depth, when the VQC architecture is extensive—the number of gates does not shrink with the circuit depth. This fast error suppression ends at the saturation towards the ultimate Helstrom limit of quantum state discrimination. On the other hand, non-extensive VQCs such as quantum convolutional neural networks are sub-optimal and fail to achieve the Helstrom limit, demonstrating a trade-off between ansatz complexity and classification performance in general. To achieve the best performance for a given VQC, the optimal classical post-processing is crucial even for a binary classification problem. To simplify VQCs for near-term implementations, we find that utilizing the symmetry of the input properly can improve the performance, while oversimplification can lead to degradation.

Zhang, Bingzhi↗
DOE OSTI2024

FY24 Progress Report: SRNL Analysis of ICCWR LCM and WAMS data for Corrosion and Cracking

Algorithms for Machine Learning (ML) and data analysis for the 3013 Surveillance Program have been developed in an ongoing collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). The objective of the algorithms is to automate the identification of corrosion and crack formation in the Inner Container Closure Weld Region (ICCWR) of the canister system used to store Pu-bearing material. Data for corrosion and cracking is collected from large binary files generated by a Laser Confocal Microscope (LCM), the Wide Area 3D Measurement System (WAMS), or,in a recent proposal, by a Scanning Electron Microscope (SEM). The ML software uses the physical attributes in the data files (e.g., one or more of: height, color, and 16-bit grayscale values as functions of position in a plane projection) to detect signs of surface corrosion and cracking after being trained on similar data, with the features to be detected. Although the initial scope included screening for broader indicators of corrosion, e.g., pitting, identification of potential cracks was prioritized for the past several years at the request of program leadership. Labeled training data is essential to developing the ML algorithm, and enhancements to data labeling capability have been developed to address this essential precursor to application of ML routines. Efficient labeling is particularly important in view of the large volume of data required to train ML algorithms and the relative rarity of cracks in the ICCWR data set. The updated program will read binary data from either LCM, WAMS or SEM files, interrogate data attributes, facilitate user labeling of data for training ML algorithms, execute ML algorithms, output parameters from trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. In FY24, hourglass neural networks (HNNs) that were initiated in FY22 were further developed and tested using available LCM data, and their performance was tested against that of the alternative U-Net Neural Network algorithm structure. HNNs along with previously developed Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) comprise a suite of ML tools for identification of cracks in the ICCWR

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS1988

Feedforward, high density, programmable read only neural network based memory system

Neural network-inspired, nonvolatile, programmable associative memory using thin-film technology is demonstrated. The details of the architecture, which uses programmable resistive connection matrices in synaptic arrays and current summing and thresholding amplifiers as neurons, are described. Several synapse configurations for a high-density array of a binary connection matrix are also described. Test circuits are evaluated for operational feasibility and to demonstrate the speed of the read operation. The results are discussed to highlight the potential for a read data rate exceeding 10 megabits/sec.

Daud, Taher↗