Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised deep learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Automated Threat Recognition For Aviation Security Applications

We have developed a framework for automated threat recognition (ATR) of explosive threat materials for both single-energy and dual-energy X-ray CT systems. Under this framework, two different types of ATR have been developed. The first type of ATR employs supervised machine learning with statistical characterization of target materials for threat identification training. The reliance only on statistical characterization information for threat training uniquely enables this style of ATR to adapt quickly to evolving threats. The second type of ATR employs deep learning through convolutional neural networks. Convolutional neural networks are attractive due to their human-like capacity for learning and strong ability to identify trends and patterns. Although this method is more powerful than the first type, it requires large amounts of training data and therefore is less agile. Each ATR performs threat characterization at the voxel neighborhood level. This approach avoids the use of threat shape as a detection criterion, which is prohibited by DHS and TSA guidelines.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Multivariate Time Series Intermittent Fault Detectionin Controller Area Network CAN

Fault detection in Controller Area Network (CAN) systems is crucial for ensuring the reliability and safety of automotive and industrial applications. This study investigates and compares the effectiveness of time series classification models for supervised fault detection in CAN data. This repository contains the code and data for our benchmarking experiment aimed at detecting intermittent faults in automotive Controller Area Network (CAN) data. The goal of this project is to compare various machine learning (ML) and deep learning (DL) models using different Time Series Cross-Validation (TSCV) techniques to evaluate their effectiveness in a streaming environment for fault detection.

Hespeler, Steven [Oak Ridge National Laboratory (O↗
NASA NTRS2019

Pixel-Based Model For High Latitude Dust Detection

Dust has implications on the energy budget, ocean biodiversity, and economy at regional and global scales. Dust detection relies on spectral sensitivity at visible (RGB) and infrared wavelengths. Radiative properties of high latitude dust and the background surface albedo in these regions (>40°N, >40°S) complicate current dust detection methods. Leveraging supervised machine learning (ML) methods, we propose a new method accounting for regional differences of dust occurrence.

High latitude dust↗
NASA NTRSDate not supplied

Cislunar Trajectory Design and Maneuver Autonomy for NASA's Moon to Mars Architecture

NASA’s Moon to Mars architecture is an ambitious roadmap of manned cislunar and deep space exploration. The extensive amount of orbital assets required will place a significant burden on ground-based resources, such as communication networks and operations facilities. Spacecraft autonomy is essential for maintaining a vast number of complex missions beyond Earth orbit. To achieve full autonomy, spacecraft must be able to employ methods of robust maneuver design without an explicit dependence on commands sent from the ground. This level of autonomy is needed not only for stationkeeping, but also for outbound transfers. To address the need of spacecraft maneuver design autonomy, this work investigates the use of neural networks (NNs) in a supervised learning environment. A supervised learning approach for NNs allows for a curated training data set, consisting exclusively of perturbations applied to a desired mission concept of operations (ConOps). The proposed approach allows humans on the ground to design a specific mission ConOps before flight, then employ NNs to fly the mission robustly and autonomously. This investigation numerically tests maneuver autonomy in four highly sensitive regions of flight: orbit raising, translunar injection burns, powered lunar flybys, and invariant manifold insertion burns. These straining cases are contextualized by testing them in a demonstration mission, targeting an Earth-Moon L3 orbit. The study first establishes feasibility by automating impulsive burn maneuvers. However, some guidance algorithms will need more intensive commands, such as inertial pointing and angular rates. To validate this method, NN maneuver autonomy is applied to a finite burn model of the demonstration mission. The use of sequential, mission specific maneuvers provide an appropriate testbed to demonstrate the robustness of a NN trained on feasible perturbed states. Moreover, these scenarios provide preliminary proof-of-concept for fully autonomous missions that execute maneuvers without dependence upon explicit command uplinks. As a result, the technological advancement proposed in this work may significantly ease the strain on ground-based mission operations. This would enable complex and autonomous mission execution in cislunar and deep space regimes, filling a technology gap required to support future manned missions.

NASA↗
NASA NTRSDate not supplied

Anomaly Detection in Flight Operational Data Using Deep Learning

In this session, we demonstrate two recently developed deep learning models for anomaly detection in flight operational data by the Data Sciences Group at NASA Ames Research Center. The first model is Convolutional Variational Auto-Encoder (CVAE) [1], which is an unsupervised deep encoder-decoder model, designed specifically for finding anomalies in heterogeneous multivariate time series data. We will demonstrate its application to finding anomalies in streaming data from NASA’s Digital Information Platform’s Fuser source. CVAE identifies data instances that are not representative of expected nominal behavior as anomalous. Since it is an unsupervised approach, the flagged anomalies will need to be reviewed by the subject matter experts (SMEs) for validation and labeling and is designed to assist with vulnerability discovery within Safety Monitoring System programs. The second model is Robust and Explainable Semi-supervised Anomaly Detection (RESAD) model [2], which builds on CVAE to allow learning from both minimally labeled data (previously reviewed by the SMEs) as well as majority unlabeled data. RESAD takes advantage of graph theoretic techniques to propagate the labels from the labeled data to the unlabeled data based on a pre-defined similarity metric and structures the learned feature space from flight time-series so that data of the same class would cluster tightly together. This model characteristic is enabled by training with an augmented loss function and allows learning of a more informative feature space for down-stream tasks such as search and active learning. We demonstrate RESAD using data from the NASA DASHlink project [3].

anomaly detection↗
DOE OSTI2022

Building surrogate models of nuclear density functional theory with Gaussian processes and autoencoders

From the lightest Hydrogen isotopes up to the recently synthesized Oganesson (Z = 118), it is estimated that as many as about 8,000 atomic nuclei could exist in nature. Most of these nuclei are too short-lived to be occurring on Earth, but they play an essential role in astrophysical events such as supernova explosions or neutron star mergers that are presumed to be at the origin of most heavy elements in the Universe. Understanding the structure, reactions, and decays of nuclei across the entire chart of nuclides is an enormous challenge because of the experimental difficulties in measuring properties of interest in such fleeting objects and the theoretical and computational issues of simulating strongly-interacting quantum many-body systems. Nuclear density functional theory (DFT) is a fully microscopic theoretical framework which has the potential of providing such a quantitatively accurate description of nuclear properties for every nucleus in the chart of nuclides. Thanks to high-performance computing facilities, it has already been successfully applied to predict nuclear masses, global patterns of radioactive decay like β or γ decay, and several aspects of the nuclear fission process such as, e.g., spontaneous fission half-lives. Yet, predictive simulations of nuclear spectroscopy—the low-lying excited states and transitions between them—or of nuclear fission, or the quantification of theoretical uncertainties and their propagation to basic or applied nuclear science applications, would require several orders of magnitude more calculations than currently possible. However, most of this computational effort would be spent into generating a suitable basis of DFT wavefunctions. Such a task could potentially be considerably accelerated by borrowing tools from the field of machine learning and artificial intelligence. In this paper, we review different approaches to applying supervised and unsupervised learning techniques to nuclear DFT.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Artificial neural network approach for multiphase segmentation of battery electrode nano-CT images

The segmentation of tomographic images of the battery electrode is a crucial processing step, which will have an additional impact on the results of material characterization and electrochemical simulation. However, manually labeling X-ray CT images (XCT) is time-consuming, and these XCT images are generally difficult to segment with histographical methods. We propose a deep learning approach with an asymmetrical depth encode-decoder convolutional neural network (CNN) for real-world battery material datasets. This network achieves high accuracy while requiring small amounts of labeled data and predicts a volume of billions voxel within few minutes. While applying supervised machine learning for segmenting real-world data, the ground truth is often absent. The results of segmentation are usually qualitatively justified by visual judgement. We try to unravel this fuzzy definition of segmentation quality by identifying the uncertainty due to the human bias diluted in the training data. Further CNN trainings using synthetic data show quantitative impact of such uncertainty on the determination of material’s properties. Nano-XCT datasets of various battery materials have been successfully segmented by training this neural network from scratch. We will also show that applying the transfer learning, which consists of reusing a well-trained network, can improve the accuracy of a similar dataset.

25 ENERGY STORAGE↗
NASA NTRS2019

Pixel Based Model For High Latitude Dust Detection

Current methods of dust detection rely on spectral sensitivity at visible (RGB) and infrared wavelengths. However, their application on different regions needs to be tuned to mitigate errors associated with background properties. High latitude dust (HLD) regions are characterized by surface with variable albedos and land cover, thus further complicating the dust detection. Leveraging supervised machine learning (ML) methods, we propose a new method accounting for regional differences of dust occurrence.

High latitude dust↗
DOE OSTI2023

Towards POI-based large-scale land use modeling: spatial scale, semantic granularity, and geographic context

The combination of spatial distribution, semantic characteristics, and sometimes temporal dynamics of POIs inside a geographic region can capture its unique land use characteristics. Most previous studies on POI-based land use modeling research focused on one geographic region and select one spatial scale and semantic granularity for land use characterization. There is a lack of understanding on the impact of spatial scale, semantic granularity, and geographic context on POI-based land use modeling, particularly large-scale land use modeling. In this study, we developed a scalable POI-based land use modeling framework and examined the impact of these three factors on POI-based land use characterization using data from three geographic regions. We developed a unified semantic representation framework for POI semantics that can help fuse heterogeneous POI data sources. Then, by combining POIs with a neural network language model, we developed a spatially explicit approach to learn the embedding representation of POIs and AOIs. We trained multiple supervised classifiers using AOI embeddings as input features to predict AOI land use at different semantic granularities. The classification performance of different land use classes was analyzed and compared across three geographic regions to identify the semantic representativeness of POI-based AOI embedding and the impact of geographic context.

58 GEOSCIENCES↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Learning neural representations for X-ray ptychography reconstruction with unknown probes

X-ray ptychography provides exceptional nanoscale resolution and is widely applied in materials science, biology, and nanotechnology. However, its full potential is constrained by the critical challenge of accurately reconstructing images when the illuminating probe is unknown. Conventional iterative methods and deep learning approaches are often suboptimal, particularly under the low-signal conditions inherent to low-dose and high-speed experiments. These limitations compromise reconstruction fidelity and restrict the broader adoption of the technique. In this work, we introduce the Ptychographic Implicit Neural Representation (PtyINR), a self-supervised framework that simultaneously addresses the object- and probe-recovery problem. By parameterizing both as continuous neural representations, PtyINR performs end-to-end reconstruction directly from raw diffraction patterns without requiring any pre-characterization of the probe. Extensive evaluations demonstrate that PtyINR achieves superior reconstruction quality on both simulated and experimental data, with remarkable robustness under challenging low-signal conditions. Furthermore, PtyINR offers a generalizable, physics-informed framework for addressing probe-dependent inverse problems, making it applicable to a wide range of computational microscopy problems.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Probing criticality with deep learning in relativistic heavy-ion collisions

Systems with different interactions could develop the same critical behavior due to the underlying symmetry and universality. Using this principle of universality, we can embed critical correlations modeled on the 3D Ising model into the simulated data of heavy-ion collisions, hiding weak signals of a few inter-particle correlations within a large particle cloud. Employing a point cloud network with dynamical edge convolution, we are able to identify events with critical fluctuations through supervised learning, and pick out a large fraction of signal particles used for decision-making in each single event.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Model metamers reveal divergent invariances between biological and artificial neural networks

Deep neural network models of sensory systems are often proposed to learn representational transformations with invariances like those in the brain. To reveal these invariances, we generated ‘model metamers’, stimuli whose activations within a model stage are matched to those of a natural stimulus. Metamers for state-of-the-art supervised and unsupervised neural network models of vision and audition were often completely unrecognizable to humans when generated from late model stages, suggesting differences between model and human invariances. Targeted model changes improved human recognizability of model metamers but did not eliminate the overall human–model discrepancy. The human recognizability of a model’s metamers was well predicted by their recognizability by other models, suggesting that models contain idiosyncratic invariances in addition to those required by the task. Metamer recognizability dissociated from both traditional brain-based benchmarks and adversarial vulnerability, revealing a distinct failure mode of existing sensory models and providing a complementary benchmark for model assessment.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

High-dimensional anomaly detection with radiative return in $\mathcal{e}$⁺$\mathcal{e}$⁻ collisions

Experiments at a future $\mathcal{e}$⁺$\mathcal{e}$⁻ collider will be able to search for new particles with masses below the nominal centre-of-mass energy by analyzing collisions with initial-state radiation (radiative return). We show that machine learning methods that use imperfect or missing training labels can achieve sensitivity to generic new particle production in radiative return events. In addition to presenting an application of the classification without labels (CWoLa) search method in $\mathcal{e}$⁺$\mathcal{e}$⁻ collisions, our study combines weak supervision with variable-dimensional information by deploying a deep sets neural network architecture. We have also investigated some of the experimental aspects of anomaly detection in radiative return events and discuss these in the context of future detector design.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

From clutter to clarity: Emergent neural operators via questionnaire metrics

Real-world datasets in chemical engineering and bioengineering processes—such as those from catalytic reactors, multiphase flows, polymerization reactors, bioreactors, and clinical trials—can often be unlabeled or disorganized, rendering the training of existing supervised learning models ineffective at learning the underlying dynamics. To salvage these datasets for decision-making, we first seek to obtain clarity from the cluttered data. Here, we present a framework for developing “structural” generative models, discovering emergent equations, and constructing efficient emulators from scrambled datasets by integrating unsupervised organizational learning techniques (Questionnaires) with advanced deep learning architectures (Deep Hidden Physics Models and Deep Operator Networks). Our approach is demonstrated on two illustrative model systems: (a) a 1D advection–diffusion partial differential equation representing a winding underground pipe and (b) an ensemble of Stuart–Landau oscillators, an agent-based system of coupled ordinary differential equations. In both cases, we successfully reconstruct meaningful spatial, temporal, and parameter embeddings from scrambled data, enabling good predictions of system dynamics. As a result, we highlight the framework’s potential for broader applications, enabling data-driven system identification in fields with inherently disorganized or hidden parameter spaces.

42 ENGINEERING↗
DOE OSTI2024

Sentinel

Network intrusion detection systems (NIDS) are commonplace in network security but they frequently employ algorithms that are computational demanding requiring hardware and software with significant power requirements. Two examples of such resource-intensive algorithms used for network security are regular expression matching and broader signature pattern matching which are commonly used in deep packet inspection (DPI). Network security algorithms that have large power requirements may be a challenge for low-power internet-of-things (IoT) environments, which generally lack the power resources to implement complex security measures like computationally expensive DPI at the edge. Furthermore, IoT environments incorporating 5G standalone networks have network latency constraints beyond just power that make DPI at the edge even more difficult. Programmable logic is ideally suited for machine learning inference for DPI because of its deep instruction level parallelism and single-cycle memory access. Machine learning approaches for DPI have been explored before using the programmable logic of field programmable gate arrays (FPGA) as a potential solution for NIDS approaches that would be power-suitable for IoT. However, those previous programmable logic NIDS approaches utilize either a supervised or unsupervised learning model. Sentinel utilizes the ensemble of these two machine learning approaches known as a semi-supervised approach which has shown promise in NIDS implementations. Sentinel provides a programmable logic implementation of a semi-supervised approach for DPI which operates at much lower power and latency than a GPU implementation with negligible loss of accuracy due to quantization through a logistic regressor.

Anderson, MatthewW [Idaho National Laboratory (INL↗
DOE OSTI2025

Simultaneous prediction of structural properties in epitaxially–grown GaN with quantum and conventional multi–output learning algorithms

Hundreds of GaN thin film crystal plasma–assisted molecular beam epitaxy synthesis experiment records spanning two decades were organized into a dataset correlating the growth experiment design parameters with discrete, binary determinations of crystallinity and surface morphology. Conventional data science techniques as well as both quantum and classical multi–output supervised machine learning algorithms were implemented to investigate the relationships between the operating parameter data and the structural figures of merit. Correlation coefficients, decision tree nodes, p–values, and SHAP values all support substrate temperature and gallium effusion cell conditions as being statistically significant for simultaneously influencing GaN crystallinity and surface morphology. Here, a conventional deep neural network learned best from the data, followed by a quantum–classical hybrid gradient boosting algorithm. When combined with calculations of uncertainty intervals based on VennAbers predictors, machine learning predictions of both structural properties show good agreement with results reported in published experimental literature.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Unleashing the Power of Industrial Big Data through Scalable Manual Labeling

Big Data plays a central role in the remarkable results achieved by Machine Learning (ML) and especially Deep Learning (DL) in the recent years. However, the difficulty in obtaining a reasonable amount of labeled samples limits ML/DL application in various domains, including industrial equipment and system monitoring. In this paper the need for methods that turn manual labeling into a scalable process is highlighted. A real world problem is analyzed for which weak supervision methods, successfully employed in other domains, did not produce acceptable results. An alternative approach based on clustering ensembles is described and tested, achieving good performance.

Paes Leao, Bruno↗