SEARCH · Engineering Papers

Results for “feature learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Feature learning and generalization in deep networks with orthogonal weights

Fully-connected deep neural networks with weights initialized from independent Gaussian distributions can be tuned to criticality, which prevents the exponential growth or decay of signals propagating through the network. However, such networks still exhibit fluctuations that grow linearly with the depth of the network, which may impair the training of networks with width comparable to depth. We show analytically that rectangular networks with tanh activations and weights initialized from the ensemble of orthogonal matrices have corresponding preactivation fluctuations which are independent of depth, to leading order in inverse width. Moreover, we demonstrate numerically that, at initialization, all correlators involving the neural tangent kernel (NTK) and its descendants at leading order in inverse width—which govern the evolution of observables during training—saturate at a depth of ~20, rather than growing without bound as in the case of Gaussian initializations. We speculate that this structure preserves finite-width feature learning while reducing overall noise, thus improving both generalization and training speed in deep networks with depth comparable to width. We provide some experimental justification by relating empirical measurements of the NTK to the superior performance of deep non-linear orthogonal networks trained under full-batch gradient descent on the MNIST and CIFAR-10 classification tasks.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2019

Feature Learning for Multispectral Satellite Imagery Classification Using Neural Architecture Search

Automated classification of remote sensing data is an integral tool for earth scientists, and deep learning has proven very successful at solving such problems. However, building deep learning models to process the data requires expert knowledge of machine learning. We introduce DELTA, a software toolkit to bridge this technical gap and make deep learning easily accessible to earth scientists. Visual feature engineering is a critical part of the machine learning lifecycle, and hence is a key area that will be automated by DELTA. Hand-engineered features can perform well, but require a cross functional team with expertise in both machine learning and the specific problem domain, which is costly in both researcher time and labor. The problem is more acute with multispectral satellite imagery, which requires considerable computational resources to process. In order to automate the feature learning process, a neural architecture search samples the space of asymmetric and symmetric autoencoders using evolutionary algorithms. Since denoising autoencoders have been shown to perform well for feature learning, the autoencoders are trained on various levels of noise and the features generated by the best performing autoencoders evaluated according to their performance on image classification tasks. The resulting features are demonstrated to be effective for Landsat-8 flood mapping, as well as benchmark datasets CIFAR10 and SVHN.

Robert Campbell↗

NASA NTRSDate not supplied

Nearest-Neighbor Machine Learning Feature Selection for Interpretation of Microbial Molecular Signatures from Isotope Ratio Mass Spectrometry Data

Mass spectrometry (MS) promises to be a powerful tool for potential biosignature detection during astrobiological missions on ocean worlds in our solar system. Accurate and generalizable machine learning methods could enhance science return on investment by predicting seawater chemistry and classifying isotopic biosignatures, either as a signature consistent with microbial life (biotic) or as a novelty (unclassified/unique). However, machine learning models are likely to be complex and involve interactions between MS features, making biosignatures difficult to interpret. Feature selection methods provide biological and chemical context that help interpret the mechanisms of machine learning models, but these methods also need the ability to detect complex interactions. Previously, we developed a machine learning feature selection algorithm called nearest-neighbor projected distance regression (NPDR) that has the ability to identify important model features that involve complex interactions and automatically reduce correlation and the dimensionality in a high-dimensional variable space. The standard distance metrics used in NPDR – Manhattan and Euclidean – assume the multivariate data are isotropic, which is often violated in real data due to differences in the covariance between variables. Thus, we extend NPDR to include a random forest distance, and other anisotropic distance metrics, for computing nearest neighbors. We also augment the isotope-ratio MS data with time-series features from the raw MS signal to improve biotic classification. We test NPDR on our novel experimental ocean world seawater analog MS data. We measure isotope fractionations of volatile CO 2 that could be measured in exospheres or plumes. Samples include baseline abiotic conditions using a range of possible seawater chemistry consistent with Europa and Enceladus, and biotic samples that include microbes in these seawaters. We use penalized NPDR with random forest proximity to identify interpretable microbial molecular signatures. We compare features with random forest importance, and we train a classifier that discriminates between biotic and abiotic samples with high accuracy. These ML-trained ocean-world analog MS data could be used to assist in identifying biosignatures during future missions.

geochemistry↗

NASA NTRS1972

An estimation algorithm with learning feature for an adaptive bit synchronizer.

Description of an on-line optimal (in the sense of linear minimum mean square error) sequential bit synchronizer assuming minimal a priori knowledge of the incoming data. The adaptive synchronization algorithms, implemented on a small computer, are capable of accurately estimating pulse transition time, pulse amplitude, and pulse width by invoking the powerful Gauss-Markov theorem. Numerous simulations indicate that the performance of the synchronizer is satisfactory from the point of view of the amount of data required and the speed of convergence.

Chow, L. R.↗

NASA NTRS1995

Capsule Aerothermodynamics. CFD Prediction Methodology; Database for CFD Validation; Capsule Flight Programs: AFE and Apollo Lessons Learned

Features of chemically reacting separated hypersonic flows are identified and issues concerning their analysis and simulation are discussed. Emphasis is placed on flows of high temperature dissociating and ionizing air and current methods for studying and characterizing these flows, including separation, are reviewed. The aeroassist orbital transfer vehicle and its flight trajectory are used for illustration. Thermochemical nonequilibrium phenomena are emphasized and extension of continuum analysis to the high altitude slip-flow regime is considered.

Deiwert, George S.↗

NASA NTRS2018

GEONEX: Progressive Conditional Generative Adversarial Training Using Transfer learning

Obtaining accurate segmentation on large scale images is an open problem in deep learning. The main problem is the amount of labeled data that exists for large scale images. Traditionally, the common solution to this problem is to crop the large images into smaller images to increase the amount of available data and train a Conditional Generative Adversarial Network (CGAN). CGANs are currently the state of the art in image to image translation and provide better accuracy than the traditional method of training a encoder based conv-net architecture to minimize the loss at each pixel. This method can produce noisy and discontinuous images with inaccurate results. We seek to solve this problem by utilizing the concepts of transfer learning and progressive training to create a CGAN that can segment large scale images with a limited amount of labeled data. In transfer learning we recognize that many learned features are applicable to many classes from multiple domains. This introduces the concept of feature reusability, which is the basis for finetuning. Progressive training got its start in training models on the same images at different resolutions. In this work we instead train a GAN on increasing image scales by transferring the weights from the smaller scales to the larger scales. The learned features at the smaller scales are continually reused and applied to larger scales to create a CGAN that can perform accurate segmentation on large scale images. We apply this method to detect building footprints on very high-resolution overhead imagery (e.g Digital Globe and high resolution airborne platforms).

GEONEX↗

DOE OSTI2026

Expanding the Domain of Applicability of Machine Learning Models with Limited Data for Drug Property Prediction

Accurate machine learning models for predicting small molecule interactions with biological targets are essential for therapeutic discovery, biothreat response, and computational drug design, but their performance is often limited for understudied targets with sparse experimental data. To address this challenge, we developed and evaluated methods to improve molecular property prediction under low-data conditions, using the NimA-related kinase (NEK) family as a proof-of-concept. This work focused on two complementary goals within the ATOM Modeling PipeLine (AMPL) and the Generative Molecular Design (GMD) loop: expanding model applicability through transfer learning, representation learning, feature scaling, sampling strategies, and active-learning-inspired compound selection; and enabling efficient virtual screening to prioritize compounds that balance predicted activity, design objectives, and synthetic accessibility.

organic↗

DOE OSTI2024

Z-Target Radiography Postprocessing With A Deep Convolution Neural Network

Analyzing X-ray radiographs is crucial for understanding target behavior in Inertial Confinement Fusion (ICF) and High Energy Density (HED) platforms. However, the density of Magneto Raleigh Taylor (MRT) bands and limitations of target materials often obscure relevant spike growth and density information. To address this issue, machine learning postprocessing techniques can be applied to remove darkened regions in radiography images. In this study, a novel method is presented for removing MRT darkened regions from z-target radiographs using a convolutional neural network (CNN). The CNN, consisting of six layers, treats the darkened regions as noise and employs a mixed loss function and end-to-end frameworks to suppress them while preserving sharpness. The six-layer architecture is designed to effectively learn features when provided with a larger volume of learning space. Each layer is optimized using a mixed loss function that combines a standard loss pixel approach with a multi-scaled structural similarity index loss, which considers luminance, contrast, and structure in local neighborhoods. This approach is particularly beneficial for capturing the stochastic structure of MRT limbs. Due to the limited availability of experimental data, training is conducted using synthetic target radiography from 3D Alegra simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS2023

Predicting Two-Dimensional Airfoil Performance Using Graph Neural Networks

Computer simulations require the use of meshes to simulate geometries. These meshes capture important geometric features of the design and can be used in machine learning modeling. This report explores the use of graph neural networks (GNNs) to learn features from two-dimensional (2D) airfoil designs represented as a set of nodes connected using edges. This type of network is common in aerospace applications: most geometries are represented as a mesh in order to perform analysis. The objective of this work is to use GNNs to predict the performance of 2D airfoils generated using the program XFOIL. The predicted performance parameters include bulk quantities such as coefficients of lift (C L ), drag (C d , C dp ), moment (C m ), and node-specific quantities such as coefficient of pressure (C p ). In this report, a spline convolutional graph-based neural network is compared with deep learning neural networks to predict both bulk and node-specific quantities. The findings indicate the GNNs are able to predict bulk quantities quite well; however, when the number of outputs is increased, the deep neural network (DNN) proves to be better in its prediction capability. Two different normalization strategies were compared in the training of both GNNs and DNNs: minmax and standard deviation. In both types of networks, standard deviation scaling proved to be the best.

machine learning↗

NASA NTRSDate not supplied

Anomaly Detection in Flight Operational Data Using Deep Learning

In this session, we demonstrate two recently developed deep learning models for anomaly detection in flight operational data by the Data Sciences Group at NASA Ames Research Center. The first model is Convolutional Variational Auto-Encoder (CVAE) [1], which is an unsupervised deep encoder-decoder model, designed specifically for finding anomalies in heterogeneous multivariate time series data. We will demonstrate its application to finding anomalies in streaming data from NASA’s Digital Information Platform’s Fuser source. CVAE identifies data instances that are not representative of expected nominal behavior as anomalous. Since it is an unsupervised approach, the flagged anomalies will need to be reviewed by the subject matter experts (SMEs) for validation and labeling and is designed to assist with vulnerability discovery within Safety Monitoring System programs. The second model is Robust and Explainable Semi-supervised Anomaly Detection (RESAD) model [2], which builds on CVAE to allow learning from both minimally labeled data (previously reviewed by the SMEs) as well as majority unlabeled data. RESAD takes advantage of graph theoretic techniques to propagate the labels from the labeled data to the unlabeled data based on a pre-defined similarity metric and structures the learned feature space from flight time-series so that data of the same class would cluster tightly together. This model characteristic is enabled by training with an augmented loss function and allows learning of a more informative feature space for down-stream tasks such as search and active learning. We demonstrate RESAD using data from the NASA DASHlink project [3].

anomaly detection↗

NASA NTRSDate not supplied

Onboard Hyperspectral Image Classification via Transfer Learning for Communication-Limited Spacecraft

Employing deep-learning and artificial-intelligence (AI) techniques onboard spacecraft can dramatically improve priority data selection to ensure more effective use of the available downlink. However, deployment of effective deep-learning models requires significant training on the ground, which may not be feasible, due to limited data available in an unexplored environment. Therefore, this research explores building robust classification models for onboard data processing where training data is highly limited using transfer-learning techniques. In this paper, we focus on the use case of hyperspectral imaging for remote sensing, a domain where the high dimensionality of the data from the sensor can rapidly saturate the downlink bandwidth. With this bottleneck, there is an impending need to autonomously and robustly classify data onboard to optimize downlink of high-impact measurements, thus maximizing the scientific utility per bit transmitted to the ground. This paper examines the use of deep neural networks onboard for hyperspectral image classification in a communication-limited scenario to analyze how the models perform with limited training data. The use of transfer learning can ameliorate the issue of poor generalization by transferring features learned from training on a large source dataset for one classification task to the target classification task with limited training data. For two deep-learning models from literature, we compare the accuracy of the models trained using transfer learning to models trained from scratch using a random weight initialization with varying amounts of training data. We demonstrate the feasibility and performance of running inference of the deep-learning models on representative flight-like hardware.

Advanced Avionics, Machine Learning, Data Processi↗

NASA NTRSDate not supplied

Compression of Solar Spectroscopic Observations: a Case Study of MgII k Spectral Line Profiles Observed by NASA’s IRIS Satellite

In this study we extract the deep features and investigate the compression of the MgII k spectral line profiles observed in quiet Sun regions by NASA’s IRIS satellite. The data set of line profiles used for the analysis was obtained on April 20th, 2020, at the center of the solar disc, and contains almost 300,000 individual MgII k line profiles after data cleaning. The data are separated into train and test subsets. The train subset was used to train the autoencoder of the varying embedding layer size. The early stopping criterion was implemented on the test subset to prevent the model from overfitting. Our results indicate that it is possible to compress the spectral line profiles more than 27 times (which corresponds to the reduction of the data dimensionality from 110 to 4) while having a 4DN average reconstruction error, which is comparable to the variations in the line continuum. The mean squared error and the reconstruction error of even statistical moments sharply decrease when the dimensionality of the embedding layer increases from 1 to 4 and almost stop decreasing for higher numbers. The observed occasional improvements in training for values higher than 4 indicate that a better compact embedding may potentially be obtained if other training strategies and longer training times are used. The features learned for the critical four-dimensional case can be interpreted. In particular, three of these four features mainly control the line width, line asymmetry, and line dip formation respectively. The presented results are the first attempt to obtain a compact embedding for spectroscopic line profiles and confirm the value of this approach, in particular for feature extraction, data compression, and denoising.

SMD↗

DOE OSTI2025

Subspace-Driven Learning for Anomaly Detection in Process Transients

Nuclear power plant (NPP) monitoring and diagnostic centers are actively investigating and implementing automated anomaly detection algorithms to help plants catch anomalies sooner, thereby preventing or reducing the duration of unexpected shutdowns. Current machine learning-based anomaly detection methods are expected to be highly effective during stable, full-power operations because NPPs typically operate as baseload power generators, meaning there are extensive operating data available from plant equipment. However, it is expected that anomaly detection methods will face significant challenges during transient conditions (i.e., when power output falls below full power) because plants only occasionally operate at these lower power levels, generating sparse transient operational data, and resulting in false alarms or missed detections. Here, to address this issue, transfer learning is used, which for this problem leverages knowledge (in the form of learned features) from stable, full-power operations to improve detection accuracy during transient conditions, even with limited data. In this effort, a novel subspace approach is developed to transfer a subset of the data features from full power operation to transients. This approach is validated through experiments using synthetic data and was found to outperform two baseline transfer learning approaches in anomaly detection performance across a range of amounts of transient data used in the training process.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

NASA NTRS1993

Knowledge acquisition and interface design for learning on demand systems

The rapid changes in our world precipitated by technology have created new problems and new challenges for education and training. A knowledge 'explosion' is occurring as our society moves toward a service oriented economy that relies on information as the major resource. Complex computer systems are beginning to dominate the workplace, causing alarming growth and change in many fields. The rapidly changing nature of the workplace, especially in fields related to information technology, requires that our knowledge be updated constantly. This characteristic of modern society poses seemingly unsolvable instructional problems involving coverage and obsolescence. The sheer amount of information to be learned is rapidly increasing, while at the same time some information becomes obsolete in light of new information. Education, therefore, must become a lifelong process that features learning of new material and skills as needed in relation to the job to be done. Because of the problems cited above, the current model of learning in advance may no longer be feasible in our high-technology world. In many cases, learning in advance is impossible because there are simply too many things to learn. In addition, learning in advance can be time consuming, and often results in decontextualized knowledge that does not readily transfer to the work environment. The large and growing discrepancy between the amount of potentially relevant knowledge available and the amount a person can know and remember makes learning on demand an important alternative to current instructional practices. Learning on demand takes place whenever an individual must learn something new in order to perform a task or make a decision. Learning on demand is a promising approach for addressing the problems of coverage and obsolescence because learning is contextualized and integrated into the task environment rather than being relegated to a separate phase that precedes work. Learning on demand allows learners to see for themselves the usefulness of new knowledge for actual problem situations, thereby increasing the motivation for learning new information. Finally, learning on demand makes new information relevant to the task at hand, leading to more informed decision making, better quality products, and improved performance.

Nelson, Wayne A.↗

DOE OSTI2026

An interactive machine learning platform for analyzing multi-particle coincidence data from cold target recoil ion momentum spectroscopy

We present SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training), a comprehensive software platform for analyzing tabulated high-dimensional multi-particle coincidence data from Cold Target Recoil Ion Momentum Spectroscopy (COLTRIMS) experiments. The software addresses critical challenges in modern momentum spectroscopy by integrating advanced machine learning techniques with physics-informed analysis in an interactive web-based environment. SCULPT implements uniform manifold approximation and projection for non-linear dimensionality reduction to reveal correlations in high-dimensional data. We also discuss potential extensions to deep autoencoders for feature learning and genetic programming for automated discovery of physically meaningful observables. A novel adaptive confidence scoring system provides quantitative reliability assessments by evaluating user-selected clustering quality metrics with predefined weights that reflect each metric’s robustness. The platform features configurable molecular profiles for different experimental systems, interactive visualization with selection tools, and comprehensive data filtering capabilities. Utilizing a subset of SCULPT’s capabilities, we analyze photo-double-ionization data measured using the COLTRIMS method for three-body dissociation of the D 2 O molecule, revealing distinct fragmentation channels and their correlations with physics parameters. The software’s modular architecture and web-based implementation make it accessible to the broader atomic and molecular physics community, significantly reducing the time required for complex multi-dimensional analyses. This opens the door to finding and isolating rare events exhibiting non-linear correlations on the fly during experimental measurements, which can help steer exploration and improve the efficiency of experiments.

Artificial neural networks↗

NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗

NASA NTRS2022

Microstructure Segmentation With Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation intersection over union accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures were trained on over 100,000 labeled microscopy images from 54 material classes. These pre-trained encoders were then embedded into multiple segmentation architectures including UNet and DeepLabV3+ to evaluate segmentation performance on created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2% reduction in relative intersection over union error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗

DOE OSTI2025

High‐Asymmetry Metasurface: A New Solution for Terahertz Resonance via Active Learning‐Augmented Diffusion Model

Terahertz (THz) metamaterials with high‐figure‐of‐merit (high‐FoM) performance resonance are essential for advancing sensors, detectors, and imagers. Conventional designs focus on symmetric or low‐asymmetry geometric structures, leaving high‐asymmetry designs largely unexplored due to the inefficiency of trial‐and‐error‐based rational design. Recent deep learning techniques offer automation and acceleration but are constrained by the need for large datasets inherent to their data‐driven nature. Here, a novel prior knowledge‐guided generative model augmented by a physics‐constrained active learning mechanism to design high‐asymmetry metamaterials. An advanced diffusion model learns features from a small set of classical structures with high‐FoM THz resonance and generates new high‐asymmetry structures. To mitigate the limited number of classical structures, the generated high‐asymmetry structures are actively selected and integrated into the initial training dataset based on their physical characteristics. Experimental results demonstrate the superior resonance performance of the generated high‐asymmetry metamaterials over classical designs, exhibiting improvements exceeding 30% in key resonance metrics. Remarkably, this performance is attained using only 68 classical structures as the initial training dataset, significantly reducing the data requirements for deep learning‐based metamaterial design. The proposed scheme for generating high‐asymmetry structures provides a new effective and efficient solution for high‐FoM resonance, expanding applications in high‐sensitivity THz metadevices.

diffusion model↗