SEARCH · Engineering Papers

Results for “Deep neural operator”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Runway Sign Classifier: A DAL C Certifiable Machine Learning System

In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) ”D”, we upgrade the system to meet the more stringent requirements of Design Assurance Level ”C”. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.

Flight Software↗

NASA NTRS2022

Spatial Grid-Based Object Localization from A Single Passive Sensor: A Deep Learning-Integrated Approach

Ongoing efforts at NASA’s Langley Research Center have produced a single passive sensor system for detecting ground objects and pinpointing their real-world location to a desired level of precision. The Langley center serves as a test range for unmanned aerial systems (UAS) and real-time knowledge about the location of people on campus is needed to inform least-risk UAS flight operations. The proposed system provides this knowledge through a camera combined with a convolutional neural network and an algorithm that projects an imaginary grid of square cells from the ground plane onto the perspective view of the camera. The position of detected objects on the camera’s projected grid determines their location in the real-world. The imaginary grid is easily mapped to a universal coordinate system, such as longitude and latitude, to provide both relative and absolute positional information of the detected objects. This simple system is shown to be accurate and effective, with decisive advantages over alternative multi-sensor and active sensor approaches. Extensions to the system are described to allow adaptation to a variety of other use cases.

object localization↗

NASA NTRS1992

Real-time antenna fault diagnosis experiments at DSS 13

Experimental results obtained when a previously described fault diagnosis system was run online in real time at the 34-m beam waveguide antenna at Deep Space Station (DSS) 13 are described. Experimental conditions and the quality of results are described. A neural network model and a maximum-likelihood Gaussian classifier are compared with and without a Markov component to model temporal context. At the rate of a state update every 6.4 seconds, over a period of roughly 1 hour, the neural-Markov system had zero errors (incorrect state estimates) while monitoring both faulty and normal operations. The overall results indicate that the neural-Markov combination is the most accurate model and has significant practical potential.

Mellstrom, J.↗

NASA NTRS2023

Damage Detection of a Pressure Vessel with Smart Sensing and Deep Learning

Structural Health Monitoring plays a crucial role in ensuring the safety and reliability of critical infrastructure, including pressure vessels involved in various applications. This research reports the damage detection of a pressure box employed in space habitat that operates in harsh environment where both structural failure and bolt joint loosening may occur. These failure modes are extremely hard to model based on first principles. We explore proper sensing mechanism and the associated inverse analysis algorithm that can elucidate the health condition of the pressure box. It is identified that piezoelectric impedance based active interrogation can provide necessary information for damage detection in such a system. Concurrently, deep learning technique leveraging spatial convolutional neural network is synthesized to analyze the raw data acquired and identify different types of damage. By training the deep learning model on a dataset of healthy and various damage scenarios, we can achieve high accuracy in identifying the presence of damage and its type. This research provides a data-driven methodology for structural damage detection using deep learning and has the potential to be extended to various systems with different failure modes.

Yang Zhang↗

NASA NTRS2019

Small target detection for search and rescue operations using distributed deep learning and synthetic data generation

It is important to find the target as soon as possible for search and rescue operations. Surveillance camera systems and unmanned aerial vehicles (UAVs) are used to support search and rescue. Automatic object detection is important because a person cannot monitor multiple surveillance screens simultaneously for 24 hours. Also, the object is often too small to be recognized by the human eye on the surveillance screen. This study used UAVs around the Port of Houston and fixed surveillance cameras to build an automatic target detection system that supports the US Coast Guard (USCG) to help find targets (e.g., person overboard). We combined image segmentation, enhancement, and convolution neural networks to reduce detection time to detect small targets. We compared the performance between the auto-detection system and the human eye. Our system detected the target within 8 seconds, but the human eye detected the target within 25 seconds. Our systems also used synthetic data generation and data augmentation techniques to improve target detection accuracy. This solution may help the search and rescue operations of the first responders in a timely manner.

Chow, Edward↗

NASA NTRSDate not supplied

Toward The Development of Hailstorm Climatologies Derived From Reanalyses and Infared/Passive Microwave Satellite Imagers

Geostationary satellite imagers, such as those of the Geostationary Operational Environmental Satellite (GOES) and Meteosat series, provide both historical and near-real-time observations of cloud top patterns that are commonly associated with severe convection. Environmental conditions favorable for severe weather are thought to be represented well by reanalyses. Predicting exactly where convection and costly storm hazards like hail will occur using models or satellite imagery alone, however, is extremely challenging. The multivariate combination of satellite-observed cloud patterns with reanalysis environmental parameters, linked to United States Next Generation Weather Radar- (NEXRAD-) estimated Maximum Expected Size of Hail (MESH) using a deep neural network (DNN), enables estimation of potentially severe hail likelihood for any observed storm cell. These estimates are specifically designed to make hail likelihood distinctions based on satellite-indicated points of deep convection within environments favorable for storm development. We seek an approach that can be used to estimate climatological hailstorm frequency and risk throughout the historical satellite data record. This presentation demonstrates that statistical distributions of convective parameters from satellite and reanalysis show separation between non-severe/severe hailstorm classes for predictors including overshooting cloud top temperature and area characteristics, convective available potential energy, vertical wind shear, 500 hPa temperature, mid-level lapse rate, precipitable water, and convective inhibition. These complex, multivariate predictor relationships are exploited within a DNN to produce a hail likelihood metric with a critical success index of 0.504 and Heidke skill score of 0.403, which is exceptional among recent analogous hail studies. Furthermore, applications of the DNN to select case studies demonstrate good qualitative agreement between hail likelihood and MESH. These hail classifications are aggregated across an 11-year GOES-12/13 image database to derive a hail frequency and severity climatology, which denotes the Central Plains, the Midwest, and northwestern Mexico as being the most hail-prone regions within the domain studied. Opportunities for training and applying DNN-based hailstorm predictions to recently developed GOES-8/10/12/13/16 and Meteosat Second Generation convective storm detection and characterization climatologies over South America and South Africa, respectively, will also be presented.

Kristopher Bedka↗

NASA NTRS2022

Radiation Tolerance and Mitigation for Neuromorphic Processors

Neuromorphic processors are designed to execute Deep Neural Networks (DNNs) at very high speed using only a fraction of the electrical power needed to run a DNN on a traditional CPU or GPU. This unique capability makes Neuromorphic processors a prime candidate for space systems, where advanced computational tasks like image analysis, depth map reconstruction, or rover control need to be executed in a power-starved environment. In contrast to the growing number of applications of Neuromorphic processors in smart phones, the automotive and robotics domain, the space environment is unforgiving because of extreme temperatures and high levels of radiation. Any space system, operating beyond LEO requires computing hardware that is resilient against radiation effects. However, Neuromorphic processors have not yet been designed or tested for their radiation tolerance. In this report, we consider traditional methods of detection of radiation events and mitigation via redundancy and gauge their effectiveness on DNNs. In contrast to traditional flight software, however, neural networks represent a statistical algorithm, which might affect its resilience against radiation events. We will focus on the analysis of the tolerance of DNNs with respect to radiation events and discuss techniques to detect radiation hits using on-chip triple modular redundancy (TMR) on an Intel Loihi neuromorphic processor and to mitigate radiation damage. We describe an architecture for on-chip TMR for the Intel Loihi and present results of initial experiments.

Neural Networks↗

NASA NTRSDate not supplied

AEGIS: Autonomous Entity Global Intelligence System for Urban Air Mobility

This paper presents a global intelligence system that synthesizes aerial vehicles’ real-time physical data, planned actions, and historical behavior into engineered data frames representing the collective state of the airspace and suitable for efficient machine learning consumption. These data frames are then learnt by a deep neural net to build a prediction model that estimates the expected evolution path of the current state, thereby identifying potential future conflicts. This approach lends itself to an automated early warning system that the aerial vehicles can implement onboard with a suitable edge computing module more efficiently and effectively than non-AI methods, and eventually take preventive or corrective measures towards self/collaborative resolution of the issues. Contrary to a centralized early warning system where all vehicles’ task-space eventually converges to a global optimum state, the presented distributed global intelligence system brings in a balance between local utility functions of each vehicle and the global operating framework. This contributes to effectively handle the potential massive scaling in urban air mobility in the near future.

Artificial Intelligence↗

NASA NTRS2017

Advances in Medical Analytics Solutions for Autonomous Medical Operations on Long-Duration Missions

A review will be presented on the progress made under STMDGame Changing Development Program Funding towards the development of a Medical Decision Support System for augmenting crew capabilities during long-duration missions, such as Mars Transit. To create an MDSS, initial work requires acquiring images and developing models that analyze and assess the features in such medical biosensor images that support medical assessment of pathologies. For FY17, the project has focused on ultrasound images towards cardiac pathologies: namely, evaluation and assessment of pericardial effusion identification and discrimination from related pneumothorax and even bladder-induced infections that cause inflammation around the heart. This identification is substantially changed due to uncertainty due to conditions of fluid behavior under space-microgravity. This talk will present and discuss the work-to-date in this Project, recognizing conditions under which various machine learning technologies, deep-learning via convolutional neural nets, and statistical learning methods for feature identification and classification can be employed and conditioned to graphical format in preparation for attachment to an inference engine that eventually creates decision support recommendations to remote crew in a triage setting.

Medical Decision Support Systems↗

NASA NTRSDate not supplied

Validation of Image-Based Neural Network Controllersthrough Adaptive Stress Testing

Neural networks have become state-of-the-art for computer vision problems because of their ability to efficiently model complex functions from large amounts of data. While neural networks can be shown to perform well empirically fora variety of tasks, their performance is difficult to guarantee.Neural network verification tools have been developed that can certify robustness with respect to a given input image; however,for neural network systems used in closed-loop controllers,robustness with respect to individual images does not address multi-step properties of the neural network controller and itsenvironment. Furthermore, neural network systems interacting in the physical world and using natural images are operating in a black-box environment, making formal verification in-tractable. This work combines the adaptive stress testing (AST)framework with neural network verification tools to search for the most likely sequence of image disturbances that cause the neural network controlled system to reach a failure. Anautonomous aircraft taxi application is presented, and results show that the AST method finds failures with more likely image disturbances than baseline methods. Further analysis of AST results revealed an explainable cause of the failure, giving insight into the problematic scenarios that should be addressed.

Adaptive Stress Testing, Marabou, Deep Neural Netw↗

NASA NTRS1989

SSME fault monitoring and diagnosis expert system

An expert system, called LEADER, has been designed and implemented for automatic learning, detection, identification, verification, and correction of anomalous propulsion system operations in real time. LEADER employs a set of sensors to monitor engine component performance and to detect, identify, and validate abnormalities with respect to varying engine dynamics and behavior. Two diagnostic approaches are adopted in the architecture of LEADER. In the first approach fault diagnosis is performed through learning and identifying engine behavior patterns. LEADER, utilizing this approach, generates few hypotheses about the possible abnormalities. These hypotheses are then validated based on the SSME design and functional knowledge. The second approach directs the processing of engine sensory data and performs reasoning based on the SSME design, functional knowledge, and the deep-level knowledge, i.e., the first principles (physics and mechanics) of SSME subsystems and components. This paper describes LEADER's architecture which integrates a design based reasoning approach with neural network-based fault pattern matching techniques. The fault diagnosis results obtained through the analyses of SSME ground test data are presented and discussed.

Ali, Moonis↗

NASA NTRSDate not supplied

Physics Informed Neural Nets for Systems Health Management

To facilitate and solve the prediction problem, awareness of the current health state of the system is key, since it is necessary to perform condition-based predictions. To accurately predict the future state of any system, it is required to possess knowledge of its current health state and future operational conditions. Development in data-driven algorithms in regression of complex nonlinear functions and classification tasks have generated a growing interest in artificial intelligence for industrial applications. Complex multi-physics models as well as digital twins, once purely built on physics and corresponding simplified lumped parameter iterations, can now benefit from machine learning algorithms to mitigate the lack of understanding of some complex behavior. The research work presents application of physics-informed neural nets application to a representative electric powertrain for unmanned aerial vehicles. The model is composed of physics-derived and empirical equations, integrated with connected networks that are strategically placed within the model to substitute equations that are subject to large uncertainty. Polynomial fit driven by heuristics or empirical observations can be substituted by more flexible networks that can minimize the error between model predictions and observations without being restricted to a predefined functional form. This modeling strategy allows training of networks deep inside the model and unknown parameters in a single learning stage.

Physics Informed↗

NASA NTRS2017

Multi-Objective Reinforcement Learning-Based Deep Neural Networks for Cognitive Space Communications

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

space archtiecture↗

NASA NTRS2017

Multi-Objective Reinforcement Learning-based Deep Neural Networks for Cognitive Space Communications

space archtiecture↗

NASA NTRSDate not supplied

Early Information Parameter-Set Analysis for Satellite Close Approaches using Machine Learning

Understanding orbital mechanics is essential in space flight and navigation applications, and leveraging modern force models for flight path projection remains an important aspect in space mission design and operation. However, force models do not capture all the dynamics or perturbations in the space environment and thus are subject to errors in predicting the state vectors. The further out the predicted miss distance between spacecraft is from the time of closest approach (TCA), the larger the propagated errors in the predicted miss distance at TCA is. The dependency on these force models for spacecraft flight state prediction calls for a more reliable method that can quantify, or even reduce, these propagated errors. With recent advances in the field artificial intelligence, specifically in machine and deep learning algorithms, a model that implements these approaches can improve on the modern force model approach. The goal for this work is to provide an early-information decision-making threshold, in order to prioritize risk assessment implementation, given the ongoing increase of space objects. In analyzing the relationship of several parameters from conjunction data messages(CDMs) and solar information, early information becomes viable in miss distance prediction with unsupervised learning techniques, which learn the parameters that are linked together with miss distance and probability of collision (Pc) variables. Another approach implemented for identifying relationships within CDMs is supervised learning, in which a shallow neural network binary classifier learns to distinguish events with Pc values¡108. These parameters detected in the unsupervised process are then applied to a regression neural network, which predicts the miss distance at TCA for a specific event within a given uncertainty bound. For the regression neural network, a Long Short Term Memory (LSTM) neural network is implemented, which yields memory about each time step in an event. Using an LSTM network, the model learns to predict miss distance within 0.2km of the value measured at TCA. Although there is a limited amount of "close miss" data to train a network, the network learns to associate parameters, like large energy dissipation rates with the secondary object, with an elevated Pc

Brianna I. Robertson↗

NASA NTRSDate not supplied

Predicting Adverse Events and their Precursors in Aviation Using Multi-Class Multiple-Instance Learning

In recent years, there has been a rapid growth in the application of machine learning techniques that leverage aviation data collected from commercial airline operations to improve safety. Anomaly detection and predictive maintenance have been the main targets for machine learning applications. However, this paper focuses on the identification of precursors, which is a relatively newer application. Precursors are events correlated with adverse events that happen prior to the adverse event itself. Therefore, precursor mining provides many benefits including understanding the reasons behind a safety incident and the ability to identify signatures, which can be tracked throughout a flight to alert the operators of an potential upcoming adverse event. This work proposes using the multiple-instance learning (MIL) framework, a weakly supervised learning task, combined with a carefully designed Multi-Head Convolutional Neural Networks-Recurrent Neural Networks (MHCNN-RNN) architecture to predict different type of adverse events for any given flights and identify their precursors with little to no post-processing.Results obtained show that the MHCNN-RNN is able to accurately forecast high speed and high path angle events during the approach, and that it is also capable of determining the aircraft’s parameters that are correlated to these events. These parameters can be considered precursors to the events.

multiple instance learning↗

NASA NTRS2019

Deep Learning Method for Detecting Precursors to Adverse Events

With the recent advancements in Deep Learning methods, the ability to model large complex heterogeneous data sets are fundamentally changing industry and research. Coupled with hardware improvements, and ease of implementation, a wide variety of deep neural network architectures can quickly be developed to solve a sweeping range of problems such as: object detection in images, automatic healthcare diagnosis using heterogenous data sources, real time language translating and sentence prediction, upscaling low resolution images, and forecasting of multivariate timeseries. Generally, many of these architectures outperform classical machine learning approaches in their respective tasks, however, this typically comes at a cost of interpretability. These black box algorithms generally suffer from lack of transparency in both model complexity as well as the rationale behind the prediction. This lack of comprehension, is driving an emerging area of interest in “Explainable AI”. An algorithm called: “Deep Temporal Multiple Instance Learning”1 was a recently developed to identify precursors to adverse events and has been applied in the aviation domain. The deep learning architecture is designed to capture the evolution of the probability of the outcome over the time preceding the adverse event using a multiple instance learning approach as illustrated in Figure 1. Precursors are defined when the probability of the event has exceeded a threshold at some point in the timeseries, at which point, a sensitivity analysis is performed to determine contributing factors. The contributing factors are used to explain and define the precursor during the periods where the probability score is high. The identified contributing factors are then presented to subject matter experts to provide objective insights into the leading factors associated with the particular adverse event. The algorithm has been tested on flight data from a commercial airline and has the ability to discover precursors to known adverse events that take the form of safety critical operations, such as unstable approach events on final approach. Apart from detecting precursors to adverse events, the converse can also be leveraged to discover corrective actions. These positive actions manifest themselves as periods in the timeseries when the precursor score has been lowered from an elevated state; meaning that if the system had been left uncorrected, it would have eventually reached the adverse event state. Characterizing these state changes can help identify successful interventions that may not have been known before. Policy makers and procedure designers can use this additional knowledge to craft more safety and efficient resilient procedures for future operations and therefore improve the overall performance of the National Airspace.

Matthews, Bryan L.↗

NASA NTRS2019

Design of Experiment for the Measurement of Aerosol Droplet Size Distribution of Temperature-Controlled Thermally Atomized Printed Electronic Inks

In-Space Manufacturing (ISM) centers around NASA’s growing need and ability to produce space technologies on demand in space. As the future of long term presence in space and deep space exploration approach, fundamental questions of our dependence on earth resupply to Low Earth Orbit (LEO) remain unanswered. ISM is leading various effort to evaluate the feasibility of producing essential spares and redundant parts on demand to enable a sustainable space-based supply chain model for part resupply. Among the parts and systems being considered, Avionics form the neural network of modern day aircraft and space vehicles providing a wealth of information ranging from Guidance, Navigation, and Control (GN&C) systems to on board Environmental Control and Life Support (ECLS) systems. Recent advances in the use of Aerosol Jet Technology to print Avionics components ranging from electrical traces on a circuit board to complex transistors and sensors raise the possibility of using such technology to reproduce or recreate electronic parts on demand with the help of custom electronics 3D printers. The challenge herein lies within the ability of such printers to generate and deposit an aerosol of electronic material utilizing processes independent of or enhanced by gravity to ensure controllably identical or improved behavior of the aerosol in an International Space Station (ISS) laboratory and on the ground. The behavior as well as the hazards and properties associated with such aerosols in a microgravity environment must be understood well in order to merit a feasible approach to utilizing them for manufacturing in space. In this report, we outline the experimental setup of a modified conventional vaping device to be used as the ideal gravity independent thermal atomization mechanism to generate aerosol. Our objective is to identify the ideal mass, density, and volume of our aerosolized droplets of ink to conclude that there exist a threshold of aerosolized ink droplet sizes that are indeed independent of the effects of gravity and remain stable after atomization. We use a Malvern Spraytec® Spray Particle Size Analyzer to perform real-time laser diffraction measurement of our ink droplets during atomization. The droplet size between conductive ink, dielectric ink and vegetable glycerin have been measured and contrasted. Furthermore, the mechanism of thermal atomization versus traditional pneumatic and ultrasonic atomization for operation in microgravity have been explored.

Wu, Chih-Hao↗