Towards Assurance for Safety-Critical, Machine Learning-enabled Aviation Systems
Explore the source record for details and available documents.
Neural networks↗
SEARCH · Engineering Papers
Results for “metric learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Anomaly Detection in Flight Operational Data Using Deep Learning
In this session, we demonstrate two recently developed deep learning models for anomaly detection in flight operational data by the Data Sciences Group at NASA Ames Research Center. The first model is Convolutional Variational Auto-Encoder (CVAE) [1], which is an unsupervised deep encoder-decoder model, designed specifically for finding anomalies in heterogeneous multivariate time series data. We will demonstrate its application to finding anomalies in streaming data from NASA’s Digital Information Platform’s Fuser source. CVAE identifies data instances that are not representative of expected nominal behavior as anomalous. Since it is an unsupervised approach, the flagged anomalies will need to be reviewed by the subject matter experts (SMEs) for validation and labeling and is designed to assist with vulnerability discovery within Safety Monitoring System programs. The second model is Robust and Explainable Semi-supervised Anomaly Detection (RESAD) model [2], which builds on CVAE to allow learning from both minimally labeled data (previously reviewed by the SMEs) as well as majority unlabeled data. RESAD takes advantage of graph theoretic techniques to propagate the labels from the labeled data to the unlabeled data based on a pre-defined similarity metric and structures the learned feature space from flight time-series so that data of the same class would cluster tightly together. This model characteristic is enabled by training with an augmented loss function and allows learning of a more informative feature space for down-stream tasks such as search and active learning. We demonstrate RESAD using data from the NASA DASHlink project [3].
anomaly detection↗
Reuse metrics and measurement: A framework
The lessons learned and experience gleaned are described by those who have started to implement the reuse metrics and measurement framework used in controlling the development of common avionics and software for its affiliated aircraft programs. The framework was developed to permit the measurement of the long term cost/benefits resulting from the creation and use of Reusable Software Objects (RSOs). The framework also monitors the efficiency and effectiveness of the Software Reuse Library (SRL). The metrics and measurement framework is defined which was established to allow some determinations and findings to be made relative to software reuse. Seven criteria are discussed which were used to guide the establishment of the proposed reuse framework. Object recapture and creation metrics are explained along with their normalized use in effort, productivity, and quality determination. A single and multiple reuse instance version of a popular cost model is presented which uses these metrics and the measurement scheme proposed to predict the software effort and duration under various reuse assumptions. Studies in using this model to predict actuals taken from the RCI data base of over 1000 completed projects is discussed.
Reifer, Donald J.↗
The widest practicable dissemination: The NASA technical report server
The search for innovative methods to distribute NASA's information lead a gross-roots team to create the NASA Technical Report Server (NTRS), which uses the World Wide Web and other popular Internet-based information systems as search engines. The NTRS is an inter-center effort which provides uniform access to various distributed publication servers residing on the Internet. Users have immediate desktop access to technical publications from NASA centers and institutes. This paper presents the NTRS architecture, usage metrics, and the lessons learned while implementing and maintaining the services over the initial 6-month period. The NTRS is largely constructed with freely available software running on existing hardware. NTRS builds upon existing hardware and software, and the resulting additional exposure for the body of literature contained will allow NASA to ensure that its institutional knowledge base will continue to receive the widest practicable and appropriate dissemination.
Nelson, Michael L.↗
The widest practicable dissemination: The NASA technical report server
The National Aeronautics and Space Act of 1958 established NASA and charged it to 'provide for the widest practicable and appropriate dissemination of information concerning...its activities and the results thereof.' The search for innovative methods to distribute NASA's information lead a grass-roots team to create the NASA Technical Report Server (NTRS), which uses the World Wide Web and other popular Internet-based information systems as search engines. The NTRS is an inter-center effort which provides uniform access to various distributed publication servers residing on the Internet. Users have immediate desktop access to technical publications from NASA centers and institutes. The NTRS is comprised of several units, some constructed especially for inclusion in NTRS, and others that are existing NASA publication services that NTRS reuses. This paper presents the NTRS architecture, usage metrics, and the lessons learned while implementing and maintaining the services over the initial six-month period. The NTRS is largely constructed with freely available software running on existing hardware. NTRS builds upon existing hardware and software, and the resulting additional exposure for the body of literature contained will allow NASA to ensure that its institutional knowledge base will continue to receive the widest practicable and appropriate dissemination.
Nelson, Michael L.↗
Issues in NASA Program and Project Management. Special Report: 1997 Conference. Project Management Now and in the New Millennium
Topics Considered Include: NASA's Shared Experiences Program; Core Issues for the Future of the Agency; National Space Policy Strategic Management; ISO 9000 and NASA; New Acquisition Initiatives; Full Cost Initiative; PM Career Development; PM Project Database; NASA Fast Track Studies; Fast Track Projects; Earned Value Concept; Value-Added Metrics; Saturn Corporation Lessons Learned; Project Manager Credibility.
Hoffman, Edward J.↗
The Widest Practicable Dissemination: The NASA Technical Report Server
The National Aeronautics and Space Act of 1958 established NASA and charged it to "provide for the widest practicable and appropriate dissemination of information concerning [...] its activities and the results thereof." The search for innovative methods to distribute NASA s information lead a grass-roots team to create the NASA Technical Report Server (NTRS), which uses the World Wide Web and other popular Internet-based information systems as search engines. The NTRS is an inter-center effort which provides uniform access to various distributed publication servers residing on the Internet. Users have immediate desktop access to technical publications from NASA centers and institutes. The NTRS is comprised of several units, some constructed especially for inclusion in NTRS, and others that are existing NASA publication services that NTRS reuses. This paper presents the NTRS architecture, usage metrics, and the lessons learned while implementing and maintaining the services over the initial 6-month period. The NTRS is largely constructed with freely available software running on existing hardware. NTRS builds upon existing hardware and software, and the resulting additional exposure for the body of literature contained will allow NASA to ensure that its institutional knowledge base will continue to receive the widest practicable and appropriate dissemination.
Nelson, Michael L.↗
The NASA Technical Report Server
The National Aeronautics and Space Act of 1958 established NASA and charged it to "provide for the widest practicable and appropriate dissemination of information concerning its activities and the results thereof." The search for innovative methods to distribute NASA's information lead a grass-roots team to create the NASA Technical Report Server (NTRS), which uses the World Wide Web and other popular Internet-based information systems as search engines. The NTRS is an inter-center effort which provides uniform access to various distributed publication servers residing on the Internet. Users have immediate desktop access to technical publications from NASA centers and institutes. The NTRS is comprised of several units, some constructed especially for inclusion in NTRS, and others that are existing NASA publication services that NTRS reuses. This paper presents the NTRS architecture, usage metrics, and the lessons learned while implementing and maintaining the service. The NTRS is largely constructed with freely available software running on existing hardware. NTRS builds upon existing hardware and software, and the resulting additional exposure for the body of literature contained ensures that NASA's institutional knowledge base will continue to receive the widest practicable and appropriate dissemination.
Nelson, Michael L.↗
Understanding software for project management
Explore the source record for details and available documents.
software management software life-cycle metrics le↗
Learning from GES DISC's MLS and OMI Data Users: Metrics Matter
It has been over 15 years since Aura research satellite launched in 2004 to observe the Earth's ozone layer, air quality, and climate from four different instruments - the High Resolution Dynamics Limb Sounder (HIRDLS), the Microwave Limb Sounder (MLS), the Ozone Monitoring Instrument (OMI), and the Tropospheric Emission Spectrometer (TES). Observations from the Aura mission have established a concrete understanding of the changing chemistry of our atmosphere.The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the official archive and distribution center for the HIRDLS, MLS, and OMI instruments. This presentation will report metrics of data usage and services on these instruments. Key to GES DISC's mission to provide better data support is gaining a better understanding of our users' needs and behaviors as they discover, access and utilize these data. We will summarize the users' needs from these instruments based on user inquiry information collected over the Aura mission lifetime and present findings from this ensemble metrics.
Metrics↗
Design for Maintenance (ILS)
Explore the source record for details and available documents.
Maintainability↗
Design for Maintenance
Design influence is accomplished through inclusion of Reliability, Maintainability, and Supportability (RMS) as part of the system’s design characteristics from program/project inception to ensure the system(s) can be supported and maintained throughout the life cycle.
Maintainability MTTR Maintenance Integrated Logist↗
CLDP/ISS Glovebox Technical Interchange Meeting
This presentation is designed to provide a high-level overview of the Microgravity Science Glovebox (MSG) and the Life Sciences Glovebox (LSG) facilities onboard the International Space Station. In addition, it provides metrics and lessons-learned information intended for the Commercial Low-Earth Orbit Development Program (CLDP) Partners.
glovebox↗
Evaluation of Classifier Complexity for Delay Tolerant Network Routing
The growing popularity of small cost effective satellites (SmallSats, CubeSats, etc.) creates the potential for a variety of new science applications involving multiple nodes functioning together or independently to achieve a task, such as swarms and constellations. As this technology develops and is deployed for missions in Low Earth Orbit and beyond, the use of delay tolerant networking (DTN) techniques may improve communication capabilities within the network. In this paper, a network hierarchy is developed from heterogeneous networks of SmallSats, surface vehicles, relay satellites and ground stations which form an integrated network. There is a tradeoff between complexity, flexibility, and scalability of user defined schedules versus autonomous routing as the number of nodes in the network increases. To address these issues, this work proposes a machine learning classifier based on DTN routing metrics. A framework is developed which will allow for the use of several categories of machine learning algorithms (decision tree, random forest and deep learning) to be applied to a dataset of historical network statistics, which allows for the evaluation of algorithm complexity versus performance to be explored. We develop the emulation of a hierarchical network, consisting of tens of nodes which form a cognitive network architecture. CORE (Common Open Research Emulator) is used to emulate the network using bundle protocol and DTN IP neighbor discovery.
Dudukovich, Rachel↗
Communicating Metrics of Land Surface Temperature Variability Using Multi-sensor Machine Learning
Land surface temperature (LST) is a key climate observable used to detect changes in the Earth’s surface energy budget that influence carbon and water cycles. Land surface temperature exhibits strong diurnal variability, which geostationary satellites can observe at scale thanks to their temporal resolution. Due to anthropogenic climate and land use changes, the surface energy balance has been considerably modified and may be described by changes in diurnal temperature range and extremes. Using high performance computing and datasets from the NASA Earth Exchange, we exploit co-located, co-temporal observations from low-earth orbit (LEO) and geostationary (GEO) sensors to develop a deep learning-based method for LEO-to-GEO algorithm emulation. Our model is trained to predict MODIS Terra LST from GOES-16 thermal bands and achieves validation error <2K. Application of the model to unseen times of day (observed by MODIS Aqua) and a new GEO sensor (Himawari-8) observing an unseen spatial domain, demonstrate the generalization of the deep learning model across space, time and spectra. Further, time series clustering approaches are examined with the objective of identifying key indicators of change in diurnal cycling and extremes on a continental scale. Communicating LST variability observed by geostationary satellites can have impacts in multiple disciplines, from understanding of snow, vegetation and soil dynamics, to recognizing trends in heat events relevant to human health.
Kate Duffy↗
There is No Free Lunch: Tradeoffs in the Utility of Learned Knowledge
With the recent introduction of learning in integrated systems, there is a need to measure the utility of learned knowledge for these more complex systems. A difficulty arrises when there are multiple, possibly conflicting, utility metrics to be measured. In this paper, we present schemes which trade off conflicting utility metrics in order to achieve some global performance objectives. In particular, we present a case study of a multi-strategy machine learning system, mutual theory refinement, which refines world models for an integrated reactive system, the Entropy Reduction Engine. We provide experimental results on the utility of learned knowledge in two conflicting metrics - improved accuracy and degraded efficiency. We then demonstrate two ways to trade off these metrics. In each, some learned knowledge is either approximated or dynamically 'forgotten' so as to improve efficiency while degrading accuracy only slightly.
Kedar, Smadar T.↗
Barefoot Rover: a Sensor-Infused Rover Wheel Demonstrating In-Situ Engineering and Science Extractions using Machine Learning
In this work, we demonstrate an instrumented wheel concept which utilizes a 2D pressure grid, an electrochemical impedance spectroscopy (EIS) sensor and machine learning (ML) to extract meaningful metrics from the interaction between the wheel and surface terrain. These include continuous slip/skid estimation, balance, and sharpness for engineering applications. Estimates of surface hydration, texture, terrain patterns, and regolith physical properties such as cohesion and angle of internal friction are additionally calculated for science applications. Traditional systems rely on post-processing of visual images and vehicle telemetry to estimate these metrics. Through in-situ sensing, these metrics can be calculated in near real time and made available to onboard science and engineering autonomy applications. This work aims to provide a deployable system for future planetary exploration missions to increase science and engineering capabilities through increased knowledge of the terrain.
Marchetti, Yuliya↗