SEARCH · Engineering Papers

Results for “system validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

All sources NASA NTRS DOE OSTI

NASA NTRS2018

Differential Adaptive Stress Testing of Airborne Collision Avoidance Systems

The next-generation Airborne Collision Avoidance System (ACAS X) is currently being developed and tested to replace the Traffic Alert and Collision Avoidance System (TCAS) as the next international standard for collision avoidance. To validate the safety of the system, stress testing in simulation is one of several approaches for analyzing near mid-air collisions (NMACs). Understanding how NMACs can occur is important for characterizing risk and informingdevelopment of the system. Recently, adaptive stress testing (AST) has been proposed as a way to find the most likely path to a failure event. The simulation-based approach accelerates search by formulating stress testing as a sequential decision process then optimizing it using reinforcement learning. The approach has been successfully applied to stress test a prototype of ACAS Xin various simulated aircraft encounters. In some applications, we are not as interestedin the system's absolute performance as its performance relative to another system. Such situations arise, for example, during regression testing or when deciding whether a new system should replace an existing system. In our collision avoidance application, we are interested in finding cases where ACAS X fails but TCAS succeeds in resolving a conflict. Existing approaches do not provide an efficient means to perform this type of analysis. This paper extends the AST approach to differential analysis by searching two simulators simultaneously and maximizing the difference between their outcomes. We call this approach differential adaptive stress testing (DAST). We apply DAST to compare a prototype of ACAS X against TCAS and show examples of encounters found by the algorithm.

Lee, Ritchie↗

NASA NTRS1996

Lessons Learned From the Flight of the NASA In-Step Cryo System Experiment

The Cryo System Experiment was developed to validate in near zero-g space a 65 K cryogenic system for focal planes, optics, or other imaging instruments that require continuous cryogenic cooling. Two key cryogenic technologies, designed to improve performance of systems for scientific, commercial and defense applications in space, were successfully demonstrated on a February 1995 Shuttle mission. The two were a 2-watt 65 K long-life low- vibration Stirling cooler and a diode oxygen heat pipe thermal switch. Lessons learned are presented.

cryogenics↗

NASA NTRS1984

MERIT - A new approach to upper air forecasting for aviation

The development of a man/computer data enhancement and management system to provide very-short range upper air forecasts is examined. The forecast accuracy and precision problems encountered with the current numerical weather prediction models are discussed. The proposed system is to utilize both radiosonde data and automated pilot reports and provide a 2-12 hour analysis/forecast with a 3 hour forecast cycle. The minimum energy routes using interactive techniques (MERIT) system is described and a diagram is povided. The preliminary testing of a modified MERIT system reveals that the system is not as accurate as the Spectral 12 hour forecast, but is more accurate than the spectral 24 hour forecast. The coplete testing and validation of the MERIT system is required before a comparison with present techniques is possible.

Steinberg, R.↗

NASA NTRS1995

Architecture for Payload Planning System (PPS) Software Distribution

The complex and diverse nature of the pay load operations to be performed on the Space Station requires a robust and flexible planning approach, and the proper software tools which tools to support that approach. To date, the planning software for most manned operations in space has been utilized in a centralized planning environment. Centralized planning is characterized by the following: performed by a small team of people, performed at a single location, and performed using single-user planning systems. This approach, while valid for short duration flights, is not conducive to the long duration and highly distributed payload operations environment of the Space Station. The Payload Planning System (PPS) is being designed specifically to support the planning needs of the large number of geographically distributed users of the Space Station. This paper problem provides a general description of the distributed planning architecture that PPS must support and describes the concepts proposed for making PPS available to the Space Station payload user community.

Howell, Eric↗

NASA NTRS2000

Software IV and V Research Priorities and Applied Program Accomplishments Within NASA

The mission of this research is to be world-class creators and facilitators of innovative, intelligent, high performance, reliable information technologies that enable NASA missions to (1) increase software safety and quality through error avoidance, early detection and resolution of errors, by utilizing and applying empirically based software engineering best practices; (2) ensure customer software risks are identified and/or that requirements are met and/or exceeded; (3) research, develop, apply, verify, and publish software technologies for competitive advantage and the advancement of science; and (4) facilitate the transfer of science and engineering data, methods, and practices to NASA, educational institutions, state agencies, and commercial organizations. The goals are to become a national Center Of Excellence (COE) in software and system independent verification and validation, and to become an international leading force in the field of software engineering for improving the safety, quality, reliability, and cost performance of software systems. This project addresses the following problems: Ensure safety of NASA missions, ensure requirements are met, minimize programmatic and technological risks of software development and operations, improve software quality, reduce costs and time to delivery, and improve the science of software engineering

Blazy, Louis J.↗

NASA NTRS2014

Airspace Safety Threshold Study: Status Update to SC-228 DAA Safety Sub-Group

Risk ratio↗

NASA NTRS2018

Trusted Communication: Utilizing Speech Communication to Enhance Human-Machine Teaming Success

An area of increasing interest for the next generation of aircraft is autonomy and the integration of increasingly autonomous systems into the national airspace. Such an integration requires humans to work closely with autonomous systems, forming teams. Our hypothesis is that a team composed of both humans and autonomous systems will operate better than either entity alone. We have existing procedures for certifying pilots to operate in the national airspace and are currently working on methods for validating the function of autonomous systems, however we have no method in place for assessing the interaction of these two disparate systems. Communication is one avenue. This paper will examine the use of language as a metric for ascertaining human-machine teaming effectiveness. A proof-of-concept of the application of two communication-based analysis techniques, Linguistic Inquiry and Word Count (LIWC) and Latent Semantic Analysis (LSA), for the prediction of success in human/chatbot teaming was conducted. By running these analyses over data from the 2014 and 2015 Loebner Prize competitions of human/chatbot teaming, numerical scores were obtained that can be associated with scores provided by human judges during the competition. Correlating their LIWC and LSA data with the scores provided by the judges, and using linear regression over this correlation, formulae were obtained that predict the score of human/chatbot interaction. These formulae were tested over the 2013 Loebner Prize transcripts, determining that, though there was strong correlation between predicted and actual scores, the predictive success of this method was not strong. However, with specialized topic spaces and lexica, as well as larger data sets, the predictive power of these metrics will improve. Given the importance of providing metrics for human-machine system team success and given the promise shown by the communication-basedLIWCand LSAmethods, continuing research in this area is necessary. After examining the potential for using communication and spoken language as a metric for the success of human/autonomous system teaming, this paper then examines aspects inherent to communication systems that may contribute to unreliability and reduced trust. Modern natural language processing tools rely on deep learning algorithms to create language rules that produce accurate results, but these rules are uninterpretable. The resulting blackbox system lacks transparency necessary for full validation and complete trust. Additionally, speech-based interfaces pose other difficulties to developing coordinated teamwork between humans and autonomous systems. Human communication is infrequently limited to speech only, instead usually relying on a combination of verbal, gestural, and general body language communication. Reducing an analysis of team effectiveness to a study of spoken language alone is problematic as it leaves these other equally important forms of communication out. This paper will examine these problems and the general deficiencies in speech-based metrics for human-machine teaming.

E L Meszaros↗

NASA NTRS1987

Fault-tolerant clock synchronization validation methodology

A validation method for the synchronization subsystem of a fault-tolerant computer system is presented. The high reliability requirement of flight-crucial systems precludes the use of most traditional validation methods. The method presented utilizes formal design proof to uncover design and coding errors and experimentation to validate the assumptions of the design proof. The experimental method is described and illustrated by validating the clock synchronization system of the Software Implemented Fault Tolerance computer. The design proof of the algorithm includes a theorem that defines the maximum skew between any two nonfaulty clocks in the system in terms of specific system parameters. Most of these parameters are deterministic. One crucial parameter is the upper bound on the clock read error, which is stochastic. The probability that this upper bound is exceeded is calculated from data obtained by the measurement of system parameters. This probability is then included in a detailed reliability analysis of the system.

Computer systems↗

NASA NTRS2016

Flight Test 4 Preliminary Results: NASA Ames SSI

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements as well as the safety substantiation and end-to-end assessment of DAA system performance. The Unmanned Aircraft System (UAS) Integration into the National Airspace System (NAS) Project conducted flight test program, referred to as Flight Test 4, at Armstrong Flight Research Center from April -June 2016. Part of the test flights were dedicated to the NASA Ames-developed Detect and Avoid (DAA) System referred to as JADEM (Java Architecture for DAA Extensibility and Modeling). The encounter scenarios, which involved NASA's Ikhana UAS and a manned intruder aircraft, were designed to collect data on DAA system performance in real-world conditions and uncertainties with four different surveillance sensor systems. Flight test 4 has four objectives: (1) validate DAA requirements in stressing cases that drive MOPS requirements, including: high-speed cooperative intruder, low-speed non-cooperative intruder, high vertical closure rate encounter, and Mode CS-only intruder (i.e. without ADS-B), (2) validate TCASDAA alerting and guidance interoperability concept in the presence of realistic sensor, tracking and navigational errors and in multiple-intruder encounters against both cooperative and non-cooperative intruders, (3) validate Well Clear Recovery guidance in the presence of realistic sensor, tracking and navigational errors, and (4) validate DAA alerting and guidance requirements in the presence of realistic sensor, tracking and navigational errors. The results will be presented at RTCA Special Committee 228 in support of final verification and validation of the DAA MOPS.

Detect-And-Avoid (DAA)↗

NASA NTRS2008

Similarity Metrics for Closed Loop Dynamic Systems

To what extent and in what ways can two closed-loop dynamic systems be said to be "similar?" This question arises in a wide range of dynamic systems modeling and control system design applications. For example, bounds on error models are fundamental to the controller optimization with modern control design methods. Metrics such as the structured singular value are direct measures of the degree to which properties such as stability or performance are maintained in the presence of specified uncertainties or variations in the plant model. Similarly, controls-related areas such as system identification, model reduction, and experimental model validation employ measures of similarity between multiple realizations of a dynamic system. Each area has its tools and approaches, with each tool more or less suited for one application or the other. Similarity in the context of closed-loop model validation via flight test is subtly different from error measures in the typical controls oriented application. Whereas similarity in a robust control context relates to plant variation and the attendant affect on stability and performance, in this context similarity metrics are sought that assess the relevance of a dynamic system test for the purpose of validating the stability and performance of a "similar" dynamic system. Similarity in the context of system identification is much more relevant than are robust control analogies in that errors between one dynamic system (the test article) and another (the nominal "design" model) are sought for the purpose of bounding the validity of a model for control design and analysis. Yet system identification typically involves open-loop plant models which are independent of the control system (with the exception of limited developments in closed-loop system identification which is nonetheless focused on obtaining open-loop plant models from closed-loop data). Moreover the objectives of system identification are not the same as a flight test and hence system identification error metrics are not directly relevant. In applications such as launch vehicles where the open loop plant is unstable it is similarity of the closed-loop system dynamics of a flight test that are relevant.

Whorton, Mark S.↗

NASA NTRS2000

Results from NASA's Near Real Time Ozone Data Assimilation Systems

Organizations in Europe, Australia, and the United States have recently broadened constituent assimilation activities beyond water vapor, which has been assimilated for years in the numerical weather prediction applications. Many of these activities have focused on ozone, with some efforts focused on the entire suite of reactive constituents that control the ozone distribution. This talk will draw from results from the near real-time ozone data assimilation system being run by NASA's Data Assimilation Office. This system utilizes ozone observations from both the TOMS and the SBUV instrument to generate global synoptic maps of ozone. The initial application of this product is to provide ozone fields to assist in the atmospheric corrections' that are necessary for the retrieval of information from other NASA instruments. The validation of the ozone assimilation system shows that the assimilated product agrees well with independent HALOE and ozonesonde observations. Aside from providing a global synoptic map, there is verifiable geophysical information at higher vertical resolution than either of the date types input into the system. This talk will establish the validation results and enumerate applications of the ozone data assimilation system. Results from exploratory research will be presented. The applications being considered include estimates of tropospheric ozone, provision of ozone fields for interactive retrievals, use of analysis increments from the assimilation to evaluate model performance, and development of long-term consistent three-dimensional global ozone fields. The results from the exploratory studies are promising, and help demonstrate how assumptions made in the development of the ozone assimilation impact the other applications. For instance, RMS errors in the current product are large near the tropopause, which is sensitive to the specification of vertical correlation functions, which in turns impacts the amount of ozone analyzed to be in the troposphere. How these sensitivities impact the different applications will also be discussed.

Rood, Richard B.↗

NASA NTRS2007

Drive System Research

An overview of the NASA Glenn Research Center Drive Systems Research will be presented. The primary purpose of this research is to improve performance, reliability, and integrity of aerospace drive systems and space mechanisms. The research is conducted through a combination of in-house, academia, and through contractors. Research is conducted through computer code development and validated through component and system testing. The drive system activity currently has four major thrust areas including: thermal behavior of high speed gearing, health and usage monitoring, advanced components, and space mechanisms.

Handschuh, Robert F.↗

NASA NTRS1968

Radar scatterometer data validation Final report

Radar scatterometer system to measure radar reflectivity or scattering cross sections of various surfaces as function of incidence angle

Craib, K. B.↗

NASA NTRS1985

Development of an expert system prototype for determining software functional requirements for command management activities at NASA Goddard

The development of an expert system prototype for determining software functional requirements for NASA Goddard's Command Management System (CMS) is described. The role of the CMS is to transform general requests into specific spacecraft commands with command execution conditions. The CMS is part of the NASA Data System which entails the downlink of science and engineering data from NASA near-earth satellites to the user, and the uplink of command and control data to the spacecraft. Subjects covered include: the problem environment of determining CMS software functional requirements; the expert system approach for handling CMS requirements development; validation and evaluation procedures for the expert system.

Liebowitz, J.↗

NASA NTRS1988

Graph-based real-time fault diagnostics

A real-time fault detection and diagnosis capability is absolutely crucial in the design of large-scale space systems. Some of the existing AI-based fault diagnostic techniques like expert systems and qualitative modelling are frequently ill-suited for this purpose. Expert systems are often inadequately structured, difficult to validate and suffer from knowledge acquisition bottlenecks. Qualitative modelling techniques sometimes generate a large number of failure source alternatives, thus hampering speedy diagnosis. In this paper we present a graph-based technique which is well suited for real-time fault diagnosis, structured knowledge representation and acquisition and testing and validation. A Hierarchical Fault Model of the system to be diagnosed is developed. At each level of hierarchy, there exist fault propagation digraphs denoting causal relations between failure modes of subsystems. The edges of such a digraph are weighted with fault propagation time intervals. Efficient and restartable graph algorithms are used for on-line speedy identification of failure source components.

Padalkar, S.↗

NASA NTRS1999

Evaluation of the Self-Nulling Rotating Eddy Current Probe System

In order to detect multi-site fatigue cracks located under flush-head rivets, automated eddy current equipment is required. To assure a reliable system, the eddy current probe must be centered easily over the installed rivets. To meet these requirements, the NDE Group at NASA LaRC developed the Self-Nulling Rotating Eddy Current Probe System (SNRECPS) which will be referred to as RPS in this document. The system was evaluated at the FAA, NDI Validation Center, in Albuquerque, New Mexico. The system was capable of detecting a 0.032 inch long crack with a 90/95% PoD. Further evaluations were conducted at Boeing in Long Beach, California. These evaluations included fatigue cracks and notches in a range from 0.025 to 0.100 inch long under flush-head aluminum rivets, and titanium or steel flush-head fasteners. The results of these tests are reported herein. Subsequently, the system was loaned to the USAF Structures Laboratory for the purpose of detecting and measuring short cracks under flush-head rivets in a variety of fatigue test specimens. The inspection task was to detect and plot crack growth from numbered fasteners in lettered rows. In January, 1998, the system was taken to Northwest Airlines Maintenance Base, in Atlanta, to inspect a DC-9, for multi-site cracks in three circumferential splices. The aircraft had 83,000 cycles. The inspection was conducted at 30 kHz from longeron 5 left to longeron 5 right. The system was calibrated using a 0,030 EDM first layer notch. The instrument gain was set to 19 mV from the notch. The reject level was set at 10 mV and the unflawed fasteners yielded a signal amplitude of 2 to 3 mV. Only one fastener location, out of about 2,500 tested, yielded a signal of 58 mV. The rivet was removed and visually evaluated. It appeared to be a slight gouge in the counter-sink zone. No fatigue cracks were detected. The same fastener locations were also inspected using the Boeing MAUS system at 60 kHz. No cracks were detected. Thus far, the rotating probe eddy current system has been found to be very user friendly and capable of detecting first layer cracks on the order of 0.030 inch long or longer.

Hagemaier, Don↗

NASA NTRS2000

Lidar Wind Measurements with the Goddard Lidar Observatory for Winds (GLOW)

We report on the development of GLOW (Goddard Lidar Observatory for Winds), a mobile Doppler lidar system which uses direct detection Doppler lidar techniques to measure wind profiles from the surface into the lower stratosphere. The system employs a Nd:YAG laser transmitter to measure winds using either aerosol backscatter at a wavelength of 1064 run or molecular backscatter at 355 nm. The system is modular in design to allow the incorporation of new technologies as they become available. GLOW is intended to be used as a deployable field system for studying atmospheric dynamics and transport and can also serve as a testbed to evaluate candidate technologies developed for use in future spaceborne systems. Finally it can be used for calibration/validation activities following launch of spaceborne wind lidar systems. A description of the mobile system is presented along with the first validated lidar wind profiles obtained with the system using a new molecular 'double edge' receiver.

Gentry, Bruce M.↗

NASA NTRS2004

Global Aerosol Profiling by Orbital Lidar, GLAS Results and Validation

The Geoscience Laser Altimeter System (GLAS) launched in 2003 is the first polar orbiting satellite lidar. The instrument was designed for high performance observations of the distribution and optical scattering cross sections of clouds and aerosol. GLAS is approaching six months of on orbit data operation. These data from thousands of orbits illustrate the ability of space lidar to accurately and dramatically measure the height distribution of global aerosol to an unprecedented degree. There were many intended science applications of the GLAS data and significant results have already been realized, profiling is a fundamentally new measurement from space with multiple applications. A most important aerosol application is providing input to global aerosol generation transport models. Another is improved measurement of aerosol optical depth. A main approach to verify the aerosol optical depth retrieval is comparison to surface measurements by Aeronet. A special feature of the GLAS satellite bus is to rapidly point the lidar instrument at off nadir targets with less than 100 m accuracy. About a dozen selected Aeronet sites were pointed at whenever the GLAS lidar came within 5 degrees of zenith. These plus a more general comparison to nearby sites support the GLAS data product values. In addition the GLAS data can be used to add vertical distribution information to Aeronet aerosol measurements. As an EOS project instrument, GLAS data products are openly available to the science community. First year results from GLAS are summarized.

Spinhirne, J. D.↗