SEARCH · Engineering Papers

Results for “Heuristic Evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2010

Evaluating Heuristics for Planning Effective and Efficient Inspections

A significant body of knowledge concerning software inspection practice indicates that the value of inspections varies widely both within and across organizations. Inspection effectiveness and efficiency can be measured in numerous ways, and may be affected by a variety of factors such as Inspection planning, the type of software, the developing organization, and many others. In the early 1990's, NASA formulated heuristics for inspection planning based on best practices and early NASA inspection data. Over the intervening years, the body of data from NASA inspections has grown. This paper describes a multi-faceted exploratory analysis performed on this · data to elicit lessons learned in general about conducting inspections and to recommend improvements to the existing heuristics. The contributions of our results include support for modifying some of the original inspection heuristics (e.g. Increasing the recommended page rate), evidence that Inspection planners must choose between efficiency and effectiveness, as a good tradeoff between them may not exist, and Identification of small subsets of inspections for which new inspection heuristics are needed. Most Importantly, this work illustrates the value of collecting rich data on software Inspections, and using it to gain insight into, and Improve, inspection practice.

Shull, Forrest J.↗

NASA NTRS1997

Evaluating a Web-Based Interface for Internet Telemedicine

The objective is to introduce the usability engineering methodology, heuristic evaluation, to the design and development of a web-based telemedicine system. Using a set of usability criteria, or heuristics, one evaluator examined the Spacebridge to Russia web-site for usability problems. Thirty-four usability problems were found in this preliminary study and all were assigned a severity rating. The value of heuristic analysis in the iterative design of a system is shown because the problems can be fixed before deployment of a system and the problems are of a different nature than those found by actual users of the system. It was therefore determined that there is potential value of heuristic evaluation paired with user testing as a strategy for optimal system performance design.

Lathan, Corinna E.↗

NASA NTRS1990

Applications of different design methodologies in navigation systems and development at JPL

The NASA/JPL deep space navigation system consists of a complex array of measurement systems, data processing systems, and support facilities, with components located both on the ground and on-board interplanetary spacecraft. From its beginings nearly 30 years ago, this system has steadily evolved and grown to meet the demands for ever-increasing navigation accuracy placed on it by a succession of unmanned planetary missions. Principal characteristics of this system are its capabilities and great complexity. Three examples in the design and development of interplanetary space navigation systems are examined in order to make a brief assessment of the usefulness of three basic design theories, known as normative, rational, and heuristic. Evaluation of the examples indicates that a heuristic approach, coupled with rational-based mathematical and computational analysis methods, is used most often in problems such as orbit determination strategy development and mission navigation system design, while normative methods have seen only limited use is such applications as the development of large software systems and in the design of certain operational navigation subsystems.

Thurman, S. W.↗

NASA NTRS1991

Learning process mapping heuristics under stochastic sampling overheads

A statistical method was developed previously for improving process mapping heuristics. The method systematically explores the space of possible heuristics under a specified time constraint. Its goal is to get the best possible heuristics while trading between the solution quality of the process mapping heuristics and their execution time. The statistical selection method is extended to take into consideration the variations in the amount of time used to evaluate heuristics on a problem instance. The improvement in performance is presented using the more realistic assumption along with some methods that alleviate the additional complexity.

Ieumwananonthachai, Arthur↗

NASA NTRS2007

Evaluation of Hands-Free Devices for Space Habitat Maintenance Procedures

Currently, International Space Station (ISS) crews use a laptop computer to display procedures for performing onboard maintenance tasks. This approach has been determined to be suboptimal. A heuristic evaluation and two studies have been completed to test commercial off-the-shelf (COTS) "near-eye" heads up displays (HUDs) for support of these types of maintenance tasks. In both studies, subjects worked through electronic procedures to perform simple maintenance tasks. As a result of the Phase I study, three HUDs were down-selected to one. In the Phase II study, the HUD was compared against two other electronic display devices - a laptop computer and an e-book reader. Results suggested that adjustability and stability of the HUD display were the most significant acceptability factors to consider for near-eye displays. The Phase II study uncovered a number of advantages and disadvantages of the HUD relative to the laptop and e-book reader for interacting with electronic procedures.

Hoffman, R. B.↗

NASA NTRS2024

Usability Evaluation of NASA TOPS Open Science 101

This report presents a usability evaluation of NASA’s Open Science 101 (OS101) interface. It offers design recommendations for improving the user experience (UX) and user interface (UI) of the initial self-paced OS101 release. The author initially conducted an independent heuristic evaluation, followed by an analysis of written user feedback to corroborate findings. Overall, the usability evaluation found that the OS101 interface offers a linear and interactive UX, with areas for improvement across five major, three minor, and two cosmetic aspects of usability.

Human-Computer Interaction↗

NASA NTRS1993

Efficient partitioning and assignment on programs for multiprocessor execution

The general problem studied is that of segmenting or partitioning programs for distribution across a multiprocessor system. Efficient partitioning and the assignment of program elements are of great importance since the time consumed in this overhead activity may easily dominate the computation, effectively eliminating any gains made by the use of the parallelism. In this study, the partitioning of sequentially structured programs (written in FORTRAN) is evaluated. Heuristics, developed for similar applications are examined. Finally, a model for queueing networks with finite queues is developed which may be used to analyze multiprocessor system architectures with a shared memory approach to the problem of partitioning. The properties of sequentially written programs form obstacles to large scale (at the procedure or subroutine level) parallelization. Data dependencies of even the minutest nature, reflecting the sequential development of the program, severely limit parallelism. The design of heuristic algorithms is tied to the experience gained in the parallel splitting. Parallelism obtained through the physical separation of data has seen some success, especially at the data element level. Data parallelism on a grander scale requires models that accurately reflect the effects of blocking caused by finite queues. A model for the approximation of the performance of finite queueing networks is developed. This model makes use of the decomposition approach combined with the efficiency of product form solutions.

Standley, Hilda M.↗

NASA NTRS1994

DTS: Building custom, intelligent schedulers

DTS is a decision-theoretic scheduler, built on top of a flexible toolkit -- this paper focuses on how the toolkit might be reused in future NASA mission schedulers. The toolkit includes a user-customizable scheduling interface, and a 'Just-For-You' optimization engine. The customizable interface is built on two metaphors: objects and dynamic graphs. Objects help to structure problem specifications and related data, while dynamic graphs simplify the specification of graphical schedule editors (such as Gantt charts). The interface can be used with any 'back-end' scheduler, through dynamically-loaded code, interprocess communication, or a shared database. The 'Just-For-You' optimization engine includes user-specific utility functions, automatically compiled heuristic evaluations, and a postprocessing facility for enforcing scheduling policies. The optimization engine is based on BPS, the Bayesian Problem-Solver (1,2), which introduced a similar approach to solving single-agent and adversarial graph search problems.

Hansson, Othar↗

NASA NTRS2004

Innovative Imagery System for Enhanced Habitability Onboard ISS: Desired Features and Possible Hardware Applications

With the advent of the ISS and the experience of Russian, European, and US crewmembers on Mir, the importance of the psychological element in long duration missions is increasingly recognized. An integrated imagery system or Magic Window System could enhance the habitability, performance, and productivity for long term stays in space. Because this is type of system is a new concept for space, functional and technical requirements need to be determined. As part of a three-year project, the functional and technical requirements for an Imagery System onboard the International Space Station (ISS) have been explored. Valuable information was gathered from a survey completed by participants that had been in analog environments (remote/isolated) such as Antarctica, Aquarius, ISS crewmember debriefs, and crew support meetings to identify key functions desired for an integrated Magic Window System. Exercise and medical care activities were identified as areas that could benefit from such a system. It was determined that for exercise, it was worth exploring the concept of displaying a dynamic screen that changes as the crewmember's speed changes while showing physiological measures in a combined display. In terms of enhancing the interfaces for medical care activities, the Magic Window System could show video clips along side procedures for just-in-time training scenarios through a heads-up display. In addition, the portability, usability, and reliability were stressed as important considerations for an integrated system of technologies or Magic Window System. In addition, a review of state-of-the-art screens and other existing technologies such as tablet PCs and Personal Digital Assistants (PDAs) was conducted and contributed to defining technical requirements and feasibility of systems. Some heuristic evaluations of large displays and PDAs were conducted. Finally, feasibility for implementation onboard ISS has been considered. Currently, specific headset units are undergoing usability testing. The outcome of these activities will be valuable to determine the best candidates for an integrated system that could accommodate different needs depending on task.

Whitmore, Mihriban↗

NASA NTRS2020

CyberGAN: Generating High-fidelity Cybersecurity Data With Generative Adversarial Networks

Machine learning for cyber defense offers the promise of detecting adversarial activity against the ground data systems managing critical space assets. A fundamental challenge facing machine learning research in cybersecurity is the lack of high-fidelity, shareable datasets for robust evaluation and testing of machine learning-based solutions. High-fidelity, real-world datasets are necessary for reliable benchmarking of nominal system behavior and malicious activity. Unfortunately, such realistic datasets of both nominal and adversarial activity are rarely shared publicly by data owners due to security and privacy concerns. Besides, the available adversarial data is sparse, which makes training models on malicious activity much harder. This situation has impeded and continues to impede the research and successful adoption of machine learning methods for cyber defense. Researchers have dealt with this problem by generating data within a low-fidelity lab environment, using classified and thus unshareable datasets, or downloading low-fidelity public datasets made available by others. We propose an innovative solution to the problem by employing machine learning methods to generate high-fidelity data. Specifically, we propose the use of Generative Adversarial Networks (GANs) to generate high-fidelity data for cybersecurity purposes. GANs have found successful image processing and natural language applications, but have not yet been investigated for cyber data generation. Our proposed approach first involves training the `discriminator' network of the GAN with a sample of real-world data consisting of malicious and nominal samples. We then use the `generator' network to generate new high-fidelity data samples consisting of an appropriate mix of malicious and nominal activity. We demonstrate applications of our architecture by generating high-fidelity cybersecurity data containing both malicious and nominal samples. We thoroughly evaluate the fidelity of our generated data using heuristics and evaluate its usefulness for machine learning applications using three different datasets. Overall, our approach results in high-fidelity, shareable datasets.

Zhang, Yuening↗

NASA NTRS1997

Automatic Generation of Heuristics for Scheduling

This paper presents a technique, called GenH, that automatically generates search heuristics for scheduling problems. The impetus for developing this technique is the growing consensus that heuristics encode advice that is, at best, useful in solving most, or typical, problem instances, and, at worst, useful in solving only a narrowly defined set of instances. In either case, heuristic problem solvers, to be broadly applicable, should have a means of automatically adjusting to the idiosyncrasies of each problem instance. GenH generates a search heuristic for a given problem instance by hill-climbing in the space of possible multi-attribute heuristics, where the evaluation of a candidate heuristic is based on the quality of the solution found under its guidance. We present empirical results obtained by applying GenH to the real world problem of telescope observation scheduling. These results demonstrate that GenH is a simple and effective way of improving the performance of an heuristic scheduler.

Morris, Robert A.↗

NASA NTRS1990

DEPEND - A design environment for prediction and evaluation of system dependability

The development of DEPEND, an integrated simulation environment for the design and dependability analysis of fault-tolerant systems, is described. DEPEND models both hardware and software components at a functional level, and allows automatic failure injection to assess system performance and reliability. It relieves the user of the work needed to inject failures, maintain statistics, and output reports. The automatic failure injection scheme is geared toward evaluating a system under high stress (workload) conditions. The failures that are injected can affect both hardware and software components. To illustrate the capability of the simulator, a distributed system which employs a prediction-based, dynamic load-balancing heuristic is evaluated. Experiments were conducted to determine the impact of failures on system performance and to identify the failures to which the system is especially susceptible.

Goswami, Kumar K.↗

NASA NTRS1987

Mathematical programming formulations for satellite synthesis

The problem of satellite synthesis can be described as optimally allotting locations and sometimes frequencies and polarizations, to communication satellites so that interference from unwanted satellite signals does not exceed a specified threshold. In this report, mathematical programming models and optimization methods are used to solve satellite synthesis problems. A nonlinear programming formulation which is solved using Zoutendijk's method and a gradient search method is described. Nine mixed integer programming models are considered. Results of computer runs with these nine models and five geographically compatible scenarios are presented and evaluated. A heuristic solution procedure is also used to solve two of the models studied. Heuristic solutions to three large synthesis problems are presented. The results of our analysis show that the heuristic performs very well, both in terms of solution quality and solution time, on the two models to which it was applied. It is concluded that the heuristic procedure is the best of the methods considered for solving satellite synthesis problems.

Bhasin, Puneet↗

NASA NTRS1993

Investigation of the launch pad abort capabilities of the HL-20 lifting body

The capability of the HL-20 lifting body spacecraft to perform an abort maneuver from the launch pad to a horizontal landing was studied at NASA Langley Research Center. This study involved both piloted and batch simulation models of the vehicle. A point-mass model of the vehicle was used for trajectory optimization studies. The piloted simulation was performed in the Visual Motion Simulator in fixed-base mode. A candidate maneuver was developed and refined for the worst-case launch-pad-to-landing-site geometry using an iterative procedure of off-line maneuver analysis followed by piloted evaluations and heuristic improvements to the candidate maneuver. The resulting maneuver demonstrates the launch site abort capability of the HL-20 and dictates requirements for nominal abort motor performance. The sensitivity of the maneuver to variations in several design parameters was documented.

Jackson, E. B.↗

NASA NTRS1994

Launch-pad abort capabilities of the HL-20 lifting body

The capability of the HL-20 lifting body to perform an abort maneuver from the launch pad to a horizontal landing was studied. The study involved both piloted and batch simulation models of the vehicle. A point-mass model of the vehicle was used for trajectory optimization studies. The piloted simulation was performed in the Langley Visual/Motion Simulator in the fixed-base mode. A candidate maneuver was developed and refined for the worst-case launch-pad-to-landing-site geometry with an iterative procedure of off-line maneuver analysis followed by piloted evaluations and heuristic improvements to the candidate maneuver. The resulting maneuver demonstrates the launch-site abort capability of the HL-20 and dictates requirements for nominal abort-motor performance. The sensitivity of the maneuver to variations in several design parameters was documented.

Jackson, E. Bruce↗

NASA NTRS1996

Site Partitioning for Redundant Arrays of Distributed Disks

Redundant arrays of distributed disks (RADD) can be used in a distributed computing system or database system to provide recovery in the presence of disk crashes and temporary and permanent failures of single sites. In this paper, we look at the problem of partitioning the sites of a distributed storage system into redundant arrays in such a way that the communication costs for maintaining the parity information are minimized. We show that the partitioning problem is NP-hard. We then propose and evaluate several heuristic algorithms for finding approximate solutions. Simulation results show that significant reduction in remote parity update costs can be achieved by optimizing the site partitioning scheme.

Mourad, Antoine N.↗

NASA NTRS1989

Optimal placement of excitations and sensors by simulated annealing

The optimal placement of discrete actuators and sensors is posed as a combinatorial optimization problem. Two examples for truss structures were used for illustration; the first dealt with the optimal placement of passive dampers along existing truss members, and the second dealt with the optimal placement of a combination of a set of actuators and a set of sensors. Except for the simplest problems, an exact solution by enumeration involves a very large number of function evaluations, and is therefore computationally intractable. By contrast, the simulated annealing heuristic involves far fewer evaluations and is best suited for the class of problems considered. As an optimization tool, the effectiveness of the algorithm is enhanced by introducing a number of rules that incorporate knowledge about the physical behavior of the problem. Some of the suggested rules are necessarily problem dependent.

Salama, Moktar↗

NASA NTRS1995

Spectral ordering techniques for incomplete LU preconditoners for CG methods

The effectiveness of an incomplete LU (ILU) factorization as a preconditioner for the conjugate gradient method can be highly dependent on the ordering of the matrix rows during its creation. Detailed justification for two heuristics commonly used in matrix ordering for anisotropic problems is given. The bandwidth reduction and weak connection following heuristics are implemented through an ordering method based on eigenvector computations. This spectral ordering is shown to be a good representation of the heuristics. Analysis and test cases in two and three dimensional diffusion problems demonstrate when ordering is important, and when an ILU decomposition will be ordering insensitive. The applicability of the heuristics is thus evaluated and placed on a more rigorous footing.

Clift, Simon S.↗