Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “HERS Raters”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

11 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2002

CRM Assessment: Determining the Generalization of Rater Calibration Training. Summary of Research Report: Gold Standards Training

The extent to which pilot instructors are trained to assess crew resource management (CRM) skills accurately during Line-Oriented Flight Training (LOFT) and Line Operational Evaluation (LOE) scenarios is critical. Pilot instructors must make accurate performance ratings to ensure that proper feedback is provided to flight crews and appropriate decisions are made regarding certification to fly the line. Furthermore, the Federal Aviation Administration's (FAA) Advanced Qualification Program (AQP) requires that instructors be trained explicitly to evaluate both technical and CRM performance (i.e., rater training) and also requires that proficiency and standardization of instructors be verified periodically. To address the critical need for effective pilot instructor training, the American Institutes for Research (AIR) reviewed the relevant research on rater training and, based on "best practices" from this research, developed a new strategy for training pilot instructors to assess crew performance. In addition, we explored new statistical techniques for assessing the effectiveness of pilot instructor training. The results of our research are briefly summarized below. This summary is followed by abstracts of articles and book chapters published under this grant.

Baker, David P.↗
NASA NTRS2001

Exploring new frontiers in multi-rater feedback

The presentation provides an overview of the Upward Feedback Program used at JPL to provide feedback to managers on their employees' perceptions of their management effectiveness.

upward feedback leadership employee assessment↗
NASA NTRS2005

The Role of Spatial Disorientation in Fatal General Aviation Accidents

In-flight Spatial Disorientation (SD) in pilots is a serious threat to aviation safety. Indeed, SD may play a much larger role in aviation accidents than the approximate 6-8% reported by the National Transportation Safety Board (NTSB) each year, because some accidents coded by the NTSB as aircraft control-not maintained (ACNM) may actually result from SD. The purpose of this study is to determine whether SD is underestimated as a cause of fatal general aviation (GA) accidents in the NTSB database. Fatal GA airplane accidents occurring between January 1995 and December 1999 were reviewed from the NTSB aviation accident database. Cases coded as ACNM or SD as the probable cause were selected for review by a panel of aerospace medicine specialists. Using a rating scale, each rater was instructed to determine if SD was the probable cause of the accident. Agreement between the raters and agreement between the raters and the NTSB were evaluated by Kappa statistics. The raters agreed that 11 out of 20 (55%) accidents coded by the NTSB as ACNM were probably caused by SD (p less than 0.05). Agreement between the raters and the NTSB did not reach significance (p greater than 0.05). The 95% C.I. for the sampling population estimated that between 33-77% of cases that the NTSB identified as ACNM could be identified by aerospace medicine experts as SD. Aerospace medicine specialists agreed that some cases coded by the NTSB as ACNM were probably caused by SD. Consequently, a larger number of accidents may be caused by the pilot succumbing to SD than indicated in the NTSB database. This new information should encourage regulating agencies to insure that pilots receive SD recognition training, enabling them to take appropriate corrective actions during flight. This could lead to new training standards, ultimately saving lives among GA airplane pilots.

Scheuring, RIchard↗
NASA NTRS2018

Integration of Optical Coherence Tomography Scan Patterns to Augment Clinical Data Suite

Vision changes identified in long duration spaceflight astronauts has led Space Medicine at NASA to adopt a more comprehensive clinical monitoring protocol. Optical Coherence Tomography (OCT) was recently implemented at NASA, including on board the International Space Station in 2013. NASA is collaborating with Heidelberg Engineering to increase the fidelity of the current OCT data set by integrating the traditional circumpapillary OCT image with radial and horizontal block images at the optic nerve head. The retinal nerve fiber layer was segmented by two experienced individuals. Intra-rater (N=4 subjects and 70 images) and inter-rater (N=4 subjects and 221 images) agreement was performed. The results of this analysis and the potential benefits will be presented.

Mason, S.↗
NASA NTRS1993

Specification-based software sizing: An empirical investigation of function metrics

For some time the software industry has espoused the need for improved specification-based software size metrics. This paper reports on a study of nineteen recently developed systems in a variety of application domains. The systems were developed by a single software services corporation using a variety of languages. The study investigated several metric characteristics. It shows that: earlier research into inter-item correlation within the overall function count is partially supported; a priori function counts, in themself, do not explain the majority of the effort variation in software development in the organization studied; documentation quality is critical to accurate function identification; and rater error is substantial in manual function counting. The implication of these findings for organizations using function based metrics are explored.

Jeffery, Ross↗
NASA NTRS1997

LOFT Debriefings: An Analysis of Instructor Techniques and Crew Participation

This study analyzes techniques instructors use to facilitate crew analysis and evaluation of their Line-Oriented Flight Training (LOFT) performance. A rating instrument called the Debriefing Assessment Battery (DAB) was developed which enables raters to reliably assess instructor facilitation techniques and characterize crew participation. Thirty-six debriefing sessions conducted at five U.S. airlines were analyzed to determine the nature of instructor facilitation and crew participation. Ratings obtained using the DAB corresponded closely with descriptive measures of instructor and crew performance. The data provide empirical evidence that facilitation can be an effective tool for increasing the depth of crew participation and self-analysis of CRM performance. Instructor facilitation skill varied dramatically, suggesting a need for more concrete hands-on training in facilitation techniques. Crews were responsive but fell short of actively leading their own debriefings. Ways to improve debriefing effectiveness are suggested.

Dismukes, R. Key↗
NASA NTRS2003

A Gold Standards Approach to Training Instructors to Evaluate Crew Performance

The Advanced Qualification Program requires that airlines evaluate crew performance in Line Oriented Simulation. For this evaluation to be meaningful, instructors must observe relevant crew behaviors and evaluate those behaviors consistently and accurately against standards established by the airline. The airline industry has largely settled on an approach in which instructors evaluate crew performance on a series of event sets, using standardized grade sheets on which behaviors specific to event set are listed. Typically, new instructors are given a class in which they learn to use the grade sheets and practice evaluating crew performance observed on videotapes. These classes emphasize reliability, providing detailed instruction and practice in scoring so that all instructors within a given class will give similar scores to similar performance. This approach has value but also has important limitations; (1) ratings within one class of new instructors may differ from those of other classes; (2) ratings may not be driven primarily by the specific behaviors on which the company wanted the crews to be scored; and (3) ratings may not be calibrated to company standards for level of performance skill required. In this paper we provide a method to extend the existing method of training instructors to address these three limitations. We call this method the "gold standards" approach because it uses ratings from the company's most experienced instructors as the basis for training rater accuracy. This approach ties the training to the specific behaviors on which the experienced instructors based their ratings.

Baker, David P.↗
NASA NTRSDate not supplied

Evidence of Participation in the Advancement of Knowledge from 25 Years of Publications for the GLOBE Program

The Global Learning and Observations to Benefit the Environment (GLOBE) Program was announced on Earth Day 1994 and began on Earth Day 1995. Over the first 25 years of its operation, there have been more than 600 formal publications and 340 conference presentations about the program, in at least 15 languages. As the 25th anniversary of the program approached, we began a focused effort to find and characterize those publications in order to better understand GLOBE’s impact to date as well as its future potential. GLOBE was envisioned from the beginning as a science and education program, so GLOBE publications cover a wide range of topics across Earth system science as well as in education and evaluation topics. To characterize the collection, a large set of dimensions was first identified from the literature, across science and education. These include codes related to: motivation and conception of the research investigation, educator practices and training, education outcomes & demonstration that these outcomes were achieved, data and study scope, characteristics of the investigation or its findings. Application of these codes was first tested by three raters (the first three authors) on a subset of the publications, then refined to capture missing aspects. The full set of publications was then rated on the full set of criteria. As a second step, subsets of the collection with particularly in depth discussion of The GLOBE Program are now being analyzed in more detail. This poster will summarize various attributes of the GLOBE bibliography and identify areas of potential for further geoscience investigations.

Lin Chambers↗
NASA NTRSDate not supplied

Defining Change Thresholds: What Change Is Outside Typical Sources of Variation?

Researchers often have a difficult time defining meaningful thresholds for change. We sometimes identify subtle changes but what amount of change is beyond typical sources of variation? This is especially complicated when trying to understand new disease pathogenesis like the constellation of eye changes leading to Spaceflight-associated Neuro-ocular Syndrome (SANS). To support decision makers in defining minimal meaningful change, we used a Bayesian hierarchical model to estimate innate sources of variability such as natural day to day variation. Healthy subjects were recruited and imaged with MRI, OCT, and US on separate days and measured by several technicians. Models were developed specifying random effects for the sources of variation – between left and right eyes, within-individuals over time, between raters, and finally between individuals. This allowed us to find the posterior distribution for the total typical variation, within an eye, which we use to define a threshold where change beyond typical sources of variation is likely. This threshold is now used as our earliest indicator of systematic increase in Total Retinal Thickness (a precursor to optic disc edema).

Millennia Young↗
NASA NTRS1991

Results from a GPS Shuttle Training Aircraft flight test

A series of Global Positioning System (GPS) flight tests were performed on a National Aeronautics and Space Administration's (NASA's) Shuttle Training Aircraft (STA). The objective of the tests was to evaluate the performance of GPS-based navigation during simulated Shuttle approach and landings for possible replacement of the current Shuttle landing navigation aid, the Microwave Scanning Beam Landing System (MSBLS). In particular, varying levels of sensor data integration would be evaluated to determine the minimum amount of integration required to meet the navigation accuracy requirements for a Shuttle landing. Four flight tests consisting of 8 to 9 simulation runs per flight test were performed at White Sands Space Harbor in April 1991. Three different GPS receivers were tested. The STA inertial navigation, tactical air navigation, and MSBLS sensor data were also recorded during each run. C-band radar aided laser trackers were utilized to provide the STA 'truth' trajectory.

Saunders, Penny E.↗