Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Expert Judgment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Eucalyptus – An Analysis Suite for Fault Trees with Uncertainty Quantification

Eucalyptus is a novel code developed at Lawrence Livermore National Laboratory to incorporate uncertainty quantification into Fault Tree Analysis (FTA). This tool addresses the challenge of imperfect knowledge in “grey-box” systems by allowing analysts to incorporate and propagate uncertainty from component-level assessments to system-level effects. Eucalyptus facilitates a consistent evaluation of the impact of subject matter expert judgment and knowledge gaps on overall system response by Monte Carlo generation of possible system fault trees, sampling probabilities of the existence of subsystems and components. Here, the code supports the specification of fault trees through text and allows export to various formats, including auto-generated images, easing analysis and reducing errors. It has undergone extensive verification testing, demonstrating its reliability and readiness for deployment, and leverages on-node parallelism for rapid analysis. Example analyses are shown that include the identification of system failure paths and quantification of the value of further information about system components.

Fault Tree Analysis↗
NASA NTRS1991

Decision paths in complex tasks

Complex real world action and its prediction and control has escaped analysis by the classical methods of psychological research. The reason is that psychologists have no procedures to parse complex tasks into their constituents. Where such a division can be made, based say on expert judgment, there is no natural scale to measure the positive or negative values of the components. Even if we could assign numbers to task parts, we lack rules i.e., a theory, to combine them into a total task representation. We compare here two plausible theories for the amalgamation of the value of task components. Both of these theories require a numerical representation of motivation, for motivation is the primary variable that guides choice and action in well-learned tasks. We address this problem of motivational quantification and performance prediction by developing psychophysical scales of the desireability or aversiveness of task components based on utility scaling methods (Galanter 1990). We modify methods used originally to scale sensory magnitudes (Stevens and Galanter 1957), and that have been applied recently to the measure of task 'workload' by Gopher and Braune (1984). Our modification uses utility comparison scaling techniques which avoid the unnecessary assumptions made by Gopher and Braune. Formula for the utility of complex tasks based on the theoretical models are used to predict decision and choice of alternate paths to the same goal.

Galanter, Eugene↗
NASA NTRS2005

Applications of Principled Search Methods in Climate Influences and Mechanisms

Forest and grass fires cause economic losses in the billions of dollars in the U.S. alone. In addition, boreal forests constitute a large carbon store; it has been estimated that, were no burning to occur, an additional 7 gigatons of carbon would be sequestered in boreal soils each century. Effective wildfire suppression requires anticipation of locales and times for which wildfire is most probable, preferably with a two to four week forecast, so that limited resources can be efficiently deployed. The United States Forest Service (USFS), and other experts and agencies have developed several measures of fire risk combining physical principles and expert judgment, and have used them in automated procedures for forecasting fire risk. Forecasting accuracies for some fire risk indices in combination with climate and other variables have been estimated for specific locations, with the value of fire risk index variables assessed by their statistical significance in regressions. In other cases, the MAPSS forecasts [23, 241 for example, forecasting accuracy has been estimated only by simulated data. We describe alternative forecasting methods that predict fire probability by locale and time using statistical or machine learning procedures trained on historical data, and we give comparative assessments of their forecasting accuracy for one fire season year, April- October, 2003, for all U.S. Forest Service lands. Aside from providing an accuracy baseline for other forecasting methods, the results illustrate the interdependence between the statistical significance of prediction variables and the forecasting method used.

Glymour, Clark↗
NASA NTRS2010

Constellation Probabilistic Risk Assessment (PRA): Design Consideration for the Crew Exploration Vehicle

Managed by NASA's Office of Safety and Mission Assurance, a pilot probabilistic risk analysis (PRA) of the NASA Crew Exploration Vehicle (CEV) was performed in early 2006. The PRA methods used follow the general guidance provided in the NASA PRA Procedures Guide for NASA Managers and Practitioners'. Phased-mission based event trees and fault trees are used to model a lunar sortie mission of the CEV - involving the following phases: launch of a cargo vessel and a crew vessel; rendezvous of these two vessels in low Earth orbit; transit to th$: moon; lunar surface activities; ascension &om the lunar surface; and return to Earth. The analysis is based upon assumptions, preliminary system diagrams, and failure data that may involve large uncertainties or may lack formal validation. Furthermore, some of the data used were based upon expert judgment or extrapolated from similar components~systemsT. his paper includes a discussion of the system-level models and provides an overview of the analysis results used to identify insights into CEV risk drivers, and trade and sensitivity studies. Lastly, the PRA model was used to determine changes in risk as the system configurations or key parameters are modified.

Prassinos, Peter G.↗
NASA NTRS2010

Reliability Quantification of Advanced Stirling Convertor (ASC) Components

The Advanced Stirling Convertor, is intended to provide power for an unmanned planetary spacecraft and has an operational life requirement of 17 years. Over this 17 year mission, the ASC must provide power with desired performance and efficiency and require no corrective maintenance. Reliability demonstration testing for the ASC was found to be very limited due to schedule and resource constraints. Reliability demonstration must involve the application of analysis, system and component level testing, and simulation models, taken collectively. Therefore, computer simulation with limited test data verification is a viable approach to assess the reliability of ASC components. This approach is based on physics-of-failure mechanisms and involves the relationship among the design variables based on physics, mechanics, material behavior models, interaction of different components and their respective disciplines such as structures, materials, fluid, thermal, mechanical, electrical, etc. In addition, these models are based on the available test data, which can be updated, and analysis refined as more data and information becomes available. The failure mechanisms and causes of failure are included in the analysis, especially in light of the new information, in order to develop guidelines to improve design reliability and better operating controls to reduce the probability of failure. Quantified reliability assessment based on fundamental physical behavior of components and their relationship with other components has demonstrated itself to be a superior technique to conventional reliability approaches based on utilizing failure rates derived from similar equipment or simply expert judgment.

Shah, Ashwin R.↗
NASA NTRS2011

Towards Measurement of Confidence in Safety Cases

Arguments in safety cases are predominantly qualitative. This is partly attributed to the lack of sufficient design and operational data necessary to measure the achievement of high-dependability targets, particularly for safety-critical functions implemented in software. The subjective nature of many forms of evidence, such as expert judgment and process maturity, also contributes to the overwhelming dependence on qualitative arguments. However, where data for quantitative measurements is systematically collected, quantitative arguments provide far more benefits over qualitative arguments, in assessing confidence in the safety case. In this paper, we propose a basis for developing and evaluating integrated qualitative and quantitative safety arguments based on the Goal Structuring Notation (GSN) and Bayesian Networks (BN). The approach we propose identifies structures within GSN-based arguments where uncertainties can be quantified. BN are then used to provide a means to reason about confidence in a probabilistic way. We illustrate our approach using a fragment of a safety case for an unmanned aerial system and conclude with some preliminary observations

Denney, Ewen↗
NASA NTRS2015

New York City Panel on Climate Change 2015 Report: Climate Observations and Projections - Chapter 1

During 2013 and 2014, numerous international (IPCC, 2013) and national (Melillo et al., 2014; Gordon, 2014) reports have concluded that human activities are changing the climate, leading to increased vulnerability and risk. Since the industrial revolution, fossil fuel burning, industrial activity, and land use changes have led to a 40% increase in heat-trapping carbon dioxide (CO2), and an approximately 150% increase in methane (CH4), another powerful greenhouse gas (GHG), has been observed. Global temperatures have increased by close to 1 C since 1880 as the upper oceans have warmed and polar ice has retreated. These and other climate changes are projected to accelerate as greenhouse gas concentrations continue to rise. In the coming decades, climate change is extremely likely to bring warmer temperatures in the New York metropolitan region (see Box. 1.1 and Fig.1.1 for key definitions and terms). Heat waves are very likely to increase; total annual precipitation will likely increase and brief, intense rainstorms are very likely to increase. Because of incomplete knowledge about exactly how much climate change will occur, choosing among policies for reducing future damages requires prudent risk management (Yohe and Leichenko, 2010; Kunreuther et al., 2013). Given differing risk tolerances among stakeholders, a risk management approach allows for a range of possible climate change outcomes to be examined with associated uncertainties surrounding their likelihoods. The New York City Panel on Climate Change 2 (NPCC2) projections can be used to inform planning across multiple governmental scales (e.g., city, county, state) in the New York metropolitan region. Such coordinated efforts can serve as test cases for successful local, state, and federal coordination for integrated climate adaptation initiatives. This chapter describes the global climate system, and presents observed temperature and precipitation trends and projections for the region. Chapter 2 (NPCC, 2015) focuses on sea level rise and possible changes in coastal storms. Chapter 3 and Chapter 4 (NPCC, 2015) describe efforts to better understand the region's vulnerability to coastal flooding during coastal storms. The treatment of likelihood related to the NPCC projections is similar to that developed by the Intergovernmental Panel on Climate Change Fourth and Fifth Assessment Reports (IPCC, 2007; 2013), with six likelihood categories (Box 1.1 and Fig. 1.1). The assignment of climate hazards to these categories is based on observed data, global climate model simulations, published literature, and expert judgment.

climate change↗
DOE OSTI2024

An Approach to Dynamic Human Reliability Analysis and Its Data Collection Framework

Human reliability analysis (HRA) is a method for evaluating human errors in a variety of complex systems such as nuclear power plants, military systems, aircraft, and chemical plants. Most HRA methods currently used by regulatory institutes or utilities are called static HRA and are carried out by simple worksheets or simple calculators. To date, there are many unsolved or intrinsic challenges in static HRA. For example, existing static HRA does not realistically model and evaluate human actions as they would be performed at actual systems. There is no method with HRA to objectively estimate the time required for human actions despite being essential to HRA processes. In addition, many HRA methods still rely on a dataset generated prior to the 1980s, from unrelated industry experience or simply from expert judgment. Accordingly, this study attempted to research how to overcome the challenges of existing HRA via dynamic risk assessment (a.k.a., simulation-based or computation-based risk assessment) techniques. First, this study developed a dynamic HRA method, named as PRocedure-based Investigation Method of EMRALD Risk Assessment – HRA (PRIMERA-HRA). The PRIMERA-HRA mainly concentrates on providing HRA analysts with specific guidelines on how to reasonably model human actions, assign human reliability data and evaluate output of simulation within a dynamic probabilistic risk assessment tool, called as Event Modeling Risk Assessment using Linked Diagrams (EMRALD). Second, this study also developed a module for performance shaping factors (i.e., the key concept in HRA quantification) applicable to dynamic HRA, then implemented it based on PRIMERA-HRA within the EMRALD tool. Third, this study developed an HRA data collection framework to support dynamic HRA, called as Simplified Human Error Experimental Program (SHEEP). Originally, the SHEEP study aimed to support static HRA and its data collection, but recently extended the scope to the new technologies such as dynamic HRA or HRA for advanced reactors. SHEEP focuses on the use of data collected from simplified simulators to complement—but not replace—data collection studies using full-scope simulators and actual operators. To date, many experiments were conducted under the SHEEP framework. Multiple analyses, such as human performance analysis, human error analysis, task complexity analysis, learning effect analysis and time distribution analysis, were also carried out using the collected data. Then, based on the major insights, an approach to inferring full-scope data based on simplified simulator data was proposed. The PRIMERA-HRA and SHEEP research are expected to evaluate human actions more realistically than existing static HRA, provide an opportunity to collect more HRA data with reasonable cost and labor, then contribute to enhance the quality of HRA.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

An Approach to Dynamic Human Reliability Analysis and Its Data Collection Framework

Human reliability analysis (HRA) is a method for evaluating human errors in a variety of complex systems such as nuclear power plants, military systems, aircraft, and chemical plants. Most HRA methods currently used by regulatory institutes or utilities are called static HRA and are carried out by simple worksheets or simple calculators. To date, there are many unsolved or intrinsic challenges in static HRA. For example, existing static HRA does not realistically model and evaluate human actions as they would be performed at actual systems. There is no method with HRA to objectively estimate the time required for human actions despite being essential to HRA processes. In addition, many HRA methods still rely on a dataset generated prior to the 1980s, from unrelated industry experience or simply from expert judgment. Accordingly, this study attempted to research how to overcome the challenges of existing HRA via dynamic risk assessment (a.k.a., simulation-based or computation-based risk assessment) techniques. First, this study developed a dynamic HRA method, named as PRocedure-based Investigation Method of EMRALD Risk Assessment – HRA (PRIMERA-HRA). The PRIMERA-HRA mainly concentrates on providing HRA analysts with specific guidelines on how to reasonably model human actions, assign human reliability data and evaluate output of simulation within a dynamic probabilistic risk assessment tool, called as Event Modeling Risk Assessment using Linked Diagrams (EMRALD). Second, this study also developed a module for performance shaping factors (i.e., the key concept in HRA quantification) applicable to dynamic HRA, then implemented it based on PRIMERA-HRA within the EMRALD tool. Third, this study developed an HRA data collection framework to support dynamic HRA, called as Simplified Human Error Experimental Program (SHEEP). Originally, the SHEEP study aimed to support static HRA and its data collection, but recently extended the scope to the new technologies such as dynamic HRA or HRA for advanced reactors. SHEEP focuses on the use of data collected from simplified simulators to complement—but not replace—data collection studies using full-scope simulators and actual operators. To date, many experiments were conducted under the SHEEP framework. Multiple analyses, such as human performance analysis, human error analysis, task complexity analysis, learning effect analysis and time distribution analysis, were also carried out using the collected data. Then, based on the major insights, an approach to inferring full-scope data based on simplified simulator data was proposed. The PRIMERA-HRA and SHEEP research are expected to evaluate human actions more realistically than existing static HRA, provide an opportunity to collect more HRA data with reasonable cost and labor, then contribute to enhance the quality of HRA.

99 - GENERAL AND MISCELLANEOUS↗
NASA NTRS1989

Expert systems should be more accurate than human experts - Evaluation procedures from human judgment and decisionmaking

Two procedures for the evaluation of the performance of expert systems are illustrated: one procedure evaluates predictive accuracy; the other procedure is complementary in that it uncovers the factors that contribute to predictive accuracy. Using these procedures, it is argued that expert systems should be more accurate than human experts in two senses. One sense is that expert systems must be more accurate to be cost-effective. Previous research is reviewed and original results are presented which show that simple statistical models typically perform better than human experts for the task of combining evidence from a given set of information sources. The results also suggest the second sense in which expert systems should be more accurate than human experts. They reveal that expert systems should share factors that contribute to human accuracy, but not factors that detract from human accuracy. Thus the thesis is that one should both require and expect systems to be more accurate than humans.

Levi, Keith↗
NASA NTRS2005

CoMET: Cost and Mass Evaluation Tool for Spacecraft and Mission Design

New technology in space exploration is often developed without a complete knowledge of its impact. While the immediate benefits of a new technology are obvious, it is harder to understand its indirect consequences, which ripple through the entire system. COMET is a technology evaluation tool designed to illuminate how specific technology choices affect a mission at each system level. COMET uses simplified models for mass, power, and cost to analyze performance parameters of technologies of interest. The sensitivity analysis that CoMET provides shows whether developing a certain technology will greatly benefit the project or not. CoMET is an ongoing project approaching a web-based implementation phase. This year, development focused on the models for planetary daughter craft, such as atmospheric probes, blimps and balloons, and landers. These models are developed through research into historical data, well established rules of thumb, and engineering judgment of experts at JPL. The model is validated by corroboration with JpL advanced mission studies. Other enhancements to COMET include adding launch vehicle analysis and integrating an updated cost model. When completed, COMET will allow technological development to be focused on areas that will most drastically improve spacecraft performance.

Bieber, Ben S.↗
DOE OSTI1969

Application of Nuclear Technology to Art Identification Problems: First Annual Report

Throughout the centuries, as man has become increasingly affluent, his interest in antiquities and objects of art, and the price he has been willing to pay for them, has increased. Inevitably, one result has been to attract the unscrupulous to the production and sale of counterfeit paintings, sculptures, and other works of art. The skill of forgers varies but often is highly sophisticated. However, dealers, museum curators, and other experts have also become more skillful in the detection of forgeries. Scientific tools of increasing sensitivity and sophistication have gradually been applied to the examination of the materials of art and archaeology. Such tools frequently support and render more reliable the judgment of the experts. The quality of forgeries may improve further as their makers become acquainted with the new methods of detection and in turn learn to circumvent or confound the methods. Therefore, the development of still more advanced methods of examination has great utility in the art world. This is particularly true if the ultimate effect is to make circumvention of the methods of examination so costly that the economic incentives for producing forgeries may be substantially reduced, if not eliminated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Observations and Machine-Learned Models of Near-Surface Permafrost along the Koyukuk River, Alaska, USA

This dataset contains GeoTIFs (raster) and GeoPackages (vector) that map observations of near-surface permafrost and not-permafrost from a field campaign conducted near the village of Huslia, AK along the Koyukuk River and its floodplain in July 2018. These data were collected as part of a campaign to understand if and how permafrost impacts riverbank erosion. This problem cannot be assessed without knowing where permafrost exists. Permafrost was observed via frost probing (to a maximum depth of one meter), coring (to a maximum depth of two meters) and bank/bar excavations. An additional boat survey was performed wherein expert (Joel Rowland) judgment assessed the presence or absence of distinctive permafrost features (e.g., overhanging tundra mats, thermoerosional niching, ice wedges, active drainage of ice melt from soils). This dataset also contains the input features and results of two machine learning models (random forest and convolutional neural network) that extrapolate the observations to the full floodplain that may be useful for building, testing, or validating other machine-learned permafrost models. Permafrost data are provided as georasters of the same shape and geovectors (polylines/polygons) and are all projected into EPSG:32605. All data can be visualized with a GIS (QGIS, ArcGIS, etc.).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1994

A new machine classification method applied to human peripheral blood leukocytes

Human beings judge images by complex mental processes, whereas computing machines extract features. By reducing scaled human judgments and machine extracted features to a common metric space and fitting them by regression, the judgments of human experts rendered on a sample of images may be imposed on an image population to provide automatic classification.

Rorvig, Mark E.↗
NASA NTRS2005

Visualization support for risk-informed decision making when planning and managing software developments

Key decisions are made in the early stages of planning and management of software developments. The information basis for these decisions is often a mix of analogy with past developments, and the best judgments of domain experts. Visualization of this information can support to such decision making by clarifying the status of the information and yielding insights into the ramifications of that information vis-a-vis decision alternatives.

technology infusion↗
NASA NTRS2006

Scheduling Software for Complex Scenarios

Preparing a vehicle and its payload for a single launch is a complex process that involves thousands of operations. Because the equipment and facilities required to carry out these operations are extremely expensive and limited in number, optimal assignment and efficient use are critically important. Overlapping missions that compete for the same resources, ground rules, safety requirements, and the unique needs of processing vehicles and payloads destined for space impose numerous constraints that, when combined, require advanced scheduling. Traditional scheduling systems use simple algorithms and criteria when selecting activities and assigning resources and times to each activity. Schedules generated by these simple decision rules are, however, frequently far from optimal. To resolve mission-critical scheduling issues and predict possible problem areas, NASA historically relied upon expert human schedulers who used their judgment and experience to determine where things should happen, whether they will happen on time, and whether the requested resources are truly necessary.

Source record↗
NASA NTRS1985

Perspectives on knowledge in engineering design

Various perspectives are given of the knowledge currently used in engineering design, specifically dealing with knowledge-based expert systems (KBES). Constructing an expert system often reveals inconsistencies in domain knowledge while formalizing it. The types of domain knowledge (facts, procedures, judgments, and control) differ from the classes of that knowledge (creative, innovative, and routine). The feasible tasks for expert systems can be determined based on these types and classes of knowledge. Interpretive tasks require reasoning about a task in light of the knowledge available, where generative tasks create potential solutions to be tested against constraints. Only after classifying the domain by type and level can the engineer select a knowledge-engineering tool for the domain being considered. The critical features to be weighed after classification are knowledge representation techniques, control strategies, interface requirements, compatibility with traditional systems, and economic considerations.

Rasdorf, W. J.↗