SEARCH · Engineering Papers

Results for “expert elicitation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

NASA NTRS2020

Burning Embers: Towards More Transparent and Robust Climate-change Risk Assessments

The Intergovernmental Panel on Climate Change (IPCC) reports provide policy-relevant insights about climate impacts, vulnerabilities and adaptation through a process of peer-reviewed literature assessments underpinned by expert judgement. An iconic output from these assessments is the burning embers diagram, first used in the Third Assessment Report to visualize reasons for concern, which aggregate climate-change-related impacts and risks to various systems and sectors. These burning embers use colour transitions to show changes in the assessed level of risk to humans and ecosystems as a function of global mean temperature. In this Review, we outline the history and evolution of the burning embers and associated reasons for concern framework, focusing on the methodological approaches and advances. While the assessment framework and figure design have been broadly retained over time, refinements in methodology have occurred, including the consideration of different risks, use of confidence statements, more formalized protocols and standardized metrics. Comparison across reports reveals that the risk level at a given temperature has generally increased with each assessment cycle, reflecting accumulating scientific evidence. For future assessments, an explicit, transparent and systematic process of expert elicitation is needed to enhance comparability, quality and credibility of burning embers.

burning embers diagram↗

NASA NTRSDate not supplied

Application of Analytical Hierarchy Process for Narrowing Down Nep Candidate Reactor Designs

Nuclear electric propulsion (NEP)-powered vehicles have been contemplated for human Mars missions. The nuclear power system contemplates using a high temperature light-weight nuclear reactor for the production of electrical power in the range of 2-5 MWe with a 3-10 year service life. A myriad of technology options exist for achieving these mission objectives. The analytic hierarchy process (AHP), a multi-attribute decision method, is being used to narrow down the candidate designs. The AHP is a structured decision process that fuses model-supplied quantitative data with subjective assessments to facilitate decisions that involve multiple competing criteria. Proven end-to-end nuclear design and systems analysis tools will be used to provide quantitative performance data such as end-to-end system reliability, system robustness to recover from off-normal conditions, system specific weight (α in kg/kWe), and the ability to meet the service life-time and power level requirements. Three high assay low enriched uranium (HALEU) reactor concepts, namely, gas-, heat pipe- and pumped liquid-cooled nuclear cores coupled to a He/Xe gas or supercritical CO2- Brayton power conversion system are being modeled using the AHP to understand the trade-offs associated with these design combinations. In addition to the reactor and power conversion options, there are a myriad of additional components and subsystems – e.g., radiators, heat exchangers, and recuperators – that also figure into the evaluation process. The choice of the right overall system is a multidimensional problem that has to include not only quantitative data, but also the so called “external factors”, examples of which include the component and subsystem Technology Readiness Levels (TRLs), the associated Advancement Degrees of Difficulty (AD2), the cost and schedule required to achieve a technical maturity consistent with mission infusion, alignment with the priorities of NASA’s Space Nuclear Propulsion program, and alignment with other ongoing government and commercial investments in micro reactors. Focused expert elicitations form the basis for qualitative data set. As a final step, decision-makers individually express their opinions regarding the relative importance of the criteria and preferences among the alternatives through pairwise comparisons. The paper will describe the AHP approach, progress to-date applying it to the NEP human Mars mission problem, and preliminary results. Use of AHP provides sufficient flexibility for incorporating industry input at different stages as the technologies evolve through additional research and development. It is expected that the decision process will be ongoing and expanded to examine additional options and technology choices, culminating in a defensible set of candidate reactor concepts that will form the basis for developing a multi-year NEP technology maturation strategy. .

Dasari V Rao↗

NASA NTRSDate not supplied

Establishing the Framework for e-VTOL Flight Mission Scenario Development for Urban Air Mobility Research using Cognitive Task Analysis

This paper focuses on applying specific protocol aspects of Cognitive Task Analysis (CTA) methods to formulate non-routine flight mission use-case scenarios in support of a potential research study at the National Aeronautics and Space Administration (NASA) Langley Research Center under the Air Traffic Management – eXploration (ATM-X) Urban Air Mobility (UAM) sub-project. The objective of use-case scenarios is to provide a hypothetical situation to elicit expert knowledge and feedback in a specific subject matter area. Use-case scenarios in the field of aviation, including UAM, can help to support the safe integration of these vehicles into the National Airspace System (NAS) through the examination of airspace procedures during non-routine events. Inherent in this type of research is the need to study in-flight non-routine scenarios that the UAM vehicle could encounter. Situations categorized as non-routine events are any events that occur outside of the original flight plan such as a mechanical malfunction or other in-flight emergency requiring special procedures, which can include diversions to a specified, or non-specified, emergency landing area. In specific instances, some non-routine events could be considered as “routine” during in-flight operations, such as the need to perform a go-around due to a balked landing. As the rate of technological expansion surrounding UAM grows rapidly in modern aviation, so does the need for research that focuses on the safe integration of these vehicles including research with a focus on the methods and development of airspace operational procedures. Research rooted in CTA methods provides a framework for developing use-case scenarios appropriate for environments where mental demands are substantial. Substantial mental demands exist while operating in the NAS for pilots, airline dispatchers, and air traffic controllers among many others. With the focus on UAM in mind, it is critical to frame a baseline use-case scenario using information as it pertains to current-day airspace operations. According to Dr. Robert R. Hoffman’s “Protocols for Cognitive Task Analysis,” one of the first steps to applying CTA methods to research is the concept of “bootstrapping” in which the researcher(s) familiarize themselves with the domain that is being studied. To formulate the non-routine flight mission use-case scenarios for the purpose of this potential NASA ATM-X UAM research study, and to inform the baseline use-case scenario, the practice of bootstrapping was utilized to acquire knowledge of current-day airspace procedures.

Heidi S Glaudel↗

NASA NTRSDate not supplied

Systemic Risk From the Perspective of Climate, Environmental and Disaster Risk Science and Practice

Understanding and managing systemic risk is more important than ever due to our immense global connectivity (e.g., between sectors, such as food-health-water-energy, countries and continents, down to individuals). Despite the fact that the notion of systemic risk is several decades old, the term is used in diverse ways across different disciplines (e.g., financial systems, medicine, earth system sciences, disaster risk research and climate science). Triggered by the repercussions of the global financial crisis of the late 2000s, and more recently the COVID-19 pandemic, which are clear realization of systemic risk, the perception of systemic risk has often been focused on global and catastrophic or even existential risks. Systemic risk, however, can be seen as a feature of systems at all possible scales (e.g., global, national, regional, local) with system boundaries varying depending on the context. Addressing current societal challenges, such as climate change, in terms of systemic risk requires integrating different systems perspectives and fostering system thinking, while implementing key intergovernmental agendas, such as the Paris Agreement, the Sendai Framework for Disaster Risk Reduction and the Sustainable Development Goals. Based on insights gained and knowledge collected from an expert workshop, literature review and expert elicitation, we give an integrated perspective of climate, environmental and disaster risk science and practice on systemic risk as summarized in a Briefing Note to the International Science Council. We provide an overview of concepts of systemic risk that have evolved over time and identify commonalities across terminologies and perspectives associated with systemic risk used in different contexts. Key attributes of systemic risk are outlined without prescribing a single definition, and information and data requirements are discussed that are essential for a better and more actionable understanding of the systemic nature of risk. Finally, the opportunities to connect research and policy for addressing systemic risk are highlighted.

systemic risk↗

DOE OSTI2020

Direct-DC Power in Buildings: Identifying the Best Applications Today for Tomorrow’s Building Sector

Driven by the increased use of direct current (DC) sources (photovoltaics, battery storage) and DC end-use devices (electronics, solid-state lighting, efficient motors), DC power distribution in buildings and DC microgrids have been proposed as a way to achieve greater efficiency, cost savings, and resiliency in a transitioning building sector. Despite these important benefits, several market and technological barriers inhibit the development of DC distribution, and the market for DC in buildings is still largely in the demonstration phase. Therefore, to jumpstart this technology, a clear path forward must emerge at this early stage of deployment. The goal of this paper is to define specific end-use cases for which DC distribution in buildings is a value proposition today by defining clear efficiency and resiliency benefits while addressing barriers to implementation. The paper begins with a technology and market assessment of DC distribution equipment, end uses, and technology standards. That is followed by results from an expert elicitation of DC power and building end-use professionals (e.g., electrical designers, building operators, engineers) and reports on-site visits and lessons learned from successful (and less successful) field deployments of DC distribution projects in North America. We present specific adoption pathways at the community and building level that can be implemented today, and evaluate them using qualitative and quantitative metrics, such as technology and market readiness, energy savings, and resiliency.

building-level electrical distribution↗

DOE OSTI2021

Direct-DC Power in Buildings: Identifying the Best Applications Today for Tomorrow’s Building Sector

Driven by the increased use of direct current (DC) sources (photovoltaics, battery storage) and DC end-use devices (electronics, solid-state lighting, efficient motors), DC power distribution in buildings and DC microgrids have been proposed as a way to achieve greater efficiency, cost savings, and resiliency in a transitioning building sector.Despite these important benefits, several market and technological barriers inhibit the development of DC distribution, and the market for DC in buildings is still largely in the demonstration phase. Therefore, to jumpstart this technology, a clear path forward must emerge at this early stage of deployment. The goal of this paper is to define specific end-use cases for which DC distribution in buildings is a value proposition today by defining clear efficiency and resiliency benefits while addressing barriers to implementation.The paper begins with a technology and market assessment of DC distribution equipment, end uses, and technology standards. That is followed by results from an expert elicitation of DC power and building end-use professionals (e.g., electrical designers, building operators, engineers) and reports on-site visits and lessons learned from successful (and less successful) field deployments of DC distribution projects in North America. We present specific adoption pathways at the community and building level that can be implemented today, and evaluate them using qualitative and quantitative metrics, such as technology and market readiness, energy savings, and resiliency.

Vossos, Evangelos↗

DOE OSTI2024

Repowering and Retrofitting of Solar Inverters: A Field Case Study [Slides]

A handful of large-scale photovoltaic (PV) plants have undergone retrofitting and/or repowering of inverters for a variety of reasons, such as rapid product life-cycle innovations with lack of reverse compatibility, original equipment manufacturers (OEMs) exiting the inverter business, weather damage, more lucrative revenue opportunities driven by high contractual offtake price, and so on. Insights has been collected via expert elicitation and relevant staff involved in these retrofitting/repowering at the case study site. Selecting and installing inverters from a different OEM at the commercially operating PV plant provides a unique opportunity to thoroughly document inverter retrofitting/repowering. The case study describes compatibility of mechanical, electrical, communications, and other important aspects. The goal is to provide a public case study that improves the general knowledge of the solar industry about what is involved in repowering/retrofitting a PV plant and how others can best prepare.

14 SOLAR ENERGY↗

DOE OSTI2024

Enhancing PV Inverter Reliability Through Predictive Maintenance: Insights from Retrofitting of PV Inverters

Photovoltaic (PV) systems represent a cornerstone in the global shift toward sustainable energy generation, with inverters serving as the crucial link between solar panels and the grid. Despite their pivotal role, inverters are susceptible to failures, contributing significantly to maintenance events and operational disruptions in large-scale PV plants. This white paper investigates the importance and methodologies of predictive maintenance strategies that monitor the component-level pre-failure signatures on PV inverters. Through a comprehensive survey of literature and industry professionals, insights on preventive maintenance and retrofitting practices have been gathered. The expert elicitation helps shed light on common challenges and opportunities for improving PV system reliability, particularly inverter reliability. By addressing these challenges, the white paper aims to enhance the long-term viability and effectiveness of solar PV plants in the renewable energy landscape, contributing to the global transition toward environmentally friendly energy generation.

14 SOLAR ENERGY↗

NASA NTRS2013

A Step-Wise Approach to Elicit Triangular Distributions

Adapt/combine known methods to demonstrate an expert judgment elicitation process that: 1.Models expert's inputs as a triangular distribution, 2.Incorporates techniques to account for expert bias and 3.Is structured in a way to help justify expert's inputs. This paper will show one way of "extracting" expert opinion for estimating purposes. Nevertheless, as with most subjective methods, there are many ways to do this.

Greenberg, Marc W.↗

DOE OSTI2023

A Bayesian model for multivariate discrete data using spatial and expert information with application to inferring building attributes

When modeling sparsely observed multivariate data, strong prior information elicited from experts can be used to bolster predictive accuracy and counteract sampling bias. Similarly, modeling autocorrelation in space can help make use of co-occurrence patterns present in many types of spatial data. To make use of both expert prior information and spatial structure, we propose a novel graphical model for a spatial Bayesian network developed specifically to address challenges in inferring the attributes of buildings from geographically sparse observational data. This model is implemented as the sum of a spatial multivariate Gaussian random field and a tabular conditional probability function in real-valued space prior to projection onto the probability simplex. This modeling form is especially suitable for the usage of prior information in the form of sets of atomic rules obtained from experts. To perform inference with missing data, we implement a Markov chain Monte Carlo scheme composed of alternating steps of Gibbs sampling of missing entries and Hamiltonian Monte Carlo for model parameters. A case study in building attribution is presented to highlight the advantages and limitations of this approach.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2005

Developing a Methodology for Eliciting Subjective Probability Estimates During Expert Evaluations of Safety Interventions: Application for Bayesian Belief Networks

The NASA Aviation Safety Program (AvSP) has defined several products that will potentially modify airline and/or ATC operations, enhance aircraft systems, and improve the identification of potential hazardous situations within the National Airspace System (NAS). Consequently, there is a need to develop methods for evaluating the potential safety benefit of each of these intervention products so that resources can be effectively invested to produce the judgments to develop Bayesian Belief Networks (BBN's) that model the potential impact that specific interventions may have. Specifically, the present report summarizes methodologies for improving the elicitation of probability estimates during expert evaluations of AvSP products for use in BBN's. The work involved joint efforts between Professor James Luxhoj from Rutgers University and researchers at the University of Illinois. The Rutgers' project to develop BBN's received funding by NASA entitled "Probabilistic Decision Support for Evaluating Technology Insertion and Assessing Aviation Safety System Risk." The proposed project was funded separately but supported the existing Rutgers' program.

Wiegmann, Douglas A.a↗

DOE OSTI2022

Expert perspectives on the wind plant of the future

Abstract Wind power technology has changed rapidly in recent years. Technology innovation, evolving power markets, and competing land and ocean uses continue to influence the design and operation of wind turbines and plants. Anticipating these trends and their impact on future facilities can inform commercial strategies and research priorities. Drawing from a recent survey of 140 of the world's foremost wind experts, we identify expectations of future wind plant design in 2035, both for onshore and offshore wind. Experts anticipate continued growth in turbine size, to 5.5 (onshore) and 17 MW (offshore), with plants located in increasingly less favorable wind and siting regimes. They expect plant sizes of 1,100 MW for fixed‐bottom and 600 MW for floating offshore wind. Experts forecast enhanced grid‐system value from wind through significant to widespread use of larger rotors, hybrid projects with batteries and hydrogen production, and more. To explain experts' perspectives on future plant design and operation, we identify five mechanisms: economies of unit, plant, and resource scale; grid‐system value economies; and production efficiencies. We characterize learning effects as a moderating influence on the strength of these mechanisms. In combination, experts predict that these design choices support levelized cost of energy reductions of 27% (onshore) and 17%–35% (floating and fixed‐bottom offshore) by 2035 compared to today, while enhancing wind energy's grid service offerings. Our findings provide a much‐needed benchmark for representing future wind technologies in power sector models and address a critical research gap by explaining the economics behind wind energy design choices.

17 WIND ENERGY↗

DOE OSTI2025

Identifying Adversarial Cyber-Activity in Operational Technology Environments Using Bayesian Networks

Critical infrastructure and other operational technology (OT) environments face increasing cybersecurity risks from adversarial behavior. This paper describes the development of a risk model using a Bayesian network to enhance the comprehension of observable cyber events caused by malicious activity in OT environments. The core of the Bayesian network is a process model that describes the stages of adversary behavior. The remainder of the model is based on the MITRE ATT&CK® for Industrial Control Systems (ICS) taxonomy, which includes tactics and techniques that may be used by the adversary. The observables provide evidence for adversary behavior through the intermediary technique and tactic nodes. One challenge in constructing this model is a lack of open-source data from cyber-attacks on OT systems. This paper discusses learning from limited data, the elicitation of expert opinion to construct the conditional probability tables when data is scarce, and the refinement of the most difficult conditional probabilities tables using several forms of sensitivity analyses. Finally, the Bayesian network is demonstrated using two historical case studies: the DarkSide ransomware attack on the Colonial Pipeline and the destructive cyberattack targeting the ThyssenKrupp blast furnace. Index Terms—Cybersecurity, industrial control systems, operational technology

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Estimating the Contributions to Human Error Probability from the Convolution of the Distribution of Time Available and Time Required

As part of their duties, Human Reliability Analysis must often evaluate if crews in nuclear power plants (NPPs) can complete tasks associated with a human-failure event within time limits. For example, the time required in NPP scenarios is determined by systematic and structured walkthroughs, feasibility studies, recorded times from training exercises, and interviews with experienced operators and experts. Typically, a point estimate is derived for the estimate (mean, maximum, or 95th percentile of time required). Using point-estimate values can mask the risk associated with variability among crews, plant conditions and set-up, environmental conditions, and other impact factors under which these actions are executed. While point estimates for time required and time available have served the industry well, without considering the uncertainty they could lead to biased understanding about the risk. The Integrated Human Event Analysis System - General Methodology (IDHEAS-G) model (developed by the US Nuclear Regulatory Commission, NRC) for human error probability calculates human error probability by summing two probabilities: insufficient time and cognitive error. As such, the model takes a more holistic approach by considering the full distributions for time required and time available to calculate the human error probability because the time available to complete the task is insufficient. In this study, we expand on the work of the NRC and discuss methods for estimating these time considerations. For example, for the time required, the impact of Performance Influencing Factors (PIFs) on the distribution was divided into impacts that are aleatory in nature, such as crew-to-crew variability, and those that are epistemic (i.e., the PIFs). Starting with the factors that introduce aleatory uncertainty, a first-order distribution was developed from a large set of time required (i.e., NPP task completion times) data for the range of operator actions that occur in the NPP control room under simulated accident conditions. The first-order distribution can then be adjusted to account for epistemic uncertainty using research associated with the impact of applicable PIFs on the time required. We also develop guidance for analysts to address the probability distributions for the time available. The guidance we developed on how to estimate time required and time available distributions is based on the identification of pertinent research and data, data analyses, and expert knowledge elicitation.

human error probability, human performance, time e↗

NASA NTRS1992

Min and max are the only continuous ampersand-, V-operations for finite logics

Experts usually express their degrees of belief in their statements by the words of a natural language (like 'maybe', 'perhaps', etc.). If an expert system contains the degrees of beliefs t(A) and t(B) that correspond to the statements A and B, and a user asks this expert system whether 'A&B' is true, then it is necessary to come up with a reasonable estimate for the degree of belief of A&B. The operation that processes t(A) and t(B) into such an estimate t(A&B) is called an &-operation. Many different &-operations have been proposed. Which of them to choose? This can be (in principle) done by interviewing experts and eliciting a &-operation from them, but such a process is very time-consuming and therefore, not always possible. So, usually, to choose a &-operation, the finite set of actually possible degrees of belief is extended to an infinite set (e.g., to an interval (0,1)), define an operation there, and then restrict this operation to the finite set. Only this original finite set is considered. It is shown that a reasonable assumption that an &-operation is continuous (i.e., that gradual change in t(A) and t(B) must lead to a gradual change in t(A&B)), uniquely determines min as an &-operation. Likewise, max is the only continuous V-operation. These results are in good accordance with the experimental analysis of 'and' and 'or' in human beliefs.

Kreinovich, Vladik↗

NASA NTRS2010

NextGen Future Safety Assessment Game

The successful implementation of the next generation infrastructure systems requires solid understanding of their technical, social, political and economic aspects along with their interactions. The lack of historical data that relate to the long-term planning of complex systems introduces unique challenges for decision makers and involved stakeholders which in turn result in unsustainable systems. Also, the need to understand the infrastructure at the societal level and capture the interaction between multiple stakeholders becomes important. This paper proposes a methodology in order to develop a holistic approach aiming to provide an alternative subject-matter expert (SME) elicitation and data collection method for future sociotechnical systems. The methodology is adapted to Next Generation Air Transportation System (NextGen) decision making environment in order to demonstrate the benefits of this holistic approach.

Ancel, Ersin↗

NASA NTRS2011

NextGen Future Safety Assessment Game

Ancel, Ersin↗

NASA NTRSDate not supplied

Ensuring Safe Decision-Making on the Moon and Mars: Cognitive Performance Assessment for Exploration Class Mission EVA

Extravehicular activity (EVA) is one of the most dangerous and cognitively demanding actions that astronauts can execute, and the cognitive demands associated with future partial gravity EVAs on the Moon and Mars are expected to be higher compared to microgravity EVAs currently conducted from the International Space Station. Decrements in cognitive performance present an important risk to crew safety during exploration mission class EVA. Yet there is currently insufficient data to characterize cognitive performance prior to, during, and following EVA. Furthermore, it is still unclear which cognitive domains are most important for conducting mission critical decisions with crew safety implications. To address this gap, we conducted a cognitive task analysis (CTA) of EVA to characterize the procedures, the cognitive demands required, and the critical safety decisions associated with decrements in cognitive performance. We conducted a cognitive task analysis with 15 astronauts and subject matter experts in EVA operations and research. Interviews focused on surface exploration EVA and elicited feedback from experts on the cognitive skills required for specific EVA tasks, including cognitive strategies, critical cues, and decision-making strategies. A cognitive demands table was assembled to consolidate and synthesize the information from all interviews. The information from this cognitive task analysis informs ongoing exploration EVA for Moon to Mars. This work identifies the specific cognitive challenges that astronauts are likely to encounter during surface exploration EVA, and provides the foundation for: (1) prioritized and targeted cognitive performance measurement and functional performance tests, (2) EVA simulation design at varying levels of cognitive workload, and (3) the development of training and other technologies that can improve safe decision-making and inform EVA planning on future spaceflight missions to the Moon and Mars.

Steven R Anderson↗