Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DECISION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Water Governance Process Assessment: Evaluating the Link between Decision Making Processes and Outcomes in the Columbia River Basin

Academics and practitioners agree that in water governance, the quality of a decision making process should influence the quality of the outcome and the degree to which it is accepted by interested parties. However, finding a feasible way to evaluate and then improve the quality of a decision making process has proven elusive. Systematically collecting evidence of a link between process and outcome is also challenging. In my dissertation, I developed a synthesis framework for evaluating and improving water governance decision making to address these two challenges. The synthesis framework, which I call the Water Governance Process Assessment (Water GPA), draws upon 22 existing frameworks rooted in resilience, adaptive governance, and good governance. From these frameworks, I identified and provided a way to evaluate four characteristics critical to good water governance decision making processes: 1) accountability, 2) inclusivity, and 3) information, and 4) context. I applied the Water GPA framework to the recent reviews of the Columbia River Treaty by the United States and Canada. I collected data for the case studies through semi-structured interviews and surveys of process participants from the federal agencies, Tribal and First Nations, state/provincial governments, local governments, stakeholder interests, and citizens. I coded and analyzed the interviews using the qualitative analysis software QSR NVivo and the characteristics identified in the Water GPA framework. I used the two case study applications to demonstrate how to use the framework. I identified what aspects of the four process categories served as barriers and building blocks for good water governance in each water governance process. I also gleaned lessons learned and recommendations including some for determining process leadership, ensuring meaningful engagement and inclusivity, addressing sovereignty issues, setting decision criteria, sharing decision authority, allocating resources in future processes in the US and BC portions of the Columbia River Basin and similar basins. In my case studies, I also investigated which characteristics of the water governance decision making process influenced the direct outcomes of those processes (the decisions) as well as other non-target outcomes (such as trust, co-produced science, new coalitions, etc.). All four characteristics outlined in the Water Governance Process Assessment (Water GPA), played some role in the development of the content and/or support of the two case study decision documents (US Regional Recommendation and BC Provincial Decision). Generally, when the characteristic of the process was done well, it improved the legitimacy and acceptance of the decision. At the same time performing poorly in one area did not necessarily torpedo the process or decision. In both case studies, it appears that the interplay between the process characteristics (that is where two or more characteristics converge) had a greater influence over the decision. Further work is needed to clearly identify what characteristics of a process are most influential in different situations. The Water GPA is one useful tool for this effort.

13 HYDRO ENERGY↗
DOE OSTI2016

Energy-Water Nexus: Balancing the Tradeoffs between Two-Level Decision Makers

Energy-water nexus has substantially increased importance in the recent years. Synergistic approaches based on systems-analysis and mathematical models are critical for helping decision makers better understand the interrelationships and tradeoffs between energy and water. In energywater nexus management, various decision makers with different goals and preferences, which are often conflicting, are involved. These decision makers may have different controlling power over the management objectives and the decisions. They make decisions sequentially from the upper level to the lower level, challenging decision making in energy-water nexus. In order to address such planning issues, a bi-level decision model is developed, which improves upon the existing studies by integration of bi-level programming into energy-water nexus management. The developed model represents a methodological contribution to the challenge of sequential decisionmaking in energy-water nexus through provision of an integrated modeling framework/tool. An interactive fuzzy optimization methodology is introduced to seek a satisfactory solution to meet the overall satisfaction of the two-level decision makers. The tradeoffs between the two-level decision makers in energy-water nexus management are effectively addressed and quantified. Application of the proposed model to a synthetic example problem has demonstrated its applicability in practical energy-water nexus management. Optimal solutions for electricity generation, fuel supply, water supply including groundwater, surface water and recycled water, capacity expansion of the power plants, and GHG emission control are generated. In conclusion, these analyses are capable of helping decision makers or stakeholders adjust their tolerances to make informed decisions to achieve the overall satisfaction of energy-water nexus management where bi-level sequential decision making process is involved.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

A conceptual framework for negotiating public involvement in municipal waste management decision-making in the UK

Highlights: • A review of public involvement in waste management decision-making was undertaken. • Public involvement seen as necessary to create stronger foundations for decisions. • The mode and level of public involvement explored in different decision contexts. • Public involvement can be negotiated based on the technological and social context. • Early public involvement favoured where issues are contentious or uncertain. - Abstract: The technical expertise that politicians relied on in the past to produce cost-effective and environmentally sound solutions no longer provides sufficient justification to approve waste facilities. Local authorities need to find more effective ways to involve stakeholders and communities in decision-making since public acceptance of municipal waste facilities is integral to delivering effective waste strategies. This paper presents findings from a research project that explored attitudes towards greater levels of public involvement in UK waste management decision-making. The study addressed questions of perception, interests, the decision context, the means of engagement and the necessary resources and capacity for adopting a participatory decision process. Adopting a mixed methods approach, the research produced an empirical framework for negotiating the mode and level of public involvement in waste management decision-making. The framework captures and builds on theories of public involvement and the experiences of practitioners, and offers guidance for integrating analysis and deliberation with public groups in different waste management decision contexts. Principles in the framework operate on the premise that the decision about ‘more’ and ‘better’ forms of public involvement can be negotiated, based on the nature of the waste problem and wider social context of decision-making. The collection of opinions from the wide range of stakeholders involved in the study has produced new insights for the design of public engagement processes that are context-dependent and ‘fit-for-purpose’; these suggest a need for greater inclusivity in the case of contentious technologies and high levels of uncertainty regarding decision outcomes.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2015

A paradigm shift - the key to optimal, defensible and transparent waste management, disposal and remediation decisions - 15236

Current methods for structuring nuclear waste management, disposal, and remediation decisions are subject to three major limitations. First, they are based on a series of extraordinarily conservative assumptions. The dominant way of thinking is that, for example, conservative models yield better, safer decisions. Second, they lack methods to meaningfully address and incorporate the 'human dimension'. The dominant way of thinking is that the current approach is sufficient to bring together the values and interests of both internal and external stakeholders in making a decision. And third, they lack a formal method through which both internal and external stakeholders are encouraged collectively to 'think through' complex problems that contain multiple competing objectives so that wise, socially acceptable, and scientifically-defensible decisions are made. To overcome these limitations, a paradigm shift is needed towards a holistic, integrated approach to structure nuclear waste management, disposal and remediation decisions. The first requirement of this paradigm shift is the realization and recognition that the problems faced are decision problems, and that there is a formal theory of decision analysis that can, and should, be used to support finding the best solutions. Decision analysis, very simply, is formalized common sense. The reason it is needed for complex problems is that there are too many moving parts (variables) for humans to manage with out the formalism. Decision analysis leads to addressing problem solving using a combination of a science-based model and a costs/values model. Expectation ('realism') and uncertainty are addressed in the science-based models, whereas stakeholder values, preferences, and desires are addressed when modeling costs and value judgments. Science-based modeling tools have been developed, improved, and used since the current regulatory setting was introduced about 30 years ago. However, modeling of costs and value judgments has not often been addressed using any formal approach, despite the openings that exist in regulations (e.g., CERCLA's modifying criteria and the concept of 'as low as reasonably achievable' referenced in DOE Orders). A discussion of the need and rationale for this paradigm shift is presented, followed by a description of the technical basics of this approach to decision modeling, and ending with an explanation of the steps that are taken in this approach to structured decision-making. A software framework program called Guided Interactive Statistical Decision Tools (GiSdT - pronounced 'gist') is introduced that supports this paradigm shift. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2015

A paradigm shift: an introduction to structured decision making for sustainable waste management and remediation - 15649

transport of contaminants and human health risk assessment. Most often these modeling activities are carried out in a conservative fashion, usually beyond even the conservatism that is built into the associated regulations and guidance. Decisions are aimed primarily at compliance with regulatory objectives, rather than optimization. This leads to unnecessarily costly remediation and under-utilization of the nation's relatively few radioactive waste disposal facilities. Risk assessment decisions under CERCLA and RCRA might seem to perform better, but if anything this is because the focus is on shorter-term decision-making because long term fate and transport modeling is not considered necessary in most cases. However, some more complex contamination problems suffer from the same basic issues as radioactive waste disposal in terms of the need for complex fate and transport modeling and subsequent risk assessment. At least CERCLA addresses optimization in the feasibility study, but it does not address how to perform that optimization in the face of uncertainty or stakeholders who have different competing objectives. Other environmental issues that fall outside the realm of DOE Orders, NRC regulations, and regulations such as CERCLA and RCRA, have found a different path towards effective decision-making. This includes land reuse and watershed management decisions, for which there is a focus on sustainability and on stakeholder involvement throughout the decision making process. EPA is using 'structured decision analysis' to achieve the primary goal of maximizing societal welfare. These same tools and approaches can be brought into environmental and waste management decisions that need to be made by DOE. Structured decision analysis provides a formal process for capturing not only the science side of the problem, but also the costs and value judgments of the stakeholders, to reach an optimal solution. But this approach requires a paradigm shift within the DOE and NRC environmental and waste management programs. That is, it requires a willingness to engage a new approach that will provide a path towards effective optimization in the decision-making process for the complex environmental problems faced by DOE and NRC. Because of the current nature of problem solving for these types of problems, which involves many layers of conservatism, this structured decision analysis path towards optimization will also realize substantial cost savings for these programs, while maintaining or improving defensibility and transparency in the decision making process. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2018

IAEA Initiatives that Integrate Stakeholders, Models and Decision Making - 18646

The International Atomic Energy Agency (IAEA) has initiated an effort to provide stakeholder support services as part of Phase II of the Constraints to Implementing Decommissioning and Environmental Remediation (CIDER II) programs. The intent of this sub-group of CIDER II is to provide members states support in areas of stakeholder communication, engagement and participation, with the goal of building trust in decommissioning and environmental remediation (D and ER) decisions for radioactive contamination and facilities. IAEA has also initiated a follow-on effort to address Modeling and Data for Radiological Impact Assessments (MODARIA II), which is aimed at enhancing the capabilities of Member States to simulate radionuclide transfer in the environment, and to ensure appropriate levels of protection from ionizing radiation. Working Group 1 of MODARIA II is focused on the integration of modeling and decision analysis. These two groups have complementary and supporting objectives, which provides an opportunity for collaboration and mutual benefit. The technical integrator is the approach of structured decision making, which follows a five-step process that combines stakeholder values with models through a technically defensible decision analysis. Briefly, the steps are: 1. Provide the context and decision context for the problem; 2. Obtain costs, values and concerns from stakeholders, to develop objectives that need to be achieved (e.g., improve human health, minimize costs, increase jobs); 3. Identify options that might achieve the objectives (e.g., cleanup, institutional controls, education); 4. Develop probabilistic models that connect options to objectives so that the objectives can be evaluated collectively (e.g., a fate and transport and risk model to address the objective of reducing human health risk, a cost model to address costs of taking action); and, 5. Model evaluation, including sensitivity analysis, to determine if an action can be taken with sufficient certainty, or if collection of more data is needed to decrease uncertainty before a decision is made. This approach also supports adaptive management as new data are collected. This approach clarifies the roles of stakeholders and subject matter experts in the decision-making process, where stakeholders specify values, and subject matter experts focus on the models that are needed to evaluate the options. It also provides a technically defensible, reproducible, transparent and traceable approach to all of the components necessary to properly solve the types of complex decision problems that plague radioactive waste management, decommissioning and environmental remediation. This paper introduces the major challenges to implementing such a broad scale of decision analysis, and approaches to overcoming those challenges, which include not only the technical challenges, but also the need to develop capacity in a range of skills not usually accommodated in environmental modeling and includes the need for the IAEA to address decision making in different countries, and with different cultures, religions, political affiliations, and languages. This paper explores the potential for IAEA to benefit from both the CIDER II and MODARIA II programs, and hence to develop a fully integrated approach to effective decision making for radioactive waste management and environmental remediation. Note that there are many people involved in these efforts at the IAEA, and credit belongs to those people for the progress that has been made to date. Our intent here is simply to introduce these IAEA programs at the WM2018 Symposium to provide our perspective and some awareness of some of the directions in which IAEA appears to be headed regarding stakeholder engagement, environmental and risk modeling, data science, and decision analysis. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2015

Development of a First-of-a-Kind Deterministic Decision-Making Tool for Supervisory Control System

Decision-making is the process of identifying and choosing alternatives where each alternative offers a different approach or path to move from a given state or condition to a desired state or condition. The generation of consistent decisions requires that a structured, coherent process be defined, immediately leading to a decision-making framework. The overall objective of the generalized framework is for it to be adopted into an autonomous decision-making framework and tailored to specific requirements for various applications. In this context, automation is the use of computing resources to make decisions and implement a structured decision-making process with limited or no human intervention. The overriding goal of automation is to replace or supplement human decision makers with reconfigurable decision- making modules that can perform a given set of tasks reliably. Risk-informed decision making requires a probabilistic assessment of the likelihood of success given the status of the plant/systems and component health, and a deterministic assessment between plant operating parameters and reactor protection parameters to prevent unnecessary trips and challenges to plant safety systems. The implementation of the probabilistic portion of the decision-making engine of the proposed supervisory control system was detailed in previous milestone reports. Once the control options are identified and ranked based on the likelihood of success, the supervisory control system transmits the options to the deterministic portion of the platform. The deterministic multi-attribute decision-making framework uses variable sensor data (e.g., outlet temperature) and calculates where it is within the challenge state, its trajectory, and margin within the controllable domain using utility functions to evaluate current and projected plant state space for different control decisions. Metrics to be evaluated include stability, cost, time to complete (action), power level, etc. The integration of deterministic calculations using multi-physics analyses (i.e., neutronics, thermal, and thermal-hydraulics) and probabilistic safety calculations allows for the examination and quantification of margin recovery strategies. This also provides validation of the control options identified from the probabilistic assessment. Thus, the thermal-hydraulics analyses are used to validate the control options identified from the probabilistic assessment. Future work includes evaluating other possible metrics and computational efficiencies.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

Stakeholder Engaged Structured Decision Making to Improve Performance and Reduce Costs While Protecting Human Health and the Environment - 18645

The objective of the Best Practice is to facilitate a paradigm shift when it comes to approaches to the decision making process for nuclear waste management, disposal, and remediation decisions. The stakeholder-engaged structured decision making (SDM) paradigm shift provides a transparent framework for developing optimal solutions to complex problems (Keeney, 1992, Gregory et al, 2012). This is a deliberative-analytical process. The deliberative side centers on understanding stakeholder concerns and values, and developing objectives while identifying options that may achieve those objectives. SDM is implemented with the aid of computer tools that are aimed specifically at capturing these deliberative aspects of a decision analysis. SDM captures this information in a structured format that allows access to the values, objectives, and options that have been identified to help. Through this approach we achieve transparency, traceability, and reproducibility. The goal of SDM at this stage is to provide a formal structure for capturing and processing deliberative information. The deliberative part sets the stage for the analytical part of SDM. A variety of Subject matter experts (SMEs) are engaged in how to evaluate the options through the objectives. Objectives often include minimizing human health risk and minimizing cost, but in the full scope of a sustainability-based approach to decision making, it can include objectives related to economic, environmental and social issues. Subject matter experts might provide other options for achieving the objectives, but their primary role is to evaluate/model the options to the endpoint defined by the objectives or in other words perform a consequence analysis. The structure of SDM makes it clear exactly what is needed from the SMEs, because the options are identified, and the endpoint (objectives) are defined - evaluations and/or models are needed to connect options to objectives. This consequence analysis completes the evaluation, and directly addresses which of the options is the best option. The same SDM approach to finding the best options can also be used for prioritization and resource allocation. Because all of the information is captured in a formal system with the help of computer tools (a software framework for implementing SDM), the decision models that are developed for an application can be fully evaluated numerically and for the insights gained from using a formal process for managing the multiple factors that need to be considered for complex decisions. This includes uncertainty and sensitivity analysis that can be used to guide the need for further data/information collection if the optimal decision is not adequately supported - that is, if there is insufficient confidence in the decision. Also, this feeds directly into adaptive management therefore if more data/information are collected then the efficacy of the current decision can be evaluated, potentially leading to a change in decisions if warranted. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

A Typology of Decision-Making Tasks for Visualization

Despite decision-making being a vital goal of data visualization, little work has been done to differentiate decision-making tasks within the field. While visualization task taxonomies and typologies exist, they often focus on more granular analytical tasks that are too low-level to describe large complex decisions, which can make it difficult to reason about and design decision-support tools. In this paper, we contribute a typology of decision-making tasks that were iteratively refined from a list of design goals distilled from a literature review. Our typology is concise and consists of only three tasks: CHOOSE, ACTIVATE, and CREATE. Although decision types originating in other disciplines exist, we provide definitions for these tasks that are suitable for the visualization community. Our proposed typology offers two benefits. First, the ability to compose and hierarchically organize the tasks enables flexible and clear descriptions of decisions with varying levels of complexities. Second, the typology encourages productive discourse between visualization designers and domain experts by abstracting the intricacies of data, thereby promoting clarity and rigorous analysis of decision-making processes. We demonstrate the benefits of our typology through four case studies, and present an evaluation of the typology from semi-structured interviews with experienced members of the visualization community who have contributed to developing or publishing decision support systems for domain experts. Our interviewees used our typology to delineate the decision-making processes supported by their systems, demonstrating its descriptive capacity and effectiveness. Finally, we present preliminary findings on the usefulness of our typology for visualization design.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Charging Decision Making for Automated Electric Vehicles

Abstract: Overview/Significance With the emerging of autonomous driving and its application in electric vehicles, the charging decision making should be taken over by the vehicles. Autonomous electric vehicles will require the capability to make charging decisions according to the battery energy state, the travel demand and also the available charging infrastructure. The introduction of autonomous driving technology would remove the challenge of co-locating charging infrastructure with driver destinations and presents a driver-free method for EVs to reach nearby charging stations. This will significantly change the charging behavior of electric vehicles. EV driver will no longer need to be present at charging stations for charging actions. Automated EVs can drive to nearby charging stations to perform charging actions by themselves when necessary. Meanwhile, connected vehicles technology makes electric vehicles have the capability to sense and obtain pertinent information from nearby charging station networks and then calculate the corresponding costs and availability for charging. All these promising capabilities bring new opportunities to construct an automatic charging decision making system and improve energy efficiency of charging actions for automatic electric vehicles. Abstract: Methodology This research introduces optimization tools and case studies on charging decision making for automated electric vehicles under personal usage scenario. The designed optimization tools aim to provide charging strategies, i.e. the optimal choice of charging station outside home and the amount of charged energy, by considering constraints from personal daily itineraries and existing charging infrastructure. Abundant real world EV data is utilized to construct stochastic energy consumption prediction model under realistic driving conditions. Advanced dynamic programming technology is utilized to model and solve the potential multi-stage decision making process by considering realistic uncertainties of EV energy cost and dynamics of itinerary information. Large-scale simulation studies using Chicago itinerary data are demonstrated to illustrate energy benefits from automatic charging decision making for future autonomous EVs. Abstract: Results/Conclusions Optimal automatic charging decision making and its energy benefits have been studied for personal automated electric vehicles. A data-driven method based on large amount of real-world data and the corresponding real-time update algorithm have been proposed to construct multi-channel stochastic energy consumption prediction model. Based on the energy cost prediction, dynamic multi-stage charging decision-making models are established for optimal strategies during a daily itinerary. Derived charging strategies can minimize the monetary and energy cost of charging actions for autonomous vehicles. Simulations and case studies show the potential ability of autonomous electric vehicles to reduce the range anxiety and charging infrastructure dependency. Automatic charging decision making can mitigate the pain from electric vehicle charging necessity outside home. In conclusion, automated electric vehicles equipped with an appropriate automatic charging decision making system will help EV driver to relieve from heavy time cost of recharging, reduce the energy cost of performing charging actions and improve the energy efficiency in future electrified transportation system.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2018

A Decentralized Bi-Level Fuzzy Two-Stage Decision Model for Flood Management

Flood, as a serious worldwide environment problem, can lead to detrimental economic losses and fatalities. Effective flood control is desired to mitigate the adverse impacts of flooding and the associated flood risk through development of cost-effective and efficient flood management decisions and policies. A bi-level fuzzy two-stage stochastic programming model, named BIFS model is developed in this study to provide decision support for economic analysis of flood management. The BIFS model is capable of not only addressing the sequential decision making issue involving the two-level decision makers, but also correcting the pre-regulated flood management decisions before the occurrence of a flood event in the two-stage environment. The probabilistic and non-probabilistic uncertainties expressed as probability density functions and fuzzy sets are quantitatively analyzed. The overall satisfaction solution is obtained for meeting the goals of the two-level decision makers by compromising, reflecting the tradeoffs among various decision makers in the two decision-making levels. The results of application of the BIFS model to a representative case study indicate informed decision strategies for flood management. Tradeoffs between economic objectives and uncertainty-averse attitudes of decision makers are quantified.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2010

Conformance Testing: Measurement Decision Rules

The goal of a Quality Management System (QMS) as specified in ISO 9001 and AS9100 is to provide assurance to the customer that end products meet specifications. Measuring devices, often called measuring and test equipment (MTE), are used to provide the evidence of product conformity to specified requirements. Unfortunately, processes that employ MTE can become a weak link to the overall QMS if proper attention is not given to the measurement process design, capability, and implementation. Documented "decision rules" establish the requirements to ensure measurement processes provide the measurement data that supports the needs of the QMS. Measurement data are used to make the decisions that impact all areas of technology. Whether measurements support research, design, production, or maintenance, ensuring the data supports the decision is crucial. Measurement data quality can be critical to the resulting consequences of measurement-based decisions. Historically, most industries required simplistic, one-size-fits-all decision rules for measurements. One-size-fits-all rules in some cases are not rigorous enough to provide adequate measurement results, while in other cases are overly conservative and too costly to implement. Ideally, decision rules should be rigorous enough to match the criticality of the parameter being measured, while being flexible enough to be cost effective. The goal of a decision rule is to ensure that measurement processes provide data with a sufficient level of quality to support the decisions being made - no more, no less. This paper discusses the basic concepts of providing measurement-based evidence that end products meet specifications. Although relevant to all measurement-based conformance tests, the target audience is the MTE end-user, which is anyone using MTE other than calibration service providers. Topics include measurement fundamentals, the associated decision risks, verifying conformance to specifications, and basic measurement decisions rules.

Mimbs, Scott M.↗
NASA NTRS2014

Analysis of Airport Ground Delay Program Decisions Using Data Mining Techniques

Air traffic service providers have to make decisions regarding changes to air traffic flow in the event of major weather disturbances and traffic congestions to maintain safety of the system. The behavior of the air traffic management system will be more predictable if consistent decisions are made under similar traffic and weather conditions. Consistency of deciding on control action depends on the weather and traffic conditions as well as accuracy in predicting these conditions. Weather parameters (defined in terms of forecast and actual weather and traffic conditions) on different days can be used to categorize days into days with little decision consistency, days with moderate decision consistency and days with high decision consistency. Four years of traffic, weather and ground delay program decisions data at major airports in the United States are used in the analysis. This paper examines performance of different data mining methods in the three regions of decision consistency. Not surprisingly, data mining methods have the best performance in the region of most decision consistency and have the poorest performance in the region of little decision consistency. In applications where data mining methods have differing performance in differing regions, it would be more useful to characterize the region specific performance instead of characterizing performance by a single parameter. Finally, the results show no significant variation in the performance of different data mining methods for this particular problem. The fact that different mining methods show no significant variation also provides further confidence in the results of data mining methods. Work in this abstract discusses initial results. This paper describes the results in terms of both forecast and actual environmental conditions and discusses how prediction errors impact decision consistency.

Kulkarni, Deepak↗
NASA NTRSDate not supplied

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗
DOE OSTI2017

Decision Analyses Related to Groundwater Remediation - 17051

Traditional decision analyses of groundwater remediation scenarios frequently fail because the probability of adverse, unanticipated events occurring is often high. This is driven by many factors including that (1) the models developed to represent the flow and transport in contaminated aquifers are typically simpler than reality and do not account for all the important governing processes, (2) probability distributions are assigned to critical performance parameters (such as model inputs or characteristics of the engineered remediation systems) even though some of these parameters might be unknown or not be very well constrained. However, there is a novel decision analysis methodology and a computational tool based on Bayesian Information-Gap Decision Theory (BIG-DT) that are designed to mitigate the shortcomings of the models and probabilistic decision analyses by leveraging a non-probabilistic decision theory -- information-gap decision theory. The new decision analysis methodology considers possible models that have not been explicitly enumerated and does not require us to commit to a particular probability distribution for model and remediation-design parameters. Both the set of possible models and the set of possible probability distributions grow as the degree of uncertainty increases. The fundamental question that BIG-DT asks is 'How large can these sets be before a particular decision results in an undesirable outcome?'. The decision that allows these sets to be the largest is considered to be the best option. In this way, BIG-DT enables robust decision-support for groundwater remediation problems. Here, we apply BIG-DT to a representative groundwater remediation scenario where different options for hydraulic containment and pump and treat are being considered. BIG-DT requires many model runs and high-performance computing resources are needed when complex models are used. We demonstrate the BIG-DT analyses on a series of synthetic problems that are designed to be consistent with real-world problems such as Los Alamos National Laboratory (LANL) contamination sites. We also discuss a general framework for how the BIG-DT analyses will be carried out at the chromium groundwater contamination site at LANL. BIG-DT is implemented in Julia (a high-level, high-performance dynamic programming language for technical computing) and is part of the MADS framework (http://mads.lanl.gov and http://madsjulia.github.io/Mads.jl). (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

Climate-aware decision-making: lessons for electric grid infrastructure planning and operations

Abstract Climate change poses significant risks to large-scale infrastructure systems and brings considerable uncertainties that challenge historical planning approaches. Here we focus on how climate awareness might be better incorporated into planning and decision-making in the electric power sector. To do so, we consider lessons from decision science literature where researchers have specifically focused on how to make better decisions under uncertainty. We perform a three-part review: of decision science literature on best practices for decision-making under uncertainty, of industry practice documents to understand how new uncertainties may affect the types of decisions electric utilities are making today, and of literature on electricity sector planning under climate change to assess how lessons from decision science might fit into sector-specific analyses. We discuss how characterizations of uncertainty from the decision science literature might guide approaches in the electricity sector to appropriately capture climate-related uncertainties. We also distill three key ideas from the decision science literature that can be incorporated into electricity sector planning to manage these new uncertainties: robustness, adaptive planning, and multi-stakeholder engagement. We offer example recommendations for how these key ideas might be incorporated into one essential type of planning activity, capacity expansion.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

The modeling of risk perception in the use of structural health monitoring information for optimal maintenance decisions

This paper proposes an approach to select a maintenance strategy from a predefined set of choices considering the decision maker’s behavioral risk profile. It is assumed that the damage state is characterized by a continuous state parameter probabilistically inferred from observable sensor data. This work applies an engineering application of consequence-based decision-making incorporating the acceptable risk intensity of the decision-maker, i.e., the decision-maker’s (individual or an organization) valuation of the outcome of a decision, using a risk profile model. The utility of a decision-maker is subjective, and this paper considers the fact that different decision-makers mentally assign a different importance factor (the utility) to the seriousness or urgency to take necessary actions with the increasing intensity of structural damage. The approach herein incorporates a layer of human psychology on selecting appropriate maintenance strategies that not only depend on the posterior distribution of unmeasurable damage state but also consider the behavioral risk profile of the decision-maker. Further, the collective decision-making of an organization consisting of many individuals is also investigated. The approach is exemplified in a case study involving life cycle monitoring of a miter gate, part of a lock system enabling navigation of inland waterways.

42 ENGINEERING↗
DOE OSTI2021

Analytical Framework for Homeowner-Focused Computational Cost-Benefit Mitigation Decision Making

As a result of increasing windstorm losses in the United States over the past 50 years, a variety of residential wind hazard mitigation strategies have been recommended. Wind hazard mitigation undoubtedly reduces windstorm losses; however, little customized information is available to support the economic decision-making process for individual stakeholders, especially at the consumer level. Therefore, there is a need for decision support tools to calculate, evaluate, and present the economic benefits of mitigation over variable decision-making time horizons for unique building characteristics in specific geographical locations. This paper presents a cost-benefit computational decision-making recommendation framework that evaluates and presents the cost effectiveness of mitigation, customized based on location, years of interest, and building characteristics. The decision-making framework consists of two cores. The user-independent core and user-dependent core which consists of eight steps, divided into four subprocesses: (1) data acquisition, (2) data evaluation, (3) cost-benefit calculation, and (4) results comparison and decision making. To demonstrate the framework, a wood-framed single-family home in light suburban terrain in Golden Meadow, Louisiana, is used as a case study, considering retrofit mitigation with a $15,000 budget and a decision-making time horizon of 30 years. The case study provides six mitigation options, with cost ranging between approximately $2,000 and $15,000 and net benefit/cost ratios ranging from 2.4 to 13. Once fully implemented in a decision making information tool, the developed mitigation recommendation framework will provide valuable information that can be used by home designers, builders, owners, and other stakeholders to better understand the economic facets of wind hazard mitigation, providing critical information as part of the overall decision-making process.

Orooji, Fatemeh↗