Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Judgment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Empirical Analysis of the Subjective Impressions and Objective Measures of Domain Scientists’ Visual Analytic Judgments

Scientists often use specific data analysis and presentation methods familiar within their domain. But does high familiarity drive better analytical judgment? This question is especially relevant when familiar methods themselves can have shortcomings: many visualizations used conventionally for scientific data analysis and presentation do not follow established best practices. This necessitates new methods that might be unfamiliar yet prove to be more effective. But there is little empirical understanding of the relationships between scientists’ subjective impressions about familiar and unfamiliar visualizations and objective measures of their visual analytic judgments. To address this gap and to study these factors, we focus on visualizations used for comparison of climate model performance. We report on a comprehensive survey-based user study with 47 climate scientists and present an analysis of : i) relationships among scientists’ familiarity, their perceived lev- els of comfort, confidence, accuracy, and objective measures of accuracy, and ii) relationships among domain experience, visualization familiarity, and post-study preference.

Dasgupta, Aritra↗
DOE OSTI2025

Confidence-weighted integration of human and machine judgments for superior decision-making

Large language models (LLMs) can surpass humans in certain forecasting tasks. What role does this leave for humans in the overall decision process? One possibility is that humans, despite performing worse than LLMs, can still add value when teamed with them. A human and machine team can surpass each individual teammate when team members’ confidence is well calibrated and team members diverge in which tasks they find difficult (i.e., calibration and diversity are needed). We simplified and extended a Bayesian approach to combining judgments using a logistic regression framework that integrates confidence-weighted judgments for any number of team members. Using this straightforward method, we demonstrated its effectiveness in both image classification and neuroscience forecasting tasks. Combining human judgments with one or more machines consistently improved overall team performance. Our hope is that this simple and effective strategy for integrating the judgments of humans and machines will lead to productive collaborations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Maximum likelihood estimation of difference scaling functions for suprathreshold judgments

Maximum likelihood estimation (MLE) has been used to produce perceptual scales from binary judgments of triads and quadruples. This method relies on Thurstone’s theory of a stochastic perceptual process where the perceived difference of two stimuli is the difference in their perceived strengths. It is possible that the perception of a suprathreshold difference is overestimated when adding smaller differences, a phenomenon referred to as diminishing returns. The current approach to construct a perceptual scale using MLE does not account for this phenomenon. We present a way to model the perception of differences using MLE and Thurstone’s theory, adapted to allow the possibility of diminishing returns. This method is validated using Monte Carlo simulated responses to experimental triads and can correctly model diminishing returns, the absence of diminishing returns, and the opposite of diminishing returns both in the cases when a perceptual scale is known and when the true perceived strengths of the stimuli are unknown. Additionally, this method was applied to empirical data sets to determine its feasibility in investigations of perception. Ultimately, it was found that this analysis allows for more accurate modeling of suprathreshold difference judgments, a more complete understanding of the perceptual processes underlying comparisons, and the evaluation of Thurstone’s theory of difference judgments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Legal Judgment Prediction Based on Multiclass Information Fusion

Legal judgment prediction (LJP), as an effective and critical application in legal assistant systems, aims to determine the judgment results according to the information based on the fact determination. In real-world scenarios, to deal with the criminal cases, judges not only take advantage of the fact description, but also consider the external information, such as the basic information of defendant and the court view. However, most existing works take the fact description as the sole input for LJP and ignore the external information. We propose a Transformer-Hierarchical-Attention-Multi-Extra (THME) Network to make full use of the information based on the fact determination. We conduct experiments on a real-world large-scale dataset of criminal cases in the civil law system. Experimental results show that our method outperforms state-of-the-art LJP methods on all judgment prediction tasks.

Zhu, Kongfan↗
DOE OSTI2025

Neural correspondence to spectrum of environmental uncertainty in multiple-cue probability judgment system with time delay

Despite state-of-the-art technologies like artificial intelligence, human judgment is critically essential in cooperative systems, such as the multi-agent system (MAS), which collect information among agents based on multiple-cue judgment. Human agents can prevent impaired situational awareness of automated agents by confirming situations under environmental uncertainty. System error caused by uncertainty can result in an unreliable system environment, and this environment affects the human agent, resulting in non-optimal decision-making in MAS. Thus, it is necessary to know how human behavior is changed to capture system reliability under uncertainty. Another issue affecting MAS is time delay, which can delay agent information transfer, resulting in low performance and instability. However, it is difficult to find studies on the influence of time delay on human agents. This study is about understanding the human decision-making process under a specific system reliability environment by uncertainty with time delay. We used concepts of expected and unexpected uncertainty to implement reliability of the system usage environment with three types of time delay conditions: no time delay, regular time delay, and irregular time delay conditions. We used electroencephalogram (EEG) for human cognitive neural mechanisms in multiple-cue judgment systems to understand human decision-making. In the reliability of system usage environment, the unreliable system environment significantly creates less memory load by less utilization of system rules for decision-making. In terms of time delay, delayed information delivery does not significantly affect memory load for decision-making.

cognitive process↗
DOE OSTI2017

2005 ACGIH Lifting TLV: Employee-Friendly Presentation and Guidance for Professional Judgment

The American Council of Governmental Industrial Hygienists (ACGIH) Lifting Threshold Limit Values (TLVs) provide a tool to reduce incidence of low back and shoulder injuries. However, application of the TLV is too complicated for floor-level workers and relies on professional judgment to assess commonly encountered tasks. This paper presents an Employee-Friendly Simplified Format of the TLV that has been adapted from Table 1 of the Lifting TLV presented in the 2005 TLVs and BEIs Based on the Documentation of the Threshold Limit Values for Chemical Substances and Physical Agents & Biological Exposure Indices. This simplified format can be employed by floor-level workers to self-assess lifting tasks. The Ergonomics Project Team also provides research-based guidance for applying professional judgment consistent with standard industry practice: Extended Work Shifts – Reduce weight by 20% for shifts lasting 8 to 12 hours; Constrained Lower Body Posture – Reduce weight by 25% when lifting in such postures; Infrequently Performed Lifts – Lift up to 15 lbs. ≤3 lifts per hour within the zones marked “No safe limit for repetitive lifting” in the TLVs Table 1; Asymmetry beyond 30° – Reduce weight by 10 lbs. for lifts with up to 60° asymmetry from sagittal plane.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Evaluating Climate Models’ Cloud Feedbacks Against Expert Judgment

Abstract The persistent and growing spread in effective climate sensitivity (ECS) across global climate models necessitates rigorous evaluation of their cloud feedbacks. Here we evaluate several cloud feedback components simulated in 19 climate models against benchmark values determined via an expert synthesis of observational, theoretical, and high‐resolution modeling studies. We find that models with smallest feedback errors relative to these benchmark values generally have moderate total cloud feedbacks (0.4–0.6 W m −2 K −1 ) and ECS (3–4 K). Those with largest errors generally have total cloud feedback and ECS values that are too large or too small. Models tend to achieve large positive total cloud feedbacks by having several cloud feedback components that are systematically biased high rather than by having a single anomalously large component, and vice versa. In general, better simulation of mean‐state cloud properties leads to stronger but not necessarily better cloud feedbacks. The Python code base provided herein could be applied to developmental versions of models to assess cloud feedbacks and cloud errors and place them in the context of other models and of expert judgment in real‐time during model development.

58 GEOSCIENCES↗
DOE OSTI2026

Criteria for Retention of 3013 S1 Containers Based on Relative Risk and Expert Judgment

An evaluation was performed to assess the suitability of thirty-three 3013 containers proposed for retention. These containers have moisture levels greater than 0.08 wt.% – the S1 population. The remainder of the S1 population stored at SRS will be down blended and disposed of by the end of 2028. Based on field surveillance and shelf-life data available to date as well as informed technical judgment, no container is currently expected to fail in its 50-year storage period. However, corrosion risk varies across the S1 population. Relative risks were evaluated using predicted Consensus Scores and their 95% Upper Prediction Limits (UPLs). The predicted values are based on a statistical model of Consensus Score as a function of moisture, chloride, and whether the packaged material was electrorefining scrap packaged at Hanford. Consensus Score has been shown to be a useful indicator of corrosion potential, and the UPL captures uncertainty in the model predictions, providing a conservative indicator of corrosion potential. Using UPLs to determine relative risks, together with expert review, three containers were identified as not suitable for retention, and the remainder were determined to be suitable.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2019

Destructive Behavior, Judgment, and Economic Decision-Making Under Thermal Stress

Accumulating evidence indicates that environmental temperature substantially affects economic outcomes and violence, but the reasons for this linkage are not well understood. We systematically evaluate the effect of thermal stress on multiple dimensions of economic decisionmaking, judgment, and destructive behavior with 2,000 participants in Kenya and the US who were randomly assigned to different temperatures in a laboratory. We find that most dimensions of decision-making are unaffected by temperature. However, heat causes individuals to voluntarily destroy other participants’ assets, with more pronounced effects during a period of heightened political conflict in Kenya.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis

Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ~85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Additionally, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Quantitative Exposure Assessment Professional Judgment Validation Study

Professional judgement is a key element of an exposure assessment program. Industrial hygienists frequently rely on it to make judgements about the acceptability of occupational exposures to chemical, biological, and physical agents. Often this must be done with little or no quantitative sampling data. This is especially true in the case of research and development, where activities may be short duration and non-routine. In this situation there are very limited opportunities for sampling, and therefore, professional judgement becomes a major tool industrial hygienists rely upon. One of the limitations of professional judgement is that exposures may be misclassified. That is, an exposure deemed to be unacceptable may in fact be acceptable and unnecessary effort and expense may be incurred as a result. On the other hand, an exposure may be judged acceptable when it is in fact unacceptable, resulting in an overexposure and potential injury or illness. It is therefore crucial that professional judgement be validated. In this study we evaluated 106 acceptable exposure determinations with retrospective quantitative exposure monitoring to validate the performance of Sandia National Laboratory industrial hygienists at performing exposure determinations with professional judgement.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Robust verification analysis

We introduce a new methodology for inferring the accuracy of computational simulations through the practice of solution verification. We demonstrate this methodology on examples from computational heat transfer, fluid dynamics and radiation transport. Our methodology is suited to both well- and ill-behaved sequences of simulations. Our approach to the analysis of these sequences of simulations incorporates expert judgment into the process directly via a flexible optimization framework, and the application of robust statistics. The expert judgment is systematically applied as constraints to the analysis, and together with the robust statistics guards against over-emphasis on anomalous analysis results. We have named our methodology Robust Verification. Our methodology is based on utilizing multiple constrained optimization problems to solve the verification model in a manner that varies the analysis' underlying assumptions. Constraints applied in the analysis can include expert judgment regarding convergence rates (bounds and expectations) as well as bounding values for physical quantities (e.g., positivity of energy or density). This approach then produces a number of error models, which are then analyzed through robust statistical techniques (median instead of mean statistics). This provides self-contained, data and expert informed error estimation including uncertainties for both the solution itself and order of convergence. Our method produces high quality results for the well-behaved cases relatively consistent with existing practice. The methodology can also produce reliable results for ill-behaved circumstances predicated on appropriate expert judgment. We demonstrate the method and compare the results with standard approaches used for both code and solution verification on well-behaved and ill-behaved simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗