SEARCH · Engineering Papers

Results for “Judgment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Technical Issues in the Comprehensive Nuclear Test Ban Treaty (CTBT) Ratification Debate: A 20 Year Retrospective

Two decades after the U.S. Senate declined in 1999 to give its consent to ratification of the Comprehensive Nuclear Test Ban Treaty (CTBT), the treaty remains in legal and political limbo. Legally, the treaty cannot enter into force without U.S. ratification. However, the Senate’s rejection does not strip the treaty of its force as an element of customary international law or alter the will of other signatories to see it enter into force. Politically, the debate over the virtues and flaws of the treaty remains as strong as ever, with some CTBT advocates pushing energetically for U.S. re-review while some opponents seek ways to “un-sign” the treaty. Meanwhile, despite the Senate’s rejection of the treaty, the United States acts in a manner consistent with its main obligations by maintaining a moratorium on explosive nuclear testing and providing financial support (as well as technical expertise) to the CTBT Organization (CTBTO)—in particular the CTBTO’s International Monitoring System (IMS). The CTBT may well remain in such limbo for a long time to come, unless some catalytic event comes along to re-shape national perceptions and priorities, one way or the other. The treaty’s future, whatever it might be, will be determined in part by how much perspectives might have changed in light of interim developments. Our purpose is to shed light on some of those developments and to assess their relevance. The 1999 ratification debate addressed a large number of issues. Rejection was driven by various policy and technical judgments. The purpose of this paper is to re-examine those technical judgments in light of what has been learned over the following two decades. On some technical matters, there was explicit recognition of extant uncertainties, as well as a conviction and hope that time would dispel them. To be doubly clear, our purpose is not to recommend a way forward on CTBT ratification. The policy judgments of 1999 and 2020 are beyond the scope of this paper. This technical re-examination proceeds as follows. It begins with a short background section to introduce the key technical matters in discussion in the U.S. ratification process. Then it moves systematically through each of those topics with a review of relevant background information, a discussion of key points in debate, an analysis of subsequent experience, and a current assessment. The historical analysis draws on many sources including the primary documents of the Congressional debate, the official article-by-article interpretation by the Department of State, and the statements of key players in the process. We also relied on key technical reports and publications by subject matter experts that have addressed in more detail many of the issues in our focus. These sources include the 2002 and 2012 reports of the National Academy of Sciences as well as publications by senior technical staff of Lawrence Livermore National Laboratory (LLNL) who were actively involved in the ratification debate, including then-Laboratory Director Bruce Tarter. We also conducted several interviews with government officials from the 1990s, past laboratory directors, and other members of the scientific community. But, except where individuals are specifically cited by name, the judgments offered here are our own and should not be attributed to any other individual or to LLNL as such.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2018

Dose–volume histogram evaluation scheme for beam orientations in intensity-modulated radiation therapy

Beam orientations in intensity-modulated radiation therapy (IMRT) is an important, but difficult NP-hard optimization problem. A considerable part of this difficulty is due to the essential prerequisite of defining, appropriately, counter-intuitive criteria for a mathematical plan evaluation that coincides with the clinical judgment of the considered plan. Moreover, the quality of beam directions depends heavily on their intensities. Usually a stochastic selector is utilized for optimizing beam orientations, and the intensities are thus calculated many thousands of times, each time for a different selection of beam directions, resulting in excessive time complexity. Therefore, selecting an appropriate set of beam directions in IMRT is still a time-consuming manual trial-and-error search procedure that depends on intuition and empirical knowledge. To overcome these difficulties, this work utilizes the concept of dose–volume histogram (DVH), which is one of the main recognized quantitative measurement tools used for plan judgment, and presents a DVH evaluation scheme that parallelizes plan evaluation with clinical plan judgment. This scheme is then utilized to optimize the intensities of each individual beam, enabling it to be represented by its DVHs, produced for the different irradiated volumes. Finally, an evolutionary algorithm is employed to search for the best combination of those DVHs reflecting the clinical optimal. The presented methods have been applied to three real clinical examples. In comparison to the standard equally spaced beam plans, improvements were reported in all three cases. In comparison with the greedy algorithm, improvements were reported in those examples requiring more than five beam directions.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

NASA NTRS1977

Judgements of relative noisiness of a supersonic transport and several commercial-service aircraft

Two laboratory experiments were conducted on the relative noisiness of takeoff and landing operations of a supersonic transport and several other aircraft in current commercial service. A total of 96 subjects made noisiness judgments on 120 tape-recorded flyover noises in the outdoor-acoustic-simulation experiment; 32 different subjects made judgments on the noises in the indoor-acoustic-simulation experiment. The judgments were made by using the method of numerical category scaling. The effective perceived noise level underestimated the noisiness of the supersonic transport by 3.5 db. For takeoff operations, no difference was found between the noisiness of the supersonic transport and the group of other aircraft for the A-weighted rating scale; however, for landing operations, the noisiness of the supersonic transport was overestimated by 3.7 db. Very high correlation was found between the outdoor-simulation experiment and the indoor-simulation experiment.

Powell, C. A.↗

NASA NTRS1980

Annoyance due to multiple airplane noise exposure

A laboratory study was conducted to investigate the annoyance effects of multiple aircraft noise exposure in which 250 subjects judged the annoyance of half-hour periods of airplane noise simulative of typical indoor home exposures. The variables of the aircraft noise exposure were the peak noise level of flyovers, which was constant within each period, and the number of flyovers. Each subject judged 5 of the possible 25 factorial combinations of level and number. Other variables investigated included the experience of the test subjects in making annoyance judgments and their home exposure to airplane noise. The annoyance judgments increased with both noise level and number of flyovers. The increased annoyance produced by doubling the number of flyovers was found to be the equivalent of a 4 to 6 db increase in noise level. The sensitivity of the subjects to changes in both noise level and number of flyovers increased with laboratory experience. Although the means of the annoyance judgments made in the laboratory were found to decrease with the subjects' home exposure to aircraft noise, the subjects' sensitivities to changes in both level and number were unaffected by their home exposure.

Powell, C. A.↗

NASA NTRS1980

Effects of magnification and visual accommodation on aimpoint estimation in simulated landings with real and virtual image displays

Twenty professional pilots observed a computer-generated airport scene during simulated autopilot-coupled night landing approaches and at two points (20 sec and 10 sec before touchdown) judged whether the airplane would undershoot or overshoot the aimpoint. Visual accommodation was continuously measured using an automatic infrared optometer. Experimental variables included approach slope angle, display magnification, visual focus demand (using ophthalmic lenses), and presentation of the display as either a real (direct view) or a virtual (collimated) image. Aimpoint judgments shifted predictably with actual approach slope and display magnification. Both pilot judgments and measured accommodation interacted with focus demand with real-image displays but not with virtual-image displays. With either type of display, measured accommodation lagged far behind focus demand and was reliably less responsive to the virtual images. Pilot judgments shifted dramatically from an overwhelming perceived-overshoot bias 20 sec before touchdown to a reliable undershoot bias 10 sec later.

Randle, R. J.↗

NASA NTRS1982

Subjective scaling of mental workload in a multi-task environment

Those factors in a multi-task environment that contribute to the operators' "sense" of mental workload were identified. The subjective judgment as conscious experience of mental effort was decided to be the appropriate method of measurement. Thurstone's law of comparative judgment was employed in order to construct interval scales of subjective mental workload from paired comparisons data. An experimental paradigm (Simulated Multi-Task Decision-Making Environment) was employed to represent the ideal experimentally controlled environment in which human operators were asked to "attend" to different cases of Tulga's decision making tasks. Through various statistical analyses it was found that, in general, a lower number of tasks-to-be-processed per unit time (a condition associated with longer interarrival times), results in a lower mental workload, a higher consistency of judgments within a subject, a higher degree of agreement among the subjects, and larger distances between the cases on the Thurstone scale of subjective mental workload. The effects of various control variables and their interactions, and the different characteristics of the subjects on the variation of subjective mental workload are demonstrated.

Daryanian, B.↗

NASA NTRS1982

A model of subjective probabilities from small groups

Methods for aggregating the opinions of individual forecasters in order to improve the quality of probabilistic forecasts are presented. Experimental results obtained by Seaver in his study of probability judgments by groups of four people are considered. The decision variable partition model of subjective probability and a simple model of the effects of interaction on judgments were used to simulate the group judgment of discrete probabilities investigated by Seaver. The initial results of the simulation are very promising in that (1) they show the principal effects Seaver observed and (2) these effects can, for the most part, be traced to specific characteristics of the models.

Ferrell, W. R.↗

NASA NTRS1987

NASA Lewis Research Center Futuring Workshop

On October 21 and 22, 1986, the Futures Group ran a two-day Futuring Workshop on the premises of NASA Lewis Research Center. The workshop had four main goals: to acquaint participants with the general history of technology forecasting; to familiarize participants with the range of forecasting methodologies; to acquaint participants with the range of applicability, strengths, and limitations of each method; and to offer participants some hands-on experience by working through both judgmental and quantitative case studies. Among the topics addressed during this workshop were: information sources; judgmental techniques; quantitative techniques; merger of judgment with quantitative measurement; data collection methods; and dealing with uncertainty.

Boroush, Mark↗

NASA NTRS1987

Absolute magnitude estimation and relative judgement approaches to subjective workload assessment

Two rating scale techniques employing an absolute magnitude estimation method, were compared to a relative judgment method for assessing subjective workload. One of the absolute estimation techniques used was an unidimensional overall workload scale and the other was the multidimensional NASA-Task Load Index technique. Thomas Saaty's Analytic Hierarchy Process was the unidimensional relative judgment method used. These techniques were used to assess the subjective workload of various single- and dual-tracking conditions. The validity of the techniques was defined as their ability to detect the same phenomena observed in the tracking performance. Reliability was assessed by calculating test-retest correlations. Within the context of the experiment, the Saaty Analytic Hierarchy Process was found to be superior in validity and reliability. These findings suggest that the relative judgment method would be an effective addition to the currently available subjective workload assessment techniques.

Vidulich, Michael A.↗

NASA NTRS1990

Symbolic enhancement of perspective displays

Two exocentric azimuth judgment experiments with a perspective display were conducted with 16 subjects. Previous work has shown these judgments to exhibit a bias possibly due to misinterpretation of the viewing parameters used to generate the display. Though geometric compensations may be used to correct for the bias, an alternate technique selected in the following 2 experiments was the introduction of symbolic enhancements in the form of compass roses. It is suggested that a compass rose with 30 deg divisions results in overall optimal azimuth estimation accuracy when accuracy and decision time are both considered. The data also suggest that the added radial lines on the compass roses may interact with normalization processes that influence the judgment errors.

Ellis, Stephen R.↗

NASA NTRS1996

Comparison of the Performance of Noise Metrics as Predictions of the Annoyance of Stage 2 and Stage 3 Aircraft Overflights

Thirty audiometrically screened test participants judged the relative annoyance of two comparison (variable level) and thirty-four standard (fixed level) signals in an adaptive paired comparison psychoacoustic study. The signal ensemble included both FAR Part 36 Stage 2 and 3 aircraft overflights, as well as synthesized aircraft noise signatures and other non-aircraft signals. All test signals were presented for judgment as heard indoors, in the presence of continuous background noise, under free-field listening conditions in an anechoic chamber. Analyses of the performance of 30 noise metrics as predictors of these annoyance judgments confirmed that the more complex metrics were generally more accurate and precise predictors than the simpler methods. EPNL was somewhat less accurate and precise as a predictor of the annoyance judgments than a duration-adjusted variant of Zwicker's Loudness Level.

Pearsons, Karl S.↗

NASA NTRS2001

Using the Human Eye to Characterize Displays

Monitor characterization has taken on new importance for non-professional users, who are not usually equipped to make photometric measurements. Our purpose was to examine some of the visual judgments used in characterization schemes that have been proposed for web users. We studied adjusting brightness to set the black level, banding effects due to digitization, and gamma estimation in the light and in the dark, and a color-matching task in the light, on a desktop CRT and a laptop LCD. Observers demonstrated the sensitivity of the visual system for comparative judgments in black-level adjustment, banding visibility, and gamma estimation. The results of the color-matching task were ambiguous. In the brightness adjustment task, the action of the adjustment was not as presumed; however, perceptual judgments were as expected under the actual conditions. When the gamma estimates of observers were compared to photometric measurements, problems with the definition of gamma were identified. Information about absolute light levels that would be important for characterizing a display, given the shortcomings of gamma in measuring apparent contrast, are not measurable by eye alone. The LCD was not studied as extensively as the CRT because of viewing-angle problems, and its transfer function did not follow a power law, rendering gamma estimation meaningless.

Gille, Jennifer↗

NASA NTRS2004

Development of an Expert Judgement Elicitation and Calibration Methodology for Risk Analysis in Conceptual Vehicle Design

A comprehensive expert-judgment elicitation methodology to quantify input parameter uncertainty and analysis tool uncertainty in a conceptual launch vehicle design analysis has been developed. The ten-phase methodology seeks to obtain expert judgment opinion for quantifying uncertainties as a probability distribution so that multidisciplinary risk analysis studies can be performed. The calibration and aggregation techniques presented as part of the methodology are aimed at improving individual expert estimates, and provide an approach to aggregate multiple expert judgments into a single probability distribution. The purpose of this report is to document the methodology development and its validation through application to a reference aerospace vehicle. A detailed summary of the application exercise, including calibration and aggregation results is presented. A discussion of possible future steps in this research area is given.

Unal, Resit↗

DOE OSTI2021

Key Constructs for Reasonable Confidence in System Validation Programs

This paper presents four key constructs developed in preparing IEEE Std P2411TM, Human Factors Engineering Guide for the Validation of System Designs and Integrated Systems Operations at Nuclear Facilities for submission. These constructs reflect judgments that responsible parties make in order to bring validation to closure. Such judgments weigh competing concerns and cost-benefits for stakeholders. Improved processes and methods may support such judgments but cannot alone eliminate uncertainty. Stating and clarifying these constructs aims to reduce uncertainty in the validation process, to better prepare the implementers and reviewers of future validation programs, both in the nuclear industry and beyond. Reasonable Confidence – A proof standard, as used to reach conclusions in a legal case. Legal proof standards are bounded between “preponderance of evidence” and “beyond reasonable doubt”. Reasonable confidence is comparable to an intermediate legal proof standard of “clear and convincing evidence.” Representative Test Set – A collection of scenarios of sufficient variety to represent the anticipated range of operational conditions, events, evolutions, and activities for validating the system(s) under test. Dispositive vs. Diagnostic Criteria – Relevant performance criteria are placed in one of two categories. Dispositive criteria are assessed, without exception, to determine whether a validation test passes or fails overall. Diagnostic criteria are assessed, subject to justified exception, to evaluate the quality or degree of some particular aspect of performance. Repeatability – Consistency in achieving passing results is to be demonstrated for each scenario in the representative test set; thus, the minimum number of formal repetitions for each scenario is two.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2016

Generation of Low Fidelity Experimental Covariance Matrices for ICSBEP Cases

In validation, statistical methods routinely incorporate information from integral experiments to improve estimations of code biases and uncertainties. One source of high quality integral experiments is the ICSBEP Handbook. While uncertainties for criticality benchmarks are well characterized in the Handbook, information pertaining to the shared uncertainty/covariance between the cases is sparse, yet required for the aforementioned methods; correlations can result from using the same fuel for example. To remedy the lack of available covariance data in the Handbook, recent efforts have been undertaken, notably, within the framework of the NEA Expert Group on Uncertainty Analysis in Criticality Safety Assessment (UACSA) under the Working Party on Nuclear Criticality Safety (WPNCS) . The UACSA activities have focused primarily on comparing methods used to generate high quality covariance matrices for a small number of evaluations. Despite these efforts, within the Database for ICSBEP DICE only about 2% of the nearly 5000 experiments in the ICSBEP Handbook have correlation coefficients, so the amount of work required to generate the missing data appears daunting. And so, to fill in the gaps where no correlations are available, it is proposed to generate low fidelity correlations by isolating the largest components of uncertainty and determining the fraction shared between cases within an evaluation. Uncertainties can be shared when they are not independently measured, or the source of uncertainty is unchanged between cases. The degree to which an uncertainty is shared is linked to the experimental procedure; an approximation is made that in the absence of additional information, similar experimental procedures will lead to a similar fraction of shared uncertainty. Between experimental procedures there is some commonality. One method, proposed here, to help elucidate similar procedures is to encode each procedure into a decision tree grouped by uncertainty type. End nodes of the tree require judgment to assign the fraction of shared uncertainty between cases associated with the node. This can also be thought of as a set of rules, such that if experimental procedures and conditions occurred, then assign the shared uncertainty as some number between 0.0 and 1.0. One advantage of such a procedure is the formation of a transparent logical link between the judgment applied and the amount of shared uncertainty. This makes the results both more reproducible and easy to update. Generating correlation coefficients is reduced to developing decision trees that categorize how the experiments were done, and apply judgment of the degree of shared uncertainty for the end node of the tree. Trees need to be constructed after reading section 1 and 2 of the evaluation and some evaluations certainly do not contain enough information to create informative decision trees nor to assign an amount of shared uncertainty. These evaluations will not be considered here. (authors)

42 ENGINEERING↗

NASA NTRS1998

Two memories for geographical slant: separation and interdependence of action and awareness

The present study extended previous findings of geographical slant perception, in which verbal judgments of the incline of hills were greatly overestimated but motoric (haptic) adjustments were much more accurate. In judging slant from memory following a brief or extended time delay, subjects' verbal judgments were greater than those given when viewing hills. Motoric estimates differed depending on the length of the delay and place of response. With a short delay, motoric adjustments made in the proximity of the hill did not differ from those evoked during perception. When given a longer delay or when taken away from the hill, subjects' motoric responses increased along with the increase in verbal reports. These results suggest two different memorial influences on action. With a short delay at the hill, memory for visual guidance is separate from the explicit memory informing the conscious response. With short or long delays away from the hill, short-term visual guidance memory no longer persists, and both motor and verbal responses are driven by an explicit representation. These results support recent research involving visual guidance from memory, where actions become influenced by conscious awareness, and provide evidence for communication between the "what" and "how" visual processing systems.

NASA Discipline Space Human Factors↗

DOE OSTI2025

Identifying preferential flow from soil moisture time series: Review of methodologies

Abstract Identifying and quantifying preferential flow (PF) through soil—the rapid movement of water through spatially distinct pathways in the subsurface—is vital to understanding how the hydrologic cycle responds to climate, land cover, and anthropogenic changes. In recent decades, methods have been developed that use measured soil moisture time series to identify PF. Because they allow for continuous monitoring and are relatively easy to implement, these methods have become an important tool for recognizing when, where, and under what conditions PF occurs. The methods seek to identify a pattern or quantification that indicates the occurrence of PF. Most commonly, the chosen signature is either (1) a nonsequential response to infiltrated water, in which soil moisture responses do not occur in order of shallowest to deepest, or (2) a velocity criterion, in which newly infiltrated water is detected at depth earlier than is possible by nonpreferential flow processes. Alternative signatures have also been developed that have certain advantages but are less commonly utilized. Choosing among these possible signatures requires attention to their pertinent characteristics, including susceptibility to errors, possible bias toward false negatives or false positives, reliance on subjective judgments, and possible requirements for additional types of data. We review 77 studies that have applied such methods to highlight important information for readers who want to identify PF from soil moisture data and to inform those who aim to develop new methods or improve existing ones. Core Ideas Soil moisture data can be used to identify the occurrence of preferential flow (PF) and its initiating conditions. Various data‐analysis methods to identify PF differ in susceptibility to error, bias, and subjectivity. These methods can utilize vast amounts of data from soil moisture monitoring networks to develop understanding of when, where, and under what conditions PF occurs. Newly developed methods may lead to better accuracy and reliability, and reduce the need for subjective judgments. Plain Language Summary Preferential flow through soil occurs when a large amount of water is suddenly available, as during an intense storm. This type of flow moves rapidly through the soil in distinct narrow pathways rather than moving evenly throughout the body of soil, with major consequences for groundwater resources, ecosystems, spreading of contaminants, and other vital concerns. Methods of detecting preferential flow have been developed that utilize measurements of soil water content made by sensors installed at various depths. This measurement technology has been widely implemented, many locations now having datasets years in length, and various methods have been developed for using these to identify preferential flow. The various methods are based on different features in the soil moisture records and vary in their advantages and shortcomings. In this review, we explain and evaluate these methods, highlighting important information for their implementation to identify preferential flow from soil moisture data and for efforts to develop new methods or improve existing ones.

Nimmo, John R↗

DOE OSTI2016

A Bayesian Machine Learning Model for Estimating Building Occupancy from Open Source Data

Understanding building occupancy is critical to a wide array of applications including natural hazards loss analysis, green building technologies, and population distribution modeling. Due to the expense of directly monitoring buildings, scientists rely in addition on a wide and disparate array of ancillary and open source information including subject matter expertise, survey data, and remote sensing information. These data are fused using data harmonization methods which refer to a loose collection of formal and informal techniques for fusing data together to create viable content for building occupancy estimation. In this paper, we add to the current state of the art by introducing the Population Data Tables (PDT), a Bayesian based informatics system for systematically arranging data and harmonization techniques into a consistent, transparent, knowledge learning framework that retains in the final estimation uncertainty emerging from data, expert judgment, and model parameterization. PDT probabilistically estimates ambient occupancy in units of people/1000ft2 for over 50 building types at the national and sub-national level with the goal of providing global coverage. The challenge of global coverage led to the development of an interdisciplinary geospatial informatics system tool that provides the framework for capturing, storing, and managing open source data, handling subject matter expertise, carrying out Bayesian analytics as well as visualizing and exporting occupancy estimation results. We present the PDT project, situate the work within the larger community, and report on the progress of this multi-year project.Understanding building occupancy is critical to a wide array of applications including natural hazards loss analysis, green building technologies, and population distribution modeling. Due to the expense of directly monitoring buildings, scientists rely in addition on a wide and disparate array of ancillary and open source information including subject matter expertise, survey data, and remote sensing information. These data are fused using data harmonization methods which refer to a loose collection of formal and informal techniques for fusing data together to create viable content for building occupancy estimation. In this paper, we add to the current state of the art by introducing the Population Data Tables (PDT), a Bayesian model and informatics system for systematically arranging data and harmonization techniques into a consistent, transparent, knowledge learning framework that retains in the final estimation uncertainty emerging from data, expert judgment, and model parameterization. PDT probabilistically estimates ambient occupancy in units of people/1000 ft 2 for over 50 building types at the national and sub-national level with the goal of providing global coverage. The challenge of global coverage led to the development of an interdisciplinary geospatial informatics system tool that provides the framework for capturing, storing, and managing open source data, handling subject matter expertise, carrying out Bayesian analytics as well as visualizing and exporting occupancy estimation results. We present the PDT project, situate the work within the larger community, and report on the progress of this multi-year project.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗