Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85

SU-F-T-138: Commissioning and Evaluating Dose Computation Models for a Dedicated Proton Line Scanning Beam Nozzle in Eclipse Treatment Planning System

Purpose: In this study, we present an effective method to derive low dose envelope of the proton in-air spot fluence at beam positions other than the isocenter to reduce amount of measurements required for planning commission. Also, we demonstrate commissioning and validation results of this method to the Eclipse treatment planning system (version 13.0.29) for a Sumitomo dedicated proton line scanning beam nozzle. Methods: The in-air spot profiles at five beam-axis positions (±200, ±100 and 0 mm) were obtained in trigger mode using a MP3 Water tank (PTW-Freiburg) and a pinpoint ionization chamber (model 31014, PTW-Freiburg). Low dose envelope (below 1% of the center dose) of the spot profile at isocenter was obtained by repeated point measurements to minimize dosimetry uncertainty. The double Gaussian (DG) model was used to fit and obtain optimal σ1, σ2 and their corresponding weightings through our in-house MATLAB (Mathworks) program. σ1, σ2 were assumed to expand linearly along the beam axis from a virtual source position calculated by back projecting fitted sigmas from the single Gaussian (SG) model. Absolute doses in water were validated using an Advanced Markus chamber at the depth of 2cm with Pristine Peak (BP) R90d ranging from 5–32 cm for 10×10 cm2 scanned fields. The field size factors were verified with square fields from 2 to 20 cm at 2cm and before BP depth. Results: The absolute dose outputs were found to be within ±3%. For field size factor, the agreement between calculated and measurement were within ±2% at 2cm and ±3% before BP, except for the field size below 2×2 cm2. Conclusion: The double Gaussian model was found to be sufficient for characterizing the Sumitomo dedicated proton line scanning nozzle. With our effective double Gaussian fitting method, we are able to save significant proton beam time with acceptable output accuracy.

60 APPLIED LIFE SCIENCES↗

Absorption Properties of Supercooled Liquid Water between 31 and 225 GHz: Evaluation of Absorption Models Using Ground-Based Observations

Microwave radiometers (MWR) are commonly used to quantify the amount of supercooled liquid water (SLW) in clouds; however, the accuracy of the SLW retrievals is limited by the poor knowledge of the SLW dielectric properties at microwave frequencies. Six liquid water permittivity models were compared with ground-based MWR observations between 31 and 225 GHz from sites in Greenland, the German Alps, and a low-mountain site; average cloud temperatures of observed thin cloud layers range from 0° to –33°C. A recently published method to derive ratios of liquid water opacity from different frequencies was employed in this analysis. These ratios are independent of liquid water path and equal to the ratio of αL at those frequencies that can be directly compared with the permittivity model predictions. The observed opacity ratios from all sites show highly consistent results that are generally within the range of model predictions; however, none of the models are able to approximate the observations over the entire frequency and temperature range. Findings in earlier published studies were used to select one specific model as a reference model for αL at 90 GHz; together with the observed opacity ratios, the temperature dependence of αL at 31.4, 52.28, 150, and 225 GHz was derived. The results reveal that two models fit the opacity ratio data better than the other four models, with one of the two models fitting the data better for frequencies below 90 GHz and the other for higher frequencies. Furthermore, these findings are relevant for SLW retrievals and radiative transfer in the 31–225-GHz frequency region.

54 ENVIRONMENTAL SCIENCES↗

An evaluation of GPT models for phenotype concept recognition

Clinical deep phenotyping and phenotype annotation play a critical role in both the diagnosis of patients with rare disorders as well as in building computationally-tractable knowledge in the rare disorders field. These processes rely on using ontology concepts, often from the Human Phenotype Ontology, in conjunction with a phenotype concept recognition task (supported usually by machine learning methods) to curate patient profiles or existing scientific literature. With the significant shift in the use of large language models (LLMs) for most NLP tasks, we examine the performance of the latest Generative Pre-trained Transformer (GPT) models underpinning ChatGPT as a foundation for the tasks of clinical phenotyping and phenotype annotation. The experimental setup of the study included seven prompts of various levels of specificity, two GPT models (gpt-3.5-turbo and gpt-4.0) and two established gold standard corpora for phenotype recognition, one consisting of publication abstracts and the other clinical observations. The best run, using in-context learning, achieved 0.58 document-level F1 score on publication abstracts and 0.75 document-level F1 score on clinical observations, as well as a mention-level F1 score of 0.7, which surpasses the current best in class tool. Without in-context learning, however, performance is significantly below the existing approaches. Our experiments show that gpt-4.0 surpasses the state of the art performance if the task is constrained to a subset of the target ontology where there is prior knowledge of the terms that are expected to be matched. While the results are promising, the non-deterministic nature of the outcomes, the high cost and the lack of concordance between different runs using the same prompt and input make the use of these LLMs challenging for this particular task.

59 BASIC BIOLOGICAL SCIENCES↗

RE-ASSUME: A Decision Maker's Guide to Evaluating Energy Scenarios, Modeling, and Assumptions

The desire for a “crystal ball” to peer into the future has perhaps never been stronger among energy and environmental planners. While the future has always been uncertain, the interconnectedness of modern economic systems serves to enhance the ripple effect of our energy actions. Today's decisions also seem to be made under heightened uncertainty, as demonstrated by the number of recent unexpected events, such as a prolonged economic recession, an unfolding shale energy revolution, the Fukushima disaster, the Arab Spring, and a notable reduction in photovoltaic costs. Policymakers have historically relied on energy scenarios and modeling to help inform decision making. While there exists a wealth of advanced model, data, and energy scenario options, important questions remain about how to effectively apply these tools to help plan for an uncertain future.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Hardware Evaluation: Abstract Machine Models and Proxy Architectures for Exascale Computing

The strategic and economic impacts of high performance computing (HPC) systems, over the last several decades, have enabled dramatic improvements in manufacturing, design, development, and research, across almost every sector of the U.S. economy, including genetic analysis for agriculture and medicine, oil exploration, electronics, aircraft, and automotive design, and of course national defense technologies. HPC has repeatedly compounded reductions in time-to-solution and time-to-market in each of these areas. In recognition of these strategic and economic impacts, the Department of Energy (DOE) created the Exascale Computing Project (ECP) as a broad-based technology project focusing on simultaneously co-evolving high-performance computing architecture, system software, and application software. While many details of the exascale architectures are undefined, communicating the form and direction of this co-evolution of advances, in each of these three components, across the project, presents a need for a common language through which these advances can be shared. In this document we present a series of abstract representations designed to allow application developers to focus on the aspects of the machine that are important or relevant to performance and code structure. These abstract machine models (AMMs) describe the proposed architectures at the component and system level and are intended as communication aids between application developers and hardware architects during the co-design process. In addition to providing a common language for communication, the constraints of protecting the intellectual property of ECP’s vendor partners requires that the architectural advances be presented in an abstract sense, rather than exposing the specifics of an individual vendor’s designs, while still providing enough detail such that application developers will be able to reason about the performance trade-offs in the design space of their applications.

97 MATHEMATICS AND COMPUTING↗

A spatially explicit individual-based modeling approach to evaluate the cumulative effects of wind energy development on the greater sage-grouse. Final Report

In order to achieve the national vision of 20% of energy being produced by wind by 2030, issues associated with wind energy development’s potential impacts on ecological resources must be addressed. In particular, wind energy and associated transmission system development may affect wildlife populations through direct mortality, habitat fragmentation, habitat degradation, and disturbance. Although individual projects may have little effect on the sustainability of wildlife populations, cumulative impacts could be significant. The wind energy industry and regulators have developed processes for assessing the impacts of individual projects and identifying mitigation requirements, but no one has developed a methodology for considering cumulative impacts in a systematic and holistic way.

17 WIND ENERGY↗

Application and Evaluation of Surrogate Models for Radiation Source Search

Surrogate models are increasingly required for applications in which first-principles simulation models are prohibitively expensive to employ for uncertainty analysis, design, or control. They can also be used to approximate models whose discontinuous derivatives preclude the use of gradient-based optimization or data assimilation algorithms. We consider the problem of inferring the 2D location and intensity of a radiation source in an urban environment using a ray-tracing model based on Boltzmann transport theory. Whereas the code implementing this model is relatively efficient, extension to 3D Monte Carlo transport simulations precludes subsequent Bayesian inference to infer source locations, which typically requires thousands to millions of simulations. Additionally, the resulting likelihood exhibits discontinuous derivatives due to the presence of buildings. To address these issues, we discuss the construction of surrogate models for optimization, Bayesian inference, and uncertainty propagation. Specifically, we consider surrogate models based on Legendre polynomials, multivariate adaptive regression splines, radial basis functions, Gaussian processes, and neural networks. We detail strategies for computing training points and discuss the merits and deficits of each method.

Cook, Jared A.↗

An Evaluation of AI Models’ Performance for Three Geothermal Sites

Current artificial intelligence (AI) applications in geothermal exploration are tailored to specific geothermal sites, limiting their transferability and broader applicability. This study aims to develop a globally applicable and transferable geothermal AI model to empower the exploration of geothermal resources. This study presents a methodology for adopting geothermal AI that utilizes known indicators of geothermal areas, including mineral markers, land surface temperature (LST), and faults. The proposed methodology involves a comparative analysis of three distinct geothermal sites—Brady, Desert Peak, and Coso. The research plan includes self-testing to understand the unique characteristics of each site, followed by dependent and independent tests to assess cross-compatibility and model transferability. The results indicate that Desert Peak and Coso geothermal sites are cross-compatible due to their similar geothermal characteristics, allowing the AI model to be transferable between these sites. However, Brady is found to be incompatible with both Desert Peak and Coso. The geothermal AI model developed in this study demonstrates the potential for transferability and applicability to other geothermal sites with similar characteristics, enhancing the efficiency and effectiveness of geothermal resource exploration. This advancement in geothermal AI modeling can significantly contribute to the global expansion of geothermal energy, supporting sustainable energy goals.

Energy & Fuels↗

Predicting ocean waves along the US east coast during energetic winter storms: sensitivity to whitecapping parameterizations

The performance of two methods for quantifying whitecapping dissipation incorporated in the Simulating Waves Nearshore (SWAN) wave model is evaluated for waves generated along and off the US east coast under energetic winter storms with a predominantly westerly wind. Parameterizing the whitecapping effect can be done using the Komen-type schemes, which are based on mean spectral parameters, or the saturation-based (SB) approach of van der Westhuysen (2007), which is based on local wave parameters and the saturation level concept of the wave spectrum (we use “Komen” and “Westhuysen” to denote these two approaches). Observations of wave parameters and frequency spectra at four National Data Buoy Center (NDBC) buoys are used to evaluate simulation results. Model–data comparisons show that when using the default parameters in SWAN, both Komen and Westhuysen methods underestimate wave height. Simulations of mean wave period using the Komen method agree with observations, but those using the Westhuysen method are substantially lower. Examination of source terms shows that the Westhuysen method underestimates the total energy transferred into the wave action equations, especially in the lower frequency bands that contain higher spectral energy. Several causes for this underestimation are identified. The primary reason is the difference between the wave growth conditions along the east coast during winter storms and the conditions used for the original whitecapping formula calibration. In addition, some deficiencies in simulation results are caused along the coast by the “slanting fetch” effect that adds low-frequency components to the 2-D wave spectra. These components cannot be simulated partly or entirely by available source terms (wind input, whitecapping, and quadruplet) in models and their interaction. Further, the effect of boundary layer instability that is not considered in the Komen and Westhuysen whitecapping wind input formulas may cause additional underestimation.

Allahdadi, Mohammad Nabi↗