Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Evaluating E3SM Global Storm‐Resolving Model Simulations of Deep Convection: Insights From DP‐SCREAM During TRACER

Global Storm-Resolving Models (GSRMs) are becoming increasingly vital for advancing climate modeling and improving the prediction of extreme weather events. Houston, a coastal region frequently affected by deep convective storms, offers an ideal setting to evaluate the ability of GSRMs to simulate deep convection. This study assesses the performance of the Doubly Periodic Simple Cloud-Resolving E3SM (Energy Exascale Earth System Model) Atmosphere Model (DP-SCREAM) using observations from the TRacking Aerosol Convection interactions ExpeRiment (TRACER) campaign. DP-SCREAM effectively reproduces the diurnal cycles of clouds and precipitation, demonstrating much greater skill than the E3SM single column model. The DP-SCREAM is demonstrated to be applicable to coastal regions, partially due to the forcing data sets already capturing the influence of breezes. DP-SCREAM also replicates biases persistent in the global version of SCREAM: the underrepresentation of boundary layer shallow clouds, a lack of mid-level congestus clouds, and the popcorn convection, characterized by small and disorganized convective cells generating the strongest precipitation. To investigate these issues, two sensitivity experiments were conducted: increasing the mixing length and scaling up the buoyancy flux within the Simplified Higher Order Closure scheme. Increasing the mixing length improved mid-level congestus representation and reduced unrealistic early morning fog occurrence. Enhancing buoyancy flux only marginally improved the bias of underproduced big convective cells. In conclusion, an additional resolution sensitivity test at 0.5 km grid spacing demonstrated that a refined horizontal resolution alone is insufficient to resolve these biases.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2009

Evaluation of Low Altitude Cloud Amount Using CALIPSO Observations

Clouds strongly modulate the transfer of solar and thermal radiation within the atmosphere. Changes in the distribution and properties of clouds in a changing climate represent climate feedbacks affecting climate sensitivity, but the predicted responses of clouds from current climate models are inconsistent. Of particular concern is the response of low clouds, and it is recognized that proper representation of low clouds in global models is in need of improvement. Accurate satellite cloud climatologies are necessary to evaluate model performance, but existing climatologies require validation and are inadequate in some respects for model evaluation. CALIPSO and CloudSat have been acquiring high vertical resolution cloud profiles since launch in 2006. These global cloud datasets represent a new tool for evaluation of both models and existing satellite climatologies. While a recently available merged CALIPSO-CloudSat cloud climatology provides the most comprehensive view of global 3-D cloud distribution, the CALIPSO dataset set on its own has several unique characteristics, including profiles of the atmospheric boundary layer with 30 meter vertical resolution. In this paper we investigate the use of CALIPSO data to characterize boundary layer clouds, looking toward the development of a reliable global dataset suitable for evaluation of model performance.

Winker, Dave↗
NASA NTRS2004

PARAGON: A Systematic, Integrated Approach to Aerosol Observation and Modeling

Aerosols are generated and transformed by myriad processes operating across many spatial and temporal scales. Evaluation of climate models and their sensitivity to changes, such as in greenhouse gas abundances, requires quantifying natural and anthropogenic aerosol forcings and accounting for other critical factors, such as cloud feedbacks. High accuracy is required to provide sufficient sensitivity to perturbations, separate anthropogenic from natural influences, and develop confidence in inputs used to support policy decisions. Although many relevant data sources exist, the aerosol research community does not currently have the means to combine these diverse inputs into an integrated data set for maximum scientific benefit. Bridging observational gaps, adapting to evolving measurements, and establishing rigorous protocols for evaluating models are necessary, while simultaneously maintaining consistent, well understood accuracies. The Progressive Aerosol Retrieval and Assimilation Global Observing Network (PARAGON) concept represents a systematic, integrated approach to global aerosol Characterization, bringing together modern measurement and modeling techniques, geospatial statistics methodologies, and high-performance information technologies to provide the machinery necessary for achieving a comprehensive understanding of how aerosol physical, chemical, and radiative processes impact the Earth system. We outline a framework for integrating and interpreting observations and models and establishing an accurate, consistent and cohesive long-term data record.

PARAGON↗
NASA NTRS1999

Crashworthy Evaluation of a 1/5-Scale Model Composite Fuselage Concept

A 1/5-scale model composite fuselage concept for light aircraft and rotorcraft has been developed to satisfy structural and flight loads requirements and to satisfy design goals for improved crashworthiness. The 1/5-scale model fuselage consists of a relatively rigid upper section which forms the passenger cabin, a stiff structural floor, and an energy absorbing subfloor which is designed to limit impact forces during a crash event. The focus of the present paper is to describe the crashworthy evaluation of the fuselage concept through impact testing and finite element simulation using the nonlinear, explicit transient dynamic code, MSC/DYTRAN. The energy absorption behavior of two different subfloor configurations was determined through quasi-static crushing tests. For the dynamic evaluation, each subfloor configuration was incorporated into a 1/5-scale model fuselage section, which was impacted at 31 ft/s vertical velocity onto a rigid surface. The experimental data demonstrate that the fuselage section with a foam-filled subfloor configuration satisfied the impact design requirement. In addition, the fuselage section maintained excellent energy absorption behavior for a 31 ft/s vertical drop test with a 15 deg-roll impact attitude. Good correlation was obtained between the experimental data and analytical results for both impact conditions.

Jackson, Karen E.↗
DOE OSTI2025

Evaluating Pretreatment Strategies with Modeling for Reducing Scaling Potential of Reverse Osmosis Concentrate: Insights from Ion Exchange and Activated Alumina

Reverse osmosis concentrate (ROC) treatment is critical for enhancing water recovery and minimizing concentrate volume for disposal, especially in regions facing water scarcity. This study investigates the application of ion exchange (IX) resins and activated alumina (AA) as pretreatment strategies to mitigate scaling in ROC due to high concentrations of total dissolved solids, hardness (Ca 2+ and Mg 2+ ), and silica. Through a series of Langmuir isotherms, continuous column experiments, and model simulation, two types of strong acid cation IX resins and three types of strong base anion IX resins alongside three types of AA were evaluated. Results indicate that AA exhibits superior performance in silica removal, achieving up to a 65% reduction and maintaining performance for up to 800 bed volume without reaching saturation. Model simulation of a secondary reverse osmosis treating ROC after the IX and AA pretreatment indicated an additional water recovery of ~70% using antiscalants. This study demonstrates the potential for achieving higher water recovery while also identifying opportunities for pretreatment improvement. Challenges such as the limited IX capacity treating ROC, which requires frequent regeneration and increases operational costs, along with the restricted regeneration capacity of AA, underscore the importance of innovation. These findings emphasize the critical need for developing advanced materials and optimized strategies to further enhance the efficiency of ROC treatment processes.

activated alumina↗
NASA NTRS1991

Development of biomechanical models for human factors evaluations

Previewing human capabilities in a computer-aided engineering mode has assisted greatly in planning well-designed systems without the cost and time involved in mockups and engineering models. To date, the computer models have focused on such variables as field of view, accessibility and fit, and reach envelopes. Program outputs have matured from simple static pictures to animations viewable from any eyepoint. However, while kinematics models are available, there are few biomechanical models available for estimating strength and motion patterns. Those, such as Crew Chief, that are available are based on strength measurements taken in specific positions. Johnson Space Center is pursuing a biomechanical model which will use strength data collected on single joints at two or three velocities to attempt to predict compound motions of several joint simultaneously and the resulting force at the end effector. Two lines of research are coming together to produce this result. One is an attempt to use optimal control theory to predict joint motion in complex motions, and another is the development of graphical representation of human capabilities. The progress to date in this research is described.

Woolford, Barbara↗
NASA NTRSDate not supplied

A Systems Approach to AI Model Integration and Performance Evaluation for the Generic UAM Simulation Framework

This paper introduces py-guam, an open-source experimentation framework developed for the NASA Generic Urban Air Mobility simulation (GUAM) environment, facilitating the integration and evaluation of advanced artificial intelligence (AI) algorithms. We present a systems approach which enables the seamless incorporation of data-driven models, including off-nominal and failure state detection, into the GUAM’s Cognitive Architecture (CA). The framework supports customizable experimentation parameters, derives Safety Performance Indicators (SPIs) from UL 4600 safety case analyses, and employs rapid UAM simulations to assess AI impacts on flight performance across diverse scenarios. Through comprehensive testing and validation experiments, we demonstrate GUAM’s capability to enhance safety and efficiency in urban air mobility operations. Additionally, the open-source nature of py-guam fosters community collaboration, ensuring continuous improvement and adaptability to evolving technological advancements. This work establishes a robust tool for developing and testing AI-driven urban air mobility (UAM) systems, advancing the safety and reliability of autonomous urban air vehicles.

Artificial Intelligence↗
NASA NTRS2016

Experimental Evaluation of the Scale Model Method to Simulate Lunar Vehicle Dynamics

As compared to driving on Earth, the presence of lower gravity and uneven terrain on planetary bodies makes high speed driving difficult. In order to maintain ground contact and control vehicles need to be designed with special attention to dynamic response. The challenge of maintaining control on the Moon was evident during high speed operations of the Lunar Roving Vehicle (LRV) on Apollo 16, as at one point all four tires were off the ground; this event has been referred to as the Lunar Grand Prix. Ultimately, computer simulation should be used to examine these phenomena during the vehicle design process; however, experimental techniques are required for the validation and elucidation of key issues. The objectives of this study were to evaluate the methodology for developing a scale model of a lunar vehicle using similitude relationships and to test how vehicle configuration, six or eight wheel pods, and local tire compliance, soft or stiff, affect the vehicles dynamic performance. A wheel pod consists of a drive and steering transmission and wheel. The Lunar Electric Rover (LER), a human driven vehicle with a pressurized cabin, was selected as an example for which a scale model was built. The scaled vehicle was driven over an obstacle and the dynamic response was observed and then scaled to represent the full-size vehicle in lunar gravity. Loss of ground contact, in terms of vehicle travel distance with tires off the ground, was examined. As expected, local tire compliance allowed ground contact to be maintained over a greater distance. However, switching from a six-tire configuration to an eight-tire configuration with reduced suspension stiffness had a negative effect on ground contact. It is hypothesized that this was due to the increased number or frequency of impacts. The development and testing of this scale model provided practical lessons for future low-gravity vehicle development.

Moon↗
NASA NTRS1994

A parametric sensitivity study for single-stage-to-orbit hypersonic vehicles using trajectory optimization

The class of hypersonic vehicle configurations with single stage-to-orbit (SSTO) capability reflect highly integrated airframe and propulsion systems. These designs are also known to exhibit a large degree of interaction between the airframe and engine dynamics. Consequently, even simplified hypersonic models are characterized by tightly coupled nonlinear equations of motion. In addition, hypersonic SSTO vehicles present a major system design challenge; the vehicle's overall mission performance is a function of its subsystem efficiencies including structural, aerodynamic, propulsive, and operational. Further, all subsystem efficiencies are interrelated, hence, independent optimization of the subsystems is not likely to lead to an optimum design. Thus, it is desired to know the effect of various subsystem efficiencies on overall mission performance. For the purposes of this analysis, mission performance will be measured in terms of the payload weight inserted into orbit. In this report, a trajectory optimization problem is formulated for a generic hypersonic lifting body for a specified orbit-injection mission. A solution method is outlined, and results are detailed for the generic vehicle, referred to as the baseline model. After evaluating the performance of the baseline model, a sensitivity study is presented to determine the effect of various subsystem efficiencies on mission performance. This consists of performing a parametric analysis of the basic design parameters, generating a matrix of configurations, and determining the mission performance of each configuration. Also, the performance loss due to constraining the total head load experienced by the vehicle is evaluated. The key results from this analysis include the formulation of the sizing problem for this vehicle class using trajectory optimization, characteristics of the optimal trajectories, and the subsystem design sensitivities.

Lovell, T. Alan↗
NASA NTRS1996

Evaluation of Subgrid-Scale Models for Large Eddy Simulation of Compressible Flows

The objective of this project was to evaluate and develop subgrid-scale (SGS) turbulence models for large eddy simulations (LES) of compressible flows. During the first phase of the project results from LES using the dynamic SGS model were compared to those of direct numerical simulations (DNS) of compressible homogeneous turbulence. The second phase of the project involved implementing the dynamic SGS model in a NASA code for simulating supersonic flow over a flat-plate. The model has been successfully coded and a series of simulations has been completed. One of the major findings of the work is that numerical errors associated with the finite differencing scheme used in the code can overwhelm the SGS model and adversely affect the LES results. Attached to this overview are three submitted papers: 'Evaluation of the Dynamic Model for Simulations of Compressible Decaying Isotropic Turbulence'; 'The effect of the formulation of nonlinear terms on aliasing errors in spectral methods'; and 'Large-Eddy Simulation of a Spatially Evolving Compressible Boundary Layer Flow'.

Blaisdell, Gregory A.↗
NASA NTRS1993

Evaluation of two-equation turbulence models for predicting transitional flows

An evaluation of the capabilities and limitations of four low-Reynolds number two-equation turbulence models for predicting bypass transition on a flat plate has been conducted. The ability of these models to reproduce the effect of the free-stream turbulence on transition was tested. The strengths and deficiencies of these models have been identified systematically. It is found that the k-epsilon models are capable of predicting the qualitative aspects of transition. However, the predictions are found to be sensitive to the initial conditions. Also, the start and end of transition were found to depend on the damping functions used in the low-Reynolds number versions of the k-epsilon models.

Abid, Ridha↗
DOE OSTI2025

Trusted Simulation: Considering Model Quality in the Context of User Trust

A high‐quality simulation model should help its users to easily and appropriately calibrate their trust in the model. Traditional evaluation metrics such as validation and robustness are necessary but insufficient for this task. Trust calibration depends on factors like the model's transparency, applicability to intended use, usability, reputation, and consideration of potential bias. This article proposes a framework for designing and evaluating system dynamics models by considering factors that contribute to the proper calibration of user trust. This framework takes inspiration from trusted artificial intelligence, broadening our traditional concept of model quality and explicitly focusing on what users need to consider a model trustworthy and to understand the model's relevance to its intended purpose. The trusted simulation framework can improve our integration of model quality activities throughout the modeling process, leading to more impactful and better‐targeted model design, development, and evaluation.

Naugle, Asmeret Bier [Sandia National Laboratories↗
NASA NTRS2004

Part 2 of a Computational Study of a Drop-Laden Mixing Layer

This second of three reports on a computational study of a mixing layer laden with evaporating liquid drops presents the evaluation of Large Eddy Simulation (LES) models. The LES models were evaluated on an existing database that had been generated using Direct Numerical Simulation (DNS). The DNS method and the database are described in the first report of this series, Part 1 of a Computational Study of a Drop-Laden Mixing Layer (NPO-30719), NASA Tech Briefs, Vol. 28, No.7 (July 2004), page 59. The LES equations, which are derived by applying a spatial filter to the DNS set, govern the evolution of the larger scales of the flow and can therefore be solved on a coarser grid. Consistent with the reduction in grid points, the DNS drops would be represented by fewer drops, called computational drops in the LES context. The LES equations contain terms that cannot be directly computed on the coarser grid and that must instead be modeled. Two types of models are necessary: (1) those for the filtered source terms representing the effects of drops on the filtered flow field and (2) those for the sub-grid scale (SGS) fluxes arising from filtering the convective terms in the DNS equations. All of the filtered-sourceterm models that were developed were found to overestimate the filtered source terms. For modeling the SGS fluxes, constant-coefficient Smagorinsky, gradient, and scale-similarity models were assessed and calibrated on the DNS database. The Smagorinsky model correlated poorly with the SGS fluxes, whereas the gradient and scale-similarity models were well correlated with the SGS quantities that they represented.

Okongo, Nora↗
NASA NTRS2001

Routine High-Resolution Forecasts/Analyses for the Pacific Disaster Center: User Manual

Enclosed herein is our HWCMO user manual. This manual constitutes the final report for our NASA/PDC grant, NASA NAG5-8730, "Routine High Resolution Forecasts/Analysis for the Pacific Disaster Center". Since the beginning of the grant, we have routinely provided experimental high resolution forecasts from the RSM/MSM for the Hawaii Islands, while working to upgrade the system to include: (1) a more robust input of NCEP analyses directly from NCEP; (2) higher vertical resolution, with increased forecast accuracy; (3) faster delivery of forecast products and extension of initial 1-day forecasts to 2 days; (4) augmentation of our basic meteorological and simplified fireweather forecasts to firedanger and drought forecasts; (5) additional meteorological forecasts with an alternate mesoscale model (MM5); and (6) the feasibility of using our modeling system to work in higher-resolution domains and other regions. In this user manual, we provide a general overview of the operational system and the mesoscale models as well as more detailed descriptions of the models. A detailed description of daily operations and a cost analysis is also provided. Evaluations of the models are included although it should be noted that model evaluation is a continuing process and as potential problems are identified, these can be used as the basis for making model improvements. Finally, we include our previously submitted answers to particular PDC questions (Appendix V). All of our initially proposed objectives have basically been met. In fact, a number of useful applications (VOG, air pollution transport) are already utilizing our experimental output and we believe there are a number of other applications that could make use of our routine forecast/analysis products. Still, work still remains to be done to further develop this experimental weather, climate, fire danger and drought prediction system. In short, we would like to be a part of a future PDC team, if at all possible, to further develop and apply the system for the Hawaiian and other Pacific Islands as well as the entire Pacific Basin.

Roads, John↗
DOE OSTI2025

Predicting critical heat flux with uncertainty quantification and domain generalization using conditional variational autoencoders and deep neural networks

Deep generative models (DGMs) can generate synthetic data samples that closely resemble the original dataset, addressing data scarcity. In this work, we developed a conditional variational autoencoder (CVAE) to augment critical heat flux (CHF) data used for the 2006 Groeneveld lookup table. To compare with traditional methods, a fine-tuned deep neural network (DNN) regression model was evaluated on the same dataset. Both models achieved small mean absolute relative errors, with the CVAE showing more favorable results. Uncertainty quantification (UQ) was performed using repeated CVAE sampling and DNN ensembling. The DNN ensemble improved performance over the baseline, while the CVAE maintained consistent results with less variability and higher confidence. Both models achieved small errors inside and outside the training domain, with slightly larger errors outside. Altogether, the CVAE performed better than the DNN in predicting CHF and exhibited better uncertainty behavior.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Performance Comparison of Machine Learning Models for Ultrasonic Nondestructive Evaluation of Alkali-Silica Reaction in Concrete

Alkali-silica reaction (ASR) causes concrete degradation, leading to cracking, rebar corrosion, and reduced structural integrity, which raises safety concerns. Ultrasonic nondestructive evaluation (NDE) effectively assesses concrete properties and monitors ASR progression. However, its deployment and analysis require specialized expertise and subjective interpretation. As computational power increases, artificial intelligence (AI) and machine learning (ML) algorithms are increasingly being used to automate NDE data analysis across various industries for AI-assisted automation. Regulatory agencies are adapting to this technological shift, prompting a need to evaluate current ML technologies’ capabilities and limitations in assessing concrete material properties and damage. This report presents a comparative analysis of four ML regression models for predicting concrete material damage induced by ASR expansion using long-term ultrasonic data monitoring. The models investigated include linear regression (LR), support vector regression (SVR), shallow neural networks (NN), and deep neural networks (DNN). LR, SVR, and shallow NN models use features extracted from ultrasonic signals, whereas the DNN model processes time-domain ultrasonic signals and frequency spectra directly. The study systematically compared the models’ performance from various perspectives, including model input, prediction performance, and generalization ability. The findings indicate significant variability in model performance, with some ML algorithms achieving very high or very low prediction accuracy depending on the preprocessing and feature engineering (extraction and selection) applied. Key insights include the observation that shallow ML models (LR, SVR, and shallow NNs) require meticulous preprocessing and feature extraction to achieve high accuracy. In contrast, the DNN model, although it bypasses the need for feature engineering, necessitates extensive preprocessing to mitigate noise and computational demands. The SVR model emerged as the top performer among the shallow models, and the DNN model exhibited superior performance on specific datasets but struggled with generalization across specimens from different batches. Additionally, the SVR model is sensitive to temperature variations, whereas the DNN model is robust in this regard. Using recurrent neural networks is recommended for future ASR expansion prediction studies. Recurrent neural networks’ inherent ability to capture temporal dependencies and long-term patterns makes them well suited for analyzing sequential ultrasonic monitoring data. Overall, the results and conclusions of this study could provide insights into the capabilities and effectiveness of ML when applied to ultrasonic NDE data and help identify best practices for using ML for ultrasonic NDE of concrete material properties.

36 MATERIALS SCIENCE↗
NASA NTRS1976

Aeroheating model advancements featuring electroless metallic plating

Discussed are advancements in wind tunnel model construction methods and hypersonic test data demonstrating the methods. The general objective was to develop model fabrication methods for improved heat transfer measuring capability at less model cost. A plated slab model approach was evaluated with cast models containing constantan wires that formed single-wire-to-plate surface thermocouple junctions with a seamless skin of electroless nickel alloy. The surface of a space shuttle orbiter model was selectively plated with scaled tiles to simulate, with high fidelity, the probable misalignments of the heatshield tiles on a flight vehicle. Initial, Mach 8 heating results indicated a minor effect of tile misalignment roughness on boundary layer transition, implying a possible relaxation of heatshield manufacturing tolerances. Some loss of the plated tiles was experienced when the model was tested at high heating rates.

Stalmach, C. J., Jr.↗
NASA NTRS1980

Linear analysis of poststall gyrations

The poststall gyrations of a high-performance aircraft are investigated by approximating the equations of motion with a linear mathematical model. Equilibrium poststall spin conditions are determined, which are not limited to small angles or spin rates and are used as reference conditions about which the linear model is developed. As a result, complete coupling between lateral and longitudinal motions is retained. Three aerodynamic models are evaluated using this linear analysis. These models include pure rotary derivatives obtained from VPI&SU stability tunnel, forced-oscillation data obtained from NASA tests, and a combination of the two which separates the unsteady and pure rotary derivatives. Results indicate that this last model gives better predictions of aircraft motion during poststall gyrations at high angles of attack and large sideslip angles. Comparisons are made with full-scale flight test results.

Hreha, M. A.↗