Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Evaluation of Classifier Complexity for Delay Tolerant Network Routing

The growing popularity of small cost effective satellites (SmallSats, CubeSats, etc.) creates the potential for a variety of new science applications involving multiple nodes functioning together or independently to achieve a task, such as swarms and constellations. As this technology develops and is deployed for missions in Low Earth Orbit and beyond, the use of delay tolerant networking (DTN) techniques may improve communication capabilities within the network. In this paper, a network hierarchy is developed from heterogeneous networks of SmallSats, surface vehicles, relay satellites and ground stations which form an integrated network. There is a tradeoff between complexity, flexibility, and scalability of user defined schedules versus autonomous routing as the number of nodes in the network increases. To address these issues, this work proposes a machine learning classifier based on DTN routing metrics. A framework is developed which will allow for the use of several categories of machine learning algorithms (decision tree, random forest and deep learning) to be applied to a dataset of historical network statistics, which allows for the evaluation of algorithm complexity versus performance to be explored. We develop the emulation of a hierarchical network, consisting of tens of nodes which form a cognitive network architecture. CORE (Common Open Research Emulator) is used to emulate the network using bundle protocol and DTN IP neighbor discovery.

Dudukovich, Rachel↗

PERSIANN-Unet: A Global Deep Learning Framework for Near-Real-Time Precipitation Estimation Using Infrared Data

Access to high-quality, high-resolution, near-real-time precipitation data is essential for hydrological and meteorological research and disaster mitigation. Traditional tools such as rain gauges and radar networks, though effective, have limitations, including sparse coverage in remote areas and high operational costs. Satellite data, with its global coverage and high spatial and temporal resolutions, mitigates limitations in coverage. Satellite precipitation products like Hydro Estimator (HE), Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG), and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) utilize both geosynchronous thermal infrared (IR) and passive microwave (PMW) data in their operation. PMW sensors offer detailed atmospheric profiles but suffer from higher latency, whereas IR sensors provide lower latency but only capture cloud-top information. Despite this constraint, IR data remains attractive for low-latency precipitation estimation. Recent advances in deep learning, particularly convolutional neural networks (CNNs), have further improved satellite precipitation retrievals. This study introduces PERSIANN-Unet (PUnet or PERSIANN V3), a quasi-global algorithm covering 60°N–60°S that combines IR data, monthly climatology, and the UNet architecture to produce half-hourly precipitation estimates at 0.04° resolution. The product is evaluated against HE, IMERG, and PDIR-Now for 2022–2023. Results show that PUnet closely matches its training target, IMERG V07 Final, at the global scale, and performance is further evaluated against Stage IV as a reference over CONUS. Training PUnet on IMERG (2016–2021) leverages a high-quality, integrated PMW IR-gauge precipitation product while developing an IR-based framework not reliant on PMW availability. By operating on a single global image, PUnet avoids tile partitioning and blending steps, reducing edge discontinuities, and produces more spatially consistent precipitation fields across hemispheres.

Phu Nguyen↗

Validation of High-Fidelity CFD/CAA Framework for Launch Vehicle Acoustic Environment Simulation against Scale Model Test Data

A hybrid Computational Fluid Dynamics and Computational Aero-Acoustics (CFD/CAA) modeling framework has been developed for launch vehicle liftoff acoustic environment predictions. The framework couples the existing highly-scalable NASA production CFD code, Loci/CHEM, with a high-order accurate discontinuous Galerkin solver developed in the same production framework, Loci/THRUST, to accurately resolve and propagate acoustic physics across the entire launch environment. Time-accurate, Hybrid RANS/LES CFD modeling is applied for predicting the acoustic generation physics at the plume source, and a high-order accurate unstructured discontinuous Galerkin (DG) method is employed to propagate acoustic waves away from the source across large distances using high-order accurate schemes. The DG solver is capable of solving 2nd, 3rd, and 4th order Euler solutions for non-linear, conservative acoustic field propagation. Initial application testing and validation has been carried out against high resolution acoustic data from the Ares Scale Model Acoustic Test (ASMAT) series to evaluate the capabilities and production readiness of the CFD/CAA system to resolve the observed spectrum of acoustic frequency content. This paper presents results from this validation and outlines efforts to mature and improve the computational simulation framework.

Liever, Peter A.↗

Validation of High-Fidelity CFD/CAA Framework for Launch Vehicle Acoustic Environment Simulation against Scale Model Test Data

A hybrid Computational Fluid Dynamics and Computational Aero-Acoustics (CFD/CAA) modeling framework has been developed for launch vehicle liftoff acoustic environment predictions. The framework couples the existing highly-scalable NASA production CFD code, Loci/CHEM, with a high-order accurate Discontinuous Galerkin solver developed in the same production framework, Loci/THRUST, to accurately resolve and propagate acoustic physics across the entire launch environment. Time-accurate, Hybrid RANS/LES CFD modeling is applied for predicting the acoustic generation physics at the plume source, and a high-order accurate unstructured mesh Discontinuous Galerkin (DG) method is employed to propagate acoustic waves away from the source across large distances using high-order accurate schemes. The DG solver is capable of solving 2nd, 3rd, and 4th order Euler solutions for non-linear, conservative acoustic field propagation. Initial application testing and validation has been carried out against high resolution acoustic data from the Ares Scale Model Acoustic Test (ASMAT) series to evaluate the capabilities and production readiness of the CFD/CAA system to resolve the observed spectrum of acoustic frequency content. This paper presents results from this validation and outlines efforts to mature and improve the computational simulation framework.

Liever, Peter A.↗

Performance and Evaluation of the Global Modeling and Assimilation Office Observing System Simulation Experiment

The National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASA/GMAO) has spent more than a decade developing and implementing a global Observing System Simulation Experiment framework for use in evaluting both new observation types as well as the behavior of data assimilation systems. The NASA/GMAO OSSE has constantly evolved to relect changes in the Gridpoint Statistical Interpolation data assimiation system, the Global Earth Observing System model, version 5 (GEOS-5), and the real world observational network. Software and observational datasets for the GMAO OSSE are publicly available, along with a technical report. Substantial modifications have recently been made to the NASA/GMAO OSSE framework, including the character of synthetic observation errors, new instrument types, and more sophisticated atmospheric wind vectors. These improvements will be described, along with the overall performance of the current OSSE. Lessons learned from investigations into correlated errors and model error will be discussed.

OSS↗

Evaluation of Transport-Equation-Based Transition Models for High-Speed Boundary Layers Using OVERFLOW

Accurate modeling of laminar-turbulent transition is crucial for the design of hypersonic flight systems. However, the current transition models used in production CFD codes are insufficient for high-speed flows. Many extensions to low-speed models have been suggested; however, a thorough verification and validation effort is needed before these models can be used in design settings. Challenges include potentially missing details of the model implementation requirements and/or a complete specification of the input parameters needed to replicate the test findings. A meaningful assessment of the generalization capability of these models is also hindered by a lack of information regarding the specific flow configurations and associated grids employed for model calibration. As a key first step toward model verification, we present an independent assessment of two recently proposed models for high-speed transition, namely, a model within the SST-$\gamma$ framework and a model based on the SST-$\gamma-\nu_L$ equations. These models are implemented in the NASA OVERFLOW 2.3e solver and their performance in predicting first mode, second mode, and crossflow transition has been evaluated for several test cases in the supersonic and hypersonic regimes. Besides the test cases employed by the model developers, which could have also been used for model calibration, the present assessment includes supplementary configurations that contribute to an unbiased assessment of the models. The outcomes presented in this study indicate the potential for the models to be applied to high-speed flight configurations. Key steps toward future improvements to these models are also outlined.

High-speed flow↗

Towards a flexible framework for community-wide ensemble forecasting tailored for major space environment impacts

To build continuously improving space weather predictive capabilities based on science and enable assessments and rapid implementations of advances in research into source-to-impact modelling systems we need: to assemble parts of the puzzle by solving problems focused on specific physical domains; to identify essential space environment quantities (ESEQs) passed between domains and linked to impacts; evaluate modeling capabilities for each ESEQ; connect all validated solutions from space weather origins on the sun to impacts on humans and critical infrastructure; to design displays for ensemble predictions tailored for major space weather user groups; to build a collaborative environment for efficient sharing of information and capabilities (models/data/expertise) and collaborative development. The presentation will overview existing community-wide space weather forecasting frameworks and research-to-operations pipelines and discuss opportunities to build a collaborative plug-and-play platform for interconnecting predictive capabilities developed under different space weather programs.

space weather↗

Towards A Flexible Framework for Community-Wide Ensemble Forecasting Tailored for Major Space Environment Impacts

To build continuously improving space weather predictive capabilities based on science and enable assessments and rapid implementations of advances in research into source-to-impact modelling systems we need: to assemble parts of the puzzle by solving problems focused on specific physical domains; to identify essential space environment quantities (ESEQs) passed between domains and linked to impacts; evaluate modeling capabilities for each ESEQ; connect all validated solutions from space weather origins on the sun to impacts on humans and critical infrastructure; to design displays for ensemble predictions tailored for major space weather user groups; to build a collaborative environment for efficient sharing of information and capabilities (models/data/expertise) and collaborative development. The presentation will overview existing community-wide space weather forecasting frameworks and research-to-operations pipelines and discuss opportunities to build a collaborative plug-and-play platform for interconnecting predictive capabilities developed under different space weather programs.

space weather↗

Fluctuation-induced conductivity of superconductors above the transition temperature - Regularization of the Maki diagram.

The Maki contribution to the conductivity above the superconducting transition temperature is regularized within the framework of the BCS theory. This is achieved through the renormalization of the impurity-scattering vertex by inclusion of the effects of pair fluctuations. The conductivity is evaluated for a thin film. It depends only on the reduced temperature and the normal resistance per square. Fair agreement is found with Al films over a wide temperature range. Agreement is not found with experiments on Bi, Pb, and Ga films, which apparently contain a strong additional pair-breaking effect. The temperature range in which interactions among fluctuations become important in the Maki conductivity is generally larger than that given by the Ginzburg criterion.

Keller, J.↗

Application of FDI metrics to detection and isolation of sensor failures in turbine engines

This paper develops a framework for the design of failure detection and isolation (FDI) algorithms. Rather than trying to apply 'optimal' techniques in a top-down manner, the system redundancies are evaluated with respect to their ability to provide reliable FDI information. Previous work of Pattipati et al. (1984) and Weiss et al. (1984) defined a useful context and several useful analytical results, which provide a basis for the FDI design methodology developed here. A general decision structure which can take advantage of redundancy evaluation is presented, and examples of typical design considerations are discussed. The operation of the decision structure is then demonstrated for a sensor FDI application involving the F-100 jet engine.

Weiss, J. L.↗

Rotorcraft Conceptual Design Environment

Requirements for a rotorcraft conceptual design environment are discussed, from the perspective of a government laboratory. Rotorcraft design work in a government laboratory must support research, by producing technology impact assessments and defining the context for research and development; and must support the acquisition process, including capability assessments and quantitative evaluation of designs, concepts, and alternatives. An information manager that will enable increased fidelity of analysis early in the design effort is described. This manager will be a framework to organize information that describes the aircraft, and enable movement of that information to and from analyses. Finally, a recently developed rotorcraft system analysis tool is described.

Johnson, Wayne↗

Rotorcraft Conceptual Design Environment

Requirements for a rotorcraft conceptual design environment are discussed, from the perspective of a government laboratory. Rotorcraft design work in a government laboratory must support research, by producing technology impact assessments and defining the context for research and development; and must support the acquisition process, including capability assessments and quantitative evaluation of designs, concepts, and alternatives. An information manager that will enable increased fidelity of analysis early in the design effort is described. This manager will be a framework to organize information that describes the aircraft, and enable movement of that information to and from analyses. Finally, a recently developed rotorcraft system analysis tool is described.

Johnson, Wayne↗

Development of an OSSE Framework for a Global Atmospheric Data Assimilation System

Observing system simulation experiments (OSSEs) are powerful tools for estimating the usefulness of various configurations of envisioned observing systems and data assimilation techniques. Their utility stems from their being conducted in an entirely simulated context, utilizing simulated observations having simulated errors and drawn from a simulation of the earth's environment. Observations are generated by applying physically based algorithms to the simulated state, such as performed during data assimilation or using other appropriate algorithms. Adding realistic instrument plus representativeness errors, including their biases and correlations, can be critical for obtaining realistic assessments of the impact of a proposed observing system or analysis technique. If estimates of the expected accuracy of proposed observations are realistic, then the OSSE can be also used to learn how best to utilize the new information, accelerating its transition to operations once the real data are available. As with any inferences from simulations, however, it is first imperative that some baseline OSSEs are performed and well validated against corresponding results obtained with a real observing system. This talk provides an overview of, and highlights critical issues related to, the development of an OSSE framework for the tropospheric weather prediction component of the NASA GEOS-5 global atmospheric data assimilation system. The framework includes all existing observations having significant impact on short-term forecast skill. Its validity has been carefully assessed using a range of metrics that can be evaluated in both the OSSE and real contexts, including adjoint-based estimates of observation impact. A preliminary application to the Aeolus Doppler wind lidar mission, scheduled for launch by the European Space Agency in 2014, has also been investigated.

Gelaro, Ronald↗

A Systematic Approach for Real-Time Operator Functional State Assessment

A task overload condition often leads to high stress for an operator, causing performance degradation and possibly disastrous consequences. Just as dangerous, with automated flight systems, an operator may experience a task underload condition (during the en-route flight phase, for example), becoming easily bored and finding it difficult to maintain sustained attention. When an unexpected event occurs, either internal or external to the automated system, the disengaged operator may neglect, misunderstand, or respond slowly/inappropriately to the situation. In this paper, we discuss an approach for Operator Functional State (OFS) monitoring in a typical aviation environment. A systematic ground truth finding procedure has been designed based on subjective evaluations, performance measures, and strong physiological indicators. The derived OFS ground truth is continuous in time compared to a very sparse estimation of OFS based on an expert review or subjective evaluations. It can capture the variations of OFS during a mission to better guide through the training process of the OFS assessment model. Furthermore, an OFS assessment model framework based on advanced machine learning techniques was designed and the systematic approach was then verified and validated with experimental data collected in a high fidelity Boeing 737 simulator. Preliminary results show highly accurate engagement/disengagement detection making it suitable for real-time applications to assess pilot engagement.

Zhang, Guangfan↗

Biomedical Simulation Models of Human Auditory Processes

Detailed acoustic engineering models that explore noise propagation mechanisms associated with noise attenuation and transmission paths created when using hearing protectors such as earplugs and headsets in high noise environments. Biomedical finite element (FE) models are developed based on volume Computed Tomography scan data which provides explicit external ear, ear canal, middle ear ossicular bones and cochlea geometry. Results from these studies have enabled a greater understanding of hearing protector to flesh dynamics as well as prioritizing noise propagation mechanisms. Prioritization of noise mechanisms can form an essential framework for exploration of new design principles and methods in both earplug and earcup applications. These models are currently being used in development of a novel hearing protection evaluation system that can provide experimentally correlated psychoacoustic noise attenuation. Moreover, these FE models can be used to simulate the effects of blast related impulse noise on human auditory mechanisms and brain tissue.

Bicak, Mehmet M. A.↗

Observing System Simulation Experiments for Fun and Profit

Observing System Simulation Experiments can be powerful tools for evaluating and exploring both the behavior of data assimilation systems and the potential impacts of future observing systems. With great power comes great responsibility - given a pure modeling framework, how can we be sure our results are meaningful? The challenges and pitfalls of OSSE calibration and validation will be addressed, as well as issues of incestuousness, selection of appropriate metrics, and experiment design. The use of idealized observational networks to investigate theoretical ideas in a fully complex modeling framework will also be discussed

Observational Networks↗

Evaluating Multispectral Snowpack Reflectivity With Changing Snow Correlation Lengths

This study investigates the sensitivity of multispectral reflectivity to changing snow correlation lengths. Matzler's ice-lamellae radiative transfer model was implemented and tested to evaluate the reflectivity of snow correlation lengths at multiple frequencies from the ultraviolet (UV) to the microwave bands. The model reveals that, in the UV to infrared (IR) frequency range, the reflectivity and correlation length are inversely related, whereas reflectivity increases with snow correlation length in the microwave frequency range. The model further shows that the reflectivity behavior can be mainly attributed to scattering rather than absorption for shallow snowpacks. The largest scattering coefficients and reflectivity occur at very small correlation lengths (approximately 10(exp -5 m) for frequencies higher than the IR band. In the microwave range, the largest scattering coefficients are found at millimeter wavelengths. For validation purposes, the ice-lamella model is coupled with a multilayer snow physics model to characterize the reflectivity response of realistic snow hydrological processes. The evolution of the coupled model simulated reflectivities in both the visible and the microwave bands is consistent with satellite-based reflectivity observations in the same frequencies. The model results are also compared with colocated in situ snow correlation length measurements (Cold Land Processes Field Experiment 2002-2003). The analysis and evaluation of model results indicate that the coupled multifrequency radiative transfer and snow hydrology modeling system can be used as a forward operator in a data-assimilation framework to predict the status of snow physical properties, including snow correlation length.

Kang, Do Hyuk↗

Observing System Simulation Experiments

Observing System Simulation Experiments can be powerful tools for evaluating and exploring both the behavior of data assimilation systems and the potential impacts of future observing systems. With great power comes great responsibility - given a pure modeling framework, how can we be sure our results are meaningful? The challenges and pitfalls of OSSE calibration and validation will be addressed, as well as issues of incestuousness, selection of appropriate metrics, and experiment design. The use of idealized observational networks to investigate theoretical ideas in a fully complex modeling framework will also be discussed.

Prive, Nikki↗