Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Concept of ASTER calibration requirement

The document of ASTER Calibration Requirement specifies the following items related to spectral and radiometric characteristics of the ASTER instrument: (1) characteristics whose knowledge is specified, (2) requirement for knowledge of the characteristics, (3) methodology for characteristics evaluation, and (4) supplementary information and data related with characteristics evaluation. This document is applicable to the document of the ASTER Instrument Specification on Observational Performances, and will be a part of the ASTER Calibration Plan. ASTER Calibration Requirement is scheduled to establish the concept and framework by March 1992 when the 5th Calibration and Data Validation Panel Meeting is held, and to determine details including requirement values and evaluation methodologies by October 1992 around which the Calibration Peer Review may be held. The ASTER Calibration Plan is planned to finish by the same time.

Ono, A.↗

Partial-Wave Representations of Laser Beams for Use in Light-Scattering Calculations

In the framework of generalized Lorenz-Mie theory, laser beams are described by sets of beam-shape coefficients. The modified localized approximation to evaluate these coefficients for a focused Gaussian beam is presented. A new description of Gaussian beams, called standard beams, is introduced. A comparison is made between the values of the beam-shape coefficients in the framework of the localized approximation and the beam-shape coefficients of standard beams. This comparison leads to new insights concerning the electromagnetic description of laser beams. The relevance of our discussion is enhanced by a demonstration that the localized approximation provides a very satisfactory description of top-hat beams as well.

Gouesbet, Gerard↗

Probabilistic Prognosis of Non-Planar Fatigue Crack Growth

Quantifying the uncertainty in model parameters for the purpose of damage prognosis can be accomplished utilizing Bayesian inference and damage diagnosis data from sources such as non-destructive evaluation or structural health monitoring. The number of samples required to solve the Bayesian inverse problem through common sampling techniques (e.g., Markov chain Monte Carlo) renders high-fidelity finite element-based damage growth models unusable due to prohibitive computation times. However, these types of models are often the only option when attempting to model complex damage growth in real-world structures. Here, a recently developed high-fidelity crack growth model is used which, when compared to finite element-based modeling, has demonstrated reductions in computation times of three orders of magnitude through the use of surrogate models and machine learning. The model is flexible in that only the expensive computation of the crack driving forces is replaced by the surrogate models, leaving the remaining parameters accessible for uncertainty quantification. A probabilistic prognosis framework incorporating this model is developed and demonstrated for non-planar crack growth in a modified, edge-notched, aluminum tensile specimen. Predictions of remaining useful life are made over time for five updates of the damage diagnosis data, and prognostic metrics are utilized to evaluate the performance of the prognostic framework. Challenges specific to the probabilistic prognosis of non-planar fatigue crack growth are highlighted and discussed in the context of the experimental results.

Leser, Patrick E.↗

Evaluating Process Effectiveness to Reduce Risk

It is well documented that government agencies do not have the same incentive as the private sector to focus on process effectiveness and continual improvement of those processes. It is also well documented whenever government agencies fail to deliver efficient, effective, consistent, and fair services to the citizens. In spite of the various "reinventing government" and "effectiveness initiatives" of the past decades, and in spite of the efforts on the part of many agencies to improve, government in general still lags behind industry in creating a culture of effective processes and systems. While the tragic events that unfolded recently in Flint, Michigan, teach us that running government "like a business" does not always take the needs of the citizenry into account, there are many lessons and techniques from the private sector that government agencies can use to improve. The incentive to improve, while mandated by various administrations1, needs to come from within the workforce, in order to effectively take root. The best, most effective incentive is to reduce, control or eliminate risk. Government agencies face some of the same risks as the private sector, while some are unique. While ISO 310002 has been around since 2009, risk has taken on increased visibility within the private sector with the advent of the emphasis on risk-based thinking in ISO 9001:20153. The relationship between risk-based thinking and effective processes is simple and direct. Those processes that are well thought out and standardized (i.e. Plan-Do-Check-Act), will have taken into account the applicable policy, statutory, regulatory, safety, quality and technical parameters, which may not occur to someone performing the process with minimal experience or training; and thus protect the employees, the public and the agency from statutory and regulatory violations; delay in providing services; non-delivery of services; harm to public or employee safety and health; cost overruns; breaches in security; loss of confidence in government; failure of publicly funded projects; damage to the environment; ethics violations, and the list goes on; with local, national and even international consequences. The Plan-Do-Check-Act process, also known as the "process approach" can be used at any time to establish and standardize a process, and it can also be used to check periodically for "process creep" (i.e., informal, unauthorized changes that have occurred over time), any necessary updates and improvements. While ISO 9001 compliance is not mandated for all government agencies, if interpreted correctly, it can be useful in establishing a framework and implementing effective management systems and processes.4 Another method that can be used to evaluate effectiveness is the scorecard definitions in Mallory's Process Management Standard5 as a basis for evaluating work on the process level on effective, and continuously improved and improving processes. With processes on the lower end of the scale, agencies are vulnerable to a great many risks, with employees and managers making up many of the rules as they go, leading to the above listed negative results. Without clear guidance for nominal operations, off-nominal situations can, and do, increase the likelihood of chaos. In an increasingly technical environment, with inter-agency communication and collaboration becoming the norm, agencies need to come to grips with the fact that processes can become rapidly outdated, and that the technical community should take on an increased role in the maturation of the agency's processes. Industry has long known that effective processes are also efficient, and process improvement methods such as Kaizen, Lean, Six Sigma, 5S, and mistake proofing lead to increased productivity, improved quality, and decreased cost. Again, government agencies have different concerns, but inefficiencies and mistakes can have dire and wide reaching consequences for the public that they serve. While no one goes to work planning to cause harm, it is up to agencies to establish upper level systems, which make establishment and compliance with processes possible. Again, Mallory provides us with a Systems Management Standard6, similar to the Process Management Standard, with a scale of 0-5 for systems effectiveness and maturity. Deming determined that "eighty-five percent of the reasons for failure are deficiencies in the systems and process rather than the employee. The role of management is to change the process rather than badgering individual employees to do better." 7 It is not just the working level employees who need effective processes, but the mid-and upper level managers as well. A disciplined management culture sets the tone for the employees, aids both routine and off-nominal decision-making, and incorporates risk -based thinking into the systems and processes as a matter of normal activity. Figure 1, illustrates the relationship between ineffective and effective processes and risk, through the use of the "stoplight" colors that are commonly used to show serious situations (red), situations which may be improving or deteriorating depending on trends (yellow), and situations that are under control and continuously improved (green).

Shepherd, Christena C.↗

Methods for Evaluating the Effectiveness of Programs to Train Pilot Monitoring

This report provides a compendium of methods for evaluation of training programs. It addresses training programs for developing pilot monitoring skills and has a focus on monitoring flight path management. It presents the Sensemaking Model of Monitoring as a framework for organizing the targets of training, and thus also the types of change in performance useful for assessing program effectiveness. It provides guidance for designing program evaluations, which can be tailored to address the specific monitoring topics or content for an application to fleet or airline needs.

monitoring↗

Evaluation of GEOS-Simulated L-Band Microwave Brightness Temperature using Aquarius Observations over Non-Frozen Land across North America

L-band brightness temperature (T(sub b)) is one of the key remotely-sensed variables that provides information regarding surface soil moisture conditions. In order to harness the information in T(sub b) observations, a radiative transfer model (RTM) is investigated for eventual inclusion into a data assimilation framework. In this study, T(sub b) estimates from the RTM implemented in the NASA Goddard Earth Observing System (GEOS) were evaluated against the nearly four-year record of daily T(sub b) observations collected by L-band radiometers onboard the Aquarius satellite. Statistics between the modeled and observed T(sub b) were computed over North America as a function of soil hydraulic properties and vegetation types. Overall, statistics showed good agreement between the modeled and observed T(sub b) with a relatively low, domain-average bias (0.79 K (ascending) and -2.79 K (descending)), root mean squared error (11.0 K (ascending) and 11.7 K (descending)), and unbiased root mean squared error (8.14 K (ascending) and 8.28 K (descending)). In terms of soil hydraulic parameters, large porosity and large wilting point both lead to high uncertainty in modeled T(sub b) due to the large variability in dielectric constant and surface roughness used by the RTM. The performance of the RTM as a function of vegetation type suggests better agreement in regions with broadleaf deciduous and needleleaf forests while grassland regions exhibited the worst accuracy amongst the five different vegetation types.

L-band radiometry↗

CARETS: An experimental regional information system using ERTS data

The U. S. Geological Survey CARETS (Central Atlantic Regional Ecological Test Site)/ERTS investigation is testing the applicability of ERTS data as input to an environmental information system for a multi-state mid-Atlantic region surrounding the Chesapeake and Delaware Bays. The information system framework encompasses a flow of information through several stages from sensor to user, and involving evaluation and feedback from several potential users. Basic assumptions of the CARETS project model are that there is a measurable environmental impact associated with land use and land use change as determined with remote sensor data, and that the ERTS derived land use data sets, when properly calibrated, may thus provide regional planners and administrators with a shortcut to an understanding of the environmental changes that are going on in their regions.

Alexander, R. H.↗

Performance Mapping Studies in Redox Flow Cells

Pumping power requirements in any flow battery system constitute a direct parasitic energy loss. It is therefore useful to determine the practical lower limit for reactant flow rates. Through the use of a theoretical framework based on electrochemical first principles, two different experimental flow mapping techniques were developed to evaluate and compare electrodes as a function of flow rate. For the carbon felt electrodes presently used in NASA-Lewis Redox cells, a flow rate 1.5 times greater than the stoichiometric rate seems to be the required minimum.

Hoberecht, M. A.↗

Profiling and Improving I/O Performance of a Large-Scale Climate Scientific Application

Exascale computing systems are soon to emerge, which will pose great challenges on the huge gap between computing and I/O performance. Many large-scale scientific applications play an important role in our daily life. The huge amounts of data generated by such applications require highly parallel and efficient I/O management policies. In this paper, we adopt a mission-critical scientific application, GEOS-5, as a case to profile and analyze the communication and I/O issues that are preventing applications from fully utilizing the underlying parallel storage systems. Through in-detail architectural and experimental characterization, we observe that current legacy I/O schemes incur significant network communication overheads and are unable to fully parallelize the data access, thus degrading applications' I/O performance and scalability. To address these inefficiencies, we redesign its I/O framework along with a set of parallel I/O techniques to achieve high scalability and performance. Evaluation results on the NASA discover cluster show that our optimization of GEOS-5 with ADIOS has led to significant performance improvements compared to the original GEOS-5 implementation.

Exascale↗

Testing a Neural Network Accelerator on a High-Altitude Balloon

The cognitive communications project has been working to re d machine learning approaches to support their deployment and sustained use in space environments. It has historically been difficult to implement such techniques on space platforms, however, due to the computational requirements they levy onto general-purpose avionics hardware. While technologies exist to accelerate the computation of aspects of neural networks, such platforms have not historically been deployed in space environments. Given that testing payloads in such environments can be both cost- and time-prohibitive, high-altitude balloons can be used as a way to approximate a space environment at a much lower cost, thus providing a cost-effective way in which to test newer approaches to hardware acceleration for artificial intelligence which may be deployed onto spacecraft more directly. This paper describes a successful test of a commercial off- the-shelf neural network accelerator on a high-altitude balloon. It begins by explaining our selection criteria when evaluating different commercial neural network acceleration techniques: primary considerations include size, weight, and power (SWaP) as well as ease of integration. Next, the paper describes the development and implementation of an experimental flight test platform: flight and ground components are discussed. Afterward, the paper discusses the experimental payload itself: this includes the experimental procedure as well as the specific image and method used for testing. Finally, the paper concludes with an evaluation of both the experimental device tested at altitude as well as the flight test framework itself, identifying how the existing platform can be used to continue tes g commercial off-the-shelf (COTS) solutions for acceleration.

Clark, Gilbert↗

Evaluating Novel Approaches for In Situ Analysis in the Search for Extraterrestrial Life

Universal approaches to detect extraterrestrial life are required to determine whether life exists outside of earth. A significant limitation of current detection strategies is the absence of a reliable, universal approach. They do not provide mechanisms to detect life that has evolved distinctly from life on Earth. We hypothesize that the most basic, essential features of life that can be detected are that they decrease their internal entropy at the expense of free energy obtained from their surroundings and that they produce heat associated with metabolic reactions. We propose that by using a thermodynamic analysis of the energy associated with these fundamental components of life, one can readily quantify the degree of free energy difference between life forms or their remnants and their abiotic surroundings. Analysis of the free energy entropy term (and thus structural complexity) and enthalpy term (as a manifestation of the energy release caused by metabolic reactions) may provide definitive detection of the presence of life. As a part of our analysis we review data available in the literature about the thermodynamics of microbial growth to assess the extent microbial metabolism and growth may contribute to negative entropy of maintenance (metabolism) and structural complexity. Additionally, we evaluate the different mechanisms by which structural complexity can be measured using calorimetry. An experimental framework is considered using concepts from previous work that could be used to further validate the calorimetric methods for life detection by potentially differentiating between life forms or their remnants from an abiotic surrounding. Results obtained demonstrate the applications of this approach and indicate that modification of a microcalorimetric technique has potential for incorporation into a space exploration platform as a life-detection payload.

Life Detection, Calorimetry↗

Developing a Customizable Composite Drought Index for Pakistan

Pakistan has experienced intense agricultural droughts in recent years, with the southern provinces experiencing the most severe drought conditions. These prolonged dry conditions often result in failed crop production, impacting families and communities. To reduce the impacts a drought may have on a community by identifying dry conditions, drought indices are used. A customizable composite drought index (CDI) is developed to improve the spatial and temporal understanding of historical agricultural droughts that have affected Pakistan. An in depth analysis using 9 input variables will provide information regarding the most important variables for differing locations. This framework can enhance drought monitoring and forecasting systems. The performance of the CDI will be evaluated by using production data from different crops that have significant economic value in Pakistan. These crops include wheat, rice, maize, cotton and barley. The expected relationship between the CDI and crop production would be to see production decrease when the CDI decreases, an indication that crops failed due to drought. The CDI is made more precise by evaluating only agricultural areas using the months of the growing season for each specific crop. This will allow for a custom drought index to be developed based on crop type and geographic location that only uses the most important variables, while still capturing the full extent of historical drought.

Caily Schwartz↗

Global Landslide Forecasting System for Hazard Assessment and Situational Awareness

Landslides triggered by extreme rainfall can be devastating, resulting in loss of life, property, and infrastructure. Landslide forecasting systems provide an opportunity to build awareness of potential hazards and ultimately take preemptive measures. There is currently a dearth of forecasting systems that provide regional or global coverage, but these systems can offer important situational awareness in data-sparse, ungauged, or large-scale catchments. A near global, primarily satellite-based system calledthe Landslide Hazard Assessment for Situational Awareness (LHASA) provides near real-time estimates of potential landslide hazard and exposure around the world. In this work, a precipitation forecast module is introduced into LHASA to complement the existing LHASA framework and provide an estimate of landslide hazard up to three days in advanceat 1km resolution. The model-based Goddard Earth Observing System-Forward Processing (GEOS-FP) precipitation forecast product is used as the forcing input for the model in place of the satellite-based Integrated Multi-satellitE Retrievals for Global Precipitation Mission product. Soil moisture and snow depth from the GEOS-FP assimilated product are also incorporated. The study period January 2020–January 2021 is usedto test the model performance against the LHASA near real-time estimates at multiple spatiotemporal scales. Validation of the model is carried out using a collection of rainfall-triggered landslide inventories from around the world as case studies to demonstrate the potential utility and limitations of this system.The rescaling of the GEOS-FP precipitation product is a critical step in incorporating the forecasted precipitation data within LHASA-Forecast (LHASA-F). Combining different streams of forecasted data within the LHASA-F framework shows promise, particularly for larger events at the 1-and 2-day lead time for events. Results indicate that for the case studies evaluated, the LHASA-F is generally able to resolve major landslide events triggered by extreme rainfall, such as from tropical cyclones. The analysis shows that landslide forecast outputs may be represented differently depending on the user’s needs. This framework serves as a first milestone in providing a global predictive view of landslide hazard.

Landslide forecasting system↗

Community Radiative Transfer Model: Implementing A New Cloud Scattering Database and Developing the Forward Radar Module

The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promises in calculating the scattering properties of particles with different shapes in the microwave frequencies. The goal of the research was to enhance the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. Given that such optical properties cannot be practically calculated on the fly, pre-computed look-up tables need to be implemented into the fast RT models to calculate the scattering properties of these particles using the DDA technique and the inputs provided by the users. Therefore, we implemented such pre-computed DDA databases into CRTM. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system will be used to evaluate the scattering improvements. Additionally, we used the backscattering information from the DDA database to implement a radar simulator into CRTM. The radar operators takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

Recent Developments in the Assimilation of Microwave and Radar Observations Into NWP Models

Microwave observations play a very important role in improving the weather forecasts. Although these observations are routinely assimilated into NWP models in clear-sky conditions, assimilation of all-sky microwave observations is very limited. Two main factors contributing to this limitation are inaccuracy in the input cloud and hydrometeor profiles used as input to the radiative transfer model and also error in scattering calculations performed by the radiative transfer model itself. The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promise in calculating the scattering properties of particles with different shapes in the microwave frequencies. This presentation focuses on recent advancements in the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system are used to evaluate the scattering improvements. Additionally, the backscattering information from the DDA database was used to implement a radar simulator into CRTM. The radar operator takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

Enhancing CRTM All-Sky Simulations and Implementation of A New Active Sensor Module

Microwave observations play a very important role in improving the weather forecasts. Although these observations are routinely assimilated into NWP models in clear-sky conditions, assimilation of all-sky microwave observations is very limited. Two main factors contributing to this limitation are inaccuracy in the input cloud and hydrometeor profiles used as input to the radiative transfer model and also error in scattering calculations performed by the radiative transfer model itself. The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promise in calculating the scattering properties of particles with different shapes in the microwave frequencies. This presentation focuses on recent advancements in the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system are used to evaluate the scattering improvements. Additionally, the backscattering information from the DDA database was used to implement a radar simulator into CRTM. The radar operator takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

CRTM Microwave Cloud Scattering Lookup Tables and Radar Simulator

Microwave observations play a very important role in improving the weather forecasts. Although these observations are routinely assimilated into NWP models in clear-sky conditions, assimilation of all-sky microwave observations is very limited. Two main factors contributing to this limitation are inaccuracy in the input cloud and hydrometeor profiles used as input to the radiative transfer model and also error in scattering calculations performed by the radiative transfer model itself. The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promise in calculating the scattering properties of particles with different shapes in the microwave frequencies. This presentation focuses on recent advancements in the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system are used to evaluate the scattering improvements. Additionally, the backscattering information from the DDA database was used to implement a radar simulator into CRTM. The radar operator takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

A Systematic Framework for Investigating Climate Impact-Drivers in NASA GISS ModelE Configurations: Towards Improving ESM Development for Impact Applications

Earth System Model (ESM) development benefits from attention to the downstream applications for nature and society. The evaluation of ESMs often focuses on a limited set of climatic impact-drivers (CIDs)1 and numerous large-scale parameters that are not directly relevant for impact analysis. To address this limitation, we propose a comprehensive framework for impact-relevant model version comparison. This framework explores both the intended and unintended consequences of shifts in model components, parameters, and resolutions, emphasizing the perspectives of the climate impacts community. We have selected a core set of CIDs as a basis for comparing model versions.

earth system models↗