Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model validation and calibration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

Studies in Software Cost Model Behavior: Do We Really Understand Cost Model Performance?

While there exists extensive literature on software cost estimation techniques, industry practice continues to rely upon standard regression-based algorithms. These software effort models are typically calibrated or tuned to local conditions using local data. This paper cautions that current approaches to model calibration often produce sub-optimal models because of the large variance problem inherent in cost data and by including far more effort multipliers than the data supports. Building optimal models requires that a wider range of models be considered while correctly calibrating these models requires rejection rules that prune variables and records and use multiple criteria for evaluating model performance. The main contribution of this paper is to document a standard method that integrates formal model identification, estimation, and validation. It also documents what we call the large variance problem that is a leading cause of cost model brittleness or instability.

data mining↗

Additional Characterization of Sonoran Pics in Support of A Stable Multi-Sensor Geostationary Imager Record

The Sonoran Desert is the most utilized Pseudo Invariant Calibration Site (PICS) located in the Americas for post-launch radiometer drift monitoring and sensor pair radiometric scaling. The Sonoran Desert is located near the US Arizona and Mexican border with an elevation of 37 m. The site has small spatial and angular variations; however, soil moisture can cause short- term periodic reflectance fluctuations. The NASA Clouds and the Earth’s Radiant Energy System (CERES) project utilizes the site to validate the GOES East and West imager calibration coefficients derived from inter-calibrating GOES with MODIS. Because the daily local noon angular solar and viewing angles repeat every year over the site, a daily exoatmospheric radiance model (DERM) can be constructed over the lifetime of a well-calibrated GOES sensor. DERM based on a well-calibrated GOES imager can then be used to determine the calibration coefficients of successive GOES imagers to provide a stable multi-sensor GOES imager record. This will ensure that the GOES retrieved clouds and derived broadband fluxes can be used to infer the regional diurnal cycle in between Terra and Aqua CERES measurements to accurately compute the daily mean regional fluxes and clouds over the 20-year CERES SYN1deg product. Although the Sonoran Desert surface reflectance and atmospheric column vary seasonally, the inter-annual variability of the seasonal cycle is small, but it adds noise to the resulting DERM calibration coefficients. We seek to reduce the uncertainty of the DERM approach by improving the clear-sky filtering and correlating the observed interannual reflectance variability with atmospheric parameters, such as precipitable water, ozone concentration, and aerosol optical depth. The additional characterization of the Sonoran Desert site DERM for both the GOES East and West positions should improve the accuracy of the CERES SYN1deg product clouds and fluxes.

Prathana Khakurel↗

Comparing Designed Training Sets to Optimize Multivariate Regression Models for Pr, Nd, and Nitric Acid Using Spectrophotometry

Chemometric regression models were developed for the quantification of praseodymium (Pr, 0–1000 µg/mL), neodymium (Nd, 0–1000 µg/mL), and nitric acid (HNO 3 , 0.1–5 M) using spectrophotometry. Designed calibration sets were composed of 20 samples each: 10 model points and 10 lack-of-fit (LOF) points. The D-optimal designs effectively minimized the number of samples required to build models, and each design resulted in similar prediction performance, suggesting that statistical design of experiments can provide a reliable framework for selecting training set samples in three-variable systems. Partial least squares regression (PLSR) models were validated against a one-factor-at-a-time validation set composed of 125 samples (three variables, five levels). The top PLS-1 models resulted in average percent root mean square error of prediction error values of 3.5%, 1.7%, and 1.2% for Pr(III), Nd(III), and HNO 3 , respectively. Power set augmentations of the model and LOF samples were investigated to optimize the number of training set samples. PLSR models built using just required model points (10) had similar predictive capabilities as models including the LOF points (20) but with fewer samples. The number of validation samples was also varied systematically to learn how many samples are needed to validate regression models. This work addresses long-standing questions in the field of chemometrics to help make this approach amenable to the near-real-time quantification of hazardous species in remote settings.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Review of a code development and calibration program in support of the aeroassist flight experiment

The code development and calibration program for the Langley Aerothermodynamic Upwind Relaxation Algorithm (LAURA) is reviewed, with emphasis directed toward support of the Aeroassist Flight Experiment (AFE). The flight project is designed to obtain data which will be used in the validation of computational fluid dynamic approximation methods. Comparisons between experimental data and numerical simulation focus on perfect-gas tests over a scale model of the AFE and on flight and ground tests which challenge some aspect of the thermochemical nonequilibrium model. In the first case, the gas model is simple, but the grid-related problems of defining the real vehicle are present. In the second case, the vehicle geometries are simple, but thermochemical processes must be modeled correctly in order to compare with the experimental data. These comparisons are described as calibration runs because they test elements of the numerical simulation, but no single data set adequately simulates the full-scale AFE flight conditions. Comparisons between computation and experiment over a broad range of data sets show generally good agreement, though some aspects of the numerical model require further development.

Gnoffo, Peter A.↗

Chemical state mapping of simulant Chernobyl lava-like fuel containing material using micro-focused synchrotron X-ray spectroscopy

Uranium speciation and redox behaviour is of critical importance in the nuclear fuel cycle. X-ray absorption near-edge spectroscopy (XANES) is commonly used to probe the oxidation state and speciation of uranium, and other elements, at the macroscopic and microscopic scale, within nuclear materials. Two-dimensional (2D) speciation maps, derived from microfocus X-ray fluorescence and XANES data, provide essential information on the spatial variation and gradients of the oxidation state of redox active elements such as uranium. In the present work, we elaborate and evaluate approaches to the construction of 2D speciation maps, in an effort to maximize sensitivity to the U oxidation state at the U L 3 -edge, applied to a suite of synthetic Chernobyl lava specimens. Our analysis shows that calibration of speciation maps can be improved by determination of the normalized X-ray absorption at excitation energies selected to maximize oxidation state contrast. The maps are calibrated to the normalized absorption of U L 3 XANES spectra of relevant reference compounds, modelled using a combination of arctangent and pseudo-Voigt functions (to represent the photoelectric absorption and multiple-scattering contributions). We validate this approach by microfocus X-ray diffraction and XANES analysis of points of interest, which afford average U oxidation states in excellent agreement with those estimated from the chemical state maps. This simple and easy-to-implement approach is general and transferrable, and will assist in the future analysis of real lava-like fuel-containing materials to understand their environmental degradation, which is a source of radioactive dust production within the Chernobyl shelter.

36 MATERIALS SCIENCE↗

LDRD for a high-current polarized-beam photogun

The electron accelerator driving the Ce+BAF conversion target requires a spin-polarized photogun capable of delivering a CW beam current of at least 1 mA. As the high beam current limits the useful operating time of the gun between photocathode reactivations due to ion back-bombardment, it is critical to increase the charge lifetime. A Jefferson Lab LDRD project aims to calibrate the simulation models needed to predict photocathode lifetime as a function of electrostatic optics and drive laser parameters in order to design a gun providing at least 1000 C charge lifetime. A prototype gun demonstrating the validity of the scaling predictions will be built and tested at the Gun Test Stand. I will describe the background and progress of this project and give an outlook on the expected performance.

Bruker, Max [Thomas Jefferson National Accelerator↗

Comments on Lambeck and Coleman - 'The earth's shape and gravity field: A report of progress from 1958 to 1982'

The accuracy and validation of global gravity models based on satellite data are discussed, responding to the statistical analysis of Lambeck and Coleman (1983) (LC). Included are an evaluation of the LC error spectra, a summary of independent-observation calibrations of the error estimates of the Goddard Earth Models (GEM) 9 and L2 (Lerch et al., 1977, 1979, 1982, 1983, and 1985), a comparison of GEM-L2 with GRIM-3B (Reigber et al., 1983), a comparison of recent models with LAGEOS laser ranging, and a summary of resonant-orbit model tests. It is concluded that the accuracy of GEMs 9, 10, and L2 is much higher than claimed by LC, that the GEMs are in good agreement with independent observations and with GRIM-3B, and that the GEM calibrations were adequate. In a reply by LC, a number of specific questions regarding the error estimates are addressed, and it is pointed out that the intermodel discrepancies of the greatest geophysical interest are those in the higher-order coefficients, not discussed in the present comment. It is argued that the differences among the geoid heights of even the most recent models are large enough to call for considerable improvements.

Lerch, F. J.↗

Flow over an all-body hypersonic aircraft - Experiment and computation

The objective of the present investigation is to establish a benchmark experimental data base for a generic hypersonic vehicle shape for validation and/or calibration of advanced computational fluid dynamics computer codes. This paper includes results from the comprehensive test program conducted in the NASA Ames 3.5-ft Hypersonic Wind Tunnel for a generic all-body hypersonic aircraft model. Experimental and computational results on flow visualization, surface pressures, surface convective heat transfer, and pilot-pressure flowfield surveys are presented. Comparisons of the experimental results with computational results from an upwind parabolized Navier-Stokes code developed at NASA Ames demonstrate the capabilities of this code.

William K. Lockman↗

Investigation of electrolyte measurement in diluted whole blood using spectroscopic and chemometric methods

The feasibility of using near-infrared (NIR) spectroscopy in combination with partial least-squares (PLS) regression was explored to measure electrolyte concentration in whole blood samples. Spectra were collected from diluted blood samples containing randomized, clinically relevant concentrations of Na+, K+, and Ca2+. Sodium was also studied in lysed blood. Reference measurements were made from the same samples using a standard clinical chemistry instrument. Partial least squares (PLS) was used to develop calibration models for each ion with acceptable results (Na+, R2 = 0.86, CVSEP = 9.5 mmol/L; K+, R2 = 0.54, CVSEP = 1.4 mmol/L; Ca2+, R2 = 0.56, CVSEP = 0.18 mmol/L). Slightly improved results were obtained using a narrower wavelength region (470-925 nm) where hemoglobin, but not water, absorbed indicating that ionic interaction with hemoglobin is as effective as water in causing measurable spectral variation. Good models were also achieved for sodium in lysed blood, illustrating that cell swelling, which is correlated with sodium concentration, is not required for calibration model development.

Non-NASA Center↗

Strain Gage Load Calibration of the Wing Interface Fittings for the Adaptive Compliant Trailing Edge Flap Flight Test

The safety-of-flight parameters for the Adaptive Compliant Trailing Edge (ACTE) flap experiment require that flap-to-wing interface loads be sensed and monitored in real time to ensure that the structural load limits of the wing are not exceeded. This paper discusses the strain gage load calibration testing and load equation derivation methodology for the ACTE interface fittings. Both the left and right wing flap interfaces were monitored; each contained four uniquely designed and instrumented flap interface fittings. The interface hardware design and instrumentation layout are discussed. Twenty-one applied test load cases were developed using the predicted in-flight loads. Pre-test predictions of strain gage responses were produced using finite element method models of the interface fittings. Predicted and measured test strains are presented. A load testing rig and three hydraulic jacks were used to apply combinations of shear, bending, and axial loads to the interface fittings. Hardware deflections under load were measured using photogrammetry and transducers. Due to deflections in the interface fitting hardware and test rig, finite element model techniques were used to calculate the reaction loads throughout the applied load range, taking into account the elastically-deformed geometry. The primary load equations were selected based on multiple calibration metrics. An independent set of validation cases was used to validate each derived equation. The 2-sigma residual errors for the shear loads were less than eight percent of the full-scale calibration load; the 2-sigma residual errors for the bending moment loads were less than three percent of the full-scale calibration load. The derived load equations for shear, bending, and axial loads are presented, with the calculated errors for both the calibration cases and the independent validation load cases.

Miller, Eric J.↗

Assessment of Satellite Radiometry in the Visible Domain

Marine reflectance and chlorophyll-a concentration are listed among the Essential Climate Variables by the Global Climate Observing System. To contribute to climate research, the satellite ocean color data records resulting from successive missions need to be consistent and well characterized in terms of uncertainties. This chapter reviews various approaches that can be used for the assessment of satellite ocean color data. Good practices for validating satellite products with in situ data and the current status of validation results are illustrated. Model-based approaches and inter-comparison techniques can also contribute to characterize some components of the uncertainty budget, while time series analysis can detect issues with the instrument radiometric characterization and calibration. Satellite data from different missions should also provide a consistent picture in scales of variability, including seasonal and interannual signals. Eventually, the various assessment approaches should be combined to create a fully characterized climate data record from satellite ocean color.

ocean color↗

The Myth of Fungible FTE: A Quantitative Assessment of Matrixed Resource Allocation

Matrix organizations allow scientific facilities to share specialized personnel across projects, operations, maintenance, and strategic initiatives. Nominal staffing allocations, however, may not capture the schedule consequences of fragmented individual commitments, limited access to specialist groups, and intermittent availability of key decision makers. We developed a stochas- tic, daily-time-step simulation of a hypothetical medium-sized accelerator-facility project com- prising sequential phases and parallel tasks. Each task requires role-specific work measured in FTE-days. Ordinary personnel may be unavailable because they contribute concurrently to other institutional activities, while designated key roles have independently specified daily un- availability probabilities. An organization-wide priority factor scales the number of people from each functional group who can effectively contribute to the project. It is interpreted as a composite proxy for project access and workforce fragmentation across competing commit- ments. We examined project completion time as a function of this factor and Project Lead unavailability using 100 Monte Carlo runs per condition. Increasing priority factor from 0.1 to 1.0 reduced median completion time from 1708.5 days (interquartile range 1681.5–1735.25) to 390 days (interquartile range 379–399). At priority factor = 0.1, increasing Project Lead unavailability from 0.5 to 0.9 increased median completion time from 1713.5 days (interquartile range 1691–1733.25) to 4,417 days (interquartile range 4271.75–4550.5). The model quantifies the commonly expected sensitivity of project schedules to fragmented resource commitments and limited coordination availability. Within this model, the results also indicate a possible threshold regime in which small increases in workforce availability yield only modest sched- ule improvements until sufficient capacity becomes accessible, after which project performance improves sharply. With further validation and calibration, this quantitative framework could support resource-allocation decisions during initial project planning and subsequent schedule rebaselining.

Bai, Mei [SLAC National Accelerator Laboratory (SL↗

MISR Calibration Issues in High-Contrast Scenes, and Empirical Corrections

We diagnose the potential causes for the Multi-angle Imaging SpectroRadiometer's (MISR) persistent high aerosol optical depth (AOD) bias at low AOD with the aid of coincident MODerate-resolution Imaging Spectroradiometer (MODIS) imagery from NASA's Terra satellite. Internal reflections within the MISR instrument are responsible for a large portion of the high AOD bias in high-contrast scenes, which are especially common as broken-cloud situations over ocean. Discrepancies between MODIS and MISR nadir-viewing near-infrared (NIR) images are used to optimize nine parameters, along with a background reflectance modulation term (that was modeled separately), to represent the observed features. Independent, surface-based AOD measurements from the AErosol RObotic NETwork (AERONET) and the Marine Aerosol Network (MAN) are compared with MISR Research Algorithm (RA) AOD retrievals for 1118 coincidences to validate the corrections when applied to the nadir and off-nadir cameras. Additionally, the calibration coefficients for the red and NIR channels used for MISR over-water aerosol retrievals were reassessed with the RA to be consistent on a camera-by-camera basis. With these corrections, plus the baseline RA corrections applied (except enhanced cloud screening), the median AOD bias in the mid-visible (green) band decreases from 0.010 to 0.002, the RMSE decreases by approx. 10%, and the slope and correlation of the MISR vs. sun photometer Ångström Exponent improves. For AOD558 nm < 0.10 and with additional cloud screening, the median bias for the RA-retrieved AOD in the green band decreases from 0.011 to 0.003, compared to ~ 0.023 for the Standard Algorithm (SA). RMSE decreases by ~ 20% compared to the baseline (uncorrected) RA and by 17-53% compared to the SA. After all corrections and cloud screening are implemented, for AOD558 nm < 0.10, which includes about half the validation data, 68% absolute AOD errors for the RA have dropped to < 0.02 (approx. 0.018).

MISR↗

Dynamic Behavior of Natural Seep Vents: Analysis of Field and Laboratory Observations and Modeling (Final Scientific/Technical Report)

In this project, we have analyzed data collected by the U.S. Department of Energy (DOE), National Energy Technology Laboratory (NETL) in a high pressure water tunnel (HPWT) and data from two research cruises to natural seeps in the Gulf of Mexico to adapt and validate a numerical model to predict the dynamics of natural seeps in the deep oceans. The HPWT data include video observations of the shrinkage rate of individual methane and natural gas bubbles under simulated deep-water conditions. Field data were collected during two cruises by the Gulf Integrated Spill Research (GISR) Consortium led by Texas A&M University and funded by the Gulf of Mexico Research Initiative (GoMRI). These data included in situ observations from a remotely operated vehicle (ROV) of gas bubbles at two natural seep sites in the Gulf and acoustic observations of the natural seep bubble flares in the ocean water column. The acoustic data were from multibeam echosounders, one mounted in a forward-looking orientation on the ROV and another mounted down-looking in the haul of the ship. All of these laboratory and field data were focused on the dynamics of natural gas bubbles at temperatures and pressures favorable for clathrate hydrate formation between the gas and water. Our analyses of this data focused on understanding the mechanisms responsible for gas bubble dissolution within the hydrate stability zone (HSZ) of the oceans. Ice-like hydrate shells may form on the bubble-water interface under these conditions, and it was unknown how this might affect the mass transfer of gas into the ocean. We were able to extract bubble shrinkage rates from the HPWT datasets. Using this data we determined that mass transfer coefficients with and without a hydrate shell match empirical values for bubbles in contaminated systems (so-called dirty bubbles contaminated by naturally occurring surfactants). We also showed that free gas, and not gas hydrate, is the dominant dissolving phase when the hydrate sub-cooling is below 11 degree Celsius (temperature difference between hydrate the hydrate formation temperature and ambient temperator) or the pressure is reducing as bubbles rise through the ocean water column. Using this mass transfer model, our numerical model of bubble dissolution matched the over 200 HPWT experiments with an average error of 10% for predicting the bubble size at the end of an experiment. From field data in the literature, we also observed that gas bubbles dissolve faster when they are initially released, following mass transfer coefficients for so-called clean-bubbles (those not yet contaminated by surfactants). Shortly after release within the HSZ, a hydrate shell forms on the bubble-water interface, and the mass transfer reduces to rates matching those of dirty bubbles. We correlated this transition time from clean to dirty bubble behavior with the initial bubble surface area and the hydrate sub-cooling. With this model for hydrate formation time and using the mass transfer coefficients deduced from the HPWT data, we validated our numerical model for predicting the rise heights of natural seep flares in the oceans. Flare heights are commonly observed in haul-mounted acoustic multibeam data. The numerical model predicts bubbles to rise high in the ocean water column owing to the slower mass transfer rates for dirty bubbles that accompany the majority of their rise time. We found that the numerical model predictions matched the observed flare heights within 5% to 10% accuracy when we compared the rise heights of the largest bubbles released from the seafloor with the bubbles acoustically visible in the multibeam data. Bubbles become acoustically transparent as they shrink to sizes of order 1 mm in diameter for the multibeam frequencies used in the field. The forward-looking multibeam on the ROV also provided data on the lateral spreading of bubbles in natural seep flares. Our analysis of this data showed that spreading follows a diffusion process, with the effective diffusivity correlating with the wobbling length scale of these ellipsoidal bubbles. When we apply this diffusivity in a random displacement model of bubble spreading, our numerical simulations match closely the lateral spread observed by the M3 in the ocean water column. Finally, we compared the seep model predictions for the acoustic properties of these natural seep plumes with that observed by the acoustic instruments in the field. The M3 and EM 302 observations were converted to relative values of target strength using a calibration we obtained in the laboratory for the M3 and using an algorithm from the manufacturer for the EM 302. Comparing the numerical seep model to these data, we obtain good agreement over the whole height of rise of these bubble flares. This further validates the numerical model. Overall, our validated seep model captures the key dynamics of gas bubbles released from natural seeps in the oceans and helps to predict the fate of methane in the water column.

03 NATURAL GAS↗

Experimental Feedback Control of Flow Induced Cavity Tones

Discrete-time, linear quadratic methods were used to design feedback controllers for reducing tones generated by flow over a cavity. The dynamics of a synthetic jet actuator mounted at the leading edge of the cavity as observed by two microphones in the cavity were modeled over a broad frequency range using state space models computed from experimental data. Variations in closed loop performance as a function of model order, control order, control bandwidth, and state estimator design were studied using a cavity in the Probe Calibration Tunnel at NASA Langley. The controller successfully reduced the levels of multiple cavity tones at the tested flow speeds of Mach 0.275, 0.35, and 0.45. In some cases, the closed loop results were limited by excitation of sidebands of the cavity tones, or the creation of new tones at frequencies away from the cavity tones. Nonetheless, the results validate the combination of optimal control and experimentally-generated state space models, and suggest this approach may be useful for other flow control problems. The models were not able to account for non-linear dynamics, such as interactions between tones at different frequencies.

Cabell, Randolph H.↗

Evaluation of vicarious calibration for airborne sensors using RadCalNet

Vicarious calibration approaches using in-situ measurements are key techniques applied to on-orbit remote sensors. The in-situ measurements have advantages in terms of traceability and quantifiable accuracy that make them ideal for inter-consistency studies of on-orbit sensors. The Committee on Earth Observation Satellites Working Group on Calibration and Validation Infrared Visible Optical Sensors initiated the Radiometric Calibration Network (RadCalNet) to provide automated surface and top-of-atmosphere (TOA) reflectance data from multiple participating ground sites to the worldwide user community. RadCalNet standardizes protocols for collecting data, processes TOA reflectance, and provides International System of Units traceable uncertainty budgets for the five currently operating sites. RadCalNet data from the Railroad Valley Playa, Nevada, site was used to provide an SI-traceable, absolute radiometric calibration of the Enhanced MODIS Airborne Simulator as part of an overflight of opportunity during the FIREX-AQ campaign. The results shown here demonstrate the utility of RadCalNet data for absolute radiometric calibration of airborne imagers making flights of opportunity without the need for coordination with a field validation team.

Sensors↗

Quantitative measurement of transverse injector and free stream interaction in a nonreacting SCRAMJET combustor using laser-induced iodine fluorescence

A preliminary quantitative study of the compressible flowfield in a steady, nonreacting model SCRAMJET combustor using laser-induced iodine fluorescence (LIIF) is reported. Measurements of density, temperature, and velocity were conducted with the calibrated, nonintrusive, optical technique for two different combustor operating conditions. First, measurements were made in the supersonic flow over a rearward-facing step without transverse injection for comparison with calculated pressure profiles. The second configuration was staged injection behind the rearward-facing step at an injection dynamic pressure ratio of 1.06. These experimental results will be used to validate computational fluid dynamic (CFD) codes being developed to model supersonic combustor flowfields.

Fletcher, D. G.↗

X-ray Inspection Model Validation with Physical Dosimetry

A non-functional printed circuit board assembly was developed using ULTIboard and a simple computational transport model was created for use with the Monte Carlo software MCNP. EBT 3 film radiography was used to compare with Monte Carlo simulation to validate the board representation. The computational transport model was then revised to include connection pins, solder balls, highly attenuating internal structures, and copper trace distribution. Results from the revised model were compared to the EBT 3 films to observe improvements to dose profiles. It was found that this method was useful in verifying the placement of components, as the dose profiles were observed to follow the same trends. The experiment was then repeated using XRQA 2 films to achieve the same level of contrast with 1% the dose of EBT 3 films. It was found that high contrast may be achieved using these films to identify major issues with the model geometry, at a cost of dose profile accuracy. A second validation method was applied to the model using 37 CaF 2 thermoluminescent dosimeters (TLDs). TLD measurements were compared with the simplified and complex transport models to identify the features that have the greatest impact on simulation accuracy. The TLD calibration to CaF 2 was found to be accurate within 5.6%, while calibration to dose in Si was found to be accurate within 4.7%. Finally, it was observed that the accurate representation of solder balls and proper modeling of highly attenuating internal structures had the greatest impact on simulation accuracy.

61 RADIATION PROTECTION AND DOSIMETRY↗