SEARCH · Engineering Papers

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2021

A review of imputation strategies for isobaric labeling-based shotgun proteomics

The throughput efficiency and increased depth of coverage provided by isobaric-labeled proteomics measurements have led to increased usage of these techniques. However, the structure of missing data is uniquely different than unlabeled studies. In this review, we compare the efficacy of nine imputation methods on a CPTAC proteomics iTRAQ dataset. Imputation methods were evaluated with regard to accuracy, variability, statistical hypothesis test inference and run time over datasets consisting of varying number of iTRAQ plexes and percentages of missing data. In general, expectation maximization and random forest imputation methods yielded the best performances, and constant-based methods performed poorly consistently across all dataset sizes and percentages of missing values. For datasets with small sample sizes and higher percentages of missing data, results indicate that statistical inference with no imputation may be preferable. Based on the findings in this review, there are core imputation methods that perform higher for isobaric-labeled proteomics data, but great care and consideration as to whether imputation should be used should be given for datasets comprised of a small number of samples, as well as to factors such as computational time and reproducibility of imputation values.

Bramer, Lisa M.↗

NASA NTRS2013

The CACAO Method for Smoothing, Gap Filling, and Characterizing Seasonal Anomalies in Satellite Time Series

Consistent, continuous, and long time series of global biophysical variables derived from satellite data are required for global change research. A novel climatology fitting approach called CACAO (Consistent Adjustment of the Climatology to Actual Observations) is proposed to reduce noise and fill gaps in time series by scaling and shifting the seasonal climatological patterns to the actual observations. The shift and scale CACAO parameters adjusted for each season allow quantifying shifts in the timing of seasonal phenology and inter-annual variations in magnitude as compared to the average climatology. CACAO was assessed first over simulated daily Leaf Area Index (LAI) time series with varying fractions of missing data and noise. Then, performances were analyzed over actual satellite LAI products derived from AVHRR Long-Term Data Record for the 1981-2000 period over the BELMANIP2 globally representative sample of sites. Comparison with two widely used temporal filtering methods-the asymmetric Gaussian (AG) model and the Savitzky-Golay (SG) filter as implemented in TIMESAT-revealed that CACAO achieved better performances for smoothing AVHRR time series characterized by high level of noise and frequent missing observations. The resulting smoothed time series captures well the vegetation dynamics and shows no gaps as compared to the 50-60% of still missing data after AG or SG reconstructions. Results of simulation experiments as well as confrontation with actual AVHRR time series indicate that the proposed CACAO method is more robust to noise and missing data than AG and SG methods for phenology extraction.

LAI↗

DOE OSTI2020

A Pattern-Recognition-Based Ensemble Data Imputation Framework for Sensors from Building Energy Systems

Building operation data are important for monitoring, analysis, modeling, and control of building energy systems. However, missing data is one of the major data quality issues, making data imputation techniques become increasingly important. There are two key research gaps for missing sensor data imputation in buildings: the lack of customized and automated imputation methodology, and the difficulty of the validation of data imputation methods. In this paper, a framework is developed to address these two gaps. First, a validation data generation module is developed based on pattern recognition to create a validation dataset to quantify the performance of data imputation methods. Second, a pool of data imputation methods is tested under the validation dataset to find an optimal single imputation method for each sensor, which is termed as an ensemble method. The method can reflect the specific mechanism and randomness of missing data from each sensor. The effectiveness of the framework is demonstrated by 18 sensors from a real campus building. The overall accuracy of data imputation for those sensors improves by 18.2% on average compared with the best single data imputation method.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS1982

Report of the subpanel on resampling functions

The state of the art and anticipated requirements for resampling in applications of remote sensing are examined. It is noted that the wide range of potential applications and expanded needs for resampling versatility in new imaging sensors leads to a concern for creating standardized test data sets and ground training sites. Several research areas are defined including the development of (1) a theory for defining an input psf of an instrument such that upon resampling of the processed data into the desired output grid the radiometric and geometric properties are a best estimate of the true upwelling radiances; (2) algorithms for estimating missing data or for ignoring missing data; (3) optimal resampling functions; (4) standard data sets; and (5) merit functions for assessing resampling techniques.

Source record↗

DOE OSTI2026

Integration of ultra-low coverage whole-genome sequences for reconstructing the evolutionary history of Galapagos giant tortoises

Genomic data from contemporary and historical samples often need to be coupled for evolutionary reconstructions of multitaxon complexes. However, the genetic data recovered from historical samples may result only in ultra-low coverage whole-genome sequences (ulcWGS; <0.15× depth), leading to inaccurate evolutionary inferences given a preponderance of missing data. Using the Galapagos giant tortoise radiation as a study system (Chelonoidis spp., composed of 13 extant and four extinct lineages), we assembled a novel methodological pipeline that removes potential noise introduced by the missing data and enhances the evolutionary signal from ulcWGS samples. We leveraged existing tools for phylogenomic placement (EPA-ng), population genomic structure (smartsnp) and admixture (Admixfrog, NGSadmix) to demonstrate that the evolutionary history of samples can be uncovered with sequencing depths as low as 0.008–0.139×. Importantly, these approaches do not use genotype imputation of the ulcWGS samples, which would require extensive reference datasets. Our application to two cases of extinct lineages of Galapagos giant tortoises, with and without references from the same lineage, demonstrates the general value of the approach. We confirm where the extinct lineages from San Cristóbal and Santa Fe islands fit into the Galapagos giant tortoise radiation, and that these lineages were evolutionarily distinct entities.

ancient DNA↗

DOE OSTI2024

Electric Field Data from the North Slope of Alaska

File contains raw as well as calibrated electric field values (V/m) from two CS110 electric field meters. The file also contains raw 2-m U and V winds at the same site. The sampling rate is 1Hz. All files contain raw electric field data, as well as calibrated values to a one week ground-based calibration setup. During this week, simultaneous upward-facing ground based electric field measurements were taken alongside the two downward facing instruments on the pole (CS110-3 at 2m and CS110-2 at 5m). Slopes and intercepts were calculated and applied to the values influenced by the pole to convert the raw data to calibrated values. These calibrated values are absolute electric field measurements. There are no missing data codes. The missong data is not included in the files, and thus any files with missing data will be shorter than the normal 86,400 values. In July 2023, the CS110-3 was replaced with an identical instrument named CS110-8, and the CS110-2 was replaced with an identical instrument called CS110-9. All the variable names remained consistent and the same.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2010

Confidence-Based Feature Acquisition

Confidence-based Feature Acquisition (CFA) is a novel, supervised learning method for acquiring missing feature values when there is missing data at both training (learning) and test (deployment) time. To train a machine learning classifier, data is encoded with a series of input features describing each item. In some applications, the training data may have missing values for some of the features, which can be acquired at a given cost. A relevant JPL example is that of the Mars rover exploration in which the features are obtained from a variety of different instruments, with different power consumption and integration time costs. The challenge is to decide which features will lead to increased classification performance and are therefore worth acquiring (paying the cost). To solve this problem, CFA, which is made up of two algorithms (CFA-train and CFA-predict), has been designed to greedily minimize total acquisition cost (during training and testing) while aiming for a specific accuracy level (specified as a confidence threshold). With this method, it is assumed that there is a nonempty subset of features that are free; that is, every instance in the data set includes these features initially for zero cost. It is also assumed that the feature acquisition (FA) cost associated with each feature is known in advance, and that the FA cost for a given feature is the same for all instances. Finally, CFA requires that the base-level classifiers produce not only a classification, but also a confidence (or posterior probability).

Wagstaff, Kiri L.↗

DOE OSTI2017

A cross-sectional study of the temporal evolution of electricity consumption of six commercial buildings

Current approaches to building efficiency diagnoses include conventional energy audit techniques that can be expensive and time consuming. In contrast, virtual energy audits of readily available 15-minute-interval building electricity consumption are being explored to provide quick, inexpensive, and useful insights into building operation characteristics. A cross sectional analysis of six buildings in two different climate zones provides methods for data cleaning, population-based building comparisons, and relationships (correlations) of weather and electricity consumption. Data cleaning methods have been developed to categorize and appropriately filter or correct anomalous data including outliers, missing data, and erroneous values (resulting in < 0.5% anomalies). The utility of a cross-sectional analysis of a sample set of building's electricity consumption is found through comparisons of baseload, daily consumption variance, and energy use intensity. Correlations of weather and electricity consumption 15-minute interval datasets show important relationships for the heating and cooling seasons using computed correlations of a Time-Specific-Averaged- Ordered Variable (exterior temperature) and corresponding averaged variables (electricity consumption)(TSAOV method). The TSAOV method is unique as it introduces time of day as a third variable while also minimizing randomness in both correlated variables through averaging. This study found that many of the pair-wise linear correlation analyses lacked strong relationships, prompting the development of the new TSAOV method to uncover the causal relationship between electricity and weather. We conclude that a combination of varied HVAC system operations, building thermal mass, plug load use, and building set point temperatures are likely responsible for the poor correlations in the prior studies, while the correlation of time-specific-averaged-ordered temperature and corresponding averaged variables method developed herein adequately accounts for these issues and enables discovery of strong linear pair-wise correlation R values. TSAOV correlations lay the foundation for a new approach to building studies, that mitigates plug load interferences and identifies more accurate insights into weather-energy relationship for all building types. Over all six buildings analyzed the TSAOV method reported very significant average correlations per building of 0.94 to 0.82 in magnitude. Our rigorous statistics-based methods applied to 15- minute-interval electricity data further enables virtual energy audits of buildings to quickly and inexpensively inform energy savings measures.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS1980

Ozone data and mission sampling analysis

A methodology was developed to analyze discrete data obtained from the global distribution of ozone. Statistical analysis techniques were applied to describe the distribution of data variance in terms of empirical orthogonal functions and components of spherical harmonic models. The effects of uneven data distribution and missing data were considered. Data fill based on the autocorrelation structure of the data is described. Computer coding of the analysis techniques is included.

Robbins, J. L.↗

NASA NTRS2012

Dig Hazard Assessment Using a Stereo Pair of Cameras

This software evaluates the terrain within reach of a lander s robotic arm for dig hazards using a stereo pair of cameras that are part of the lander s sensor system. A relative level of risk is calculated for a set of dig sectors. There are two versions of this software; one is designed to run onboard a lander as part of the flight software, and the other runs on a PC under Linux as a ground tool that produces the same results generated on the lander, given stereo images acquired by the lander and downlinked to Earth. Onboard dig hazard assessment is accomplished by executing a workspace panorama command sequence. This sequence acquires a set of stereo pairs of images of the terrain the arm can reach, generates a set of candidate dig sectors, and assesses the dig hazard of each candidate dig sector. The 3D perimeter points of candidate dig sectors are generated using configurable parameters. A 3D reconstruction of the terrain in front of the lander is generated using a set of stereo images acquired from the mast cameras. The 3D reconstruction is used to evaluate the dig goodness of each candidate dig sector based on a set of eight metrics. The eight metrics are: 1. The maximum change in elevation in each sector, 2. The elevation standard deviation in each sector, 3. The forward tilt of each sector with respect to the payload frame, 4. The side tilt of each sector with respect to the payload frame, 5. The maximum size of missing data regions in each sector, 6. The percentage of a sector that has missing data, 7. The roughness of each sector, and 8. Monochrome intensity standard deviation of each sector. Each of the eight metrics forms a goodness image layer where the goodness value of each sector ranges from 0 to 1. Goodness values of 0 and 1 correspond to high and low risk, respectively. For each dig sector, the eight goodness values are merged by selecting the lowest one. Including the merged goodness image layer, there are nine goodness image layers for each stereo pair of mast images.

Rankin, Arturo L.↗

NASA NTRS1975

Statistical theory and methodology for remote sensing data analysis with special emphasis on LACIE

Crop proportion estimators for determining crop acreage through the use of remote sensing were evaluated. Several studies of these estimators were conducted, including an empirical comparison of the different estimators (using actual data) and an empirical study of the sensitivity (robustness) of the class of mixture estimators. The effect of missing data upon crop classification procedures is discussed in detail including a simulation of the missing data effect. The final problem addressed is that of taking yield data (bushels per acre) gathered at several yield stations and extrapolating these values over some specified large region. Computer programs developed in support of some of these activities are described.

Odell, P. L.↗

NASA NTRS2006

Equations for Scoring Rules When Data Are Missing

A document presents equations for scoring rules in a diagnostic and/or prognostic artificial-intelligence software system of the rule-based inference-engine type. The equations define a set of metrics that characterize the evaluation of a rule when data required for the antecedence clause(s) of the rule are missing. The metrics include a primary measure denoted the rule completeness metric (RCM) plus a number of subsidiary measures that contribute to the RCM. The RCM is derived from an analysis of a rule with respect to its truth and a measure of the completeness of its input data. The derivation is such that the truth value of an antecedent is independent of the measure of its completeness. The RCM can be used to compare the degree of completeness of two or more rules with respect to a given set of data. Hence, the RCM can be used as a guide to choosing among rules during the rule-selection phase of operation of the artificial-intelligence system..

James, Mark↗

NASA NTRS2007

Generating Scenarios When Data Are Missing

The Hypothetical Scenario Generator (HSG) is being developed in conjunction with other components of artificial-intelligence systems for automated diagnosis and prognosis of faults in spacecraft, aircraft, and other complex engineering systems. The HSG accepts, as input, possibly incomplete data on the current state of a system (see figure). The HSG models a potential fault scenario as an ordered disjunctive tree of conjunctive consequences, wherein the ordering is based upon the likelihood that a particular conjunctive path will be taken for the given set of inputs. The computation of likelihood is based partly on a numerical ranking of the degree of completeness of data with respect to satisfaction of the antecedent conditions of prognostic rules. The results from the HSG are then used by a model-based artificial- intelligence subsystem to predict realistic scenarios and states.

Mackey, Ryan↗

DOE OSTI2015

Genome-Wide Association Study Based on Multiple Imputation with Low-Depth Sequencing Data: Application to Biofuel Traits in Reed Canarygrass

Genotyping by sequencing allows for large-scale genetic analyses in plant species with no reference genome, but sets the challenge of sound inference in presence of uncertain genotypes. We report an imputation-based genome-wide association study (GWAS) in reed canarygrass (Phalaris arundinacea L., Phalaris caesia Nees), a cool-season grass species with potential as a biofuel crop. Our study involved two linkage populations and an association panel of 590 reed canarygrass genotypes. Plants were assayed for up to 5228 single nucleotide polymorphism markers and 35 traits. The genotypic markers were derived from lowdepth sequencing with 78% missing data on average. To soundly infer marker-trait associations, multiple imputation (MI) was used: several imputes of the marker data were generated to reflect imputation uncertainty and association tests were performed on marker effects across imputes. A total of nine significant markers were identified, three of which showed significant homology with the Brachypodium dystachion genome. Because no physical map of the reed canarygrass genome was available, imputation was conducted using classification trees. In general, MI showed good consistency with the complete-case analysis and adequate control over imputation uncertainty. A gain in significance of marker effects was achieved through MI, but only for rare cases when missing data were ,45%. In addition to providing insight into the genetic basis of important traits in reed canarygrass, this study presents one of the first applications of MI to genome-wide analyses and provides useful guidelines for conducting GWAS based on genotyping-by-sequencing data.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

Combined Mesonet and Tracker

Title: Combined Mesonet and Trackers (UNL Mobile Mesonets) Authors University of Nebraska PI: Adam Houston, UNL Professor (ahouston2@unl.edu) Mailing Address: 126 Bessey Hall P.O. Box 880340 Lincoln, NE 68588-0340 CoMeT Overview The University of Nebraska-Lincoln operates three Combined Mesonet and Tracker (CoMeTs). CoMeTs are Ford Explorers (model years 2015, 2017, and 2019) with forward-mounted suites of meteorological sensors and dual moonroofs, combining the capability of a mobile mesonet to collect near-surface observations with the capability of an unmanned aircraft systems (UAS) tracker vehicle, which enables an observer in the second row of seats to see the aircraft and maintain compliance with Federal Aviation Administration policies on UAS operation. The CoMeTs collect observations of slow temperature and humidity at ~2 m above ground level (AGL) using a Vaisala HMP155A, fast temperature at ~2 m AGL using a Campbell Scientific 109SS-L thermistor, pressure at ~2.5 m AGL using a Vaisala PTB210, wind speed and direction at ~3.25 m AGL using an R.M. Young 05103 propeller anemometer, and vehicle heading using a KVH Industries C-100 fluxgate compass (Barbieri et al. 2019). The HMP155A and 109SS-L thermistor are shielded and aspirated within a U-tube (Waugh and Frederickson 2010; Houston et al. 2016). This list of sensors is also included in the CoMeT data file metadata. Manufacturer specifications for these instruments are given in Table 1 of Hanft and Houston (2018). The reported measured quantities are summarized below.CoMeT-3 was funded through an equipment allocation included in the NSF TORUS award (AGS-1824649). Instrument Description The specific sensors included on each CoMeT are summarized in the table at the end of this section. In general each CoMeT collects observations of slow temperature and humidity at ~2 m above ground level (AGL) using a Vaisala HMP155, fast temperature at ~2 m AGL using a Campbell Scientific 109SS-L thermistor, pressure at ~2.5 m AGL using a Vaisala PTB210 barometer with a Gill pressure port, wind speed and direction at ~3.25 m AGL using an R.M. Young 05103 propeller anemometer, position using a Garmin 19x HVS receiver, and vehicle heading using a KVH Industries C-100 fluxgate compass. The HMP155 and 109SS are shielded and aspirated within a U-tube (Waugh and Frederickson 2010; Houston et al. 2016). Fast temperature and corrected RH measurements (using sensors housed within the U-tube) have a time constant of 10-12 s based on data collected across a temperature and RH shock during the CLOUD-MAP 2017 calibration/validation tests on June 26, 2017. Vehicle speed was < 10 kts for this test. CoMeT-1 CoMeT-2 CoMeT-3 Slow Temperature Slow RH Vaisala HMP155A-L20-PT Part #: 22280-7 Vaisala HMP155E Part #: E1AA11A0B1A1A0A Vaisala HMP155E Part #: E1AA11A0B1A1A0A Fast temperature Campbell Scientific 109SS-L20-PT Part #: 21448-3 Campbell Scientific 109SS-L12-PW Part #: 21448-109 Campbell Scientific 109SS-L12-PW Part #: 21448-150 Pressure Vaisala PTB-210 Part #: A1A1B Gill Pressure Port Part #: 61002 Vaisala PTB-210 Part #: A1A1B Gill Pressure Port Part #: 61002 Vaisala PTB-210 Part #: A1A1B Gill Pressure Port Part #: 61002 Wind RM Young 05103-L20-PT Part #: 18435-310 RM Young 05103-L20-PW Part #: 18435-244 RM Young 05103-L20-PW Part #: 18435-244 GPS Garmin GPS 19x HVS (NMEA 0183) Part #: 010-01010-00 Garmin GPS 19x HVS (NMEA 0183) Part #: 010-01010-00 Garmin GPS 19x HVS (NMEA 0183) Part #: 010-01010-00 Compass KVH C-100 Part #: 01-0177-15 KVH C-100 Part #: 01-0177-15 KVH C-100 Part #: 01-0177-15 Logger Campbell Scientific CR6-NA-XT-SW Part #: 28385-9 Campbell Scientific CR6-WIFI-XT-SW Part #: 28385-6 Campbell Scientific CR6-WIFI-XT-SW Part #: 28385-6 Data Collection and Real-Time Processing The reported measured quantities are summarized in the table below. Quantity Units Source Epoch time Seconds GPS Latitude and longitude Degrees GPS Altitude m GPS Pressure hPa PTB210 Temperature (fast) deg C 109SS-L Temperature (slow) deg C HMP155 RH (slow) % HMP155 Vehicle speed m/s GPS Vehicle heading deg C-100 and GPS In addition to the measured variables, several derived variables are calculated. Corrected/fast relative humidity (%) Relative humidity is adjusted to the fast temperature following Richardson et al. (1998) and Houston et al. (2016). Water vapor mixing ratio (g/kg) Dew point temperature (°C) Potential temperature (Kelvin) Virtual potential temperature (Kelvin) Equivalent potential temperature (Kelvin) Regular intercomparisons between all three CoMeTs were performed during TORUS 2019. Comparisons were also conducted between CoMeT-1 and CoMeT-2 during LAPSE-RATE (2018) on 14 July. In these intercomparisons, the vehicles were parked adjacent to each other aligned perpendicular to (and facing into) the wind. To minimize engine heating effects, intercomparisons were only conducted when the wind speed was >10 kts. Data Format Original data files for each deployment are saved as text files and then converted to NetCDF. NetCDF versions have units that are CF compliant and may not match the original units in the txt files. The naming convention for the NetCDF files is as follows: UNL.CoMeT3.{deployment date YYYYMMDD}.{start time of observation collection in UTC HHMM}.L2.{post-processing codes}.cdf example: UNL.CoMeT3.20190627.1931.L2.g1.f1.cdf Post-processing codes are included to track modifications to the raw data. These codes are closely connected to error flags associated with each record. Each letter corresponds to a particular instrument: g: GPS p: Barometer tf: Fast temperature ts: Slow temperature rh: Relative humidity f: Compass w: Wind monitor a: All instruments Each number corresponds to a particular post-processing action described more below. Measured and derived variables are included in the following table. Variable Heading Standard Name Units time Time seconds since 00:00:00, 01-01-1970 Alt Altitude meters lat Latitude degrees north lon Longitude degrees east fast_temp Air Temperature kelvin slow_temp Air Temperature kelvin pressure Air Pressure pascals logger_RH Relative Humidity percent calc_corr_RH Relative Humidity percent wind_speed Wind Speed meters per second wind_dir Wind From Direction degrees vehicle_dir Vehicle Direction degrees dewpoint Dew Point Temperature kelvin mixing_ratio Humidity Mixing Ratio g/g theta Air Potential Temperature kelvin theta_v Virtual Potential Temperature kelvin theta_e Equivalent Potential Temperature Kelvin error_flag The error_flag variable is a string that matches the post-processing codes listed above. All instruments will have an associated code, but will have a “0” if the datum is unchanged from the initial processed value. Error Codes The following table summarizes the error codes for data collected before 2020: Error Code Relevant CoMeT Description g1 1,2,3 Exact correction. GPS position and time reprocessed from raw data g2 1 As far as we can tell this is an exact correction to an error in the GPS time. During the correct time periods the time suddenly went backwards ~250s and stayed at this offset for 750s when it corrected itself. The offset was applied to the “time warp” period. p1 2 Approximate correction. Hole in the pressure tube connecting the pressure port to the barometer. Resulted in erroneously low air pressure measurements when the vehicle was in motion. Derived variables recalculated (dew point temperature [e depends on qv and p], water vapor mixing ratio, potential temperature, virtual potential temperature, equivalent potential temperature) a1 3 Exact correction. Missing data reprocessed from raw data a2 1 Bug fix to bias correction for ts1, ts2, and rh1: water vapor mixing ratio was off by a factor of 10 and virtual potential temperature was wrong because of this. f1 3 No correction, missing data. Fluxgate compass inoperable. Wind speed and direction calculated using GPS-derived vehicle heading instead. rh1 1 Approximate correction. Constant bias of +1.7% removed from relative humidity. Derived variables recalculated (corrected/fast relative humidity, dew point temperature, water vapor mixing ratio, virtual potential temperature, equivalent potential temperature) ts1 1 Approximate correction. Constant bias of +0.6 K removed from slow temperature. Derived variables recalculated (corrected/fast relative humidity, dew point temperature, water vapor mixing ratio, virtual potential temperature, equivalent potential temperature) ts2 1 Approximate correction. Constant bias of +1.0 K removed. Derived variables recalculated (corrected/fast relative humidity, dew point temperature, water vapor mixing ratio, virtual potential temperature, equivalent potential temperature) References Bolton, D., 1980: The Computation of Equivalent Potential Temperature. Mon. Wea. Rev., 108, 1046–1053, https://doi.org/10.1175/1520-0493(1980)108<1046:TCOEPT>2.0.CO;2. Hanft, W., and A. L. Houston, 2018: An Observational and Modeling Study of Mesoscale Air Masses with High Theta-E. Mon. Wea. Rev., 146, 2503–2524, https://doi.org/10.1175/MWR-D-17-0389.1.Wexler Houston, A. L., R. J. Laurence III, T. W. Nichols, S. Waugh, B. Argrow, and C. L. Ziegler, 2016: Intercomparison of unmanned aircraft-borne and mobile mesonet atmospheric sensors. Journal of Atmospheric and Oceanic Technology. 33, 1569-1582, doi: 10.1175/JTECH-D-15-0178.1. Lowe, P. R., 1977: An Approximating Polynomial for the Computation of Saturation Vapor Pressure. J. Applied Meteorology, 16, 100–103. Richardson, S. J., S. E. Frederickson, F. V. Brock, and J. A. Brotzge, 1998: Combination temperature and relative humidity probes: Avoiding large air temperature errors and associated relative humidity errors. Preprints, 10th Symp. On Meteorological Observations and Instrumentation, Phoenix, AZ, Amer. Meteor. Soc., 278–283. Waugh, S., and S. E. Frederickson, 2010: An improved aspirated temperature system for mobile meteorological observations, especially in severe weather. 25th Conf. on Severe Local Storms, Denver, CO, Amer. Meteor. Soc., P5.2. [Available online at https://ams.confex.com/ams/25SLS/techprogram/paper_176205.htm.]

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

On‐site surrogates for large‐scale calibration

Abstract Motivated by a computer model calibration problem from the oil and gas industry, involving the design of a honeycomb seal, we develop a new Bayesian methodology to cope with limitations in the canonical apparatus stemming from several factors. We propose a new strategy of on‐site design and surrogate modeling for a computer simulator acting on a high‐dimensional input space that, although relatively speedy, is prone to numerical instabilities, missing data, and nonstationary dynamics. Our aim is to strike a balance between data‐faithful modeling and computational tractability in a calibration framework—tailoring the computer model to a limited field experiment. Situating our on‐site surrogates within the canonical calibration apparatus requires updates to that framework. We describe a novel yet intuitive Bayesian setup that carefully decomposes otherwise prohibitively large matrices by exploiting the sparse blockwise structure. Empirical illustrations demonstrate that this approach performs well on toy data and our motivating honeycomb example.

Huang, Jiangeng↗

DOE OSTI2018

A novel low-rank matrix completion approach to estimate missing entries in Euclidean distance matrix

A Euclidean distance matrix (EDM) is a table of distance-square between points on a k-dimensional Euclidean space, with applications in many fields (e.g., engineering, geodesy, economics, genetics, biochemistry, and psychology). A problem that often arises is the absence (or uncertainty) of some EDM elements. In many situations, only a subset of all pairwise distances is available and it is desired to have some procedure to estimate the missing distances. In this paper, we address the problem of missing data in EDM through low-rank matrix completion techniques. We exploit the fact that the rank of a EDM is at most k+2 and does not depend on the number of points, which is, in general, much bigger then k. We use a singular value decomposition approach that considers the rank of the matrix to be completed and computes, in each iteration, a parameter that controls the convergence of the method. After performing a number of computational experiments, we could observe that our proposal was able to recover, with high precision, random EDMs with more than 1000 points and up to 98% of missing data in few minutes. In addition, our method required a smaller number of iterations when compared to other competitive state-of-art technique.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

QA/QC-ed BCI meteorological drivers

Time series raw datasets of Temperature, Solar Radiation, Relative Humidity, Rainfall, Barometric Pressure, Wind Speed, and Wind Direction for the Barro Colorado (PA-BCI) site for the period from 2003-01-01 to 2016-12-31 were downloaded from the STRI website. The QA/QC protocol of raw datasets included the detection and removal of NAs, outliers and bad data points, imputation of missing data, and creation of the time series with equidistant time stamps. The time series for different meteorological parameters were aligned into a single database to be used for modeling. Statistical QA/QC analysis was performed using a series of R libraries in Rstudio. The changes in the dataset were flagged. The file BCI_met_drivers_2003-2016_QAQC_summary_report.docx provides the details of the QA/QC data analysis.

54 ENVIRONMENTAL SCIENCES↗