Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

The atmospheric SO2 budget for Pinatubo derived from NOAA-11 SBUV/2 spectral data

Spectral scan data from the NOAA-11 SBUV/2 instrument were used to derive SO2 for three days following the eruption of Mt. Pinatubo - June 19, July 1, and July 17, 1991. Band structure between 300 and 310 nm observed in the backscattered albedo uniquely identifies the presence of SO2. Band ratios are used to infer SO2 amounts to better accuracy (10-20 percent) and sensitivity (about 0.5 milli-atm-cm of SO2) than the TOMS retrieval, but with relatively poor spatial coverage because the measurement is nadir only. Only 7 scans showed detectable SO2 on June 19 when the cloud was still very localized. On July 1 there were 29 scans between 35N and 12S with SO2, with the highest concentration detected over the Atlantic, and on July 17 SO2 was detected in 30 scans around the world, but in decreased concentration. Estimates of the total SO2 budget made after the cloud had spread sufficiently for the sparse SBUV/2 sampling to be adequate indicated that there were 8.4 million metric tons (MMT) of SO2 in the stratosphere on July 1, 1991, and 4.1 MMT remaining on July 17. This corresponds to an e-folding time of about 24 days for the conversion of SO2 to aerosol, and is consistent with an initial injection into the stratosphere of 12-15 MMT of SO2.

Mcpeters, Richard D.↗
NASA NTRS2000

Spacebased Observations of the Oceanic Responses to Monsoons in South China Sea and Arabian Sea

A large percentage of the world's population and their agrarian economy must endure the vagaries of the monsoons over the tropical oceans between Africa and the Philippines. We know very little about the oceanic responses to changes of the monsoon in the South China Sea (SCS), which is under the influence of the East Asian Monsoon System, and the Arabian Sea (AS), which is dominated by the Indian Monsoon System; oceanic observations are sparse in both regions. Data from spaceborne microwave scatterometers and radiometers have been used to estimate the two major atmospheric forcing, momentum flux and latent heat flux (LHF), which change with the monsoon winds. Spaceborne sensors also observed the surface signatures of the oceanic response: SST and sea level changes (SLC. Sufficient durations of these data have recently become available to allow the meaningful studies of the annual cycles and interannual anomalies. In SCS, the winter monsoon is strong and steady but the summer monsoon is weak and has large intraseasonal fluctuations. In AS, the summer monsoon is much stronger than the winter monsoon. Significant correlations between LHF and SST tendency, and between curl of wind stress and SLC are found in both oceans. In the north SCS, winds are strong and dry, LHF is high, and ocean cooling is also large in fall; LHF is low and the ocean warms up in spring. In AS, LHF and SST tendency have a semi annual period; LHF is high in summer when the wind is strong and in winter when the wind is dry. Along the coast of Oman, the strong summer southwest monsoon causes intense upwelling, low SST and LHF in summer; such wind-driven SST changes is not as obvious along the Vietnam coast because of the weaker summer monsoon. The negative correlation between curl of wind stress and SLC found in the central basins of both SCS and AS agrees with a simple Ekman pumping scenario. Cyclonic winds drive surface divergence and upwelling in the ocean; the rise of the thermocline causes lower sea levels. Anticyclonic winds cause higher SLC. The exceptions (positive correlations) are found in the coastal regions in the north and the south of SCS, off the west coast of India between 5N and 10N, and along the coast of Somalia.

Xie, Xiao-Su↗
DOE OSTI2020

Network-Scale Ubiquitous Volume Estimation Using Tree-Based Ensemble Learning Methods

Currently ubiquitous volume data for roadway networks remains the key missing dimension in traffic operations. Most volume data are average annual daily traffic (AADT) measures derived from the Highway Performance Monitoring System (HPMS). Although methods to factor the AADT to hourly averages for typical day of week exist, actual volume data is limited to a sparse collection of locations in which volumes are continuously recorded. This paper/poster explores the use of state-of-art machine learning techniques to estimate accurate volume measures that span the highway network providing ubiquitous coverage in space, and point-in-time measures for a specific date and time. Three tree-based ensemble learning models, random forest (RF), gradient boost machine (GBM), and extreme gradient boost (XGBoost), were tested for volume estimation by learning from combined dataset of commercial probe data provided by TomTom, the FHWA's Travel Monitoring Analysis System (TMAS) data, and other infrastructure attributes such as number of lanes, speed limit, and weather. The methods were tested on major corridors and freeways in the metropolitan area of Denver. All three machine learning methods were able to provide hourly volume estimates 24 hours a day, 7 days a week, and 365 days a year with around 18% mean absolute error to true volume and about 5% of error with respect to roadway capacity. The low error measures allow the potential application by transportation agencies.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

A Hybrid Framework Combining Model-Based and Data-Driven Methods for Hierarchical Decentralized Robust Dynamic State Estimation

This paper combines model-based and data-driven methods to develop a hierarchical, decentralized, robust dynamic state estimator (DSE). A two-level hierarchy is proposed, where the lower level consists of robust, model-based, decentralized DSEs. The state estimates sent from the lower level are received at the upper level, where they are filtered by a robust data-driven DSE after a principled sparse selection. This selection allows us to shrink the dimension of the problem at the upper level and hence significantly speed up the computational time. The proposed hybrid framework does not depend on the centralized infrastructure of the control centers; thus it can be completely embedded into the wide-area measurement systems. This feature will ultimately facilitate the placement of hierarchical decentralized control schemes at the phasor data concentrator locations. Also, the network model is not necessary; thus, a topology processor is not required. Finally, there is no assumption on the dynamics of the electric loads. The proposed framework is tested on the 2,000-bus synthetic Texas system, and shown to be capable of reconstructing the dynamic states of the generators with high accuracy, and of forecasting in the advent of missing data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

$S$ Hmax orientation in the Alpine region from observations of stress-induced anisotropy of nonlinear elasticity

The orientation of $S$ Hmax is commonly estimated from in situ borehole breakouts and earthquake focal mechanisms. Borehole measurements are expensive, and therefore sparse, and earthquake measurements can only be made in regions with many well-characterized earthquakes. Here, we derive the stress-field orientation using stress-induced anisotropy in nonlinear elasticity. In this method, we measure the strain derivative of velocity as a function of azimuth. We use a natural pump-probe (NPP) approach which consists of measuring elastic wave speed using empirical Green’s functions (probe) at different points of the earth tidal strain cycle (pump). The approach is validated using a larger data set in the Northern Alpine Foreland region where the orientation of maximum horizontal compressive stress is known from borehole breakouts and drilling-induced fractures. The technique resolves NNW-SSW to N-S directed $S$ Hmax which is in good agreement with conventional methods and the recent crustal stress model. We confirm that the NPP method can be applied to dense large-scale seismic arrays. The technique is then applied to the Southern Alps to understand the contemporary stress pattern associated with the ongoing deformation due to counterclockwise rotation of the Adriatic plate with respect to the European plate. Our results explain why the two major faults in Northeastern Italy, the Giudicarie Fault and the Periadriatic Line (Pustertal–Gailtal Fault) are currently inactive, while the currently acting stress field allows faults in Slovenia to deform actively. We have demonstrated that the pump-probe method has the potential to fill in the measurement gap left by conventional approaches, both in terms of regional coverage and in depth.

58 GEOSCIENCES↗
NASA NTRS1989

Regression and ratio estimators to integrate AVHRR and MSS data

Regression and ratio estimators are used to integrate AVHRR-Global Area Coverage (GAC) and Landsat MSS digital data to estimate forest area in the continental United States. Forestlands are enumerated for the 48 contiguous states using five different AVHRR-GAC data sets. Results indicated that the GAC and MSS forest estimates were not highly correlated. Although the ratio of means and linear regression corrections were, on the average, closer to national U.S. Forest Service forest area estimates, these correction procedures did not consistently improve GAC estimates of forest area. GAC forest area estimates tended to be high in densely forested regions such as the northeast and low in sparsely forested areas.

Nelson, Ross↗
NASA NTRS2011

Estimation of Surface Air Temperature Over Central and Eastern Eurasia from MODIS Land Surface Temperature

Surface air temperature (T(sub a)) is a critical variable in the energy and water cycle of the Earth.atmosphere system and is a key input element for hydrology and land surface models. This is a preliminary study to evaluate estimation of T(sub a) from satellite remotely sensed land surface temperature (T(sub s)) by using MODIS-Terra data over two Eurasia regions: northern China and fUSSR. High correlations are observed in both regions between station-measured T(sub a) and MODIS T(sub s). The relationships between the maximum T(sub a) and daytime T(sub s) depend significantly on land cover types, but the minimum T(sub a) and nighttime T(sub s) have little dependence on the land cover types. The largest difference between maximum T(sub a) and daytime T(sub s) appears over the barren and sparsely vegetated area during the summer time. Using a linear regression method, the daily maximum T(sub a) were estimated from 1 km resolution MODIS T(sub s) under clear-sky conditions with coefficients calculated based on land cover types, while the minimum T(sub a) were estimated without considering land cover types. The uncertainty, mean absolute error (MAE), of the estimated maximum T(sub a) varies from 2.4 C over closed shrublands to 3.2 C over grasslands, and the MAE of the estimated minimum Ta is about 3.0 C.

Shen, Suhung↗
DOE OSTI2022

Dynamic signal recovery in distribution grids using compressive lossy measurements

Distribution system state estimation requires reliable aggregation of the measured data. However, the large volume of the measured data imposes a significant stress on the underlying communication infrastructure. With the challenges associated with measurement availability, current distribution systems are typically unobservable. To cope with the unobservability issue, compressive sensing theory allows us to recover system state information from a small number of measurements provided the states of the distribution system exhibit sparsity. In this paper, we evaluate the robustness of an updated Kalman filtered modified compressive sensing (KF-ModCS) technique that dynamically estimates the grid states using a small fraction of measured data. In practice, measurements used for sparsity based state estimation may also be intermittent due to communication network induced losses. Further, to understand the effect of packet losses on KF-ModCS, we provide an upper bound for the expected variances of the state estimation error for a given rate of information loss. This upper bound is further improved if the support set of the sparse signal that characterizes the state dynamics does not change over time and/or the reduced model is observable. Simulations based on two practical data sets collected from actual customers in a distribution grid validate the theoretical results.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2010

Transfer of Satellite Rainfall Uncertainty from Gauged to Ungauged Regions at Regional and Seasonal Timescales

Hydrologists and other users need to know the uncertainty of the satellite rainfall data sets across the range of time/space scales over the whole domain of the data set. Here, uncertainty' refers to the general concept of the deviation' of an estimate from the reference (or ground truth) where the deviation may be defined in multiple ways. This uncertainty information can provide insight to the user on the realistic limits of utility, such as hydrologic predictability, that can be achieved with these satellite rainfall data sets. However, satellite rainfall uncertainty estimation requires ground validation (GV) precipitation data. On the other hand, satellite data will be most useful over regions that lack GV data, for example developing countries. This paper addresses the open issues for developing an appropriate uncertainty transfer scheme that can routinely estimate various uncertainty metrics across the globe by leveraging a combination of spatially-dense GV data and temporally sparse surrogate (or proxy) GV data, such as the Tropical Rainfall Measuring Mission (TRMM) Precipitation Radar and the Global Precipitation Measurement (GPM) mission Dual-Frequency Precipitation Radar. The TRMM Multi-satellite Precipitation Analysis (TMPA) products over the US spanning a record of 6 years are used as a representative example of satellite rainfall. It is shown that there exists a quantifiable spatial structure in the uncertainty of satellite data for spatial interpolation. Probabilistic analysis of sampling offered by the existing constellation of passive microwave sensors indicate that transfer of uncertainty for hydrologic applications may be effective at daily time scales or higher during the GPM era. Finally, a commonly used spatial interpolation technique (kriging), that leverages the spatial correlation of estimation uncertainty, is assessed at climatologic, seasonal, monthly and weekly timescales. It is found that the effectiveness of kriging is sensitive to the type of uncertainty metric, time scale of transfer and the density of GV data within the transfer domain. Transfer accuracy is lowest at weekly timescales with the error doubling from monthly to weekly.However, at very low GV data density (<20% of the domain), the transfer accuracy is too low to show any distinction as a function of the timescale of transfer.

Tang, Ling↗
NASA NTRS1979

Results of a statistical approach to rainfall estimation using Nimbus 5 6.7 micrometers and 11.5 micrometers THIR data

Nimbus 5 6.7 mm and 11.5 mm temperature humidity infrared radiometer (THIR) data were used in a simple multiple regression scheme to test the feasibility of using these data to estimate hourly rainfall. Throughout the test area (85 W to 105 W and 45 N to 30 N) subareas (8 deg x 6 deg) were chosen from which point to point and areal statistics were obtained. Four subsets of data were used. The first consisted of only those surface stations indicating precipitation whose latitude and longitude coincided with the THIR grid points. A second used surface stations 0.1 degree from the THIR grid points. The third was a combination of subsets one and two. A reciprocal distance weighting scheme was used to derive precipitation values in data sparse areas. A fourth subset was made using these data combined with the data from subsets one and two. Point estimates resulted in negative correlations between estimated and grid derived "surface" precipitation. One degree areal estimates showed a slight improvement with a correlation coefficient of approximately 0.11. Single regression areal estimates resulted in correlations of approximately 0.11 and 0.20 for the 6.7 mm and 11.5 mm data respectively. These poor results were attributed to problems which are inherent in the satellite data (location errors, short temporal span of data, wavelength of sensors, etc.) and the lack of sufficient surface data to better verify the satellite estimate.

Ormsby, J. P.↗
DOE OSTI2022

Physics-Informed Gaussian Process Regression for States Estimation and Forecasting in Power Grids

Real-time state estimation and forecasting are critical for the efficient operation of power grids. In this paper, a physics-informed Gaussian process regression (PhI-GPR) method is presented and used for forecasting and estimating the phase angle, angular speed, and wind mechanical power of a three-generator power grid system using sparse measurements. In standard data-driven Gaussian process regression (GPR), parameterized models for the prior statistics are fit by maximizing the marginal likelihood of observed data. In the PhI-GPR method, we propose to compute the prior statistics offline by solving stochastic differential equations (SDEs) governing the power grid dynamics. The short-term forecast of a power grid system dominated by wind generation is complicated by the stochastic nature of the wind and the resulting uncertainty in wind mechanical power. Here, we assume that the power grid dynamics are governed by swing equations, with the wind mechanical power fluctuating randomly in time. We solve these equations for the mean and covariances of the power grid states using the Monte Carlo simulation method. We demonstrate that the proposed PhI-GPR method can accurately forecast and estimate observed and unobserved states. For the considered problem, PhI-GPR has computational advantages over the ensemble Kalman filter (EnKF) method: In PhI-GPR, ensembles are computed offline and independently of the data acquisition process, whereas for EnFK, ensembles are computed online with data acquisition, rendering real-time forecast more challenging. We also demonstrate that the PhI-GPR forecast is more accurate than the EnKF forecast when the random mechanical wind power is non-Markovian. In contrast, the two methods produce similar forecasts for the Markovian mechanical wind power. For observed states, we show that PhI-GPR provides a forecast comparable to the standard data-driven GPR; both forecasts are significantly more accurate than the autoregressive integrated moving average (ARIMA) forecast. We also show that the ARIMA forecast is more sensitive to observation frequency and measurement errors than the PhI-GPR forecast.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2022

Remote sensing-based vegetation and soil moisture constraints reduce irrigation estimation uncertainty

Understanding the human water footprint and its impact on the hydrological cycle is essential to inform water management under climate change. Despite efforts in estimating irrigation water withdrawals in earth system models, uncertainties and discrepancies exist within and across modeling systems conditioned by model structure, irrigation parameterization, and the choice of input datasets. Achieving model reliability could be much more challenging for data-sparse regions, given limited access to ground truth for parameterization and validation. Here, we demonstrate the potential of utilizing remotely sensed vegetation and soil moisture observations in constraining irrigation estimation in the Noah-MP land surface model. Results indicate that the two constraints together can effectively reduce model sensitivity to the choice of irrigation parameterization by 7%–43%. It also improves the characterization of the spatial patterns of irrigation and its impact on evapotranspiration and surface soil moisture by correcting for vegetation conditions and irrigation timing. This study highlights the importance of utilizing remotely sensed soil moisture and vegetation measurements in detecting irrigation signals and correcting for vegetation growth. Integrating the two remote sensing datasets into the model provides an effective and less feature engineered approach to constraining the uncertainty of irrigation modeling. Such strategies can be potentially transferred to other modeling systems and applied to regions across the globe.

Wanshu Nie↗
NASA NTRS1994

Comparison of the seasonal and interannual variability of phytoplankton pigment concentrations in the Peru and California Current systems

Monthly composite images from the global coastal zone color scanner (CZCS) data set are used to provide an initial illustration and comparison of seasonal and interannual variability of phytoplankton pigment concentration along the western coasts of South and North America in the Peru Current system (PCS) and California Current system (CCS). The analysis utilizes the entire time series of available data (November 1978 to June 1986) to form a mean annual cycle and an index of interannual variability for a series of both latitudinal and cross-shelf regions within each current system. Within 100 km of the coast, the strongest seasonal cycles in the CCS are in two regions, one between 34 deg and 45 deg N and the second between 24 deg and 29 deg N, each with maximum concentrations (greater than 3.0 mg m(exp-3)) in May-June. Weaker seasonal variability is present north of 45 deg N and in the Southern California Bight region (32 deg N). Within the PCS, in the same 100-km-wide coastal region, highest (greater than 45 deg S) and lowest (less than 20 deg S) latitude regions have a similar seasonal cycle with maximum concentrations (greater than 1.5 mg m(exp -3)) during the austral spring, summer, and fall, matching that evident throughout the CCS. Between these regions, off northern and central Chile, the seasonal maximum occurs during July-August (austral winter), contrary to the influence of upwelling favorable winds. Within the CCS, the dominant feature of interannual variability in the 8-year time series is a strong negative concentration anomaly in 1983, an El Nino year. The relative value of this negative anomaly is strongest off central California and is followed by an even stronger negative anomaly is strongest off central California and is followed by an even stronger negative anomaly in 1984 off Baja, California. In the PCS, strong negative anomalies during the 1982-1983 El Nino period are evident only off the Peruvian coast and are evident there only in the regions 100 km or more from the coast. Although negative anomalies associated with the El Nino were not present at higher latitudes (more than approximately 20 deg S) in the PCS, the extremely sparse sampling weakens our confidence in the results of the interannual analysis in this region. An upper estimate of the systematic winter bias remaining in the global CZCS data after reprocessing with the multiple scattering algorithm is given in the appendix.

Thomas, A. C.↗
NASA NTRS2010

Potential Utility of the Real-Time TMPA-RT Precipitation Estimates in Streamflow Prediction

We investigate the potential utility of the real-time Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA-RT) data for streamflow prediction, both through direct comparisons of TMPA-RT estimates with a gridded gauge product, and through evaluation of streamflow simulations over four tributaries of La Plata Basin (LPB) in South America using the two precipitation products. Our assessments indicate that the relative accuracy and the hydrologic performance of TMPA-RT-based streamflow simulations generally improved after February 2005. The improvements in TMPA-RT since 2005 are closely related to upgrades in the TMPA-RT algorithm in early February, 2005 which include use of additional microwave sensors (AMSR-E and AMSU-B) and implementation of different calibration schemes. Our work suggests considerable potential for hydrologic prediction using purely satellite-derived precipitation estimates (no adjustments by in situ gauges) in parts of the globe where in situ observations are sparse.

Su, Fengge↗
NASA NTRS2016

Global Estimates of Fine Particulate Matter Using a Combined Geophysical-Statistical Method with Information from Satellites, Models, and Monitors

We estimated global fine particulate matter (PM(sub 2.5)) concentrations using information from satellite-, simulation- and monitor-based sources by applying a Geographically Weighted Regression (GWR) to global geophysically-based satellite-derived PM(sub 2.5) estimates. Aerosol optical depth from multiple satellite products (MISR, MODIS Dark Target, MODIS and SeaWiFS Deep Blue, and MODIS MAIAC) was combined with simulation (GEOS-Chem) based upon their relative uncertainties as determined using ground-based sun photometer (AERONET) observations for 1998−2014. The GWR predictors included simulated aerosol composition and land use information. The resultant PM(sub 2.5) estimates were highly consistent (R(sup 2) equals 0.81) with out-of-sample cross-validated PM(sub 2.5) concentrations from monitors. The global population-weighted annual average PM(sub 2.5) concentrations were 3-fold higher than the 10 micrograms per cubic meter WHO guideline, driven by exposures in Asian and African regions. Estimates in regions with high contributions from mineral dust were associated with higher uncertainty, resulting from both sparse ground-based monitoring, and challenging conditions for retrieval and simulation. This approach demonstrates that the addition of even sparse ground-based measurements to more globally continuous PM(sub 2.5) data sources can yield valuable improvements to PM(sub 2.5) characterization on a global scale.

aerosols↗
NASA NTRS2017

Effective Assimilation of SMAP Observations Using Statistical Techniques

Statistical techniques permit the retrieval of soil moisture estimates in a model climatology while retaining the spatial and temporal signatures of the satellite observations. As a consequence, the need for bias correction prior to an assimilation of these estimates is reduced, which could result in a more effective use of the independent information provided by the satellite observations. In this study, a statistical neural network (NN) retrieval algorithm is calibrated using SMAP brightness temperature observations and modeled soil moisture estimates (similar to those used to calibrate the SMAP Level 4 DA system). Daily values of surface soil moisture are estimated using the NN and then assimilated into the NASA Catchment model. The skill of the assimilation estimates is assessed based on a comprehensive comparison to in situ measurements from the SMAP core and sparse network sites as well as the International Soil Moisture Network. The NN retrieval assimilation is found to significantly improve the model skill, particularly in areas where the model does not represent processes related to agricultural practices. Additionally, the NN method is compared to assimilation experiments using traditional bias correction techniques. The NN retrieval assimilation is found to more effectively use the independent information provided by SMAP resulting in larger model skill improvements than assimilation experiments using traditional bias correction techniques.

Kolassa, J.↗
NASA NTRS2021

Trend Detection of Atmospheric Time Series: Incorporating Appropriate Uncertainty Estimates and Handling Extreme Events

This paper is aimed at atmospheric scientists without formal training in statistical theory. Its goal is to, 1) provide a critical review of the rationale for trend analysis of the time series typically encountered in the field of atmospheric chemistry; 2) describe a range of trend-detection methods; and 3) demonstrate effective means of conveying the results to a general audience. Trend detections in atmospheric chemical composition data are often challenged by a variety of sources of uncertainty, which often behave differently to other environmental phenomena such as temperature, precipitation rate, or stream flow, and may require specific methods depending on the science questions to be addressed. Some sources of uncertainty can be explicitly included in the model specification, such as autocorrelation and seasonality, but some inherent uncertainties are difficult to quantify, such as data heterogeneity and measurement uncertainty due to the combined effect of short- and long-term natural variability, instrumental stability, and aggregation of data from sparse sampling frequency. Failure to account for these uncertainties might result in an inappropriate inference of the trends and their estimation errors. On the other hand, the variation in extreme events might be interesting for different scientific questions, for example, the frequency of extremely high surface ozone events and their relevance to human health. In this study we aim to, 1) review trend detection methods for addressing different levels of data complexity in different chemical species; 2) demonstrate that the incorporation of scientifically interpretable covariates can outperform pure numerical curve fitting techniques in terms of uncertainty reduction and improved predictability; 3) illustrate the study of trends based on extreme quantiles that can provide insight beyond standard mean or median based trend estimates; and 4) present an advanced method of quantifying regional trends based on the inter-site correlations of multi-site data. All demonstrations are based on time series of observed trace gases relevant to atmospheric chemistry, but the methods can be applied to other environmental data sets.

Trace gas↗