SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66

All sources NASA NTRS DOE OSTI

NASA NTRS1995

Imaging spectrometer measurement of water vapor in the 400 to 2500 nm spectral region

The Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) measures the total upwelling spectral radiance from 400 to 2500 nm sampled at 10 nm intervals. The instrument acquires spectral data at an altitude of 20 km above sea level, as images of 11 by up to 100 km at 17x17 meter spatial sampling. We have developed a nonlinear spectral fitting algorithm coupled with a radiative transfer code to derive the total path water vapor from the spectrum, measured for each spatial element in an AVIRIS image. The algorithm compensates for variation in the surface spectral reflectance and atmospheric aerosols. It uses water vapor absorption bands centered at 940 nm, 1040 nm, and 1380 nm. We analyze data sets with water vapor abundances ranging from 1 to 40 perceptible millimeters. In one data set, the total path water vapor varies from 7 to 21 mm over a distance of less than 10 km. We have analyzed a time series of five images acquired at 12 minute intervals; these show spatially heterogeneous changes of advocated water vapor of 25 percent over 1 hour. The algorithm determines water vapor for images with a range of ground covers, including bare rock and soil, sparse to dense vegetation, snow and ice, open water, and clouds. The precision of the water vapor determination approaches one percent. However, the precision is sensitive to the absolute abundance and the absorption strength of the atmospheric water vapor band analyzed. We have evaluated the accuracy of the algorithm by comparing several surface-based determinations of water vapor at the time of the AVIRIS data acquisition. The agreement between the AVIRIS measured water vapor and the in situ surface radiometer and surface interferometer measured water vapor is 5 to 10 percent.

Green, Robert O.↗

DOE OSTI2021

Materials Fingerprinting Classification

Significant progress in many classes of materials could be made with the availability of experimentally-derived large datasets composed of atomic identities and three-dimensional coordinates. Methods for visualizing the local atomic structure, such as atom probe tomography (APT), which routinely generate datasets comprised of millions of atoms, are an important step in realizing this goal. However, state-of-the-art APT instruments generate noisy and sparse datasets that provide information about elemental type, but obscure atomic structures, thus limiting their subsequent value for materials discovery. The application of a materials fingerprinting process, a machine learning algorithm coupled with topological data analysis, provides an avenue by which here-to-fore unprecedented structural information can be extracted from an APT dataset. As a proof of concept, the material fingerprint is applied to high-entropy alloy APT datasets containing body-centered cubic (BCC) and face-centered cubic (FCC) crystal structures. A local atomic configuration centered on an arbitrary atom is assigned a topological descriptor, with which it can be characterized as a BCC or FCC lattice with near perfect accuracy, despite the inherent noise in the dataset. Finally, this successful identification of a fingerprint is a crucial first step in the development of algorithms which can extract more nuanced information, such as chemical ordering, from existing datasets of complex materials.

36 MATERIALS SCIENCE↗

DOE OSTI2021

pyDRESCALk

Modern data scientists are tasked to analyze ever-growing data sets with increasingly complex relationships. Tensor decompositions have come to play a central role in identifying underlying latent structures in higher-order data. The problem of fitting tensor models to different distributions is complicated by the combinations of size, dimensionality, and sparsity present in real world data. The situation demands efficient algorithms designed for shared-memory and distributed systems. This work will present new research that tackles these challenges on several different fronts, leveraging optimizations in numerical algorithms and sparse tensor representations in heterogeneous high performance computing environments.

Bhattarai, Manish↗

DOE OSTI2023

Cloud and Precipitation Experiment at Kennaook (Cape-K) Science Plan

The Cloud and Precipitation Experiment at Kennaook (CAPE-K) will augment ongoing measurements at the Kennaook/Cape Grim Baseline Air Pollution Station (KCG) with components of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility’s second Mobile Facility (AMF2) in a deployment that is scheduled to extend from April 2024 through September 2025. Located at Kennaook/Cape Grim on the northwestern tip of Tasmania (40.68° S, 144.69° E), KCG has produced the longest and most consequential record of Southern Hemisphere aerosol and gas-phase chemistry; however, extensive cloud and precipitation measurements have not been collected at this site. CAPE-K is motivated by the fact that model uncertainty in the surface radiation budget due to clouds and precipitation remain significant in this region, where strong latitudinal gradients in cloud feedbacks are found and there are strong and well-documented seasonal variations in marine aerosol properties. Satellite data suggest that cloud properties co-vary with the seasonal cycle in aerosol in the Southern Ocean (SO), yet detailed vertically resolved measurements of cloud and precipitation in the marine boundary layer are sparse. CAPE-K will provide the first such seasonal cycle of measurements in this important latitude band. CAPE-K will provide detailed cloud and precipitation observations over two winter seasons and align with complementary high-value-add activities planned by Australian colleagues, including a voyage of the Research Vessel Investigator (RVI) in May 2025 that will conduct measurements just offshore of KCG.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2012

Can Asian Dust Trigger Phytoplankton Blooms in the Oligotrophic Northern South China Sea?

Satellite data estimate a high dust deposition flux (approximately 18 g m(exp-2 a(exp-1) into the northern South China Sea (SCS). However, observational evidence concerning any biological response to dust fertilization is sparse. In this study, we combined long-term aerosol and chlorophyll-a (Chl-a) measurements from satellite sensors (MODIS and SeaWiFS) with a 16-year record of dust events from surface PM10 observations to investigate dust transport, flux, and the changes in Chl-a concentration over the northern SCS. Our result revealed that readily identifiable strong dust events over this region, although relatively rare (6 cases since 1994) and accounting for only a small proportion of the total dust deposition (approximately 0.28 g m(exp-2 a(exp-1), do occur and could significantly enhance phytoplankton blooms. Following such events, the Chl-a concentration increased up to 4-fold, and generally doubled the springtime background value (0.15 mg m(exp-3). We suggest these heavy dust events contain readily bioavailable iron and enhance the phytoplankton growth in the oligotrophic northern SCS.

northern South China sea↗

NASA NTRSDate not supplied

Compressive Sensing Based Data Acquisition Architecture for Transient Stellar Events in Crowded Star Fields

Compressive sensing is a mathematical technique for simultaneous data acquisition and compression. In this work, we show a CS based architecture for acquiring and reconstructing transient stellar events. This architecture recon-structs a differenced image itself, eliminating the need for any sparse domain transforms, otherwise required for traditional CS reconstruction. The resulting reconstructed differenced image is of critical importance as the information required for generating a time-series photometric light curve is obtained only from a differenced image. Hence, reconstructing a crowded star spatial image, followed by differencing is wasteful. This architecture eliminates the need to 1.) transform an image to a sparse domain, 2.) Reconstruct a dense field, and then apply differencing on the image to obtain the image of critical value. We study the case of microlensing to depict a star source experiencing magnification in time. Our results show that this architecture is able to reconstruct star source magnitudes with magnification factors greater than 1 for clean images with error less than 2% using only 10% of the Nyquist rate samples.

Asmita Korde-patel↗

NASA NTRS2021

Assessing Disaggregated SMAP Soil Moisture Products in the United States

A soil moisture (SM) disaggregation algorithm based on thermal inertia (TI) theory was implemented to downscale the Soil Moisture Active Passive (SMAP) Enhanced product(SPL2SMPE) from 9 km to 1 km over the continental United States. The algorithm applies land surface temperature and normalized difference vegetation index from Moderate Resolution Imaging Spectroradiometer (MODIS) at higher spatial resolution to estimate relative soil wetness within a coarse SMAP grid -this MODIS-derived relative wetness is then used to produce the downscaled SMAP SM. Results from the algorithm were evaluated in terms of their spatiotemporal coverage and accuracy using in situ measurements from SMAP Core Validation Sites (CVS), the US Department of Agriculture Soil Climate Analysis Network (USDA-SCAN), and the National Oceanic and Atmospheric Administration Climate Reference Network(NOAA-CRN). Results were also compared with the baselineSPL2SMPE and the SMAP/Sentinel-1 (SPL2SMAPS) 1kmproduct. Overall, the unbiased root mean square error (ubRMSE)of the disaggregated SM at the CVS using the TI approach is approximately 0.04 m3=m3, which is the SMAP mission requirement for the baseline products. The TI approach out performs the SMAP/Sentinel SL2SMAPS 1km product by approximately0.02 m3=m3. Over the agriculture/crop areas from SCAN and CRN sparse network stations, the TI approach exhibits better ubRMSE compared to SPL2SMPE andSPL2SMAPS byabout0.01 and 0.02 m3=m3, indicating its advantage in these areas. However, a drawback of this approach is that there are data gaps due to cloud cover as optical sensors cannot have a clear view of the land surface.

Soil moisture↗

DOE OSTI2022

Modeling Spatial Distribution of Snow Water Equivalent by Combining Meteorological and Satellite Data with Lidar Maps

Abstract An accurate characterization of the water content of snowpack, or snow water equivalent (SWE), is necessary to quantify water availability and constrain hydrologic and land surface models. Recently, airborne observations (e.g., lidar) have emerged as a promising method to accurately quantify SWE at high resolutions (scales of ∼100 m and finer). However, the frequency of these observations is very low, typically once or twice per season in the Rocky Mountains of Colorado. Here, we present a machine learning framework that is based on random forests to model temporally sparse lidar-derived SWE, enabling estimation of SWE at unmapped time points. We approximated the physical processes governing snow accumulation and melt as well as snow characteristics by obtaining 15 different variables from gridded estimates of precipitation, temperature, surface reflectance, elevation, and canopy. Results showed that, in the Rocky Mountains of Colorado, our framework is capable of modeling SWE with a higher accuracy when compared with estimates generated by the Snow Data Assimilation System (SNODAS). The mean value of the coefficient of determination R 2 using our approach was 0.57, and the root-mean-square error (RMSE) was 13 cm, which was a significant improvement over SNODAS (mean R 2 = 0.13; RMSE = 20 cm). We explored the relative importance of the input variables and observed that, at the spatial resolution of 800 m, meteorological variables are more important drivers of predictive accuracy than surface variables that characterize the properties of snow on the ground. This research provides a framework to expand the applicability of lidar-derived SWE to unmapped time points. Significance Statement Snowpack is the main source of freshwater for close to 2 billion people globally and needs to be estimated accurately. Mountainous snowpack is highly variable and is challenging to quantify. Recently, lidar technology has been employed to observe snow in great detail, but it is costly and can only be used sparingly. To counter that, we use machine learning to estimate snowpack when lidar data are not available. We approximate the processes that govern snowpack by incorporating meteorological and satellite data. We found that variables associated with precipitation and temperature have more predictive power than variables that characterize snowpack properties. Our work helps to improve snowpack estimation, which is critical for sustainable management of water resources.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Sparse regression for plasma physics

Many scientific problems can be formulated as sparse regression, i.e., regression onto a set of parameters when there is a desire or expectation that some of the parameters are exactly zero or do not substantially contribute. This includes many problems in signal and image processing, system identification, optimization, and parameter estimation methods such as Gaussian process regression. Sparsity facilitates exploring high-dimensional spaces while finding parsimonious and interpretable solutions. In the present work, we illustrate some of the important ways in which sparse regression appears in plasma physics and point out recent contributions and remaining challenges to solving these problems in this field. Further, a brief review is provided for the optimization problem and the state-of-the-art solvers, especially for constrained and high-dimensional sparse regression.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS2014

The Unusual Multiwavelength Properties of the Gamma-Ray Source PMN J1603-4904

Context. We investigate the nature and classification of PMNJ1603-4904, a bright radio source close to the Galactic plane, which is associated with one of the brightest hard-spectrum gamma-ray sources detected by Fermi/LAT. It has previously been classified as a low peaked BL Lac object based on its broadband emission and the absence of optical emission lines. Optical measurements, however, suffer strongly from extinction and the absence of pronounced short-time gamma-ray variability over years of monitoring is unusual for a blazar. Aims. In this paper, we are combining new and archival multi-wavelength data of PMNJ1603-4904 in order to reconsider the classification and nature of this unusual gamma-ray source. Methods. For the first time, we study the radio morphology of PMNJ1603-4904 at 8.4GHz and 22.3GHz, and its spectral properties on milliarcsecond scales, based on VLBI observations from the TANAMI program. We combine the resulting images with multi-wavelength data in the radio, IR, optical/UV, X-ray, and gamma-ray regimes. Results. PMNJ1603-4904 shows a symmetric brightness distribution at 8.4 GHz on milliarcsecond scales, with the brightest, and most compact component in the center of the emission region. The morphology is reminiscent of a compact symmetric object (CSO). Such objects, thought to be young radio galaxies, have been predicted to produce gamma-ray emission but have not been detected as a class by the Fermi gamma-ray telescope so far. Sparse (u, v)-coverage at 22.3GHz prevents an unambiguous modeling of the source morphology at this higher frequency. Moreover, infrared measurements reveal an excess in the spectral energy distribution (SED), which can be modeled with a blackbody with a temperature of about 1600 K, and which is usually not present in blazar SEDs. Conclusions. The TANAMI VLBI data and the shape of the broadband SED challenge the current blazar classification of one of the brightest gamma-ray sources in the sky. PMNJ1603-4904 seems to be either a highly peculiar BL Lac object or a misaligned jet source. In the latter case, the intriguing VLBI structure opens room for a possible classification of PMNJ1603-4904 as a gamma-ray bright CSO.

Multiwavelength↗

NASA NTRS2018

GES DISC Long-Term Data Analysis Services Using AIRS and CMS Methane Data as an Example

NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) archives and distributes rich collections of data on atmospheric greenhouse gases from multiple satellite missions and model results. Among those greenhouse gases, atmospheric methane is a powerful greenhouse gas contributing ~0.5 (W/m^2) to total radiative forcing, and its concentration has increased by about 150% since 1750. Observations or estimates of methane emissions typically have sparse spatial and temporal coverage. The lack of comprehensive spatial and temporal coverage of methane source and sink observations has made analyzing atmospheric methane trends challenging. In this study the GES DISC aims to provide the community with the resources to better understand changes in atmospheric methane concentrations and the underlying causes. We will utilize methane datasets from Atmospheric Infrared Sounder (AIRS) retrieved methane concentration and three Carbon Monitoring System (CMS) methane emission datasets (in regions of North America, Canada, and Mexico) to compare AIRS methane growth with corresponding CMS regional methane emissions. Comparisons of AIRS methane growth rates and CMS methane emissions suggests wetland emissions may impact methane growth rate trends over North America. As the record for CMS methane data is extended, both datasets can be used in conjunction to better understand impacts on atmospheric methane trends. GES DISCs new anomaly tool can also be used on select datasets to further quantify trends in atmospheric greenhouse gases.

long-term data analysis↗

DOE OSTI2024

Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects

Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.

Ren, Huiying↗

NASA NTRS2017

Full-Coverage High-Resolution Daily PM(sub 2.5) Estimation using MAIAC AOD in the Yangtze River Delta of China

Satellite aerosol optical depth (AOD) has been used to assess population exposure to fine particulate matter (PM (sub 2.5)). The emerging high-resolution satellite aerosol product, Multi-Angle Implementation of Atmospheric Correction(MAIAC), provides a valuable opportunity to characterize local-scale PM(sub 2.5) at 1-km resolution. However, non-random missing AOD due to cloud snow cover or high surface reflectance makes this task challenging. Previous studies filled the data gap by spatially interpolating neighboring PM(sub 2.5) measurements or predictions. This strategy ignored the effect of cloud cover on aerosol loadings and has been shown to exhibit poor performance when monitoring stations are sparse or when there is seasonal large-scale missngness. Using the Yangtze River Delta of China as an example, we present a Multiple Imputation (MI) method that combines the MAIAC high-resolution satellite retrievals with chemical transport model (CTM) simulations to fill missing AOD. A two-stage statistical model driven by gap-filled AOD, meteorology and land use information was then fitted to estimate daily ground PM(sub 2.5) concentrations in 2013 and 2014 at 1 km resolution with complete coverage in space and time. The daily MI models have an average R(exp 2) of 0.77, with an inter-quartile range of 0.71 to 0.82 across days. The overall Ml model 10-fold cross-validation R(exp 2) (root mean square error) were 0.81 (25 gm(exp 3)) and 0.73 (18 gm(exp 3)) for year 2013 and 2014, respectively. Predictions with only observational AOD or only imputed AOD showed similar accuracy.Comparing with previous gap-filling methods, our MI method presented in this study performed bette rwith higher coverage, higher accuracy, and the ability to fill missing PM(sub 2.5) predictions without ground PM(sub 2.5) measurements. This method can provide reliable PM(sub 2.5)predictions with complete coverage that can reduce biasin exposure assessment in air pollution and health studies.

PM2.5↗

NASA NTRS2019

Rotary-motion-extended Array Synthesis (R-MXAS)

R-MXAS is a revolutionary aerospace architecture for realizing a synthetic aperture imaging radiometer (SAIR) with dramatically lower SWaP than existing state-of-the-art (SOTA) methods. The space-based component of the RMXAS system (Figure 1) is a single platform comprising a 1-D sparse / decimated antenna array on a rigid tether (deployed parallel to the horizon) and one or more additional tethered antennas that rotate in a plane orthogonal to the 1-D array.The processing that correlates the data from these two antenna systems and performs image reconstruction has both space-based and ground-based components. The processing exploits the interferometric baselines formed between the rotating tethered antenna at radius R and each of the antennas of the 1-D array on the rigid tether.

Radiometer↗

DOE OSTI2023

symPACK: A GPU-Capable Fan-Out Sparse Cholesky Solver

Sparse symmetric positive definite systems of equations are ubiquitous in scientific workloads and applications. Parallel sparse Cholesky factorization is the method of choice for solving such linear systems. Therefore, the development of parallel sparse Cholesky codes that can efficiently run on today’s large-scale heterogeneous distributed-memory platforms is of vital importance. Modern supercomputers offer nodes that contain a mix of CPUs and GPUs. To fully utilize the computing power of these nodes, scientific codes must be adapted to offload expensive computations to GPUs. We present symPACK, a GPU-capable parallel sparse Cholesky solver that uses one-sided communication primitives and remote procedure calls provided by the UPC++ library. We also utilize the UPC++ "memory kinds" feature to enable efficient communication of GPU-resident data. We show that on a number of large problems, symPACK outperforms comparable state-of-the-art GPU-capable Cholesky factorization codes by up to 14x on the NERSC Perlmutter supercomputer.

Bellavita, Julian↗

DOE OSTI2024

Weak-form latent space dynamics identification

Recent work in data-driven modeling has demonstrated that a weak formulation of model equations enhances the noise robustness of a wide range of computational methods. In this paper, we demonstrate the power of the weak form to enhance the LaSDI (Latent Space Dynamics Identification) algorithm, a recently developed data-driven reduced order modeling technique. We introduce a weak form-based version WLaSDI (Weak-form Latent Space Dynamics Identification). WLaSDI first compresses data, then projects onto the test functions and learns the local latent space models. Notably, WLaSDI demonstrates significantly enhanced robustness to noise. With WLaSDI, the local latent space is obtained using weak-form equation learning techniques. Compared to the standard sparse identification of nonlinear dynamics (SINDy) used in LaSDI, the variance reduction of the weak form guarantees a robust and precise latent space recovery, hence allowing for a fast, robust, and accurate simulation. We demonstrate the efficacy of WLaSDI vs. LaSDI on several common benchmark examples including viscid and inviscid Burgers', radial advection, and heat conduction. For instance, in the case of 1D inviscid Burgers' simulations with the addition of up to 100% Gaussian white noise, the relative error remains consistently below 6% for WLaSDI, while it can exceed 10,000% for LaSDI. Similarly, for radial advection simulations, the relative errors stay below 15% for WLaSDI, in stark contrast to the potential errors of up to 10,000% with LaSDI. Moreover, speedups of several orders of magnitude can be obtained with WLaSDI. For example applying WLaSDI to 1D Burgers' yields a 140X speedup compared to the corresponding full order model.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

STAIR 2.0: A Generic and Automatic Algorithm to Fuse Modis, Landsat, and Sentinel-2 to Generate 10 m, Daily, and Cloud-/Gap-Free Surface Reflectance Product

Remote sensing datasets with both high spatial and high temporal resolution are critical for monitoring and modeling the dynamics of land surfaces. However, no current satellite sensor could simultaneously achieve both high spatial resolution and high revisiting frequency. Therefore, the integration of different sources of satellite data to produce a fusion product has become a popular solution to address this challenge. Many methods have been proposed to generate synthetic images with rich spatial details and high temporal frequency by combining two types of satellite datasets—usually frequent coarse-resolution images (e.g., MODIS) and sparse fine-resolution images (e.g., Landsat). In this paper, we introduce STAIR 2.0, a new fusion method that extends the previous STAIR fusion framework, to fuse three types of satellite datasets, including MODIS, Landsat, and Sentinel-2. In STAIR 2.0, input images are first processed to impute missing-value pixels that are due to clouds or sensor mechanical issues using a gap-filling algorithm. The multiple refined time series are then integrated stepwisely, from coarse- to fine- and high-resolution, ultimately providing a synthetic daily, high-resolution surface reflectance observations. We applied STAIR 2.0 to generate a 10-m, daily, cloud-/gap-free time series that covers the 2017 growing season of Saunders County, Nebraska. Moreover, the framework is generic and can be extended to integrate more types of satellite data sources, further improving the quality of the fusion product. View Full-Text

47 OTHER INSTRUMENTATION↗

DOE OSTI2021

On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

We consider the problem of reconstructing an infinite set of sparse, finite-dimensional vectors, that share a common sparsity pattern, from incomplete measurements. This is in contrast to the work (Daubechies et al., Pure Appl. Math. 57(11), 1413–1457, 2004), where the single vector signal can be infinite-dimensional, and (Fornasier and Rauhut, SIAM J. Numer. Anal. 46(2), 577613, 2008), which extends the aforementioned work to the joint sparse recovery of finite number of infinite-dimensional vectors. In our case, to take account of the joint sparsity and promote the coupling of nonvanishing components, we employ a convex relaxation approach with mixed norm penalty ℓ 2,1 . This paper discusses the computation of the solutions of linear inverse problems with such relaxation by a forward-backward splitting algorithm. However, since the solution matrix possesses infinitely many columns, the arguments of Daubechies et al. (Pure Appl. Math. 57(11), 1413–1457, 2004) no longer apply. As such, we establish new strong convergence results for the algorithm, in particular when the set of jointly sparse vectors is infinite.

97 MATHEMATICS AND COMPUTING↗