Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

Preventing Reverse Engineering of Critical Industrial Data with DIOD

Business analytics augmented by artificial intelligence and machine learning (AI/ML) have revolutionized the role of data in the modern world. In recent years, businesses have incorporated data into their decision-making process for better prediction, risk-assessment, content creation, etc. While such businesses often seek to leverage the full use of their data through third-party AI/ML services, they are often hampered by the risks of data leaks, reverse-engineering, stolen technology, etc. that often have disastrous consequences for businesses and their stakeholders alike. Thus, there arises a need for data masking prior to its transmission that obfuscates proprietary information while preserving the information relevant for AI/ML applications. In order to meet the needs of industrial data which are significantly different from those of data warehouses, previous work proposed an efficient time and space-scalable data masking paradigm known as the deceptive infusion of data (DIOD) methodology. The present work expands upon this work by leveraging existing reverse-engineering capabilities to facilitate the decomposition of industrial data into its proprietary and AI/ML-relevant parts, referred to as fundamental and inference metadata respectively. Both sets of metadata are further obfuscated in accordance with the DIOD methodology to create the DIOD rendition of the industrial data, which is rendered immune to reverse-engineering by discarding proprietary information and only preserving AI/ML-relevant information. Additionally, constraints of the original DIOD manuscript are relaxed using mutual information by configuring the methodology to the target AI/ML application to unlock the full potential of the DIOD methodology. As an example, data from a nuclear reactor is transformed into that from a nonlinear spring-mass system with different levels of data masking as required by the generic system and the target application.

97 MATHEMATICS AND COMPUTING↗

Using MODIS and GLAS Data to Develop Timber Volume Estimates in Central Siberia

The boreal forest is the Earth's largest terrestrial biome, covering some 12 million km2 and accounting for about one third of this planet's total forest area. Mapping of boreal forest's type, structure parameters and biomass are critical for understanding the boreal forest's significance in the carbon cycle, its response to and impact on global climate change. Ground based forest inventories, have much uncertainty in the inventory data, particularly in remote areas of Siberia where sampling is sparse and/or lacking. In addition, many of the forest inventories that do exist for Siberia are now a decade or more old. Thus, available forest inventories fail to capture the current conditions. Changes in forest structure in a particular forest-type and region can change significantly due to changing environment conditions, and natural and anthropogenic disturbance. Remote sensing methods can potentially overcome these problems. Multispectral sensors can be used to provide vegetation cover maps that show a timely and accurate geographic distribution of vegetation types rather than decade old ground based maps. Lidar sensors can be used to directly obtain measurements that can be used to derive critical forest structure information (e.g., height, density, and volume). These in turn can used to estimate biomass components using allometric equations without having to use out dated forest inventory. Finally, remote sensing data is ideally suited to provide a sampling basis for a rigorous statistical estimate of the variance and error bound on forest structure measures. In this study, new remote sensing methods were applied to develop estimates timber volume using NASA's MODerate resolution Imaging Spectroradiometer (MODIS) and unique waveform data of the geoscience laser altimeter system (GLAS) for a 10 deg x 10 deg area in central Siberia. Using MODIS and GLAS data, maps were produced for cover type and timber volume for 2003, and a realistic variance (error bound) for timber volume was calculated for the study area. In this 'study we used only GLAS footprints that had a slope value of less than 10 deg. This was done to avoid large errors due to the effect of slope on the GLAS models. The method requires the integration of new remote sensing methods with available ground studies of forest timber volume conducted in Russian forests. The results were compared to traditional ground forest inventory methods reported in the literature and to ground truth collected in the study area.

Ranson, K. Jon↗

Potential of Pest and Host Phenological Data in the Attribution of Regional Forest Disturbance Detection Maps According to Causal Agent

Near real time forest disturbance detection maps from MODIS NDVI phenology data have been produced since 2010 for the conterminous U.S., as part of the on-line ForWarn national forest threat early warning system. The latter has been used by the forest health community to identify and track many regional forest disturbances caused by multiple biotic and abiotic damage agents. Attribution of causal agents for detected disturbances has been a goal since project initiation in 2006. Combined with detailed cover type maps, geospatial pest phenology data offer a potential means for narrowing the candidate causal agents responsible for a given biotic disturbance. U.S. Aerial Detection Surveys (ADS) employ such phenology data. Historic ADS products provide general locational data on recent insect-induced forest type specific disturbances that may help in determining candidate causal agents for MODIS-based disturbance maps, especially when combined with other historic geospatial disturbance data (e.g., wildfire burn scars and drought maps). Historic ADS disturbance detection polygons can show severe and extensive regional forest disturbances, though they also can show polygons with sparsely scattered or infrequent disturbances. Examples will be discussed that use various historic disturbance data to help determine potential causes of MODIS-detected regional forest disturbance anomalies.

Spruce, Joseph↗

Improvements to TOVS retrievals over sea ice and applications to estimating Arctic energy fluxes

Modeling studies suggest that polar regions play a major role in modulating the Earth's climate and that they may be more sensitive than lower latitudes to climate change. Until recently, however, data from meteorological stations poleward of 70 degs have been sparse, and consequently, our understanding of air-sea-ice interaction processes is relatively poor. Satellite-borne sensors now offer a promising opportunity to observe polar regions and ultimately to improve parameterizations of energy transfer processes in climate models. This study focuses on the application of the TIROS-N operational vertical sounder (TOVS) to sea-ice-covered regions in the nonmelt season. TOVS radiances are processed with the improved initialization inversion ('3I') algorithm, providng estimates of layer-average temperature and moisture, cloud conditions, and surface characteristics at a horizontal resolution of approximately 100 km x 100 km. Although TOVS has flown continuously on polar-orbiting satellites since 1978, its potential has not been realized in high latitudes because the quality of retrievals is often significantly lower over sea ice and snow than over the surfaces. The recent availability of three Arctic data sets has provided an opportunity to validate TOVS retrievals: the first from the Coordinated Eastern Arctic Experiment (CEAREX) in winter 1988/1989, the second from the LeadEx field program in spring 1992, and the third from Russian drifting ice stations. Comparisons with these data reveal deficiencies in TOVS retrievals over sea ice during the cold season; e.g., ice surface temperature is often 5 to 15 K too warm, microwave emissivity is approximately 15% too low at large view angles, clear/cloudy scenes are sometimes misidentified, and low-level inversions are often not captured. In this study, methods to reduce these errors are investigated. Improvements to the ice surface temperature retrieval have reduced rms errors from approximately 7 K to 3 K; correction of microwaves (50 GHz) brightness temperatures for view angle dependence now allows the surface type (open water versus sea ice) to be determined; modifications to the clear/cloud tests have improved cloud detection over sea ice, especially where it is inhomogeneous; and improved surface temperature estimates have resulted in more successful diagnoses of low-level stratification. Applications of improved retrievals to studies of the Arctic energy budget are encouraging. Preliminary calculations of air-ice stress vectors, 10-m wind speed, and horizontal advection of heat and moisture are presented.

Francis, Jennifer A.↗

Blended SST fields combined SMMR/ship data (summary of technique)

All remote sensors have bias problems which vary in space and time. The infrared measurements are very sparse in cloudy areas: the SMMR has coverage limitations due to the interference of land areas and only half time operation. If one simply averaged the data from these various sources, the combination of the biases and varying sampling would introduce artifacts into the analyzed product which would reflect the sampling and which could be quite misleading. The remote sensors provide reasonable estimates of gradients of SST over length scales greater than the sensor resolution but smaller than hemispheric. Ships can be considered unbiased but are poorly distributed on small scales except in the densest shipping lanes. An analysis scheme based on a suggestion by Holl was developed which exploits just this approximately complementarity to provide an improved SST analysis. The resulting accuracy appears to be in the required approx. 0.5 C range for the particular case of using SMMR and ship data to produce two degree latitude by two degrees longitude monthly analyses.

Wilheit, T. T.↗

Recursive Dynamic Compressive Sensing in Smart Distribution Systems

With a transition to a smarter grid, we are witnessing a significant growth in smart metering infrastructure and sensor deployment in the distribution system. The underlying communication infrastructure is stressed due to the large volume of data that is generated by the smart meters/sensors. Furthermore, real time operations such as state estimation and control are impaired due to the lack of reliable aggregation of the data. In this paper, we exploit the underlying sparsity in grid data to implement two recursive dynamic compressive sensing (CS) approaches-streaming modified weighted-`1 CS and Kalman filtered CS. These approaches aim to reconstruct the sparse signal using the current underdetermined measurements and the prior information about the sparse signal and its support set. Slow signal and support change is in distribution grid data is validated using Pecan Street data. Both the IEEE 34 node test feeder system and PecanStreet data are considered as two examples to validate the superior performance of the two recursive CS techniques relative to classic CS.

Karimi, Hazhar Sufi↗

Demonstration of precise estimation of polar motion parameters with the global positioning system: Initial results

Data from the Global Positioning System (GPS) were used to determine precise polar motion estimates. Conservatively calculated formal errors of the GPS least squares solution are approx. 10 cm. The GPS estimates agree with independently determined polar motion values from very long baseline interferometry (VLBI) at the 5 cm level. The data were obtained from a partial constellation of GPS satellites and from a sparse worldwide distribution of ground stations. The accuracy of the GPS estimates should continue to improve as more satellites and ground receivers become operational, and eventually a near real time GPS capability should be available. Because the GPS data are obtained and processed independently from the large radio antennas at the Deep Space Network (DSN), GPS estimation could provide very precise measurements of Earth orientation for calibration of deep space tracking data and could significantly relieve the ever growing burden on the DSN radio telescopes to provide Earth platform calibrations.

Lichten, S. M.↗

Deep learning-accelerated 3D carbon storage reservoir pressure forecasting based on data assimilation using surface displacement from InSAR

Fast forecasting of the reservoir pressure distribution during geologic carbon storage (GCS) by assimilating monitoring data is a challenging problem. Due to high drilling cost, GCS projects usually have spatially sparse measurements from few wells, leading to high uncertainties in reservoir pressure prediction. To address this challenge, we use low-cost Interferometric Synthetic-Aperture Radar (InSAR) data as monitoring data to infer reservoir pressure build up. We develop a deep learning-accelerated workflow to assimilate surface displacement maps interpreted from InSAR and to forecast dynamic reservoir pressure. Employing an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates three-dimensional (3D) geologic properties and predicts reservoir pressure with quantified uncertainties. We use a synthetic commercial-scale GCS model with bimodally distributed permeability and porosity to demonstrate the efficacy of the workflow. A two-step CNN-PCA approach is employed to parameterize the bimodal fields. The computational efficiency of the workflow is boosted by two residual U-Net based surrogate models for surface displacement and reservoir pressure predictions, respectively. The workflow can complete data assimilation and reservoir pressure forecasting in half an hour on a personal computer.

25 ENERGY STORAGE↗

sparse_bias

This is a python package used to fit an unknown function from data that potential contains systematic biases related to metadata. The model fits the unknown function and uses a Bayesian horseshoe prior model to impose sparsity on the bias terms. This code has been generalized from research code developed for AIACHNE into a package that should have more general application in a wider class of statistical models.

Walton, Noah↗

Identifying Differential Equations in Fourier Domain (FourierIdent)

We investigate identifying differential equations in the frequency domain. Fourier analysis is an important tool in theoretical analysis and numerical solvers of differential equations, yet there is limited work in exploring this connection in the identification of differential equations. This paper aims to identify the underlying differential equation in the frequency domain, from a given single realization of the differential equation perturbed by noise. Such setting imposes difficulties which are different from other identification methods where computation is carried out in the physical domain. We propose several ways to mitigate the challenges arising from noise in data and large differences in the magnitudes of frequency responses. The main takeaways are that identifying differential equations solely in the frequency domain is challenging, the method we propose is based on a form of domain partitions in the frequency domain, and this method shows benefits for complex data even with high level of noise. We introduce a Fourier feature denoising, and define the meaningful data region and the core regions of features to reduce the effect of noise in the frequency domain and to enhance the accuracy in coefficient identification. The proposed method is tested on various differential equations with linear, nonlinear, and high-order derivative feature terms, and shows advantages on complex data with many frequency modes, even under high level of noise.

97 MATHEMATICS AND COMPUTING↗

Machine learning methods for probabilistic locked-mode predictors in tokamak plasmas

A rotating tokamak plasma can interact resonantly with the external helical magnetic perturbations, also known as error fields. This can lead to locking and then to disruptions. We leverage machine learning (ML) methods to predict the locking events. We use a coupled third-order nonlinear ordinary differential equation model to represent the interaction of the magnetic perturbation and the plasma rotation with the error field. This model is sufficient to describe qualitatively the locking and unlocking bifurcations. Here, we explore using ML algorithms with the simulation data and experimental data, focusing on the methods that can be used with sparse datasets. These methods lead to the possibility of the avoidance of locking in real-time operations. We describe the operational space in terms of two control parameters: the magnitude of the error field and the rotation frequency associated with the momentum source that maintains the plasma rotation. The outcomes are quan- tified by order parameters that completely characterize the state, whether locked or unlocked. We use unsupervised ML methods to classify locked/unlocked states and note the usefulness of a certain normalization of the order parameters. Three supervised ML classifiers are used in suite to estimate the probability of locking in the region of control parameter space with hysteresis, i.e., the set of control parameters for which both locked and unlocked states can exist. The results show that a neural network gives the best estimate of the locking probability. An analogy of the present locking model with the van der Waals equation of state is also provided.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Robust Group Subspace Recovery: A New Approach for Multi-Modality Data Fusion

Robust Subspace Recovery (RoSuRe) algorithm was recently introduced as a principled and numerically efficient algorithm that unfolds underlying Unions of Subspaces (UoS) structure, present in the data. The union of Subspaces (UoS) is capable of identifying more complex trends in data sets than simple linear models. In this work, we build on and extend RoSuRe to prospect the structure of different data modalities individually. We propose a novel multi-modal data fusion approach based on group sparsity which we refer to as Robust Group Subspace Recovery (RoGSuRe). Relying on a bi-sparsity pursuit paradigm and non-smooth optimization techniques, the introduced framework learns a new joint representation of the time series from different data modalities, respecting an underlying UoS model. We subsequently integrate the obtained structures to form a unified subspace structure. The proposed approach exploits the structural dependencies between the different modalities data to cluster the associated target objects. The resulting fusion of the unlabeled sensors’ data from experiments on audio and magnetic data has shown that our method is competitive with other state of the art subspace clustering methods. The resulting UoS structure is employed to classify newly observed data points, highlighting the abstraction capacity of the proposed method.

47 OTHER INSTRUMENTATION↗

Terrain Portrayal for Synthetic Vision Systems Head-Down Displays Evaluation Results

A critical component of SVS displays is the appropriate presentation of terrain to the pilot. At the time of this study, the relationship between the complexity of the terrain presentation and resulting enhancements of pilot SA and pilot performance had been largely undefined. The terrain portrayal for SVS head-down displays (TP-HDD) simulation examined the effects of two primary elements of terrain portrayal on the primary flight display (PFD): variations of digital elevation model (DEM) resolution and terrain texturing. Variations in DEM resolution ranged from sparsely spaced (30 arc-sec) to very closely spaced data (1 arc-sec). Variations in texture involved three primary methods: constant color, elevation-based generic, and photo-realistic, along with a secondary depth cue enhancer in the form of a fishnet grid overlay.

Hughes, Monica F.↗

Mapping Yearly Fine Resolution Global Surface Ozone through the Bayesian Maximum Entropy Data Fusion of Observations and Model Output for 1990–2017

Estimates of ground-level ozone concentrations are necessary to determine the human health burden of ozone. To support the Global Burden of Disease Study, we produce yearly fine resolution global surface ozone estimates from 1990 to 2017 through a data fusion of observations and models. As ozone observations are sparse in many populated regions, we use a novel combination of the M3Fusion and Bayesian Maximum Entropy (BME) methods. With M3Fusion, we create a multi-model composite by bias-correcting and weighting nine global atmospheric chemistry models based on their ability to predict observations (8,834 sites globally)in each region and year. BME is then used to integrate observations, such that estimates match observations at each monitoring site with the observational influence decreasing smoothly across space and time until the output matches the multi-model composite. After estimating at 0.5° resolution using BME, we add fine spatial detail from an additional model, yielding estimates at 0.1° resolution. Observed ozone is predicted more accurately (R2=0.81 at test point, 0.63 at 0.1°,0.62 at 0.5°) than the multi-model mean (R2=0.28 at 0.5°). Global ozone exposure is estimated to be increasing, driven by highly populated regions of Asia and Africa, despite decreases in the United States and Russia.

environmental modeling↗

Air Pollution Scenario over Pakistan: Characterization and Ranking of Extremely Polluted Cities using Long-Term Concentrations of Aerosols and Trace Gases

Pakistan ranks third in the world in terms of mortality attributable to air pollution, with aerosol mass concentrations (PM2.5) consistently well above WHO (World Health Organization) air quality guidelines (AQG). However, regulation is dependent on a sparse network of air quality monitoring stations and insufficient ground data. This study utilizes long-term observations of aerosols and trace gases to characterize and rank the air pollution scenarios and pollution characteristics of 80 selected cities in Pakistan. Datasets used include (1) the Aqua and Terra (AquaTerra) MODIS (Moderate Resolution Imaging Spectroradiometer) Level 2 Collection 6.1 merged Dark Target and Deep Blue (DTB) aerosol optical depth (AOD) retrieval products; (2) the CAMS (Copernicus Atmosphere Monitoring Service) reanalysis PM1, PM2.5, and PM10 data; (3) the MERRA-2 (Modern-Era Retrospective analysis for Research and Applications, Version 2) reanalysis PM2.5 data, (4) the OMI (Ozone Monitoring Instrument) tropospheric vertical column density (TVCD) of nitrogen dioxide (NO2), and VCD of sulfur dioxide (SO2) in the Planetary Boundary Layer (PBL), (5) the VIIRS (Visible Infrared Imaging Radiometer Suite) Nighttime Lights data, (6) MODIS Collection 6 Version 2 global monthly fire location data (MCD14ML), (7) population density, (8) MODIS Level 3 Collection 6 land cover types, (9) AERONET (AErosol RObotic NETwork) Version 3 Level 2.0 data, and (10) ground-based PM2.5 concentrations from air quality monitoring stations. Potential Source Contribution Function (PSCF) analyses were performed by integrating with ground-based PM2.5 concentrations and the NOAA (National Oceanic and Atmospheric Administration) HYSPLIT (Hybrid Single-Particle Lagrangian Integrated Trajectory) air parcel back trajectories to identify potential pollution source areas which are responsible for extreme air pollution in Pakistan. Results show that the ranking of the top polluted cities depends on the type of pollutant considered and the metric used. For example, Jhang, Multan, and Vehari were characterized as the top three polluted cities in Pakistan when considering AquaTerra DTB AOD products; for PM1, PM2.5, and PM10 Lahore, Gujranwala, and Okara were the top three; for tropospheric NO2 VCD Lahore, Rawalpindi, and Islamabad and for PBL SO2 VCD Lahore, Mirpur, and Gujranwala. The results demonstrate that Pakistan’s entire population has been exposed to high PM2.5 concentrations for many years, with a mean annual value of 54.7 μg/cu. m, over all Pakistan from 2003 to 2020. This value exceeds Pakistan’s National Environmental Quality Standards (Pak-NEQS, i.e., <15 μg/cu. m annual mean) for ambient air defined by the Pakistan Environmental Protection Agency (Pak-EPA) as well as the WHO Interim Target-1 (i.e., mean annual PM2.5 <35 μg/cu. m). The spatial analyses of the concentrations of aerosols and trace gases in terms of population density, nighttime lights, land cover types, and fire location data, and the PSCF analysis indicate that Pakistan’s air quality is strongly affected by anthropogenic sources inside of Pakistan, with contributions from surrounding countries. Statistically significant positive (increasing) trends in PM1, PM2.5, PM10, tropospheric NO2 VCD, and SO2 VCD were observed in ~89%, ~67%, ~48%, 91%, and ~88% of the Pakistani cities (80 cities), respectively. This comprehensive analysis of aerosol and trace gas levels, their characteristics in spatio-temporal domains, and their trends over Pakistan, is the first of its kind. Results will be helpful to the Ministry of Climate Change (Government of Pakistan), Pak-EPA, SUPARCO (Pakistan Space and Upper Atmosphere Research Commission), policymakers, and the local research community to mitigate air pollution and its effects on human health.

Muhammad Bilal↗

Comment on "Observation of Large and All-Season Ozone Losses Over the Tropics" [AIP Advances 12, 075066 (2022)]

As discussed above, and supported by extensive literature, there is no robust, credible observational evidence for substantial ozone depletion (i.e., an “ozone hole”) in the tropics. It is well known that climatological total ozone in the tropics is much lower than that in the mid-latitudes (e.g., Sahai et al., 2000; Weber et al., 2022). Satellite and ozonesonde measurements indicate a 3%–5% per decade decline of tropical lower stratosphere ozone prior to 2000, far smaller than that reported by L2022. The stronger decline reported by L2022 is caused by inappropriate use of the gap-filled version of the TOST ozone dataset, which is based on sparse tropical ozone sondes before the 1990s. This misuse of data (TOST and total column ozone) shows the importance of collaboratively engaging with groups who obtain the measurements and create climatological datasets before performing such analyses. Furthermore, the study by L2022 has multiple flaws in its discussion of atmospheric chemistry and dynamics, particularly in the proposed, and previously refuted (see Sec. III A), cosmicray- driven electron induced (CRE) mechanism. Evidence for the occurrence of tropical stratospheric clouds, as needed for the tropical CRE mechanism, is lacking, nor do CFC-12 observations show signatures of depletion in the tropical lower stratosphere, which could be associated with dissociative electron attachment-induced loss of CFC-12 on particulate matter (i.e., the CRE mechanism). Finally, it is worth reiterating that the CRE mechanism is also not responsible for polar LS ozone depletion. Polar ozone loss can be well explained by the gas phase and heterogeneous chemistry, based on extensive observations and modeling studies documented in many thousands of scientific papers on the topic [e.g., see WMO (2018) and references therein], which is not acknowledged by L2022. L2022’s research paper is a severely flawed one. There is no tropical ozone hole, and the CRE mechanism does not explain observed changes in stratospheric ozone either in the polar regions or in the tropics.

Quality assurance↗

Semantic segmentation with a sparse convolutional neural network for event reconstruction in MicroBooNE

We present the performance of a semantic segmentation network, SparseSSNet, that provides pixel-level classification of MicroBooNE data. The MicroBooNE experiment employs a liquid argon time projection chamber for the study of neutrino properties and interactions. SparseSSNet is a submanifold sparse convolutional neural network, which provides the initial machine learning based algorithm utilized in one of MicroBooNE's ν e -appearance oscillation analyses. The network is trained to categorize pixels into five classes, which are re-classified into two classes more relevant to the current analysis. The output of SparseSSNet is a key input in further analysis steps. This technique, used for the first time in liquid argon time projection chambers data and is an improvement compared to a previously used convolutional neural network, both in accuracy and computing resource utilization. Here, the accuracy achieved on the test sample is ≥ 99%. For full neutrino interaction simulations, the time for processing one image is ≈ 0.5 sec, the memory usage is at 1 GB level, which allows utilization of most typical CPU worker machine.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Sparse Matrix Approach for Simultaneous Quantification of Nystagmus and Saccade

The vestibulo-ocular reflex (VOR) consists of two intermingled non-linear subsystems; namely, nystagmus and saccade. Typically, nystagmus is analysed using a single sufficiently long signal or a concatenation of them. Saccade information is not analysed and discarded due to insufficient data length to provide consistent and minimum variance estimates. This paper presents a novel sparse matrix approach to system identification of the VOR. It allows for the simultaneous estimation of both nystagmus and saccade signals. We show via simulation of the VOR that our technique provides consistent and unbiased estimates in the presence of output additive noise.

Kukreja, Sunil L.↗