Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Validation of the Cloud_CCI (Cloud Climate Change Initiative) cloud products in the Arctic

The role of clouds in the Arctic radiation budget is not well understood. Ground-based and airborne measurements provide valuable data to test and improve our understanding. However, the ground-based measurements are intrinsically sparse, and the airborne observations are snapshots in time and space. Passive remote sensing measurements from satellite sensors offer high spatial coverage and an evolving time series, having lengths potentially of decades. However, detecting clouds by passive satellite remote sensing sensors is challenging over the Arctic because of the brightness of snow and ice in the ultraviolet and visible spectral regions and because of the small brightness temperature contrast to the surface. Consequently, the quality of the resulting cloud data products needs to be assessed quantitatively. In this study, we validate the cloud data products retrieved from the Advanced Very High Resolution Radiometer (AVHRR) post meridiem (PM) data from the polar-orbiting NOAA-19 satellite and compare them with those derived from the ground-based instruments during the sunlit months. The AVHRR cloud data products by the European Space Agency (ESA) Cloud Climate Change Initiative (Cloud_CCI) project uses the observations in the visible and IR bands to determine cloud properties. The ground-based measurements from four high-latitude sites have been selected for this investigation: Hyytiälä (61.84°N, 24.29°E), North Slope of Alaska (NSA; 71.32°N, 156.61°W), Ny-Ålesund (Ny-Å; 78.92°N, 11.93°E), and Summit (72.59°N, 38.42°W). The liquid water path (LWP) ground-based data are retrieved from microwave radiometers, while the cloud top height (CTH) has been determined from the integrated lidar–radar measurements. The quality of the satellite products, cloud mask and cloud optical depth (COD), has been assessed using data from NSA, whereas LWP and CTH have been investigated over Hyytiälä, NSA, Ny-Å, and Summit.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1971

Interhemispheric comparison of atmospheric circulation features as evaluated from NIMBUS satellite data

Findings are presented for IRIS data from NIMBUS 3 in mapping the global ozone distribution. The seasonal and regional variations of ozone, especially in the Southern Hemisphere, reveal features that were not evident from the sparse ground-based ozone observation network in this hemisphere. A regression analysis was undertaken for temperature and height fields on radiance data. Spectrum analyses of upper wind data from the North American section and Australia were completed.

Reiter, E. R.↗
DOE OSTI2021

Seasonal Disorder in Urban Traffic Patterns: A Low Rank Analysis

This article proposes several advances to sparse nonnegative matrix factorization (SNMF) as a way to identify large-scale patterns in urban traffic data. The input to our model is traffic counts organized by time and location. Nonnegative matrix factorization additively decomposes this information, organized as a matrix, into a linear sum of temporal signatures. Penalty terms encourage this factorization to concentrate on only a few temporal signatures, with weights which are not too large. Our interest here is to quantify and compare the regularity of traffic behavior, particularly across different broad temporal windows. In addition to the rank and error, we adapt a measure introduced by Hoyer to quantify sparsity in the representation. Combining these, we construct several curves which quantify error as a function of rank (the number of possible signatures) and sparsity; as rank goes up and sparsity goes down, the approximation can be better and the error should decreases. Plots of several such curves corresponding to different time windows leads to a way to compare disorder/order at different time scalewindows. In this paper, we apply our algorithms and procedures to study a taxi traffic dataset from New York City. In this dataset, we find weekly periodicity in the signatures, which allows us an extra framework for identifying outliers as significant deviations from weekly medians. We then apply our seasonal disorder analysis to the New York City traffic data and seasonal (spring, summer, winter, fall) time windows. We do find seasonal differences in traffic order.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1998

Data Mining and Optimization Tools for Developing Engine Parameters Tools

This project was awarded for understanding the problem and developing a plan for Data Mining tools for use in designing and implementing an Engine Condition Monitoring System. From the total budget of $5,000, Tricia and I studied the problem domain for developing ail Engine Condition Monitoring system using the sparse and non-standardized datasets to be available through a consortium at NASA Lewis Research Center. We visited NASA three times to discuss additional issues related to dataset which was not made available to us. We discussed and developed a general framework of data mining and optimization tools to extract useful information from sparse and non-standard datasets. These discussions lead to the training of Tricia Erhardt to develop Genetic Algorithm based search programs which were written in C++ and used to demonstrate the capability of GA algorithm in searching an optimal solution in noisy datasets. From the study and discussion with NASA LERC personnel, we then prepared a proposal, which is being submitted to NASA for future work for the development of data mining algorithms for engine conditional monitoring. The proposed set of algorithm uses wavelet processing for creating multi-resolution pyramid of the data for GA based multi-resolution optimal search. Wavelet processing is proposed to create a coarse resolution representation of data providing two advantages in GA based search: 1. We will have less data to begin with to make search sub-spaces. 2. It will have robustness against the noise because at every level of wavelet based decomposition, we will be decomposing the signal into low pass and high pass filters.

Dhawan, Atam P.↗
DOE OSTI2020

Quality Control of Additively Manufactured Metallic Structures with Machine Learning of Thermography Images

Additive manufacturing (AM) of high-strength metals is currently based on the laser powder bed fusion (LPBF) process, which can introduce internal material flaws, such as pores and anisotropy. Quality control (QC) requires nondestructive evaluation of actual AM structures. Flash thermography is a potentially promising QC technique because it is scalable to arbitrary structure size. However, the detection sensitivity of this method is limited by noise. We investigate separation of signal from noise in thermography images using several machine learning (ML) methods, including new spatial–temporal blind source separation and spatial–temporal sparse dictionary learning methods. Performance of the ML methods is benchmarked using thermography data obtained from imaging stainless steel 316L and Inconel 718 specimens produced by the LPBF method with imprinted calibrated porosity defects. The ML methods are ranked by F-score and execution runtime. Finally, the ML methods with higher accuracy require a longer runtime. However, this runtime is sufficiently short to perform QC within a realistic time frame.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Active learning of reactive Bayesian force fields applied to heterogeneous catalysis dynamics of H/Pt

Abstract Atomistic modeling of chemically reactive systems has so far relied on either expensive ab initio methods or bond-order force fields requiring arduous parametrization. Here, we describe a Bayesian active learning framework for autonomous “on-the-fly” training of fast and accurate reactive many-body force fields during molecular dynamics simulations. At each time-step, predictive uncertainties of a sparse Gaussian process are evaluated to automatically determine whether additional ab initio training data are needed. We introduce a general method for mapping trained kernel models onto equivalent polynomial models whose prediction cost is much lower and independent of the training set size. As a demonstration, we perform direct two-phase simulations of heterogeneous H 2 turnover on the Pt(111) catalyst surface at chemical accuracy. The model trains itself in three days and performs at twice the speed of a ReaxFF model, while maintaining much higher fidelity to DFT and excellent agreement with experiment.

42 ENGINEERING↗
DOE OSTI2023

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Abstract Physics-informed neural networks (PINNs) have been shown to be effective in solving partial differential equations by capturing the physics induced constraints as a part of the training loss function. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE. It also shows how physical regularizations based on continuity criteria and conservation laws fail to address this issue and rather introduce problems of their own causing the deep network to converge to a physics-obeying local minimum instead of the global minimum. We introduce Gaussian process (GP) based smoothing that recovers the performance of a PINN and promises a robust architecture against noise/errors in measurements. Additionally, we illustrate an inexpensive method of quantifying the evolution of uncertainty based on the variance estimation of GPs on boundary data. Robust PINN performance is also shown to be achievable by choice of sparse sets of inducing points based on sparsely induced GPs. We demonstrate the performance of our proposed methods and compare the results from existing benchmark models in literature for time-dependent Schrödinger and Burgers’ equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Imagery Applications for Advanced Event Analytics (WBS 24.3.1.3.3-IDC FY202 Final Project Report)

Accurate event locations and replicability of location analyses are essential for assessing the nature of an event, its context, ambient site conditions, and proximity to relevant facilities and infrastructure. Additionally, accurate event locations provide valuable information that reduce uncertainties, improve confidence in event analyses, and inform in-field verification activities. However, event location/relocation and replicability are difficult due to a number of factors, including spatially-sparse network coverage in some areas of the globe and variability in seismic data processing. This team proposed that the incorporation of high-fidelity imagery as a data backbone to the analytical assessment of a suspected underground explosion and/or an advanced seismic event bulletin produced by the International Data Centre (IDC) of the Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO PrepCom) could reduce uncertainties and improve confidence in analyses. Specifically, temporally-separated images can reduce uncertainty by identifying areas where change has occurred (e.g., building construction or demolition, road or facilities improvements). The primary goal of this project was to develop an automated geospatial processing script for imagery change detection to better reflect needs of the technical community (including the IDC) and to make the use of such a tool accessible in a variety of settings across platforms. Technical experts at Los Alamos National Laboratory successfully built GAIA: the Geospatial Automated Imagery Analysis tool, to fill this need. GAIA combines five tool components to produce orthorectified time separated imagery and imagery change detection maps. Our toolkit (1) reduces error by providing a standardized workflow for image analyses and (2) significantly reduces processing time from between 7 and 24+ hours to approximately 5 minutes. Technical experts at Sandia National Laboratories supported GAIA via beta-testing and by introducing a web-based system approach for increased applicability. To test the function, performance, broad application, and ease-of-use of GAIA, we applied it to four separate test cases. The results of this preliminary investigation show promise in reducing uncertainty in seismic event locations: if satellite imagery can show regions where operations that produce seismic activity likely occurred, then pursuing imagery to locate epicenters of seismic nuclear events could reduce the time needed to find the true epicenter location.

58 GEOSCIENCES↗
DOE OSTI2020

Imagery Applications for Advanced Event Analytics (WBS 24.3.1.3.3-IDC FY2020 Final Project Report)

Accurate event locations and replicability of location analyses are essential for assessing the nature of an event, its context, ambient site conditions, and proximity to relevant facilities and infrastructure. Additionally, accurate event locations provide valuable information that reduce uncertainties, improve confidence in event analyses, and inform in-field verification activities. However, event location/relocation and replicability are difficult due to a number of factors, including spatially-sparse network coverage in some areas of the globe and variability in seismic data processing. This team proposed that the incorporation of high-fidelity imagery as a data backbone to the analytical assessment of a suspected underground explosion and/or an advanced seismic event bulletin produced by the International Data Centre (IDC) of the Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO PrepCom) could reduce uncertainties and improve confidence in analyses. Specifically, temporally-separated images can reduce uncertainty by identifying areas where change has occurred (e.g., building construction or demolition, road or facilities improvements). The primary goal of this project was to develop an automated geospatial processing script for imagery change detection to better reflect needs of the technical community (including the IDC) and to make the use of such a tool accessible in a variety of settings across platforms. Technical experts at Los Alamos National Laboratory successfully built GAIA: the Geospatial Automated Imagery Analysis tool, to fill this need. GAIA combines five tool components to produce orthorectified time-separated imagery and imagery change detection maps. Our toolkit (1) reduces error by providing a standardized workflow for image analyses and (2) significantly reduces processing time from between 7 and 24+ hours to approximately 5 minutes. Technical experts at Sandia National Laboratories supported GAIA via beta-testing and by introducing a web-based system approach for increased applicability. To test the function, performance, broad application, and ease-of-use of GAIA, we applied it to four separate test cases. The results of this preliminary investigation show promise in reducing uncertainty in seismic event locations: if satellite imagery can show regions where operations that produce seismic activity likely occurred, then pursuing imagery to locate epicenters of seismic nuclear events could reduce the time needed to find the true epicenter location.

58 GEOSCIENCES↗
DOE OSTI2026

Financial Exposure After a Sealed-Source Release: Insurance Limits, Federal Cost Pathways, and Implications for Gamma Irradiator Substitution

This report assesses whether private insurance and existing federal authorities would likely provide meaningful financial protection to private operators of self-shielded gamma irradiators after a major sealed-source release. It does not answer this question quantitatively because publicly observable data on premiums, limits, uptake, and claims outcomes for this risk class appear sparse. Instead, it uses a qualitative structural analysis based on prior literature, review of relevant federal authorities, limited observable insurance-market evidence, expert outreach, and historical analogs. The analysis finds that available public and private mechanisms do not combine into a clear, dependable, or readily verifiable compensation structure for ordinary private sealed-source operators. Institutions therefore should not assume that either insurance or government response will make them financially whole after a severe incident. Source reduction and replacement remain more dependable than post-event financial mechanisms for reducing institutional exposure and broader radiological risk.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS1993

The growth of the UniTree mass storage system at the NASA Center for Computational Sciences

In October 1992, the NASA Center for Computational Sciences made its Convex-based UniTree system generally available to users. The ensuing months saw the growth of near-online data from nil to nearly three terabytes, a doubling of the number of CPU's on the facility's Cray YMP (the primary data source for UniTree), and the necessity for an aggressive regimen for repacking sparse tapes and hierarchical 'vaulting' of old files to freestanding tape. Connectivity was enhanced as well with the addition of UltraNet HiPPI. This paper describes the increasing demands placed on the storage system's performance and throughput that resulted from the significant augmentation of compute-server processor power and network speed.

Tarshish, Adina↗
NASA NTRS2012

Application of Satellite Gravimetry for Water Resource Vulnerability Assessment

The force of Earth's gravity field varies in proportion to the amount of mass near the surface. Spatial and temporal variations in the gravity field can be measured via their effects on the orbits of satellites. The Gravity Recovery and Climate Experiment (GRACE) is the first satellite mission dedicated to monitoring temporal variations in the gravity field. The monthly gravity anomaly maps that have been delivered by GRACE since 2002 are being used to infer changes in terrestrial water storage (the sum of groundwater, soil moisture, surface waters, and snow and ice), which are the primary source of gravity variability on monthly to decadal timescales after atmospheric and oceanic circulation effects have been removed. Other remote sensing techniques are unable to detect water below the first few centimeters of the land surface. Conventional ground based techniques can be used to monitor terrestrial water storage, but groundwater, soil moisture, and snow observation networks are sparse in most of the world, and the countries that do collect such data rarely are willing to share them. Thus GRACE is unique in its ability to provide global data on variations in the availability of fresh water, which is both vital to life on land and vulnerable to climate variability and mismanagement. This chapter describes the unique and challenging aspects of GRACE terrestrial water storage data, examples of how the data have been used for research and applications related to fresh water vulnerability and change, and prospects for continued contributions of satellite gravimetry to water resources science and policy.

Rodell, Matthew↗
NASA NTRS2020

Evaluating Drought Indices for Early Warning in East and Southern Africa

Sparsely populated regions of east and southern Africa often have little ground based data to monitor drought, crops, and water resources. The Regional Hydrologic Extremes Assessment System (RHEAS) is a NASA supported data assimilation framework that combines both a hydrologic and crop model, which provide another monitoring approach. RHEAS can also be run in forecast mode, providing outlook on drought and crop yield.

Miller, Sara↗
DOE OSTI2025

Learning Physically Interpretable Atmospheric Models From Data With WSINDy

The multiscale and turbulent nature of Earth's atmosphere has historically rendered accurate weather modeling a hard problem. Recently, there has been an explosion of interest surrounding data-driven approaches to weather modeling, which in many cases show improved forecasting accuracy and computational efficiency when compared to traditional methods. However, many of the current data-driven approaches employ highly parameterized neural networks, often resulting in uninterpretable models and limited gains in scientific understanding. In this work, we address the interpretability problem by explicitly discovering partial differential equations governing atmospheric phenomena, identifying symbolic mathematical models with direct physical interpretations. The purpose of this paper is to demonstrate that, in particular, the weak-form sparse identification of nonlinear dynamics (WSINDy) algorithm can learn effective atmospheric models from both simulated and assimilated data. Our approach adapts the standard WSINDy algorithm to work with high-dimensional fluid data of arbitrary spatial dimension.

58 GEOSCIENCES↗
DOE OSTI2022

Stellar kinematics of dwarf galaxies from multi-epoch spectroscopy: application to Triangulum II

ABSTRACT We present new MMT/Hectochelle spectroscopic measurements for 257 stars observed along the line of sight to the ultrafaint dwarf galaxy Triangulum II (Tri II). Combining results from previous Keck/DEIMOS spectroscopy, we obtain a sample that includes 16 likely members of Tri II, with up to 10 independent redshift measurements per star. To this multi-epoch kinematic data set, we apply methodology that we develop in order to infer binary orbital parameters from sparsely sampled radial velocity curves with as few as two epochs. For a previously identified (spatially unresolved) binary system in Tri II, we infer an orbital solution with period $296.0_{-3.3}^{+3.8} \rm ~ d$, semimajor axis $1.12^{+0.41}_{-0.24}\rm ~au$, and systemic velocity $-380.0 \pm 1.7 \rm ~km ~s^{-1}$ that we then use in the analysis of Tri II’s internal kinematics. Despite this improvement in the modelling of binary star systems, the current data remain insufficient to resolve the velocity dispersion of Tri II. We instead find a 95 per cent confidence upper limit of $\sigma _{v} \lesssim 3.4 \rm ~km~s^{-1}$.

Buttry, Rachel (ORCID:0000000155017609)↗
DOE OSTI2023

Wildfire smoke exposure and early childhood respiratory health: a study of prescription claims data

Abstract Wildfire smoke is associated with short-term respiratory outcomes including asthma exacerbation in children. As investigations into developmental wildfire smoke exposure on children’s longer-term respiratory health are sparse, we investigated associations between developmental wildfire smoke exposure and first use of respiratory medications. Prescription claims from IBM MarketScan Commercial Claims and Encounters database were linked with wildfire smoke plume data from NASA satellites based on Metropolitan Statistical Area (MSA). A retrospective cohort of live infants (2010–2016) born into MSAs in six western states (U.S.A.), having prescription insurance, and whose birthdate was estimable from claims data was constructed ( N = 184,703); of these, gestational age was estimated for 113,154 infants. The residential MSA, gestational age, and birthdate were used to estimate average weekly smoke exposure days ( smoke-day ) for each developmental period: three trimesters, and two sequential 12-week periods post-birth. Medications treating respiratory tract inflammation were classified using active ingredient and mode of administration into three categories:: 'upper respiratory', 'lower respiratory', 'systemic anti-inflammatory'. To evaluate associations between wildfire smoke exposure and medication usage, Cox models associating smoke-days with first observed prescription of each medication category were adjusted for infant sex, birth-season, and birthyear with a random intercept for MSA. Smoke exposure during postnatal periods was associated with earlier first use of upper respiratory medications (1–12 weeks: hazard ratio (HR) = 1.094 per 1-day increase in average weekly smoke-day, 95%CI: (1.005,1.191); 13–24 weeks: HR = 1.108, 95%CI: (1.016,1.209)). Protective associations were observed during gestational windows for both lower respiratory and systemic anti-inflammatory medications; it is possible that these associations may be a consequence of live-birth bias. These findings suggest wildfire smoke exposure during early postnatal developmental periods impact subsequent early life respiratory health.

Dhingra, Radhika↗
NASA NTRS1985

Assessment of Computer-based Geologic Mapping of Rock Units in the LANDSAT-4 Scene of Northern Death Valley, California

Geologists obtain low accuracy levels when maps derived from LANDSAT MSS data are compared with those made by conventional methods. Procedures developed for the IDIMS computer system and used to classify a subset of a TM image of the Death Valley, California - Nevada border are described. Despite the superior resolution, broader spectral coverage, and greater sensitivity inherent to the TM, the actual recorded measured accuracy was in the same narrow range (30 to 60%) recorded for MSS data from earlier LANDSATs. The supervised classification approach appears to be superior to the unsupervised approach when applied to vegetation-sparse surfaces composed of spectrally contrasting rock/soil units distributed in relatively flat to low relief terrain. As spatial resolution improves and optimal spectral bands for identifying rock materials are specified, use of classified multispectral remote sensing data from air and space when coupled with supporting field calibration and checks should become the dominant way in which geologic mapping is carried out in future decades.

Short, N. M.↗