Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Introducing and Evaluating the Climate Hazards Center IMERG with Stations (CHIMES) Timely Station-Enhanced Integrated Multisatellite Retrievals for Global Precipitation Measurement

As human exposure to hydroclimatic extremes increase and the number of in situ precipitation observations declines, precipitation estimates, such as those provided by the Integrated Multisatellite Retrievals for Global Precipitation Measurement (GPM) (IMERG) mission, provide a critical source of information. Here, we present a new gauge-enhanced dataset [the Climate Hazards Center IMERG with Stations (CHIMES)] designed to support global crop and hydrologic modeling and monitoring. CHIMES enhances the IMERG Late Run product using an updated Climate Hazards Center (CHC) high-resolution climatology (CHPclim) and low-latency rain gauge observations. CHPclim differs from other products because it incorporates long-term averages of satellite precipitation, which increases CHPclim ’s fidelity in data-sparse areas with complex terrain. This fidelity translates into performance increases in unbiased IMERGlate data, which we refer to as CHIME. This is augmented with gauge observations to produce CHIMES. The CHC’s curated rain gauge archive contains valuable contributions from many countries. There are two versions of CHIMES: preliminary and final. The final product has more copious and better-curated station data. Every pentad and month, bias-adjusted IMERGlate fields are combined with gauge observations to create pentadal and monthly CHIMESprelim and CHIMESfinal. Comparisons with pentadal, high-quality gridded station data show that IMERG late performs well (r = 0.75), but has some systematic biases which can be reduced. Monthly cross-validation results indicate that unbiasing increases the variance explained from 50% to 63% and decreases the mean absolute error from 48 to 39 mm month −1. Gauge enhancement then increases the variance explained to 75%, reducing the mean absolute error to 27 mm month −1.

Chris C funk↗
NASA NTRS2000

The Use of Additional GPS Frequencies to Independently Determine Tropospheric Water Vapor Profiles

It is well known that the currently employed L1 and L2 GPS/MET frequencies (1.2 - 1.6) Ghz) do not allow for the separation of water vapor and density (or temperature) from active microwave occultation measurements in regions of the troposphere warmer than 240 K Therefore, additional information must be used, from other types of measurements and weather analyses, to recover water vapor (and temperature) profiles. Thus in data sparse regions, these inferred profiles can be subject to larger errors than would result in data rich regions. The use of properly selected additional GPS frequencies enables a direct, independent measurement of the absorption associated with the water vapor profile, which may then be used in the standard GPS/MET retrievals to obtain a more accurate determination of atmospheric temperature throughout the water vapor layer. This study looks at the use of microwave crosslinks in the region of the 22 Ghz water vapor absorption line for this purpose. An added advantage of using 22 Ghz frequencies is that they are only negligibly affected by the ionosphere in contrast to the large effect at the GPS frequencies. The retrieval algorithm uses both amplitude and phase measurements to obtain profiles of atmospheric pressure, temperature and water water vapor pressure with a vertical resolution of 1 km or better. This technique also provides the cloud liquid water content along the ray path, which is in itself an important element in climate monitoring. Advantages of this method include the ability to make measurements in the presence of clouds and the use of techniques and technology proven through the GPS/MET experiment and several of NASA's planetary exploration missions. Simulations demonstrating this method will be presented for both clear and cloudy sky conditions.

Herman, B.M.↗
NASA NTRS1979

UHF coplanar-slot antenna for aircraft-to-satellite data communications

The initiative for starting the Aircraft-to-Satellite Data Relay (ASDAR) Program came from a recognition that much of the world's weather originates in the data sparse area of the tropics which are primarily ocean. The ASDAR system consists of (1) a data acquisition and control unit to acquire, store and format these data; (2) a clock to time the data sampling and transmission periods; and (3) a transmitter and low-profile upper hemisphere coverage antenna to relay the formatted data via satellite to the National Weather Service ground stations, as shown schematically. The low-profile antenna is a conformal antenna based on the coplanar-slot approach. The antenna is circular polarized and has an on-axis gain of nearly 2.5 dB and a HPBW greater than 90 deg. The discussion covers antenna design, radiation characteristics, flight testing, and system performance.

Myhre, R. W.↗
DOE OSTI2025

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗
DOE OSTI2024

Non-linear relationships between daily temperature extremes and US agricultural yields uncovered by global gridded meteorological datasets

Global agricultural commodity markets are highly integrated among major producers. Prices are driven by aggregate supply rather than what happens in individual countries in isolation. Furthermore, estimating the effects of weather-induced shocks on production, trade patterns and prices hence requires a globally representative weather data set. Recently, two data sets that provide daily or hourly records, GMFD and ERA5-Land, became available. Starting with the US, a data rich region, we formally test whether these global data sets are as good as more fine-scaled country-specific data in explaining yields and whether they estimate similar response functions. While GMFD and ERA5-Land have lower predictive skill for US corn and soybeans yields than the fine-scaled PRISM data, they still correctly uncover the underlying non-linear temperature relationship. All specifications using daily temperature extremes under any of the weather data sets outperform models that use a quadratic in average temperature. Correctly capturing the effect of daily extremes has a larger effect than the choice of weather data. In a second step, focusing on Sub Saharan Africa, a data sparse region, we confirm that GMFD and ERA5-Land have superior predictive power to CRU, a global weather data set previously employed for modeling climate effects in the region.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1980

Applications of the TIROS-N sounding and cloud motion wind enhancement for the FGGE 'special effort'

In January of 1978, a panel of experts recommended that a 'special effort' be made to enhance and edit satellite soundings and cloud tracked winds in data sparse regions. It was felt that these activities would be necessary to obtain maximum benefits from an evaluation of satellite data during the Global Weather Experiment (FGGE). The 'special effort' is being conducted for the two special observing periods of FGGE. More than sixty cases have been selected for enhancement on the basis of meteorological interest. These cases include situations of blocking, cutoff low development, cyclogenesis, and tropical circulations. The sounding data enhancement process consists of supplementing the operational satellite sounding data set with higher resolution soundings in meteorologically active regions, and with new soundings where data voids or soundings of questionable quality exist.

Atlas, R.↗
NASA NTRS1992

Sparse Matrices in MATLAB: Design and Implementation

The matrix computation language and environment MATLAB is extended to include sparse matrix storage and operations. The only change to the outward appearance of the MATLAB language is a pair of commands to create full or sparse matrices. Nearly all the operations of MATLAB now apply equally to full or sparse matrices, without any explicit action by the user. The sparse data structure represents a matrix in space proportional to the number of nonzero entries, and most of the operations compute sparse results in time proportional to the number of arithmetic operations on nonzeros.

Gilbert, John R.↗
NASA NTRS1988

Integration of space and in situ observations to study global climate change

The use of model-based global data sets of atmospheric circulation for studying fundamental dynamical and physical processes is discussed, focusing on limitations of the available model-based data sets. Data from the Global Weather Experiment in 1979 were analyzed by two authorized level IIIb data centers in 1980 and in 1981. The analyses led to difference in data-sparse regions such as the tropics. Study areas which can be addressed by an internally-consistent long-term multivariate data set for the atmospheric circulation are considered, including mean climate, forcing for the ocean models, global hydrological cycle, atmospheric energetics, intraseasonal variability, land surface processes, and structure and variability of vertical velocity, divergence, and diabatic heating. It is concluded that the most comprehensive technique for integrating space and in situ observations to produce this type of data set would be a four-dimensional data assimilation system with a realistic physical model of the type employed in operational numerical weather prediction.

Bengtsson, L.↗
NASA NTRS2011

Evaluation of Long-Range Lightning Detection Networks Using TRMM/LIS Observations

Recent advances in long-range lightning detection technologies have improved our understanding of thunderstorm evolution in the data sparse oceanic regions. Although the expansion and improvement of long-range lightning datasets have increased their applicability, these applications (e.g., data assimilation, atmospheric chemistry, and aviation weather hazards) require knowledge of the network detection capabilities. Toward this end, the present study evaluates data from the World Wide Lightning Location Network (WWLLN) using observations from the Lightning Imaging Sensor (LIS) aboard the Tropical Rainfall Measurement Mission (TRMM) satellite. The study documents the WWLLN detection efficiency and location accuracy relative to LIS observations, describes the spatial variability in these performance metrics, and documents the characteristics of LIS flashes that are detected by WWLLN. Improved knowledge of the WWLLN detection capabilities will allow researchers, algorithm developers, and operational users to better prepare for the spatial and temporal coverage of the upcoming GOES-R Geostationary Lightning Mapper (GLM).

Rudlosky, Scott D.↗
NASA NTRS2012

Relationships Between Long-Range Lightning Networks and TRMM/LIS Observations

Recent advances in long-range lightning detection technologies have improved our understanding of thunderstorm evolution in the data sparse oceanic regions. Although the expansion and improvement of long-range lightning datasets have increased their applicability, these applications (e.g., data assimilation, atmospheric chemistry, and aviation weather hazards) require knowledge of the network detection capabilities. The present study intercompares long-range lightning data with observations from the Lightning Imaging Sensor (LIS) aboard the Tropical Rainfall Measurement Mission (TRMM) satellite. The study examines network detection efficiency and location accuracy relative to LIS observations, describes spatial variability in these performance metrics, and documents the characteristics of LIS flashes that are detected by the long-range networks. Improved knowledge of relationships between these datasets will allow researchers, algorithm developers, and operational users to better prepare for the spatial and temporal coverage of the upcoming GOES-R Geostationary Lightning Mapper (GLM).

Rudlosky, Scott D.↗
DOE OSTI2025

A Markov chain Monte Carlo (MCMC) Bayesian inference approach to analyze apparent activation barriers and reaction orders from microreactor data

Statistical analysis of steady-state catalytic kinetic data is often limited by data sparsity due to the slow pace at which the data is collected. Data sparsity and limitations in statistical analysis make it difficult to differentiate between mechanistic models and catalytic sites. A Bayesian inference tool is reported for catalysis researchers to estimate error in the determination of reaction orders from steady state microreactor data. The benefits of a Bayesian inference approach are discussed, as an alternative to the more common frequentist approach. The approach incorporates prior knowledge of the system and the data collected to form an error estimate on reaction orders. We investigated the effects of three distinct data treatments—individual fitting of trials, pooled analysis, and constrained regression methods—on the precision and uncertainty of reaction order determinations. To assess the robustness of our findings, we conducted sensitivity analyses to evaluate the influence of Bayesian parameters on uncertainty estimation. Additionally, we utilized synthetic data to illustrate how data quality impacts the precision of uncertainty assessments. We show Bayesian analysis can obtain a more precise estimation of error with a sparse data set than a frequentist analysis. Finally, this work provides strong evidence that the adoption of Bayesian analysis of kinetic data may help researchers make more precise arguments as to the strength of their evidence for a particular mechanistic hypothesis, or in comparing across different catalysts.

42 ENGINEERING↗
NASA NTRS1984

The Use of Available Potential Energy to Evaluate Two GLAS FGGE Data Sets

Meteorological data derived from satellites provided an essential component of the FGGE data sets, especially in data sparse regions of the globe. The impact of satellite derived observations on objective analyses and numerical model predictions are investigated. The FGGE data sets and resulting numerical forecasts and the ability to define the available potential energy of the atmosphere, and its relationship to synoptic scale features such as extratropical cyclones are studied.

Horn, L. H.↗
DOE OSTI2026

A scalable framework for efficient coupling of thermal and microstructural simulations in additive manufacturing

Predicting microstructure evolution in metal additive manufacturing (AM) is important for process optimization, but spatiotemporal scale disparities between thermal transport and microstructure evolution create significant challenges for efficient data transfer between simulation codes. To address this, we present Stork, a scalable framework for coupling thermal and microstructural simulations. Stork uses a sparse data representation to identify and store active solidification sub-volumes, enabling highly parallel quad-linear interpolation from coarse thermal grids to fine microstructure grids without large intermediate storage. We demonstrate the framework by coupling the semi-analytic heat transfer code 3DThesis with the time-parallel cellular automata code Toucan. This approach achieves over two orders of magnitude reduction in data generation time and file size compared to prior workflows. Numerical studies show that quad-linear interpolation preserves grain morphology and crystallographic texture in laser powder bed fusion (LPBF) simulations for coarsening ratios up to 16. Overall, Stork provides a scalable pathway for high-throughput, component-scale AM simulations on modern high-performance computing systems.

36 MATERIALS SCIENCE↗
NASA NTRS1997

Intelligent System Development Using a Rough Sets Methodology

The purpose of this research was to examine the potential of the rough sets technique for developing intelligent models of complex systems from limited information. Rough sets a simple but promising technology to extract easily understood rules from data. The rough set methodology has been shown to perform well when used with a large set of exemplars, but its performance with sparse data sets is less certain. The difficulty is that rules will be developed based on just a few examples, each of which might have a large amount of noise associated with them. The question then becomes, what is the probability of a useful rule being developed from such limited information? One nice feature of rough sets is that in unusual situations, the technique can give an answer of 'I don't know'. That is, if a case arises that is different from the cases the rough set rules were developed on, the methodology can recognize this and alert human operators of it. It can also be trained to do this when the desired action is unknown because conflicting examples apply to the same set of inputs. This summer's project was to look at combining rough set theory with statistical theory to develop confidence limits in rules developed by rough sets. Often it is important not to make a certain type of mistake (e.g., false positives or false negatives), so the rules must be biased toward preventing a catastrophic error, rather than giving the most likely course of action. A method to determine the best course of action in the light of such constraints was examined. The resulting technique was tested with files containing electrical power line 'signatures' from the space shuttle and with decompression sickness data.

Anderson, Gray T.↗
NASA NTRS2012

The GEOS-iODAS: Description and Evaluation

This report documents the GMAO's Goddard Earth Observing System sea ice and ocean data assimilation systems (GEOS iODAS) and their evolution from the first reanalysis test, through the implementation that was used to initialize the GMAO decadal forecasts, and to the current system that is used to initialize the GMAO seasonal forecasts. The iODAS assimilates a wide range of observations into the ocean and sea ice components: in-situ temperature and salinity profiles, sea level anomalies from satellite altimetry, analyzed SST, and sea-ice concentration. The climatological sea surface salinity is used to constrain the surface salinity prior to the Argo years. Climatological temperature and salinity gridded data sets from the 2009 version of the World Ocean Atlas (WOA09) are used to help constrain the analysis in data sparse areas. The latest analysis, GEOS ODAS5.2, is diagnosed through detailed studies of the statistics of the innovations and analysis departures, comparisons with independent data, and integrated values such as volume transport. Finally, the climatologies of temperature and salinity fields from the Argo era, 2002-2011, are presented and compared with the WOA09.

Climate↗
NASA NTRS2020

NASA Global Satellite and Model Data Products and Services for Tropical Meteorology and Climatology

Satellite remote sensing and model data play an important role in research and applications of tropical meteorology and climatology over vast, data sparse oceans and remote continents. Since the first weather satellite was launched by NASA in 1960, a large collection of NASA's Earth science data is freely available to the research and application communities around the world, significantly improving our overall understanding of the Earth system and environment. Established in the mid-80s, the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), located in Maryland, USA, is a data archive center for multidisciplinary, satellite and model assimilation data products. As one of the 12 NASA data centers in Earth Sciences, GES DISC hosts several important NASA satellite missions for tropical meteorology and climatology such as the Tropical Rainfall Measuring Mission (TRMM), the Global Precipitation Measurement (GPM) mission and the Modern-Era Retrospective analysis for Research and Applications (MERRA). Over the years, GES DISC has developed data services to facilitate data discovery, access, distribution, analysis and visualization, including Giovanni, an online analysis and visualization tool without the need to download data and software. Despite many efforts for improving data access, still quite a number of challenges remain, such as finding datasets and services for a specific research topic or project, especially for inexperienced users or users outside the remote sensing community. In this article, we list and describe major NASA satellite remote sensing and model datasets and services for tropical meteorology and climatology along with examples of using the data and services, in hope that may help users better utilize the information in their research and applications.

tropical meteorology and climatology, data, servic↗
NASA NTRS1996

Variance Analysis of Unevenly Spaced Time Series Data

We have investigated the effect of uneven data spacing on the computation of delta (sub chi)(gamma). Evenly spaced simulated data sets were generated for noise processes ranging from white phase modulation (PM) to random walk frequency modulation (FM). Delta(sub chi)(gamma) was then calculated for each noise type. Data were subsequently removed from each simulated data set using typical two-way satellite time and frequency transfer (TWSTFT) data patterns to create two unevenly spaced sets with average intervals of 2.8 and 3.6 days. Delta(sub chi)(gamma) was then calculated for each sparse data set using two different approaches. First the missing data points were replaced by linear interpolation and delta (sub chi)(gamma) calculated from this now full data set. The second approach ignored the fact that the data were unevenly spaced and calculated delta(sub chi)(gamma) as if the data were equally spaced with average spacing of 2.8 or 3.6 days. Both approaches have advantages and disadvantages, and techniques are presented for correcting errors caused by uneven data spacing in typical TWSTFT data sets.

Hackman, Christine↗
DOE OSTI2025

Frictionless knowledge injection for few-shot learning

Cutting-edge machine learning methods often require large volumes of curated training data, precluding their use in national security problems with rare events in massive datasets. We present a method for incorporating abstract knowledge into models tailored for sparse data. A subject matter expert defines salient concepts using data examples, which are encoded in the model’s embedding space. Models are then trained to respect these concepts. This method enables knowledge injection, yielding effective models with limited labeled data and the ability to assess model sensitivity for subject matter expertise across the nonproliferation mission space, as demonstrated with Raman spectra analysis.

Stomps, Jordan [ORNL] (ORCID:0000000178114479)↗