SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

NASA NTRS2011

Evaluation of Long-Range Lightning Detection Networks Using TRMM/LIS Observations

Recent advances in long-range lightning detection technologies have improved our understanding of thunderstorm evolution in the data sparse oceanic regions. Although the expansion and improvement of long-range lightning datasets have increased their applicability, these applications (e.g., data assimilation, atmospheric chemistry, and aviation weather hazards) require knowledge of the network detection capabilities. Toward this end, the present study evaluates data from the World Wide Lightning Location Network (WWLLN) using observations from the Lightning Imaging Sensor (LIS) aboard the Tropical Rainfall Measurement Mission (TRMM) satellite. The study documents the WWLLN detection efficiency and location accuracy relative to LIS observations, describes the spatial variability in these performance metrics, and documents the characteristics of LIS flashes that are detected by WWLLN. Improved knowledge of the WWLLN detection capabilities will allow researchers, algorithm developers, and operational users to better prepare for the spatial and temporal coverage of the upcoming GOES-R Geostationary Lightning Mapper (GLM).

Rudlosky, Scott D.↗

NASA NTRS2012

Relationships Between Long-Range Lightning Networks and TRMM/LIS Observations

Recent advances in long-range lightning detection technologies have improved our understanding of thunderstorm evolution in the data sparse oceanic regions. Although the expansion and improvement of long-range lightning datasets have increased their applicability, these applications (e.g., data assimilation, atmospheric chemistry, and aviation weather hazards) require knowledge of the network detection capabilities. The present study intercompares long-range lightning data with observations from the Lightning Imaging Sensor (LIS) aboard the Tropical Rainfall Measurement Mission (TRMM) satellite. The study examines network detection efficiency and location accuracy relative to LIS observations, describes spatial variability in these performance metrics, and documents the characteristics of LIS flashes that are detected by the long-range networks. Improved knowledge of relationships between these datasets will allow researchers, algorithm developers, and operational users to better prepare for the spatial and temporal coverage of the upcoming GOES-R Geostationary Lightning Mapper (GLM).

Rudlosky, Scott D.↗

DOE OSTI2025

A Markov chain Monte Carlo (MCMC) Bayesian inference approach to analyze apparent activation barriers and reaction orders from microreactor data

Statistical analysis of steady-state catalytic kinetic data is often limited by data sparsity due to the slow pace at which the data is collected. Data sparsity and limitations in statistical analysis make it difficult to differentiate between mechanistic models and catalytic sites. A Bayesian inference tool is reported for catalysis researchers to estimate error in the determination of reaction orders from steady state microreactor data. The benefits of a Bayesian inference approach are discussed, as an alternative to the more common frequentist approach. The approach incorporates prior knowledge of the system and the data collected to form an error estimate on reaction orders. We investigated the effects of three distinct data treatments—individual fitting of trials, pooled analysis, and constrained regression methods—on the precision and uncertainty of reaction order determinations. To assess the robustness of our findings, we conducted sensitivity analyses to evaluate the influence of Bayesian parameters on uncertainty estimation. Additionally, we utilized synthetic data to illustrate how data quality impacts the precision of uncertainty assessments. We show Bayesian analysis can obtain a more precise estimation of error with a sparse data set than a frequentist analysis. Finally, this work provides strong evidence that the adoption of Bayesian analysis of kinetic data may help researchers make more precise arguments as to the strength of their evidence for a particular mechanistic hypothesis, or in comparing across different catalysts.

42 ENGINEERING↗

NASA NTRS1984

The Use of Available Potential Energy to Evaluate Two GLAS FGGE Data Sets

Meteorological data derived from satellites provided an essential component of the FGGE data sets, especially in data sparse regions of the globe. The impact of satellite derived observations on objective analyses and numerical model predictions are investigated. The FGGE data sets and resulting numerical forecasts and the ability to define the available potential energy of the atmosphere, and its relationship to synoptic scale features such as extratropical cyclones are studied.

Horn, L. H.↗

DOE OSTI2026

A scalable framework for efficient coupling of thermal and microstructural simulations in additive manufacturing

Predicting microstructure evolution in metal additive manufacturing (AM) is important for process optimization, but spatiotemporal scale disparities between thermal transport and microstructure evolution create significant challenges for efficient data transfer between simulation codes. To address this, we present Stork, a scalable framework for coupling thermal and microstructural simulations. Stork uses a sparse data representation to identify and store active solidification sub-volumes, enabling highly parallel quad-linear interpolation from coarse thermal grids to fine microstructure grids without large intermediate storage. We demonstrate the framework by coupling the semi-analytic heat transfer code 3DThesis with the time-parallel cellular automata code Toucan. This approach achieves over two orders of magnitude reduction in data generation time and file size compared to prior workflows. Numerical studies show that quad-linear interpolation preserves grain morphology and crystallographic texture in laser powder bed fusion (LPBF) simulations for coarsening ratios up to 16. Overall, Stork provides a scalable pathway for high-throughput, component-scale AM simulations on modern high-performance computing systems.

36 MATERIALS SCIENCE↗

NASA NTRS1997

Intelligent System Development Using a Rough Sets Methodology

The purpose of this research was to examine the potential of the rough sets technique for developing intelligent models of complex systems from limited information. Rough sets a simple but promising technology to extract easily understood rules from data. The rough set methodology has been shown to perform well when used with a large set of exemplars, but its performance with sparse data sets is less certain. The difficulty is that rules will be developed based on just a few examples, each of which might have a large amount of noise associated with them. The question then becomes, what is the probability of a useful rule being developed from such limited information? One nice feature of rough sets is that in unusual situations, the technique can give an answer of 'I don't know'. That is, if a case arises that is different from the cases the rough set rules were developed on, the methodology can recognize this and alert human operators of it. It can also be trained to do this when the desired action is unknown because conflicting examples apply to the same set of inputs. This summer's project was to look at combining rough set theory with statistical theory to develop confidence limits in rules developed by rough sets. Often it is important not to make a certain type of mistake (e.g., false positives or false negatives), so the rules must be biased toward preventing a catastrophic error, rather than giving the most likely course of action. A method to determine the best course of action in the light of such constraints was examined. The resulting technique was tested with files containing electrical power line 'signatures' from the space shuttle and with decompression sickness data.

Anderson, Gray T.↗

DOE OSTI2022

Automated identification of local contamination in remote atmospheric composition time series

Abstract. Atmospheric observations in remote locations offer a possibility of exploring trace gas and particle concentrations in pristine environments. However, data from remote areas are often contaminated by pollution from local sources. Detecting this contamination is thus a central and frequently encountered issue. Consequently, many different methods exist today to identify local contamination in atmospheric composition measurement time series, but no single method has been widely accepted. In this study, we present a new method to identify primary pollution in remote atmospheric datasets, e.g., from ship campaigns or stations with a low background signal compared to the contaminated signal. The pollution detection algorithm (PDA) identifies and flags periods of polluted data in five steps. The first and most important step identifies polluted periods based on the derivative (time derivative) of a concentration over time. If this derivative exceeds a given threshold, data are flagged as polluted. Further pollution identification steps are a simple concentration threshold filter, a neighboring points filter (optional), a median, and a sparse data filter (optional). The PDA only relies on the target dataset itself and is independent of ancillary datasets such as meteorological variables. All parameters of each step are adjustable so that the PDA can be “tuned” to be more or less stringent (e.g., flag more or fewer data points as contaminated). The PDA was developed and tested with a particle number concentration dataset collected during the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition in the central Arctic. Using strict settings, we identified 62 % of the data as influenced by local contamination. Using a second independent particle number concentration dataset also collected during MOSAiC, we evaluated the performance of the PDA against the same dataset cleaned by visual inspection. The two methods agreed in 94 % of the cases. Additionally, the PDA was successfully applied to a trace gas dataset (CO2), also collected during MOSAiC, and to another particle number concentration dataset, collected at the high-altitude background station Jungfraujoch, Switzerland. Thus, the PDA proves to be a useful and flexible tool to identify periods affected by local contamination in atmospheric composition datasets without the need for ancillary measurements. It is best applied to data representing primary pollution. The user-friendly and open-access code enables reproducible application to a wide suite of different datasets. It is available at https://doi.org/10.5281/zenodo.5761101 (Beck et al., 2021).

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2012

The GEOS-iODAS: Description and Evaluation

This report documents the GMAO's Goddard Earth Observing System sea ice and ocean data assimilation systems (GEOS iODAS) and their evolution from the first reanalysis test, through the implementation that was used to initialize the GMAO decadal forecasts, and to the current system that is used to initialize the GMAO seasonal forecasts. The iODAS assimilates a wide range of observations into the ocean and sea ice components: in-situ temperature and salinity profiles, sea level anomalies from satellite altimetry, analyzed SST, and sea-ice concentration. The climatological sea surface salinity is used to constrain the surface salinity prior to the Argo years. Climatological temperature and salinity gridded data sets from the 2009 version of the World Ocean Atlas (WOA09) are used to help constrain the analysis in data sparse areas. The latest analysis, GEOS ODAS5.2, is diagnosed through detailed studies of the statistics of the innovations and analysis departures, comparisons with independent data, and integrated values such as volume transport. Finally, the climatologies of temperature and salinity fields from the Argo era, 2002-2011, are presented and compared with the WOA09.

Climate↗

NASA NTRS2020

NASA Global Satellite and Model Data Products and Services for Tropical Meteorology and Climatology

Satellite remote sensing and model data play an important role in research and applications of tropical meteorology and climatology over vast, data sparse oceans and remote continents. Since the first weather satellite was launched by NASA in 1960, a large collection of NASA's Earth science data is freely available to the research and application communities around the world, significantly improving our overall understanding of the Earth system and environment. Established in the mid-80s, the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), located in Maryland, USA, is a data archive center for multidisciplinary, satellite and model assimilation data products. As one of the 12 NASA data centers in Earth Sciences, GES DISC hosts several important NASA satellite missions for tropical meteorology and climatology such as the Tropical Rainfall Measuring Mission (TRMM), the Global Precipitation Measurement (GPM) mission and the Modern-Era Retrospective analysis for Research and Applications (MERRA). Over the years, GES DISC has developed data services to facilitate data discovery, access, distribution, analysis and visualization, including Giovanni, an online analysis and visualization tool without the need to download data and software. Despite many efforts for improving data access, still quite a number of challenges remain, such as finding datasets and services for a specific research topic or project, especially for inexperienced users or users outside the remote sensing community. In this article, we list and describe major NASA satellite remote sensing and model datasets and services for tropical meteorology and climatology along with examples of using the data and services, in hope that may help users better utilize the information in their research and applications.

tropical meteorology and climatology, data, servic↗

NASA NTRS1996

Variance Analysis of Unevenly Spaced Time Series Data

We have investigated the effect of uneven data spacing on the computation of delta (sub chi)(gamma). Evenly spaced simulated data sets were generated for noise processes ranging from white phase modulation (PM) to random walk frequency modulation (FM). Delta(sub chi)(gamma) was then calculated for each noise type. Data were subsequently removed from each simulated data set using typical two-way satellite time and frequency transfer (TWSTFT) data patterns to create two unevenly spaced sets with average intervals of 2.8 and 3.6 days. Delta(sub chi)(gamma) was then calculated for each sparse data set using two different approaches. First the missing data points were replaced by linear interpolation and delta (sub chi)(gamma) calculated from this now full data set. The second approach ignored the fact that the data were unevenly spaced and calculated delta(sub chi)(gamma) as if the data were equally spaced with average spacing of 2.8 or 3.6 days. Both approaches have advantages and disadvantages, and techniques are presented for correcting errors caused by uneven data spacing in typical TWSTFT data sets.

Hackman, Christine↗

DOE OSTI2025

Frictionless knowledge injection for few-shot learning

Cutting-edge machine learning methods often require large volumes of curated training data, precluding their use in national security problems with rare events in massive datasets. We present a method for incorporating abstract knowledge into models tailored for sparse data. A subject matter expert defines salient concepts using data examples, which are encoded in the model’s embedding space. Models are then trained to respect these concepts. This method enables knowledge injection, yielding effective models with limited labeled data and the ability to assess model sensitivity for subject matter expertise across the nonproliferation mission space, as demonstrated with Raman spectra analysis.

Stomps, Jordan [ORNL] (ORCID:0000000178114479)↗

DOE OSTI2024

Multi-year assessment of the impact of ship-borne radiosonde observations on polar WRF forecasts in the Arctic

Abstract To compensate for the lack of conventional observations over the Arctic Ocean, ship-borne radiosonde observations have been regularly carried out during summer Arctic expeditions and the observed data have been broadcast via the global telecommunication system since 2017. With these data obtained over the data-sparse Arctic Ocean, observing system experiments were carried out using a polar-optimized version of the Weather Research and Forecasting (WRF) model and the WRF Data Assimilation (WRFDA) system to investigate their effects on analyses and forecasts over the Arctic. The results of verification against reanalysis data reveal: (1) DA effects on analyses and forecasts; (2) the reason for the year-to-year variability of DA effects; and (3) the possible role of upper-level potential vorticity in delayed DA effects. The overall assimilation effects of the extra data on the analyses and forecasts over the Arctic are positive. Initially, the DA effects are the most apparent in the temperature variables in the middle/lower troposphere, which spread to the wind variables in the upper troposphere. The effects decrease with time but reappear after approximately 120 h, even in the 240-h forecasts. The effects on forecasts vary depending on the proximity of the radiosonde observation locations to the high synoptic variability. The upper-level potential vorticity is known to play an important role in the development of Arctic cyclones, and it is suggested as a possible explanation for the delayed DA effects after about 120 h.

Geology↗

NASA NTRS2007

The Impact of Atmospheric InfraRed Sounder (AIRS) Profiles on Short-term Weather Forecasts

The Atmospheric Infrared Sounder (AIRS), together with the Advanced Microwave Sounding Unit (AMSU), represents one of the most advanced spacebased atmospheric sounding systems. The combined AlRS/AMSU system provides radiance measurements used to retrieve temperature profiles with an accuracy of 1 K over 1 km layers under both clear and partly cloudy conditions, while the accuracy of the derived humidity profiles is 15% in 2 km layers. Critical to the successful use of AIRS profiles for weather and climate studies is the use of profile quality indicators and error estimates provided with each profile Aside form monitoring changes in Earth's climate, one of the objectives of AIRS is to provide sounding information of sufficient accuracy such that the assimilation of the new observations, especially in data sparse region, will lead to an improvement in weather forecasts. The purpose of this paper is to describe a procedure to optimally assimilate highresolution AIRS profile data in a regional analysis/forecast model. The paper will focus on the impact of AIRS profiles on a rapidly developing east coast storm and will also discuss preliminary results for a 30-day forecast period, simulating a quasi-operation environment. Temperature and moisture profiles were obtained from the prototype version 5.0 EOS science team retrieval algorithm which includes explicit error information for each profile. The error profile information was used to select the highest quality temperature and moisture data for every profile location and pressure level for assimilation into the ARPS Data Analysis System (ADAS). The AIRS-enhanced analyses were used as initial fields for the Weather Research and Forecast (WRF) system used by the SPORT project for regional weather forecast studies. The ADASWRF system will be run on CONUS domain with an emphasis on the east coast. The preliminary assessment of the impact of the AIRS profiles will focus on quality control issues associated with AIRS, intelligent use of the quality indicators, and forecast verification.

Chou, Shih-Hung↗

DOE OSTI2022

Inference of bipolar neutrino flavor oscillations near a core-collapse supernova based on multiple measurements at Earth

Neutrinos in compact-object environments, such as core-collapse supernovae, can experience various kinds of collective effects in flavor space, engendered by neutrino-neutrino interactions. These include “bipolar” collective oscillations, which are exhibited by neutrino ensembles where different flavors dominate at different energies. Considering the importance of neutrinos in the dynamics and nucleosynthesis in these environments, it is desirable to ascertain whether an Earth-based detection could contain signatures of bipolar oscillations that occurred within a supernova envelope. To that end, we, in this study, continue examining a cost-function formulation of statistical data assimilation (SDA) to infer solutions to a small-scale model of neutrino flavor transformation. SDA is an inference paradigm designed to optimize a model with sparse data. Our model consists of two monoenergetic neutrino beams with different energies emanating from a source and coherently interacting with each other and with a matter background, with radially varying interaction strengths. We attempt to infer flavor transformation histories of these beams using simulated measurements of the flavor content at locations “in vacuum” (that is, far from the source), which could in principle correspond to Earth-based detectors. Within the scope of this small-scale model, we found that: (i) based on such measurements, the SDA procedure is able to infer whether bipolar oscillations had occurred within the protoneutron star envelope, and (ii) if the measurements sample the full amplitude of the neutrino oscillations in vacuum, then the amplitude of the prior bipolar oscillations is well predicted. This result intimates that the inference paradigm can well complement numerical integration codes, via its ability to infer flavor evolution at physically inaccessible locations.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Towards A Geo-Data Science Method for Assessing Rare Earth Element and Critical Mineral Occurrences in Coal and Other Sedimentary Systems

While preliminary analyses of data from open-source resources (Ekmann, 2012) show promising concentrations of REE in individual coal samples from a number of sites and basins in the U.S., other sparse data for REE in domestic coal-related strata suggest that many occurrences are low, “subeconomic” concentrations. At present, there is no method for systematically assessing potential sedimentary occurrences of REE. However, the geologic processes responsible for REE occurrences in coal-related strata are systematic; the unpredictability of REE resources in coal-related strata is due to poorly quantified spatial resource trends and the lack of an exploration method tailored to these resources. Thus, there exists a need for a systematic assessment approach that incorporates knowledge of geological variation in the mechanisms of REE enrichment within coal basins to help minimize geologic uncertainty and reduce commercial exploration risk.

01 COAL, LIGNITE, AND PEAT↗

NASA NTRS2009

Transitioning NPOESS Data to Weather Offices: The SPoRT Paradigm with EOS Data

Real-time satellite information provides one of many data sources used by NWS weather forecast offices (WFOs) to diagnose current weather conditions and to assist in short-term forecast preparation. While GOES satellite data provides relatively coarse spatial resolution coverage of the continental U.S. on a 10-15 minute repeat cycle, polar orbiting imagery has the potential to provide snapshots of weather conditions at high-resolution in many spectral channels. Additionally, polar orbiting sounding data can provide additional information on the thermodynamic structure of the atmosphere in data sparse regions of at asynoptic observation times. The NASA Short-term Prediction Research and Transition (SPoRT) project has demonstrated the utility of polar orbiting MODIS and AIRS data on the Terra and Aqua satellites to improve weather diagnostics and short-term forecasting on the regional and local scales. SPoRT scientists work directly forecasters at selected WFOS in the Southern Region (SR) to help them ingest these unique data streams into their AWIPS system, understand how to use the data (through on-site and distance learn techniques), and demonstrate the utility of these products to address significant forecast problems. This process also prepares forecasters for the use of similar observational capabilities from NPOESS operational sensors. NPOESS environmental data records (EDRs) from the Visible 1 Infrared Imager I Radiometer Suite (VIIRS), the Cross-track Infrared Sounder (CrlS) and Advanced Technology Microwave Sounder (ATMS) instruments and additional value-added products produced by NESDIS will be available in near real-time and made available to WFOs to extend their use of NASA EOS data into the NPOESS era. These new data streams will be integrated into the NWs's new AWIPS II decision support tools. The AWIPS I1 system to be unveiled in WFOs in 2009 will be a JAVA-based decision support system which preserves the functionality of the existing systems and offers unique development opportunities for new data sources and applications in the Service Orientated Architecture ISOA) environment. This paper will highlight some of the SPoRT activities leading to the integration of VllRS and CrIS/ATMS data into the display capabilities of these new systems to support short-term forecasting problems at WFOs.

Jedlovec, Gary↗

NASA NTRS1981

On the sensitivity of numerical weather prediction to remotely sensed marine surface wind data - A simulation study

The reported investigation has the objective to assess the potential impact on numerical weather prediction (NWP) of remotely sensed surface wind data. Other investigations conducted with similar objectives have not been satisfactory in connection with a use of procedures providing an unrealistic distribution of initial errors. In the current study, care has been taken to duplicate the actual distribution of information in the conventional observing system, thus shifting the emphasis from accuracy of the data to the data coverage. It is pointed out that this is an important consideration in assessing satellite observing systems since experience with sounder data has shown that improvements in forecasts due to satellite-derived information is due less to a general error reduction than to the ability to fill data-sparse regions. The reported study concentrates on the evaluation of the observing system simulation experimental design and on the assessment of the potential of remotely sensed marine surface wind data.

Cane, M. A.↗

NASA NTRS1980

Compatibility study of the Magsat data and aeromagnetic data in the eastern Piedmont of US

Data from a 2 day period recorded by Magsat were used to produce world magnetic maps of the scalar total field and three vector component total fields. Subtracting the reference field of Magsat 6/80, a scalar anomalous field and three vector component anomalous fields were also mapped. After removing 718 bad points from the original data, every fifth point was picked for contouring. While the main geomagnetic field of the Earth is surprisingly well mapped considering the short data period, the anomaly maps suffer from data sparseness. The entire Magsat file collected at altitudes of 500-700 m in nonmountainous terrain and at 900-1,000 m in mountainous terrain was averaged to reduce the total data to 6,500 measurements, yielding a 0.1 deg sampling interval along the flight path. A U.S. aeromagnetic anomaly surface map was produced and the field was upward continued to a 300 km altitude. Differences in anomaly structure between the POGO data and the map produced were attributed to insufficient removal of the reference field. Reprocessing of the data using the GSFC reference field (9/80-2) should remove the low harmonic field and improve the anomalous field structure.

Won, I. J.↗