SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Uncertainty-Guided Prediction Horizon of Phase-Resolved Ocean Wave Forecasting Under Data Sparsity: Experimental and Numerical Evaluation

Accurate short-term wave forecasting is critical for the safe and efficient operation of marine structures that rely on real-time, phase-resolved ocean wave information for control and monitoring purposes (e.g., digital twins). These systems often depend on environmental sensors (e.g., waverider buoys, wave-sensing LIDAR). Challenges arise when upstream sensor data are missing, sparse, or phase-shifted due to drift. This study investigates the performance of two machine learning models, time-series dense encoder (TiDE) and long short-term memory (LSTM), for forecasting phase-resolved ocean surface elevations under varying degrees of data degradation. We introduce the τ-trimming algorithm, which adapts the prediction horizon based on uncertainty thresholds derived from historical forecasts. Numerical wave tank (NWT) and wave basin experiments are used to benchmark model performance under short- and long-term data masking, spatially coarse sensor grids, and upstream phase shifts. Results show under a 50% probability of upstream data loss, the τ-trimmed TiDE model achieves a 46% reduction in error at the most upstream target, compared to 22% for LSTM. Furthermore, phase misalignment in upstream data introduces a near-linear increase in forecast error. Under moderate model settings, a ±3 s misalignment increases the mean absolute error by approximately 0.5 m, while the same error is accumulated at ±4 s using the more conservative approach. These findings inform the design of resilient, uncertainty-aware wave forecasting systems suited for realistic offshore sensing environments.

42 ENGINEERING↗

DOE OSTI2026

Grid Topology Discovery Algorithm Evaluation of Suitability for Utility Deployment (CRADA 606 Final Report)

This work presents the results of a field-informed demonstration aimed at evaluating the practical suitability of a topology discovery algorithm for utility environments. We demonstrated an algorithm that uses a graph-theory-informed state estimation approach for model selection. In collaboration with Survalent and Peninsula Light Co., the algorithm was applied to real feeder models and field measurements from supervisory control and data acquisition (SCADA) and advanced metering infrastructure (AMI) systems to identify the operational topology of a power distribution system. The demonstration assessed the algorithm’s performance under realistic data conditions, including sparse and noisy measurements, and examined its ability to identify the most likely network configurations. The results confirmed that the approach can effectively narrow down feasible topologies, providing operators with improved situational awareness of network status. Key lessons learned emphasize the need for systematic data validation and strategic sensor placement to enhance observability. These insights inform future deployment strategies and guide refinements for broader adoption in utility operations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2023

Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopy

Abstract Modern scanning microscopes can image materials with up to sub-atomic spatial and sub-picosecond time resolutions, but these capabilities come with large volumes of data, which can be difficult to store and analyze. We report the Fast Autonomous Scanning Toolkit (FAST) that addresses this challenge by combining a neural network, route optimization, and efficient hardware controls to enable a self-driving experiment that actively identifies and measures a sparse but representative data subset in lieu of the full dataset. FAST requires no prior information about the sample, is computationally efficient, and uses generic hardware controls with minimal experiment-specific wrapping. We test FAST in simulations and a dark-field X-ray microscopy experiment of a WSe 2 film. Our studies show that a FAST scan of <25% is sufficient to accurately image and analyze the sample. FAST is easy to adapt for any scanning microscope; its broad adoption will empower general multi-level studies of materials evolution with respect to time, temperature, or other parameters.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Accelerated Nano-Optical Imaging through Sparse Sampling

The integration time and signal-to-noise ratio are inextricably linked when performing scanning probe microscopy based on raster scanning. This often yields a large lower bound on the measurement time, for example, in nano-optical imaging experiments performed using a scanning near-field optical microscope (SNOM). Here, in this study, we utilize sparse scanning augmented with Gaussian process regression to bypass the time constraint. We apply this approach to image charge-transfer polaritons in graphene residing on ruthenium trichloride (α-RuCl 3 ) and obtain key features such as polariton damping and dispersion. Critically, nano-optical SNOM imaging data obtained via sparse sampling are in good agreement with those extracted from traditional raster scans but require 11 times fewer sampled points. As a result, Gaussian process-aided sparse spiral scans offer a major decrease in scanning time.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Using AI to build a hydrobiogeochemical soil model

Soil water content is a function of inputs from precipitation and outputs via evaporation, transpiration, lateral flow, and vertical percolation, and is sensitive to biogeochemical processes. As such, soils serve as an ideal integrator of atmospheric, hydrological, and biogeochemical processes affecting the water cycle. In addition, soil water retention capacity, infiltration rates, and hydraulic conductivity can buffer or exacerbate the effects of extreme precipitation events (e.g., flooding, runoff, subsurface transport, erosion, greenhouse gas emissions) and mitigate the impact of droughts and heat waves on land systems (e.g., fire, crop failure). However, integrating water cycle measurements spanning different land atmosphere compartments across scales is a fundamental barrier for numerical model predictability. A significant challenge is that each domain (soil, hydrology, biology, and atmosphere) typically collects different sets of data at different temporal and spatial frequencies/scales, and even different dimensionalities (2D vs 3D). To implement soil as an integrator of the water cycle in land models, we suggest that novel machine learning (ML) tools can be developed to effectively simulate complex landscapes across various domains and scales, extended to regions with sparse or no data. The ultimate goals are to improve predictive understanding of land-atmosphere interactions and to extend the predictability of current Earth System Models (ESMs) through better integration of hydrological and biogeochemical data. We envision a framework in which: (1) ML-aided data reconstructions enable the merger of data sources into a unified geospatial product; (2) automated detection techniques are used to improve the knowledge of complex soil processes and interactions; and (3) this knowledge is leveraged and incorporated into models through AI-based emulators to distinctly connect the land and atmospheric compartments of the water cycle in models.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1995

Reduction and analysis of seasons 15 and 16 (1991 - 1992) Pioneer Venus radio occulation data and correlative studies with observations of the near infrared emission of Venus

In this study, we sought to characterize variations in the abundance and distribution of subcloud H2SO4(g) in the Venus atmosphere by using a number of 13cm radio occultation measurements conducted with the Pioneer Venus Orbiter near the inferior conjunction of 1991. A total of ten data sets were examined and analyzed, producing vertical profiles of temperature and pressure in the neutral atmosphere, and sulfuric acid vapor abundance below the main cloud layer. Two of the vertical profiles of the abundance of H2SO4(g) were correlated with NIR images of the night side of Venus made during the same period of time by Boris Ragent (under a separate PVO Guest Investigator Grant). Initially, we had hoped that the combination of these two different types of data would make it possible to constrain or identify the composition of the large particles causing the features observed in the NIR images. However, the sparseness of the radio occultation data set, combined with the sparseness of the NIR data set (one image per day over an 8 day period) made it impossible to draw strong conclusions. Considered on their own, however, the parameters retrieved from the radio occultation experiments are valuable science products.

Jenkins, Jon M.↗

DOE OSTI2026

A ModEx Framework for Watershed Subsurface Investigation With Limited Geophysical Data Using Machine Learning and Hydrologic Modeling

Abstract Subsurface heterogeneity influences watershed hydrology strongly but remains difficult to characterize at catchment scales with sparse and costly field data. Geophysical surveys such as electromagnetic induction (EMI) provide local spatial subsurface images yet scaling them to watershed scales and converting EMI‐derived resistivity into hydraulic properties remains a challenge. We present a Model–Experiment (ModEx) framework that integrates limited EMI data with machine learning (ML) and hydrologic modeling to improve process representation and guide field investigations. Sparse EMI surveys were scaled to the catchment scale using a Random Forest model, and the resulting resistivity fields were combined with nearby borehole constraints to parameterize a hydrologic model. The EMI‐informed hydrological simulations improved predictions of streamflow sustained by subsurface flow and shallow saturation patterns. By combining EMI data and ML with hydrologic modeling, the ModEx framework guides future subsurface surveys, providing a transferable and efficient strategy for data–model integration across diverse watersheds. Plain Language Summary Mapping the underground network of soil and rock that controls water is essential for predicting floods and droughts, but seeing underground is difficult and expensive. We cannot drill everywhere, so scientists use geophysical tools to scan broad areas. There are two key challenges: these geophysical scans are often sparse across the whole watershed, and the geophysical data is hard to translate into water‐related properties. We used artificial intelligence to solve these problems. We taught a computer to find patterns linking the limited geophysical data to the land surface properties. This allowed it to fill in the gaps and create a complete, useful subsurface map for the entire watershed. This new map improves hydrologic simulations, leading to more accurate predictions of water movement in the watershed. It also helps scientists build better models with less data and generates a priority map showing where to measure next, making future investigations more efficient. Key Points Limited EMI scaled with ML improves catchment‐scale subsurface parameterization for hydrologic models The framework integrates hydrologic modeling with limited geophysical data to support subsurface investigation design ModEx framework offers a transferable data–model integration strategy that quantifies and reduces uncertainty guiding watershed studies

Chen, Hang↗

DOE OSTI2021

PLANC: Parallel Low-rank Approximation with Nonnegativity Constraints

In this work, we consider the problem of low-rank approximation of massive dense nonnegative tensor data, for example, to discover latent patterns in video and imaging applications. As the size of data sets grows, single workstations are hitting bottlenecks in both computation time and available memory. We propose a distributed-memory parallel computing solution to handle massive data sets, loading the input data across the memories of multiple nodes, and performing efficient and scalable parallel algorithms to compute the low-rank approximation. We present a software package called Parallel Low-rank Approximation with Nonnegativity Constraints, which implements our solution and allows for extension in terms of data (dense or sparse, matrices or tensors of any order), algorithm (e.g., from multiplicative updating techniques to alternating direction method of multipliers), and architecture (we exploit GPUs to accelerate the computation in this work). We describe our parallel distributions and algorithms, which are careful to avoid unnecessary communication and computation, show how to extend the software to include new algorithms and/or constraints, and report efficiency and scalability results for both synthetic and real-world data sets.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2015

The Parameterization of Top-Hat Particle Sensors with Microchannel-Plate-Based Detection Systems and its Application to the Fast Plasma Investigation on NASA's Magnetospheric MultiScale Mission

The most common instrument for low energy plasmas consists of a top-hat electrostatic analyzer geometry coupled with a microchannel-plate (MCP)-based detection system. While the electrostatic optics for such sensors are readily simulated and parameterized during the laboratory calibration process, the detection system is often less well characterized. Furthermore, due to finite resources, for large sensor suites such as the Fast Plasma Investigation (FPI) on NASA's Magnetospheric Multiscale (MMS) mission, calibration data are increasingly sparse. Measurements must be interpolated and extrapolated to understand instrument behavior for untestable operating modes and yet sensor inter-calibration is critical to mission success. To characterize instruments from a minimal set of parameters we have developed the first comprehensive mathematical description of both sensor electrostatic optics and particle detection systems. We include effects of MCP efficiency, gain, scattering, capacitive crosstalk, and charge cloud spreading at the detector output. Our parameterization enables the interpolation and extrapolation of instrument response to all relevant particle energies, detector high voltage settings, and polar angles from a small set of calibration data. We apply this model to the 32 sensor heads in the Dual Electron Sensor (DES) and 32 sensor heads in the Dual Ion Sensor (DIS) instruments on the 4 MMS observatories and use least squares fitting of calibration data to extract all key instrument parameters. Parameters that will evolve in flight, namely MCP gain, will be determined daily through application of this model to specifically tailored in-flight calibration activities, providing a robust characterization of sensor suite performance throughout mission lifetime. Beyond FPI, our model provides a valuable framework for the simulation and evaluation of future detection system designs and can be used to maximize instrument understanding with minimal calibration resources.

Instrument-model↗

NASA NTRS2006

Temperature-dependent Absolute Refractive Index Measurements of Synthetic Fused Silica

Using the Cryogenic, High-Accuracy Refraction Measuring System (CHARMS) at NASA's Goddard Space Flight Center, we have measured the absolute refractive index of five specimens taken from a very large boule of Corning 7980 fused silica from temperatures ranging from 30 to 310 K at wavelengths from 0.4 to 2.6 microns with an absolute uncertainty of plus or minus 1 x 10 (exp -5). Statistical variations in derived values of the thermo-optic coefficient (dn/dT) are at the plus or minus 2 x 10 (exp -8)/K level. Graphical and tabulated data for absolute refractive index, dispersion, and thermo-optic coefficient are presented for selected wavelengths and temperatures along with estimates of uncertainty in index. Coefficients for temperature-dependent Sellmeier fits of measured refractive index are also presented to allow accurate interpolation of index to other wavelengths and temperatures. We compare our results to those from an independent investigation (which used an interferometric technique for measuring index changes as a function of temperature) whose samples were prepared from the same slugs of material from which our prisms were prepared in support of the Kepler mission. We also compare our results with sparse cryogenic index data from measurements of this material from the literature.

Leviton, Douglas B.↗

NASA NTRS1987

Monitoring global vegetation using Nimbus-7 37 GHz data - Some empirical relations

The difference of the vertically and horizontally polarized brightness temperatures observed by the 37 GHz channel of the SMMR on board the Nimbus-7 satellite are correlated temporally with three indicators of vegetation density, namely the temporal variation of the atmospheric CO2 concentration at Mauna Loa (Hawaii), rainfall over the Sahel and the normalized difference vegetation index derived from the AVHRR on board the NOAA-7 satellite. SMMR 37 GHz and AVHRR provide complementary data sets for monitoring global vegetation, the 37 GHz data being more suitable for arid and semiarid regions as these data are more sensitive to changes in sparse vegetation. The 37-GHz data might be useful for understanding desertification and indexing Co2 exchange between the biosphere and the atmosphere.

Choudhury, B. J.↗

NASA NTRS2019

Online Visualization and Analysis of NASA Satellite-Based Global and Regional Precipitation Products Through Giovanni

Observational data are essential for Earth science research and applications. Traditional ground-based observations suffer from many limitations (e.g. costly deployment). As a result, data are often sparse and inconsistent, especially over vast oceans that cover nearly 71% of the Earth's surface, and for remote continents. Precipitation is one of the important physical parameters in the global hydrological cycle and other disciplines. Each year, severe floods and droughts happen in different parts of the world and cause significant damage to the economy, as well as human casualties (e.g. Hurricane Katrina, the Dust Bowl). Accurate and timely precipitation observations and predictions are important for research and applications. However, ground-based precipitation observations are quite limited, especially in remote and mountainous regions. Since the satellite era began, satellite-based precipitation products have gained popularity in Earth science research, applications, and education. Accessing satellite products can be a daunting task to many users, especially those who do not have prior experience or knowledge with satellite data. Recognizing this obstacle, the NASA Goddard Earth Sciences and Data and Information Services Center (GES DISC), home to data archives for the NASA-JAXA Tropical Rainfall Measuring Mission (TRMM) and Global Precipitation Measurement (GPM), has developed data services including an online visualization and analysis tool, Giovanni (the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure), enabling users at different levels to access, explore, and evaluate NASA satellite-based data products without downloading either data and software, or requiring coding. Currently, global and regional precipitation products from different satellite missions (TRMM, GPM) and projects (e.g. the Modern Era Retrospective-analysis for Research and Applications Version 2 (MERRA-2), and the North American Land Data Assimilation System (NLDAS)), ranging from half-hourly to monthly temporal resolution, are available in Giovanni. There are over 1900 variables in Giovanni, covering measurements in precipitation, hydrology, atmospheric dynamics, atmospheric chemistry, etc. In this poster presentation, we will provide a live demonstration of Giovanni and its latest development, including precipitation-related variables, and new basic features such as polar projections. The session will also provide a Q&A opportunity for attendees.

Giovanni↗

NASA NTRSDate not supplied

Why Developing A Full-Sun UV Spectrograph Sounding Rocket Is Difficult but Necessary

For a conference focused on a 20-year perspective of where we have been and where we are going, this talk will highlight specifically why we need a sounding rocket to lead the way in obtaining high-resolution UV spectroscopy of our sun-as-a-star. The currently available data is either sparse or at a much lower resolution when compared with Hubble (HST) spectra of other stars. Because of this, we cannot say with certainty if our Sun’s variability is typical. This limitation is hindering our quest to place our Sun in it’s rightful place on our star-charts. Ultraviolet light is a key component in determining this. Elements such as Hydrogen and Iron have strong emission lines here, and from them we can determine many things about the star from which they originate. The width of these spectral features can tell us about the relative plasma activity on the surface of the star, the abundance of those elements, and of course the temperatures. This has of course been done with many instruments that have very narrow field of views and/or narrow bandwidths. The only way to directly compare our Sun’s UV ”fingerprint” with other stars is to take an image of it in the same manner by which HST does: as a point source object. Our team’s aim has been to develop a sounding rocket capable of doing just that. Along the way, the many challenges we face are producing many solutions. A unique take on the traditional Rowland circle spectrograph will allow us to cover a large range of UV spectra with minimal moving parts. A novel in-situ gain calibration technique will be used in order to maintain the high level of radiometric accuracy desired. Naturally, complications arise when one takes UV measurements as you are flying out of earth’s atmosphere and back into it. Though it may be minor, the absorbed UV spectra from the atmosphere will have to be accounted for. In preparing for this, we have stumbled across a way in which those ”telluric” absorption lines may prove useful for spectral calibration purposes. In addition, raw data from previous sounding rocket missions may have hidden within them useful information for the atmospheric-physics community. In this talk, we will highlight the unique calibration problems and solutions we have faced, and how those are of greater importance to the greater heliophysics and atmospheric communities. With these amazing developments at hand, the future of heliophysics instrumentation is looking bright. This material is based upon work supported by the NSF EPSCoR RII-Track-1.2a (Non-invasive plasma diagnostics for LTP) Cooperative Agreement OIA-1655280. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

NASA, UAH, MSU, Sounding Rocket, Aeronomy, Instrum↗

DOE OSTI2023

Artificial Intelligence Guided Studies of van der Waals Magnets

A materials informatics framework to explore a large number of candidate van der Waals (vdW) materials is developed. In particular, in this study a large space of monolayer transition metal halides is investigated by combining high-throughput density functional theory calculations and artificial intelligence (AI) to accelerate the discovery of stable materials and the prediction of their magnetic properties. Here, the formation energy is used as a proxy for chemical stability. Semi-supervised learning is harnessed to mitigate the challenges of sparsely labeled materials data in order to improve the performance of AI models. This approach creates avenues for the rapid discovery of chemically stable vdW magnets by leveraging the ability of AI to recognize patterns in data, to learn mathematical representations of materials from data and to predict materials properties. Using this approach, previously unexplored vdW magnetic materials with potential applications in data storage and spintronics are identified.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Machine Learning Eliminates Reanalysis Warm Bias and Reveals Weaker Winter Surface Cooling Over Arctic Sea Ice

The surface energy budget governs Arctic sea-ice growth/melt, yet observations are sparse, and reanalysis data sets suffer from systematic biases. Here, we train a neural network with observational data to bias-correct hourly ERA5 fluxes over Arctic ice-covered regions (≥70°N; sea-ice concentration >80%) for 1994–2024. Training data cover two full seasonal cycles and different sea-ice regimes. The neural network reduces RMSE for net shortwave radiation by ∼40%, downward longwave radiation by ∼16% and the total surface energy budget by ∼55%, eliminating the wintertime warm bias of ∼4 K in ERA5. Wintertime surface cooling is reduced by ∼50%, yielding thermodynamic ice-growth estimates of ∼80–120 cm, consistent with SMOS–CryoSat satellite thickness increases and in contrast to the 150–200 cm growth implied by ERA5. Our bias-corrected data capture the observed clear/cloudy states of the winter boundary layer and can be used to study Arctic climatology, evaluate climate models and drive sea-ice-ocean models.

Hossain, Akil [Alfred Wegener Institute for Polar ↗

DOE OSTI2024

Efficient learning of accurate surrogates for simulations of complex systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse or dynamic data. Here, we introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data, and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation-of-state model can be reliably autogenerated from expensive calculations using few model evaluations.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Bayesian Gaussian process inference for neutron spin echo measurement

Neutron spin echo (NSE) spectroscopy provides unique access to microscopic dynamics, but its application is often constrained by low neutron flux, long acquisition times, and significant noise. Here, we present a Bayesian inference approach based on Gaussian process regression (GPR) to reconstruct high-quality spin echo signals from sparse and noisy data by exploiting correlations in reciprocal space. Benchmarks on synthetic datasets and validation with experimental NSE measurements of dendrimers show that GPR suppresses noise, interpolates missing intensity values, and accommodates irregular observations. The method improves accuracy, shortens acquisition times, and enables high-throughput and real-time studies. Beyond NSE, the framework is broadly applicable to other low signal-to-noise ratio scattering techniques, thereby extending the scope of neutron spectroscopy.

Tung, Chi-Huan [Oak Ridge National Laboratory (ORN↗

DOE OSTI2025

Measurement of the mean excitation energy of liquid argon

The mean excitation energy (I-value) of liquid argon is a critical input for energy estimation in neutrino oscillation experiments. It is measured to be 199 +4 -5 eV using the range of 402.2 MeV protons from the Fermilab Linac. This compares to the author's recent evaluation of (197 ± 7) eV based on a combination of an oscillator strength distribution analysis, gaseous argon range measurements, sparse stopping power data on solid argon, and an extrapolation of data on the effect of phase from other substances. Using all sources of information, we recommend a value of (198 ± 4) eV for liquid argon, which is significantly higher than 188 eV, from ICRU-37's gaseous argon evaluation, commonly used in Monte Carlo codes such as Geant4.

47 OTHER INSTRUMENTATION↗