SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

All sources NASA NTRS DOE OSTI

DOE OSTI2022

The High Latitude Spectroscopic Survey on the Nancy Grace Roman Space Telescope

The Nancy Grace Roman Space Telescope will conduct a High Latitude Spectroscopic Survey (HLSS) over a large volume at high redshift, using the near-IR grism (1.0–1.93 μm, R = 435–865) and the 0.28 deg 2 wide-field camera. We present a reference HLSS that maps 2000 deg 2 and achieves an emission-line flux limit of 10 -16 erg s -1 cm -2 at 6.5σ, requiring ~0.6 yr of observing time. We summarize the flowdown of the Roman science objectives to the science and technical requirements of the HLSS. We construct a mock redshift survey over the full HLSS volume by applying a semianalytic galaxy formation model to a cosmological N-body simulation and use this mock survey to create pixel-level simulations of 4 deg 2 of HLSS grism spectroscopy. We find that the reference HLSS would measure ~10 million H α galaxy redshifts that densely map large-scale structure at z = 1–2 and 2 million [O III] galaxy redshifts that sparsely map structures at z = 2–3. We forecast the performance of this survey for measurements of the cosmic expansion history with baryon acoustic oscillations and the growth of large-scale structure with redshift-space distortions. We also study possible deviations from the reference design and find that a deep HLSS at f line > 7 × 10 -17 erg s -1 cm -2 over 4000 deg 2 (requiring ~1.5 yr of observing time) provides the most compelling stand-alone constraints on dark energy from Roman alone. This provides a useful reference for future optimizations. The reference survey, simulated data sets, and forecasts presented here will inform community decisions on the final scope and design of the Roman HLSS.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2001

A TRMM-Calibrated Infrared Rainfall Algorithm Over Brazil

An improved version of the Convective/Stratiform Technique (CST) and its application over Brazil are presented. Keeping the major components of traditional CST, we have modified the scheme for classing convective and stratiform rainfall and tested various schemes to eliminate non-raining cirrus clouds. The parameters of the technique are calibrated using Tropical Rainfall Measuring Mission (TRMM) multi-sensor observations, including TMI, PR and Visible Infrared Scanner (VIRS). This technique takes advantage of the high temporal sampling rate of geosynchronous infrared channel and the better instantaneous rain observation of TRMM Microwave Imager (TMI) and PR. Moreover, sparse TMI rain estimates are used to dynamically calibrate the parameters of the technique to improve its performance. The technique is applied to make rainfall estimates on various time scales and to study rainfall statistics such as the distributions of rain intensity and storm duration over a four-month period beginning January 1999. The study period coincides with the TRMM/Large-Scale Biosphere-Atmosphere experiment in Amazonia (LBA) ground validation experiment, and observations from the LBA radar are used to validate the technique. Results show the mean diurnal cycle of precipitation over the LBA area, and are compared to radar data from the Tropical Oceans and Global Atmosphere (TOGA) radar. When the larger geographic region is considered, the analysis of the mean hourly estimates revealed the pronounced effects of rivers, topography and local circulations on the rainfall.

Negri, Andrew J.↗

NASA NTRSDate not supplied

Development of the GeoNEX Level 2G Products: Exploiting the Diurnal Variability of TOA Reflectance in Atmospheric Correction

This study develops a new atmospheric correction algorithm to generate the Level 2G products, in particular the gap-filled Surface Reflectance at 10-minute time steps, for the Geostationary-NASA Earth Exchange (GeoNEX) project. The algorithm is based on the MODIS MAIAC (Multi-Angle Implementation of Atmospheric Correction) framework but with significant modifications to exploit angular/temporal information from the diurnal variability of the GeoNEX L1G TOA (Top-of-Atmosphere) reflectance. The algorithm starts by evaluating the roughness/smoothness of the diurnal time series of the TOA reflectance. Because rapid changes in TOA reflectance are generally caused by passing clouds or shadows, rough segments of the time series are automatically filtered out while the smooth segments are further tested for brightness and temperature to identify clear-sky and snow-free observations. Next the algorithm runs the MAIAC RTM (Radiative Transfer Model) to retrieve the Ross-Thick-Li-Sparse (RTLS) BRDF model parameters and the daily-mean atmospheric optical depth (AOD) that allow the RTM to optimally simulate the observed diurnal variability of clear-sky TOA reflectance. Once the initial RTLS parameters are retrieved after the algorithm’s burn-in period, they are used as the prior information to predict the AOD level for the next days, while the subsequent clear-sky observations are used to make necessary adjustments to the RTLS parameters in an continuous fashion. This “prediction-analysis” cycle is then iterated to process the full time series of the L1G data, skipping only total-cloudy days or when surface snow is detected. We tested the algorithm over a list of selected AERONET sites. The retrieved results (the daily mean AOD and the RTLS parameters) reasonably agree with the ground-based measurements. Importantly, the results indicate that the diurnal cycles of surface reflectance are continuous functions of the illumination-view geometry. Thus we can use the retrieved RTLS model to accurately fill in data gaps on partial cloudy days. Also, our algorithm is totally independent from the traditional approaches based on the use of spectral band ratios between the shortwave infrared (e.g., 2.2µm) and the visible (e.g., 0.47µm and 0.64µm) bands. Our results thus demonstrate that the high-frequent diurnal geostationary observations contain unique information that helps us improve atmospheric correction of remote sensing data.

GeoNEX↗

DOE OSTI2023

Non-asymptotic analysis of ensemble Kalman updates: effective dimension and localization

Many modern algorithms for inverse problems and data assimilation rely on ensemble Kalman updates to blend prior predictions with observed data. Ensemble Kalman methods often perform well with a small ensemble size, which is essential in applications where generating each particle is costly. This paper develops a non-asymptotic analysis of ensemble Kalman updates, which rigorously explains why a small ensemble size suffices if the prior covariance has moderate effective dimension due to fast spectrum decay or approximate sparsity. Here, we present our theory in a unified framework, comparing everal implementations of ensemble Kalman updates that use perturbed observations, square root filtering and localization. As part of our analysis, we develop new dimension-free covariance estimation bounds for approximately sparse matrices that may be of independent interest.

Mathematics↗

NASA NTRS2014

Comparison of Land Skin Temperature from a Land Model, Remote Sensing, and In-situ Measurement

Land skin temperature (Ts) is an important parameter in the energy exchange between the land surface and atmosphere. Here hourly Ts from the Community Land Model Version 4.0, MODIS satellite observations, and in-situ observations in 2003 were compared. Compared with the in-situ observations over four semi-arid stations, both MODIS and modeled Ts show negative biases, but MODIS shows an overall better performance. Global distribution of differences between MODIS and modeled Ts shows diurnal, seasonal, and spatial variations. Over sparsely vegetated areas, the model Ts is generally lower than the MODIS observed Ts during the daytime, while the situation is opposite at nighttime. The revision of roughness length for heat and the constraint of minimum friction velocity from Zeng et al. [2012] bring the modeled Ts closer to MODIS during the day, and have little effect on Ts at night. Five factors contributing to the Ts differences between the model and MODIS are identified, including the difficulty in properly accounting for cloud cover information at the appropriate temporal and spatial resolutions, and uncertainties in surface energy balance computation, atmospheric forcing data, surface emissivity, and MODIS Ts data. These findings have implications for the cross-evaluation of modeled and remotely sensed Ts, as well as the data assimilation of Ts observations into Earth system models.

Modeling↗

NASA NTRS2011

Sparse Solutions for Single Class SVMs: A Bi-Criterion Approach

In this paper we propose an innovative learning algorithm - a variation of One-class nu Support Vector Machines (SVMs) learning algorithm to produce sparser solutions with much reduced computational complexities. The proposed technique returns an approximate solution, nearly as good as the solution set obtained by the classical approach, by minimizing the original risk function along with a regularization term. We introduce a bi-criterion optimization that helps guide the search towards the optimal set in much reduced time. The outcome of the proposed learning technique was compared with the benchmark one-class Support Vector machines algorithm which more often leads to solutions with redundant support vectors. Through out the analysis, the problem size for both optimization routines was kept consistent. We have tested the proposed algorithm on a variety of data sources under different conditions to demonstrate the effectiveness. In all cases the proposed algorithm closely preserves the accuracy of standard one-class nu SVMs while reducing both training time and test time by several factors.

Das, Santanu↗

NASA NTRS2010

Sparse Superpixel Unmixing for Hyperspectral Image Analysis

Software was developed that automatically detects minerals that are present in each pixel of a hyperspectral image. An algorithm based on sparse spectral unmixing with Bayesian Positive Source Separation is used to produce mineral abundance maps from hyperspectral images. A superpixel segmentation strategy enables efficient unmixing in an interactive session. The algorithm computes statistically likely combinations of constituents based on a set of possible constituent minerals whose abundances are uncertain. A library of source spectra from laboratory experiments or previous remote observations is used. A superpixel segmentation strategy improves analysis time by orders of magnitude, permitting incorporation into an interactive user session (see figure). Mineralogical search strategies can be categorized as supervised or unsupervised. Supervised methods use a detection function, developed on previous data by hand or statistical techniques, to identify one or more specific target signals. Purely unsupervised results are not always physically meaningful, and may ignore subtle or localized mineralogy since they aim to minimize reconstruction error over the entire image. This algorithm offers advantages of both methods, providing meaningful physical interpretations and sensitivity to subtle or unexpected minerals.

Castano, Rebecca↗

NASA NTRS2018

Exoplanet Biosignatures: At the Dawn of a New Era of Planetary Observations

The rapid rate of discoveries of exoplanets has expanded the scope of the science possible for the remote detection of life beyond Earth. The Exoplanet Biosignatures Workshop-Without-Walls (EBWWW) held in 2016 engaged the international scientific community across diverse scientific disciplines, to assess the state of the science and technology in the search for life on exoplanets, and to identify paths for progress. The workshop activities resulted in five (5) major review papers which provide: 1) an encyclopedic review of known and proposed biosignatures and models used to ascertain them; 2) an in-depth review of O2 as a biosignature, rigorously examining the nuances of false positives and negatives for evidence of life; 3) a Bayesian framework to comprehensively organize current understanding to quantify confidence in biosignature assessments; 4) an extension of that Bayesian framework in anticipation of increasing planetary data and novel concepts of biosignatures, and 5) a review of the upcoming telescope capabilities to characterize exoplanets and their environment. Because of the immense content of these review papers, this summary provides a guide to their complementary scope and highlights salient features. Strong themes that emerged from the workshop were that biosignatures must be interpreted in the context of their environment, and that frameworks must be developed to link diverse forms of scientific understanding of that context to quantify the likelihood that a biosignature has been observed. Models are needed to explore the parameter space where measurements will be widespread but sparse in detail. Given the technological prospects for large ground-based telescopes and direct imaging from space-based observatories, the detection of atmospheric signatures of a few potentially habitable planets may come before 2030.

spectral imaging↗

NASA NTRS2018

Evaluation of Opportunistic Contact Graph Routing in Random Mobility Environments

Routing in networks where nodes move randomly is particularly challenging due their potentially unpredictable, and rapidly changing topology. Several routing algorithms have been presented in the literature to address the needs of such networks, most of them implementing variants of controlled network ﬂooding in the hope of successful data delivery. In this note, we compare the results of previous routing algorithms with Opportunistic Contact Graph Routing (OCGR), an enhanced version of Contact Graph Routing (CGR) that is suitable for networks where contacts cannot always be scheduled ahead of time. To perform the benchmark, we simulate a network of nodes moving in a certain space according to the Random Waypoint Mobility Model, and then take measurements of bundle delivery probabilty and overhead ratio as metrics of performance and cost respectively. Through this exercise, we demonstrate that the performance of OCGR is highly dependent on the type of network under consideration (e.g. very sparse vs. densely connected) and the assumed mobility model.

Burleigh, Scott↗

DOE OSTI2022

A QUBO formulation for top-τ eigencentrality nodes

The efficient calculation of the centrality or “hierarchy” of nodes in a network has gained great relevance in recent years due to the generation of large amounts of data. The eigenvector centrality (aka eigencentrality) is quickly becoming a good metric for centrality due to both its simplicity and fidelity. In this work we lay the foundations for solving the eigencentrality problem of ranking the importance of the nodes of a network with scores from the eigenvector of the network, using quantum computational paradigms such as quantum annealing and gate-based quantum computing. The problem is reformulated as a quadratic unconstrained binary optimization (QUBO) that can be solved on both quantum architectures. The results focus on correctly identifying a given number of the most important nodes in numerous networks given by the sparse vector solution of our QUBO formulation of the problem of identifying the top- τ highest eigencentrality nodes in a network on both the D-Wave and IBM quantum computers.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Tucana B: A Potentially Isolated and Quenched Ultra-faint Dwarf Galaxy at $D$ ≈ 1.4 Mpc*

We report the discovery of Tucana B, an isolated ultra-faint dwarf galaxy at a distance of $D$ = 1.4 Mpc. Tucana B was found during a search for ultra-faint satellite companions to the known dwarfs in the outskirts of the Local Group, although its sky position and distance indicate the nearest galaxy to be ~500 kpc distant. Deep ground-based imaging resolves Tucana B into stars, and it displays a sparse red giant branch consistent with an old, metal-poor stellar population analogous to that seen in the ultra-faint dwarf galaxies of the Milky Way, albeit at fainter apparent magnitudes. Tucana B has a half-light radius of 80 ± 40 pc and an absolute magnitude of ${M}_{V}=-{6.9}_{-0.6}^{+0.5}$ mag (${L}_{V}=({5}_{-2}^{+4})\times {10}^{4}$ L⊙), which is again comparable to the Milky Way's ultra-faint satellites. There is no evidence for a population of young stars, either in the optical color–magnitude diagram or in GALEX archival ultraviolet imaging, with the GALEX data indicating $\mathrm{log}({\mathrm{SFR}}_{\mathrm{NUV}}/{M}_{\odot }\,{\mathrm{yr}}^{-1})\lt -5.4$ for star formation on ≲100 Myr timescales. Given its isolation and physical properties, Tucana B may be a definitive example of an ultra-faint dwarf that has been quenched by reionization, providing strong confirmation of a key driver of galaxy formation and evolution at the lowest mass scales. It also signals a new era of ultra-faint dwarf galaxy discovery at the extreme edges of the Local Group.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS1999

Middle Atmosphere Transport Properties of Assimilated Datasets

One of the most compelling reasons for performing data assimilation in the middle atmosphere is to obtain global, balanced datasets for studies of trace gas transport and chemistry. This is a major motivation behind the Goddard Earth observation System-Data Assimilation System (GEOS-DAS). Previous studies have shown that while this and other data assimilation systems can generally obtain good estimates of the extratropical rotational velocity field, the divergent part of the dynamical field is deficient; this impacts the "residual circulation" and leads to spurious trace gas transport on seasonal and interannual timescales. These problems are impacted by the quality and the method of use of the observational data and by deficiencies in the atmospheric general circulation model. Whichever the cause at any place and time, the "solution" is to introduce non-physical forcing terms into the system (the so-called incremental analysis updates); these can directly (thermal) or indirectly (mechanical) affect the residual circulation. This paper will illustrate how the divergent circulation is affected by deficiencies in both observations and models. Theoretical considerations will be illustrated with examples from the GEOS-DAS and from simplified numerical experiments. These are designed to isolate known problems, such as the inability of models to sustain a quasi-biennial oscillation and sparse observational constraints on tropical dynamics, or radiative inconsistencies in the presence of volcanic aerosols.

Pawson, Steven↗

NASA NTRS1999

Middle Atmospheric Transport Properties of Assimilated Datasets

Pawson, Steven↗

NASA NTRS2011

Estimating Evapotranspiration Using an Observation Based Terrestrial Water Budget

Evapotranspiration (ET) is difficult to measure at the scales of climate models and climate variability. While satellite retrieval algorithms do exist, their accuracy is limited by the sparseness of in situ observations available for calibration and validation, which themselves may be unrepresentative of 500m and larger scale satellite footprints and grid pixels. Here, we use a combination of satellite and ground-based observations to close the water budgets of seven continental scale river basins (Mackenzie, Fraser, Nelson, Mississippi, Tocantins, Danube, and Ubangi), estimating mean ET as a residual. For any river basin, ET must equal total precipitation minus net runoff minus the change in total terrestrial water storage (TWS), in order for mass to be conserved. We make use of precipitation from two global observation-based products, archived runoff data, and TWS changes from the Gravity Recovery and Climate Experiment satellite mission. We demonstrate that while uncertainty in the water budget-based estimates of monthly ET is often too large for those estimates to be useful, the uncertainty in the mean annual cycle is small enough that it is practical for evaluating other ET products. Here, we evaluate five land surface model simulations, two operational atmospheric analyses, and a recent global reanalysis product based on our results. An important outcome is that the water budget-based ET time series in two tropical river basins, one in Brazil and the other in central Africa, exhibit a weak annual cycle, which may help to resolve debate about the strength of the annual cycle of ET in such regions and how ET is constrained throughout the year. The methods described will be useful for water and energy budget studies, weather and climate model assessments, and satellite-based ET retrieval optimization.

Rodell, Matthew↗

DOE OSTI2025

Variance Preserving Spectral Subsampling

Generating statistically faithful short-duration gamma-ray spectra from a single long measurement is essential in nuclear safeguards, supporting tasks such as algorithm development and machine-learning applications, especially when list-mode data are unavailable. Existing subsampling methods often distort the statistical characteristics of genuine short-duration measurements, leading to biased or unreliable analytical outcomes and thereby undermining downstream tasks. In this work, we compare five subsampling approaches using a benchmark set of 156 genuine replicate spectra collected with a high-purity germanium detector. We evaluate each method with respect to run-to-run variance, channel-to-channel variance, and preservation of total counts (losslessness). Across a wide range of subsampling ratios, only binomial subsampling without replacement consistently reproduces the statistical properties of genuine short-duration spectra, maintaining proper dispersion even in sparse spectral regions and perfectly preserving total counts. These results provide a mathematically principled and practically validated framework for generating synthetically shortened spectra when true short-duration measurements are unavailable.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

NASA NTRS1994

Parallel Preconditioning for CFD Problems on the CM-5

Up to today, preconditioning methods on massively parallel systems have faced a major difficulty. The most successful preconditioning methods in terms of accelerating the convergence of the iterative solver such as incomplete LU factorizations are notoriously difficult to implement on parallel machines for two reasons: (1) the actual computation of the preconditioner is not very floating-point intensive, but requires a large amount of unstructured communication, and (2) the application of the preconditioning matrix in the iteration phase (i.e. triangular solves) are difficult to parallelize because of the recursive nature of the computation. Here we present a new approach to preconditioning for very large, sparse, unsymmetric, linear systems, which avoids both difficulties. We explicitly compute an approximate inverse to our original matrix. This new preconditioning matrix can be applied most efficiently for iterative methods on massively parallel machines, since the preconditioning phase involves only a matrix-vector multiplication, with possibly a dense matrix. Furthermore the actual computation of the preconditioning matrix has natural parallelism. For a problem of size n, the preconditioning matrix can be computed by solving n independent small least squares problems. The algorithm and its implementation on the Connection Machine CM-5 are discussed in detail and supported by extensive timings obtained from real problem data.

Simon, Horst D.↗

NASA NTRS2002

Effects of Ordering Strategies and Programming Paradigms on Sparse Matrix Computations

The Conjugate Gradient (CG) algorithm is perhaps the best-known iterative technique to solve sparse linear systems that are symmetric and positive definite. For systems that are ill-conditioned, it is often necessary to use a preconditioning technique. In this paper, we investigate the effects of various ordering and partitioning strategies on the performance of parallel CG and ILU(O) preconditioned CG (PCG) using different programming paradigms and architectures. Results show that for this class of applications: ordering significantly improves overall performance on both distributed and distributed shared-memory systems, that cache reuse may be more important than reducing communication, that it is possible to achieve message-passing performance using shared-memory constructs through careful data ordering and distribution, and that a hybrid MPI+OpenMP paradigm increases programming complexity with little performance gains. A implementation of CG on the Cray MTA does not require special ordering or partitioning to obtain high efficiency and scalability, giving it a distinct advantage for adaptive applications; however, it shows limited scalability for PCG due to a lack of thread level parallelism.

Oliker, Leonid↗

DOE OSTI2025

Knowledge Oriented Graph Unified Transformer (KOGUT) v0.1

KOGUT — Knowledge Oriented Graph Unified Transformer KOGUT implements the Relational Graph Transformer (RelGT) architecture for knowledge graph link prediction in biological domains, with a primary focus on microbial growth media prediction. While the original RelGT (arXiv:2505.10960) targets relational tables, time series, and multi-table databases, KOGUT adapts this architecture for heterogeneous biological knowledge graphs, providing first-in-class AI predictive models for microbial cultivation. Key Adaptations Beyond Original RelGT: - Knowledge Graph Focus: Applied to biological KGs with semantic node types (taxa, chemicals, media, phenotypes, environments) versus generic relational database tables, trained on the KG-Microbe knowledge graph (1.3M entities, 2.9M edges, 24 relation types). - Multimodal Node Encoding: Integrates node labels, categories, descriptions, and synonyms from KG metadata through learned embedding layers—adapting relational column features to graph node attributes with textual semantics. - Extended K-Hop Subgraph Strategy: Optimized neighborhood sampling (3-hop default, configurable up to 200 nodes) tuned for sparse biological networks, building on the original local-global attention framework with biological relation preservation. - Biolink Predicate Preservation: Type-specific transformations for 24 biological edge semantics (occurs_in, consumes, produces, has_phenotype, subclass_of) beyond standard relational foreign keys, enabling multi-relation link prediction. - Inductive Learning Support: Enables zero-shot predictions for novel taxa through feature-based embeddings (temperature, oxygen requirements, gram stain, cell shape), extending the original transductive relational benchmark scope to uncultured microorganisms. CheapSOTA Performance Optimizations (This Distribution): - VQ-EMA Centroid Attention: Vector quantization with exponential moving average for improved global context modeling (+5-10% MRR improvement). - HDF5 Precomputed Data Loading: One-time preprocessing of k-hop subgraphs to eliminate redundant graph traversals (2-5× training speedup). - Distributed Data Parallel Training: Multi-GPU support for scaling to larger knowledge graphs (tested on 4× NVIDIA A100 GPUs at NERSC Perlmutter). - Mixed Precision Training: Automatic mixed precision (AMP) for memory efficiency and faster training. Advantages Over Standard Knowledge Graph Embedding Models: Combines RelGT's proven multi-element tokenization (features, type, hop, structure) with graph-native biological representations, enabling interpretable link prediction across heterogeneous entities that standard embedding models (TransE, RotatE, ComplEx) and table-based transformers cannot directly model. Achieves near-perfect performance on microbial growth media prediction (MRR: 0.9966, Precision@1: 0.9932, Hit@10: 1.0000) while maintaining explainability through attention-based reasoning over biological pathways. Training Data: - KG-Microbe merged knowledge graph: 1,379,337 nodes, 2,960,472 edges - 24 biological relation types including taxonomic hierarchies, metabolic interactions, phenotype associations, and environmental relationships - Primary prediction task: Growth media suitability for microbial taxa (biolink:occurs_in, 50K edges) - Multi-relation capability: Predicts links for any of the 24 relation types, including chemical consumption/production, phenotype associations, and taxonomic classification Citation: Original RelGT Architecture: Dwivedi et al., "Relational Graph Transformer", arXiv:2505.10960, 2025 KOGUT Implementation: Knowledge Oriented Graph Unified Transformer for Microbial Growth Media Prediction Developed at Lawrence Berkeley National Laboratory (LBNL) Trained on NERSC Perlmutter supercomputer

Joachimiak, Marcin [Lawrence Berkeley National Lab↗