Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparse Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

FLASH: FPGA-Accelerated Smart Switches with GCN Case Study

Some communication switches, e.g., the Mellanox SHArP and those in the IBM BlueGene clusters, are augmented to process packets at the application level with fixed-function collectives. This approach, however, lacks flexibility, which limits their applicability in diverse and dynamic workloads. Recently, a new type of programmable packet processor, which uses high-level languages, e.g., P4, has emerged as possible candidates. P4-based switches, however, fall short in certain applications, including machine learning, where capabilities not currently supported by P4 are needed. These include more complex calculation, such as sparse computation and fused multiply-accumulate, data-intensive floating point operations, data reuse, and significant memory. The problem addressed here is that such a switch augmentation needs to support: a large amount of state, significant flexible compute capability, and ease of programming, all while maintaining full functionality, including ensuring high throughput, and demonstrating utility. In this work, we propose a programmable look-aside-type accelerator that can be embedded into, or attached to, existing communication switch pipelines and that is capable of processing packets at line-rate. The proposed in-switch accelerator is based on mixing an ISA (subset of RISC-V instructions) with dataflow graphs (found in CGRAs). To augment performance, vector instructions are also supported. To facilitate usability, we have developed a complete toolchain to compile user-provided C/C++ codes to appropriate back-end instructions for configuring the accelerator. While this approach is flexible enough to support various workloads, in this paper, we consider Graph Convolutional Networks (GCNs) as a case study. Experimental results show that this approach considerably improves the performance of distributed GCN applications.

Haghi, Pouya↗
NASA NTRS1978

Structural performance analysis and redesign

Program performs stress buckling and vibrational analysis of large, linear, finite-element systems in excess of 50,000 degrees of freedom. Cost, execution time, and storage requirements are kept reasonable through use of sparse matrix solution techniques, and other computational and data management procedures designed for problems of very large size.

Whetstone, W. D.↗
NASA NTRS1988

Asteroid and comet flux in the neighborhood of the earth

Significant advances in the knowledge and understanding of the flux of large solid objects in the neighborhood of Earth have occurred. The best estimates of the collision rates with Earth of asteroids and comets and the corresponding production of impact craters are presented. Approximately 80 Earth-crossing asteroids were discovered through May 1988. Among 42 new Earth-crossing asteroids found in the last decade, two-thirds were discovered from observations at Palomar Observatory and 15 were discovered or independently detected in dedicated surveys with the Palomar Observatory and 15 were discovered or independently detected in dedicated surveys with the Palomar 46 cm Schmidt. Probabilities of collision with Earth have been calculated for about two-thirds of the known Earth-crossing asteroids. When multiplied by the estimated population of Earth-crossers, this yields an estimated present rate of collision about 65 pct higher than that previously reported. Spectrophotometric data obtained chiefly in the last decade show that the large majority of obvserved Earth-crossers are similar to asteroids found in the inner part of the main belt. The number of discovered Earth-crossing comets is more than 4 times greater than the number of known Earth-crossing asteroids, but reliable data on the sizes of comet nuclei are sparse. The flux of comets almost certainly was highly variable over late geologic time, owing to the random perturbation of the Oort comet cloud by stars in the solar neighborhood.

Shoemaker, Eugene M.↗
NASA NTRS1989

Computational architecture for integrated controls and structures design

To facilitate the development of control structure interaction (CSI) design methodology, a computational architecture for interdisciplinary design of active structures is presented. The emphasis of the computational procedure is to exploit existing sparse matrix structural analysis techniques, in-core data transfer with control synthesis programs, and versatility in the optimization methodology to avoid unnecessary structural or control calculations. The architecture is designed such that all required structure, control and optimization analyses are performed within one program. Hence, the optimization strategy is not unduly constrained by cold starts of existing structural analysis and control synthesis packages.

Belvin, W. Keith↗
NASA NTRS2000

Assimilation of Combined Microwave and Lightning Measurement in a Mesoscale Weather Prediction Model

Intermittent measurements of precipitation and integrated water vapor (IWV) distributions were retrieved from the Special Sensor Microwave/Imager (SSM/I) and Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) radiometers. Lightning generates very low frequency (VLF) radio noise pulses called sferics. Those pulses propagate over large distances so that they can be continuously monitored with a sparse network of ground based radio receivers. Sferics data, tuned with intermittent spaceborne microwave radiometer data, were used to generate estimated rainfall that was assimilated into a mesoscale weather prediction model. Both continuous latent heating adjustment and a variational technique are applied as assimilation procedures to evaluate the impact of lightning observations on the forecast of an intense winter squall line over the Gulf of Mexico. Sensitivities to the assimilation of additional measurements such as IWV and sea surface temperature (SST), and measurement errors will also be discussed.

Chang, Dong-Eon↗
NASA NTRS2004

Arctic Strato-Mesospheric Temperature and Wind Variations

Upper stratosphere and mesosphere rocket measurements are actively used to investigate interaction between the neutral, electrical, and chemical atmospheres and between lower and upper layers of these regions. Satellite temperature measurements from HALOE and from inflatable falling spheres complement each other and allow illustrations of the annual cycle to 85 km altitude. Falling sphere wind and temperature measurements reveal variability that differs as a function of altitude, location, and time. We discuss the state of the Arctic atmosphere during the summer 2002 (Andoya, Norway) and winter 2003 (ESRANGE, Sweden) campaigns of MaCWAVE. Balloon-borne profiles to 30 km altitude and sphere profiles between 50 and 90 km show unique small-scale structure. Nonetheless, there are practical implications that additional measurements are very much needed to complete the full vertical profile picture. Our discussion concentrates on the distribution of temperature and wind and their variability. However, reliable measurements from other high latitude NASA programs over a number of years are available to help properly calculate mean values and the distribution of the individual measurements. Since the available rocket data in the Arctic's upper atmosphere are sparse the results we present are basically a snapshot of atmospheric structure.

Schmidlin, F. J.↗
NASA NTRS1998

Method and system for data clustering for very large databases

Multi-dimensional data contained in very large databases is efficiently and accurately clustered to determine patterns therein and extract useful information from such patterns. Conventional computer processors may be used which have limited memory capacity and conventional operating speed, allowing massive data sets to be processed in a reasonable time and with reasonable computer resources. The clustering process is organized using a clustering feature tree structure wherein each clustering feature comprises the number of data points in the cluster, the linear sum of the data points in the cluster, and the square sum of the data points in the cluster. A dense region of data points is treated collectively as a single cluster, and points in sparsely occupied regions can be treated as outliers and removed from the clustering feature tree. The clustering can be carried out continuously with new data points being received and processed, and with the clustering feature tree being restructured as necessary to accommodate the information from the newly received data points.

Zhang, Tian↗
DOE OSTI2022

Simurgh: A Framework for Cad-Driven Deep Learning Based X-Ray CT Reconstruction

High-resolution X-ray computed tomography (XCT) is an important technique for the inspection of additively manufactured (AM) parts. While XCT is typically used off-line to inspect a subset of manufactured parts, significantly accelerating measurement speed while retaining accuracy would enable use of XCT for in-line inspection to rapidly identify defects in each part as it is manufactured. Here, we propose a deep learning (DL) based approach that uses computer aided design (CAD) models of the AM parts and physics-based information to rapidly produce high-quality reconstructions from sparse XCT measurements without high quality ground truth data. Our approach uses a generative adversarial neural network (GAN) to produced realistic training data from the CAD-based simulations and a deep neural network that is trained using data from the first stage to produce accurate 3D reconstructions. Using experimental XCT data of metal parts, we demonstrate enhanced defect detection capabilities while dramatically reducing the scan time.

Ziabari, Amir↗
NASA NTRS1994

Multiple View Zenith Angle Observations of Reflectance From Ponderosa Pine Stands

Reflectance factors (RF(lambda)) from dense and sparse ponderosa pine (Pinus ponderosa) stands, derived from radiance data collected in the solar principal plane by the Advanced Solid-State Array Spectro-radiometer (ASAS), were examined as a function of view zenith angle (theta(sub v)). RF(lambda) was maximized with theta(sub v) nearest the solar retrodirection, and minimized near the specular direction throughout the ASAS spectral region. The dense stand had much higher RF anisotropy (ma)dmurn RF is minimum RF) in the red region than did the sparse stand (relative differences of 5.3 vs. 2.75, respectively), as a function of theta(sub v), due to the shadow component in the canopy. Anisotropy in the near-infrared (NIR) was more similar between the two stands (2.5 in the dense stand and 2.25 in the sparse stand); the dense stand exhibited a greater hotspot effect than 20 the sparse stand in this spectral region. Two common vegetation transforms, the NIR/red ratio and the normalized difference vegetation index (NDVI), both showed a theta(sub v) dependence for the dense stand. Minimum values occurred near the retrodirection and maximum values occurred near the specular direction. Greater relative differences were noted for the NIR/red ratio (2.1) than for the NDVI (1.3). The sparse stand showed no obvious dependence on theta(sub v) for either transform, except for slightly elevated values toward the specular direction.

Johnson, Lee F.↗
NASA NTRS2016

Soil Moisture Active Passive Mission L4_SM Data Product Assessment (Version 2 Validated Release)

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public Version 2 validated release scheduled for 29 April 2016. The assessment of the Version 2 L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to up-scaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the wide geographic range of the sparse network sites, and the global assessment of the assimilation diagnostics, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 2 validation and supports the validated release of the data. An analysis of the time average surface and root zone soil moisture shows that the global pattern of arid and humid regions are captured by the L4_SM estimates. Results from the core validation site comparisons indicate that "Version 2" of the L4_SM data product meets the self-imposed L4_SM accuracy requirement, which is formulated in terms of the ubRMSE: the RMSE (Root Mean Square Error) after removal of the long-term mean difference. The overall ubRMSE of the 3-hourly L4_SM surface soil moisture at the 9 km scale is 0.035 cubic meters per cubic meter requirement. The corresponding ubRMSE for L4_SM root zone soil moisture is 0.024 cubic meters per cubic meter requirement. Both of these metrics are comfortably below the 0.04 cubic meters per cubic meter requirement. The L4_SM estimates are an improvement over estimates from a model-only SMAP Nature Run version 4 (NRv4), which demonstrates the beneficial impact of the SMAP brightness temperature data. L4_SM surface soil moisture estimates are consistently more skillful than NRv4 estimates, although not by a statistically significant margin. The lack of statistical significance is not surprising given the limited data record available to date. Root zone soil moisture estimates from L4_SM and NRv4 have similar skill. Results from comparisons of the L4_SM product to in situ measurements from nearly 400 sparse network sites corroborate the core validation site results. The instantaneous soil moisture and soil temperature analysis increments are within a reasonable range and result in spatially smooth soil moisture analyses. The O-F residuals exhibit only small biases on the order of 1-3 degrees Kelvin between the (re-scaled) SMAP brightness temperature observations and the L4_SM model forecast, which indicates that the assimilation system is largely unbiased. The spatially averaged time series standard deviation of the O-F residuals is 5.9 degrees Kelvin, which reduces to 4.0 degrees Kelvin for the observation-minus-analysis (O-A) residuals, reflecting the impact of the SMAP observations on the L4_SM system. Averaged globally, the time series standard deviation of the normalized O-F residuals is close to unity, which would suggest that the magnitude of the modeled errors approximately reflects that of the actual errors. The assessment report also notes several limitations of the "Version 2" L4_SM data product and science algorithm calibration that will be addressed in future releases. Regionally, the time series standard deviation of the normalized O-F residuals deviates considerably from unity, which indicates that the L4_SM assimilation algorithm either over- or under-estimates the actual errors that are present in the system. Planned improvements include revised land model parameters, revised error parameters for the land model and the assimilated SMAP observations, and revised surface meteorological forcing data for the operational period and underlying climatological data. Moreover, a refined analysis of the impact of SMAP observations will be facilitated by the construction of additional variants of the model-only reference data. Nevertheless, the “Version 2” validated release of the L4_SM product is sufficiently mature and of adequate quality for distribution to and use by the larger science and application communities.

SMAP L4_SM↗
NASA NTRS2018

The SMAP Mission Combined Active-Passive Soil Moisture Product at 9 Km and 3 Km Spatial Resolutions

The NASA Soil Moisture Active Passive (SMAP) mission was launched on January 31st, 2015. The spacecraft was to provide high-resolution (3 km and 9 km) global soil moisture estimates at regular intervals by combining for the first time L-band radiometer and radar observations. On July 7th, 2015, a component of the SMAP radar failed and the radar ceased operation. However, before this occurred the mission was able to collect and process ~2.5 months of the SMAP high-resolution active-passive soil moisture data (L2SMAP) that coincided with the Northern Hemisphere's vegetation green-up and crop growth season. In this study, we evaluate the SMAP high-resolution soil moisture product derived from several alternative algorithms against in situ data from core calibration and validation sites (CVS), and sparse networks. The baseline algorithm had the best comparison statistics against the CVS and sparse networks. The overall unbiased root-mean-square-difference is close to the 0.04 cu. m/cu. m the SMAP mission requirement. A 3 km spatial resolution soil moisture product was also examined. This product had an unbiased root-mean-square-difference of ~0.053 cu. m/cu. m. The SMAP L2SMAP product for ~2.5 months is now validated for use in geophysical applications and research and available to the public through the NASA Distributed Active Archive Center (DAAC) at the National Snow and Ice Data Center (NSIDC). The L2SMAP product is packaged with the geo-coordinates, acquisition times, and all requisite ancillary information. Although limited in duration, SMAP has clearly demonstrated the potential of using a combined L-band radar-radiometer for proving high spatial resolution and accurate global soil moisture.

high resolution soil moisture↗
DOE OSTI2021

Online Voltage Event Detection Using Synchrophasor Data with Structured Sparsity-Inducing Norms

This paper develops an accurate and computationally efficient data-driven framework to detect voltage events from PMU data streams. It develops an innovative Proximal Bilateral Random Projection (PBRP) algorithm to quickly decompose the PMU data matrix into a low-rank matrix, a row-sparse event-pattern matrix and a noise matrix. Here, the row-sparse pattern matrix significantly distinguishes events from normal behavior. These matrices are then fed into a clustering algorithm to separate voltage events from normal operating conditions. Large-scale numerical study results on real-world PMU data show that the proposed algorithm is computationally more efficient and achieves higher F scores than state-of-the-art benchmarks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

A nudged hybrid analysis and modeling approach for realtime wake-vortex transport and decay prediction

Here, we put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements for air traffic improvements. Toward emerging applications of digital twins in aviation, the proposed approach allows for constructing a realtime predictive tool for wake-vortex transport and decay systems. We build on the fact that in realistic application, there are uncertainties in initial and boundary conditions, model parameters, as well as measurements. Moreover, conventional nonlinear ROMs based on Galerkin projection (GROMs) suffer from imperfection and solution instabilities, especially for advection-dominated flows with slow decay in the Kolmogorov n-width. In the presented LSTM nudging (LSTM-N) approach, we fuse forecasts from a combination of imperfect GROM and uncertain state estimates, with sparse Eulerian sensor measurements to provide more reliable predictions in a dynamical data assimilation framework. We illustrate our concept by solving the two-dimensional vorticity transport equation. We investigate the effects of measurements noise and state estimate uncertainty on the performance of the LSTM-N behavior. We also demonstrate that it can sufficiently handle different levels of temporal and spatial measurement sparsity, and offer a huge potential in developing next-generation digital twin technologies for aerospace applications.

42 ENGINEERING↗
DOE OSTI2023

Analyzing Hardware and Software Common Cause Failures in Digital Instrumentation and Control Systems using Dual Error Propagation Method

This paper develops a methodology for quantifying software common cause failures (CCFs) in digital instrumentation and control (I&C) systems of nuclear power plants. To support the transition of analog I&C systems to digital in nuclear power plants, probabilistic risk assessment (PRA) techniques are used. The hardware components of the I&C systems have reliability databases that can be used in the PRA studies. However, the failure data for redundant software components of the systems is sparse. Failure of components constitutes a CCF, wherein two or more components or systems fail due to a single shared cause and coupling mechanism. This paper proposes a quantification approach that can simultaneously model hardware and software components, incorporate the CCFs of software systems in the models, and bridge the gap between the failure quantification of models and the development of CCF parametric databases. We demonstrate the dual error propagation method (DEPM) by developing I&C systems failure models for a representative digital reactor trip system. The DEPM models are built to simulate the control and data flows within the systems and can accommodate failure states. By expanding DEPM to software CCFs, we generated alpha factor parameter estimates for each of the modeled error propagation mechanisms.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1997

Application-Controlled Demand Paging for Out-of-Core Visualization

In the area of scientific visualization, input data sets are often very large. In visualization of Computational Fluid Dynamics (CFD) in particular, input data sets today can surpass 100 Gbytes, and are expected to scale with the ability of supercomputers to generate them. Some visualization tools already partition large data sets into segments, and load appropriate segments as they are needed. However, this does not remove the problem for two reasons: 1) there are data sets for which even the individual segments are too large for the largest graphics workstations, 2) many practitioners do not have access to workstations with the memory capacity required to load even a segment, especially since the state-of-the-art visualization tools tend to be developed by researchers with much more powerful machines. When the size of the data that must be accessed is larger than the size of memory, some form of virtual memory is simply required. This may be by segmentation, paging, or by paged segments. In this paper we demonstrate that complete reliance on operating system virtual memory for out-of-core visualization leads to poor performance. We then describe a paged segment system that we have implemented, and explore the principles of memory management that can be employed by the application for out-of-core visualization. We show that application control over some of these can significantly improve performance. We show that sparse traversal can be exploited by loading only those data actually required. We show also that application control over data loading can be exploited by 1) loading data from alternative storage format (in particular 3-dimensional data stored in sub-cubes), 2) controlling the page size. Both of these techniques effectively reduce the total memory required by visualization at run-time. We also describe experiments we have done on remote out-of-core visualization (when pages are read by demand from remote disk) whose results are promising.

Cox, Michael↗
DOE OSTI2020

On‐site surrogates for large‐scale calibration

Abstract Motivated by a computer model calibration problem from the oil and gas industry, involving the design of a honeycomb seal, we develop a new Bayesian methodology to cope with limitations in the canonical apparatus stemming from several factors. We propose a new strategy of on‐site design and surrogate modeling for a computer simulator acting on a high‐dimensional input space that, although relatively speedy, is prone to numerical instabilities, missing data, and nonstationary dynamics. Our aim is to strike a balance between data‐faithful modeling and computational tractability in a calibration framework—tailoring the computer model to a limited field experiment. Situating our on‐site surrogates within the canonical calibration apparatus requires updates to that framework. We describe a novel yet intuitive Bayesian setup that carefully decomposes otherwise prohibitively large matrices by exploiting the sparse blockwise structure. Empirical illustrations demonstrate that this approach performs well on toy data and our motivating honeycomb example.

Huang, Jiangeng↗
NASA NTRS2017

Crater Morphometry and Crater Degradation on Mercury: Mercury Laser Altimeter (MLA) Measurements and Comparison to Stereo-DTM Derived Results

Two types of measurements of Mercury's surface topography were obtained by the MESSENGER (MErcury Surface Space ENvironment, GEochemisty and Ranging) spacecraft: laser ranging data from Mercury Laser Altimeter (MLA) [1], and stereo imagery from the Mercury Dual Imaging System (MDIS) camera [e.g., 2, 3]. MLA data provide precise and accurate elevation meaurements, but with sparse spatial sampling except at the highest northern latitudes. Digital terrain models (DTMs) from MDIS have superior resolution but with less vertical accuracy, limited approximately to the pixel resolution of the original images (in the case of [3], 15-75 m). Last year [4], we reported topographic measurements of craters in the D=2.5 to 5 km diameter range from stereo images and suggested that craters on Mercury degrade more quickly than on the Moon (by a factor of up to approximately 10×). However, we listed several alternative explanations for this finding, including the hypothesis that the lower depth/diameter ratios we observe might be a result of the resolution and accuracy of the stereo DTMs. Thus, additional measurements were undertaken using MLA data to examine the morphometry of craters in this diameter range and assess whether the faster crater degradation rates proposed to occur on Mercury is robust.

morphometry↗
DOE OSTI2025

Hyperspectral segmentation of plants in fabricated ecosystems

Hyperspectral imaging provides a powerful tool for analyzing above-ground plant characteristics in fabricated ecosystems, offering rich spectral information across diverse wavelengths. This study presents an efficient workflow for hyperspectral data segmentation and subsequent data analytics, minimizing the need for user annotation through the use of ensembles of sparse mixed scale convolution neural networks. The segmentation process leverages the diversity of ensembles to achieve high accuracy with minimal labeled data, reducing labor-intensive annotation efforts. To further enhance robustness, we incorporate image alignment techniques to address spatial variability in the dataset. Downstream analysis focuses on using the segmented data for processing spectral data, enabling monitoring of plant health. This approach provides a scalable solution for spectral segmentation, and facilitates actionable insights into plant conditions in complex, controlled environments. Our results demonstrate the utility of combining advanced machine learning techniques with hyperspectral analytics for high-throughput plant monitoring.

Zwart, Petrus H.↗