Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sparse data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Sparse Reconstruction of Electric Fields from Radial Magnetic Data

Accurate estimates of the horizontal electric field on the Sun’s visible surface are important not only for estimating the Poynting flux of magnetic energy into the corona but also for driving time-dependent magnetohydrodynamic models of the corona. In this paper, a method is developed for estimating the horizontal electric field from a sequence of radial-component magnetic field maps. This problem of inverting Faraday’s law has no unique solution. Unfortunately, the simplest solution (a divergence-free electric field) is not realistically localized in regions of nonzero magnetic field, as would be expected from Ohm’s law. Our new method generates instead a localized solution, using a basis pursuit algorithm to find a sparse solution for the electric field. The method is shown to perform well on test cases where the input magnetic maps are flux balanced in both Cartesian and spherical geometries. However, we show that if the input maps have a significant imbalance of flux—usually arising from data assimilation—then it is not possible to find a localized, realistic, electric field solution. This is the main obstacle to driving coronal models from time sequences of solar surface magnetic maps.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Generating synthetic signaling networks for in silico modeling studies

Predictive models of signaling pathways have proven to be difficult to develop. Reasons include the uncertainty in the number of species, the complexity in species’ interactions, and the sparseness and uncertainty in experimental data. Traditional approaches to developing mechanistic models rely on collecting experimental data and fitting a single model to that data. This approach works for simple systems but has proven unreliable for complex systems such as biological signaling networks. For example, uncertainty and sparseness of the data often result in overfitted models that have little predictive value beyond recapitulating the experimental data itself. Thus, there is a need to develop new approaches to create predictive mechanistic models of complex systems. However, to determine the effectiveness of any new algorithm, a baseline model is needed to test its performance. To meet this need, we developed a method for generating artificial synthetic networks that are reasonably realistic and thus can be treated as ground truth models. These synthetic models can then be used to generate synthetic data for developing and testing algorithms designed to recover the underlying network topology and associated parameters. Here, we describe a simple approach for generating synthetic signaling networks that can be used for this purpose.

42 ENGINEERING↗
DOE OSTI2019

Spatial Signal Detection Using Continuous Shrinkage Priors

Motivated by the problem of detecting changes in two-dimensional X-ray diffraction data, we propose a Bayesian spatial model for sparse signal detection in image data. Our model places considerable mass near zero and has heavy tails to reflect the prior belief that the image signal is zero for most pixels and large for an important subset. We show that the spatial prior places mass on nearby locations simultaneously being zero, and also allows for nearby locations to simultaneously be large signals. The form of the prior also facilitates efficient computing for large images. We conduct a simulation study to evaluate the properties of the proposed prior and show that it outperforms other spatial models. As a result, we apply our method in the analysis of X-ray diffraction data from a two-dimensional area detector to detect changes in the pattern when the material is exposed to an electric field.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Computing Sparse Tensor Decompositions via Chapel and C++/MPI Interoperability without Intermediate I/O

We extend an existing approach for efficient use of shared mapped memory across Chapel and C++ for graph data stored as 1-D arrays to sparse tensor data stored using a combination of 2-D and 1-D arrays. We describe the specific extensions that provide use of shared mapped memory tensor data for a particular C++ tensor decomposition tool called GentenMPI. We then demonstrate our approach on several real-world datasets, providing timing results that illustrate minimal overhead incurred using this approach. Finally, we extend our work to improve memory usage and provide convenient random access to sparse shared mapped memory tensor elements in Chapel, while still being capable of leveraging high performance implementations of tensor algorithms in C++.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Evaluation of a Class of Simple and Effective Uncertainty Methods for Sparse Samples of Random Variables and Functions

When very few samples of a random quantity are available from a source distribution of unknown shape, it is usually not possible to accurately infer the exact distribution from which the data samples come. Under-estimation of important quantities such as response variance and failure probabilities can result. For many engineering purposes, including design and risk analysis, we attempt to avoid under-estimation with a strategy to conservatively estimate (bound) these types of quantities -- without being overly conservative -- when only a few samples of a random quantity are available from model predictions or replicate experiments. This report examines a class of related sparse-data uncertainty representation and inference approaches that are relatively simple, inexpensive, and effective. Tradeoffs between the methods' conservatism, reliability, and risk versus number of data samples (cost) are quantified with multi-attribute metrics use d to assess method performance for conservative estimation of two representative quantities: central 95% of response; and 10 -4 probability of exceeding a response threshold in a tail of the distribution. Each method's performance is characterized with 10,000 random trials on a large number of diverse and challenging distributions. The best method and number of samples to use in a given circumstance depends on the uncertainty quantity to be estimated, the PDF character, and the desired reliability of bounding the true value. On the basis of this large data base and study, a strategy is proposed for selecting the method and number of samples for attaining reasonable credibility levels in bounding these types of quantities when sparse samples of random variables or functions are available from experiments or simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Optical Variability of ICRF3 Quasars in the Pan-STARRS 3Pi Survey with Functional Principal Component Analysis

We make use of individual (epoch) detection data from the Pan-STARRS “3π” survey for 2863 optical ICRF3 counterparts in the five wavelength bands g, r, i, z, and y, published as part of the Data Release 2. A dedicated method based on the Functional Principal Component Analysis is developed for these sparse and irregularly sampled data. With certain regularization and normalization constraints, it allows us to obtain uniform and compatible estimates of the variability amplitudes and average magnitudes between the passbands and objects. We find that the starting assumption of affinity of the light curves for a given object at different wavelengths is violated for several percent of the sample. The distributions of rms variability amplitudes are strongly skewed toward small values, peaking at ∼0.1 mag with tails stretching to 2 mag. Statistically, the lowest variability is found for the r band and the largest for the reddest y band. A small “brighter-redder” effect is present, with amplitudes in y greater than amplitudes in g in 57% of the sample. The variability versus redshift dependence shows a strong decline with z toward redshift 3, which we interpret as the time dilation of the dominant time frequencies. The colors of radio-loud ICRF3 quasars are correlated with redshift in a complicated, wavy pattern governed by the emergence of brightest emission lines within the five passbands.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Use of Machine Learning on PMU Data for Transmission System Fault Analysis

Synchrophasor technology has been used for monitoring, control, and protection of bulk power system for over 10 years. Deployment of phasor measurement units (PMUs) in the USA power system has surpassed 3000 units installed in the transmission substations as stand-alone intelligent electronic devices (IEDs) or as a software add-on to other devices such as digital protective relays (DPRs) or digital fault recorders (DFRs). By now, thousands of terabytes of PMU data may have been captured and stored by various transmission system operators (TSOs) and independent system operators (ISOs). This creates an opportunity to deploy advanced machine learning (ML) techniques to detect and classify faults recorded by PMUs automatically to be used by the system operators for rapid, critical decision-making when manual analysis of the past or unfolding events is not feasible. In this paper we offer a brief background on how the automated fault analysis may be done using DPR and/or DFR data, and compare some of the legacy approaches to the new ML approaches in the context of the system-wide PMU recordings. We then offer insights from developing practical ML solutions that have been applied on field recordings captured by close to 450 PMUs from all three US interconnections (Western, Eastern and ERCOT) over two years (2016-2017). We identify and illustrate ML challenges we addressed: inaccurate data, data with scarce and temporally imprecise fault labels, data recorded by PMUs sparsely located at substations resulting in the fault records taken afar from the ends of the faulted lines, data containing only positive sequence values, and data taken at different voltage levels. We then illustrate the ML model results for fault analysis under different application scenarios. The novelty of this study is not only in the design, implementation, and performance analysis of the ML algorithms, but also in the use of advanced fault modelling and simulation approaches to improve the training results when developing supervised ML models for fault detection and classification. Extensive simulations of faults were conducted on a 14-bus power system to create a training dataset with over 1400 accurately labelled faults. This dataset was applied to enhance the accuracy of fault detection and classification of machine learning-based models trained with small number of labelled faults in large datasets recorded in the grid interconnections ranging from 5,000 to 70,000 buses.

Synchrophasors, Machine Learning, Fault Analysis, ↗
DOE OSTI2016

An ensemble of ocean reanalyses for 1815-2013 with sparse observational input

This paper describes a new eight-member ensemble of ocean reanalyses spanning nearly 200 years from 1815 to 2013 generated using the Simple Ocean Data Assimilation system with sparse observational input (SODAsi) to explore long-term changes in the oceans. The eight ensemble members assimilate surface temperature observations and use surface boundary conditions from an atmospheric reanalysis that is loosely coupled to the ocean reanalysis. In this work, both surface and subsurface quantities, such as dynamic height and heat content, show a broad spectrum of variability. Surface temperature trends from 1815 to 2013 are positive in most regions, with some important exceptions; the central Tropical Pacific, around Antarctica, and in the Gulf Stream and Kuroshio extension regions all show cooling trends. A near-global average shows warming of about 0.8°C over the full period, with most of the warming occurring after 1920. There is pronounced multidecadal variability in both the midlatitude and tropical oceans. In the North Atlantic Ocean, temperature variability is highly correlated with the meridional overturning stream function, with the largest correlation occurring when the stream function is advanced by 9 years. Trends of upper ocean heat content and dynamic height from the 1950s onward compare well with previously published values. Globally averaged heat content of the upper 700 m shows a nearly linear rise after the 1920s, requiring a net downward surface heat flux increase of 0.47 W m -2 into the ocean. This is close to published estimates of the increased flux required to explain the heat content increase from 1971 to 2010.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Multi-Frame X-Ray Imaging Using a Streak Camera with a Patterned Photocathode, LAO-008-16, Year 1 of 2

Presentation describes development of a new time-resolved x-ray imaging system based on compressed ultrafast photography (CUP) to achieve multi-frame imaging for energies in the 1–10 keV range with a frame rate between 1 to 10 frames per nanosecond depending on the sweep time. A patterned gold photocathode was constructed for use with an existing Kentech x-ray streak camera. Significant work in reconstruction algorithms was performed to enable reconstructions of data that are much less sparse. These algorithms were tested with optical streak camera CUP data acquired at Livermore Operations.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗