SEARCH · Engineering Papers

Results for “data reduction techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Baryonic Effects in Cosmic Shear Tomography: PCA Parameterization and the Importance of Extreme Baryonic Models

Baryonic effects are amongst the most severe systematics to the tomographic analysis of weak lensing data which is the principal probe in many future generations of cosmological surveys like LSST, Euclid etc.. Modeling or parameterizing these effects is essential in order to extract valuable constraints on cosmological parameters. In a recent paper, Eifler et al. (2015) suggested a reduction technique for baryonic effects by conducting a principal component analysis (PCA) and removing the largest baryonic eigenmodes from the data. In this article, we conducted the investigation further and addressed two critical aspects. Firstly, we performed the analysis by separating the simulations into training and test sets, computing a minimal set of principle components from the training set and examining the fits on the test set. We found that using only four parameters, corresponding to the four largest eigenmodes of the training set, the test sets can be fitted thoroughly with an RMS $\sim 0.0011$. Secondly, we explored the significance of outliers, the most exotic/extreme baryonic scenarios, in this method. We found that excluding the outliers from the training set results in a relatively bad fit and degraded the RMS by nearly a factor of 3. Therefore, for a direct employment of this method to the tomographic analysis of the weak lensing data, the principle components should be derived from a training set that comprises adequately exotic but reasonable models such that the reality is included inside the parameter domain sampled by the training set. The baryonic effects can be parameterized as the coefficients of these principle components and should be marginalized over the cosmological parameter space.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS1987

Speckle noise reduction of 1-look SAR imagery

Speckle noise is inherent to synthetic aperture radar (SAR) imagery. Since the degradation of the image due to this noise results in uncertainties in the interpretation of the scene and in a loss of apparent resolution, it is desirable to filter the image to reduce this noise. In this paper, an adaptive algorithm based on the calculation of the local statistics around a pixel is applied to 1-look SAR imagery. The filter adapts to the nonstationarity of the image statistics since the size of the blocks is very small compared to that of the image. The performance of the filter is measured in terms of the equivalent number of looks (ENL) of the filtered image and the resulting resolution degradation. The results are compared to those obtained from different techniques applied to similar data. The local adaptive filter (LAF) significantly increases the ENL of the final image. The associated loss of resolution is also lower than that for other commonly used speckle reduction techniques.

Nathan, Krishna S.↗

DOE OSTI2020

Learning to Solve Large-Scale Security-Constrained Unit Commitment Problems

Security-constrained unit commitment (SCUC) is a fundamental problem in power systems and electricity markets. In practical settings, SCUC is repeatedly solved via mixed-integer linear programming (MIP), sometimes multiple times per day, with only minor changes in input data. In this work, we propose a number of machine learning techniques to effectively extract information from previously solved instances in order to significantly improve the computational performance of MIP solvers when solving similar instances in the future. Based on statistical data, we predict redundant constraints in the formulation, good initial feasible solutions, and affine subspaces where the optimal solution is likely to lie, leading to a significant reduction in problem size. Computational results on a diverse set of realistic and large-scale instances show that using the proposed techniques, SCUC can be solved on average 4.3 times faster with optimality guarantees and 10.2 times faster without optimality guarantees, with no observed reduction in solution quality. Out-of-distribution experiments provide evidence that the method is somewhat robust against data-set shift. Summary of Contribution. The paper describes a novel computational method, based on a combination of mixed-integer linear programming (MILP) and machine learning (ML), to solve a challenging and fundamental optimization problem in the energy sector. The method advances the state-of-the-art, not only for this particular problem, but also, more generally, in solving discrete optimization problems via ML. We expect that the techniques presented can be readily used by practitioners in the energy sector and adapted, by researchers in other fields, to other challenging operations research problems that are solved routinely.

Machine Learning↗

NASA NTRS1990

Study of the crust and mantle using magnetic surveys by Magsat and other satellites

This article summarizes some of the methodology and results for studies of the earth's mantle and crust. Mantle conductivity studies can be made either by studying signals impressed on the earth from outside, e.g., the ionosphere or magnetosphere, or by studing signals originating in the core and transmitted through the mantle. Crustal field studies begin with a careful selection of the data and subsequent removal of core and external fields by some sort of filtering. Average maps from different local times sometimes differ, presumably due to the remaining presence of fields of external origin. Several techniques for further filtering are discussed. Where large-area aeromagnetic maps are available, crustal maps derived from satellite data can be compared with upward continued data. In general, the comparisons show agreement, with some differences, particularly in and near the auroral belts. The satellite data are further reduced by various methods of inverse and forward modeling, sometimes including reduction to the pole. These techniques are generally unstable at the equator. Common methods of stabilizing the inversions include principle components analysis and ridge regression. Because of the presence of the core field, the entire crustal contribution from the field is not known. Also, there is a basic nonuniqueness to the inverse solutions. Nevertheless, magnetizations that are interpretable can be derived.

Langel, Robert A.↗

NASA NTRS2000

T1 VSAT Fade Compensation Statistical Results

New satellite communication systems are steadily seeking to use higher frequency bands to accommodate the requirements for additional capacity. At these higher frequencies, propagation impairments that did not significantly affect the signal at lower frequencies begin to have considerable impact. In Ka-band. the next logical commercial frequency band to be used for satellite communication, attenuation of the signal due to rain is a primary concern. An experimental satellite built by NASA, the Advanced Communication Technology Satellite (ACTS). launched in September 1993, is the first U.S. communication satellite operating in the Ka-band. In addition to higher carrier frequencies, a number of other new technologies, including on-board baseband processing. multiple beam antennas, and rain fade detection and compensation techniques, were designed into the ACTS. Verification experiments have been conducted since the launch to characterize the new technologies. The focus of this paper is to characterize the method used by the ACTS TI Very Small Aperture Terminal (TI VSAT) ground stations in detecting the presence of fade in the communication signal and to adaptively compensate for it by the addition of burst rate reduction and forward error correction. Measured data obtained from the ACTS program was used to validate the compensation technique. A software process was developed and demonstrated to statistically characterize the increased availability achieved by the compensation techniques in terms of the bit error rate time enhancement factor. Several improvements to the ACTS technique are discussed and possible implementations for future Ka band system are offered.

Johnson, Sandra K.↗

NASA NTRS2001

Analysis of Fade Detection and Compensation Experimental Results in a Ka-Band Satellite System

The frequency bands being used for new satellite communication systems are constantly increasing to accommodate the requirements for additional capacity. At these higher frequencies, propagation impairments that did not significantly affect the signal at lower frequencies begin to have considerable impact. In Ka-band, the next logical commercial frequency band to be used for satellite communication, attenuation of the signal due to rain is a primary concern. An experimental satellite built by NASA, the Advanced Communication Technology Satellite (ACTS), launched in September 1993, is the first US communication satellite operating in the Ka-band. In addition to higher carrier frequencies, a number of other new technologies, including onboard baseband processing, multiple beam antennas, and rain fade detection and compensation techniques, were designed into the ACTS. Verification experiments have been conducted since the launch to characterize the new technologies. The focus of this thesis is to describe and validate the method used by the ACTS Very Small Aperture Terminal (VSAT) ground stations in detecting the presence of fade in the communication signal and to adaptively compensate for it by the addition of burst rate reduction and forward error correction. Measured data obtained from the ACTS program is used to validate the compensation technique. In this thesis, models in MATLAB are developed to statistically characterize the increased availability achieved by the compensation techniques in terms of the bit error rate time enhancement factor. Several improvements to the ACTS technique are discussed and possible implementations for future Ka-band systems are also presented.

Johnson, Sandra↗

NASA NTRS1988

Application of optical correlation techniques to particle imaging velocimetry

Pulsed laser sheet velocimetry yields nonintrusive measurements of velocity vectors across an extended 2-dimensional region of the flow field. The application of optical correlation techniques to the analysis of multiple exposure laser light sheet photographs can reduce and/or simplify the data reduction time and hardware. Here, Matched Spatial Filters (MSF) are used in a pattern recognition system. Usually MSFs are used to identify the assembly line parts. In this application, the MSFs are used to identify the iso-velocity vector contours in the flow. The patterns to be recognized are the recorded particle images in a pulsed laser light sheet photograph. Measurement of the direction of the partical image displacements between exposures yields the velocity vector. The particle image exposure sequence is designed such that the velocity vector direction is determined unambiguously. A global analysis technique is used in comparison to the more common particle tracking algorithms and Young's fringe analysis technique.

Wernet, Mark P.↗

NASA NTRS1988

Application of optical correlation techniques to particle imaging

Pulsed laser sheet velocimetry yields noninstrusive measurements of velocity vectors across an extended 2-dimensional region of the flow field. The application of optical correlation techniques to the analysis of multiple exposure laser light sheet photographs can reduce and/or simplify the data reduction time and hardware. Here, Matched Spatial Filters (MSF) are used in a pattern recognition system. Usuallay MSFs are used to identify the assembly line parts. In this application, the MSFs are used to identify the iso-velocity vector contours in the flow. The patterns to be recognized are the recorded particle images in a pulsed laser light sheet photograph. Measurement of the direction of the particle image displacements between exposures yields the velocity vector. The particle image exposure sequence is designed such that the velocity vector direction is determined unambiguously. A global analysis technique is used in comparison to the more common particle tracking algorithms and Young's fringe analysis technique.

Wernet, Mark P.↗

DOE OSTI2019

Deep multiscale model learning

The objective of this article is to design novel multi-layer neural networks for multiscale simulations of flows taking into account the observed fine data and physical modeling concepts. Our approaches use deep learning techniques combined with local multiscale model reduction methodologies to predict flow dynamics. Using reduced-order model concepts is important for constructing robust deep learning architectures since the reduced-order models provide fewer degrees of freedom. We consider flow dynamics in porous media as multi-layer networks in this work. More precisely, the solution (e.g., pressures and saturation) at the time instant n+1 depends on the solution at the time instant n and input parameters, such as permeability fields, forcing terms, and initial conditions. One can regard the solution as a multi-layer network, where each layer, in general, is a nonlinear forward map and the number of layers relates to the internal time steps. We will rely on rigorous model reduction concepts to define unknowns and connections between layers. It is critical to use reduced-order models for this purpose, which will identify the regions of influence and the appropriate number of variables. Furthermore, due to the lack of available observed fine data, the reduced-order model can provide us sufficient inexpensive data as needed. The designed deep neural network will be trained using both coarse simulation data which is obtained from the reduced-order model and observed fine data. We will present the main ingredients of our approach and numerical examples. Numerical results show that using deep learning with data generated from multiscale models as well as available observed fine data, we can obtain an improved forward map which can better approximate the fine scale model.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Multivariate Pointwise Information-Driven Data Sampling and Visualization

With increasing computing capabilities of modern supercomputers, the size of the data generated from the scientific simulations is growing rapidly. As a result, application scientists need effective data summarization techniques that can reduce large-scale multivariate spatiotemporal data sets while preserving the important data properties so that the reduced data can answer domain-specific queries involving multiple variables with sufficient accuracy. While analyzing complex scientific events, domain experts often analyze and visualize two or more variables together to obtain a better understanding of the characteristics of the data features. Therefore, data summarization techniques are required to analyze multi-variable relationships in detail and then perform data reduction such that the important features involving multiple variables are preserved in the reduced data. To achieve this, in this work, we propose a data sub-sampling algorithm for performing statistical data summarization that leverages pointwise information theoretic measures to quantify the statistical association of data points considering multiple variables and generates a sub-sampled data that preserves the statistical association among multi-variables. Using such reduced sampled data, we show that multivariate feature query and analysis can be done effectively. The efficacy of the proposed multivariate association driven sampling algorithm is presented by applying it on several scientific data sets.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1982

A wake traverse technique for use in a 2 dimensional transonic flexible walled test section

Reported two dimensional validation data from the Transonic Self-Streamlining Wind Tunnel (TSWT) concerns model lift. The models tested provided data on their pressure distributions. This information was numerically integrated over the model surface to determine lift, pressure drag and pitching moment. However, the pressure drag is only a small component of the total drag at nominal angles of attack and cannot be used to assess the quality of flow simulation. An intrusive technique for obtaining information on the total drag of a model in TSWT is described. The technique adopted is the wake traverse method. The associated tunnel hardware and control and data reduction software are outlined and some experimental results are presented for discussion.

Wolf, S. W. D.↗

NASA NTRS1987

Microgravity research applications of interferometry to heat and mass transfer

Major space flight experiments which have used interferometric measurement as a major support system are described together with similar ground-based research. Space experiments include those flown on Spacelab 3 ad on the D-1 mission. Included in the ground-based experiments are aircraft studies, Spacelab 3 data reduction and evaluation, and supportive laboratory hardware development. It is shown that optical measurement techniques are crucial to microgravity research.

Owen, Robert B.↗

NASA NTRS1993

Two protogalaxy candidates in one night

In spite of a lookback time of 77 to 89 percent of the age of the universe, Lilly provided evidence that the dominant stellar population in 0902 was surprisingly old, over 1 Gyr. This age estimate was based on the values K = 18.8 and I-K = 4.5 in the central 3.5 x 3.5 inch. At Lilly's suggestion, we reobserved 0902 at K, and in initial reductions of our data found only an upper limit (K greater than 19.5). After extensive experimentation with reduction techniques and correction for low-level systematic effects, we succeeded in extracting a four sigma detection at K = 19.9 in a 4 inch diameter circular aperture. The morphology and location of this detection were more reminescent of the Ly alpha image than of CCD continuum images, and in Mar. 1992, we obtained a narrow band image which included redshifted (OIII) 4959,5007 A line emission at 2.20 microns. This (OIII) image gave a strong detection which demonstrates that most of the central K flux, already reduced by a magnitude, is due to line emission. Our formal line corrected values in a 4 inch aperture are K = 21.5 and R-K = 1.9. Hence, 0902 has been transformed from relaxed maturity into unsettled youth.

Eisenhardt, Peter↗

NASA NTRS1993

Automated Analysis of a Large-Scale Sky Survey: The SKICAT System

We describe the application of decision tree based classification techniques to the development of an automated tool for the reduction of a large scientific data set. The primary benefits of the SKICAT approach are increased data reduction throughput, repeatability, and consistency of classification.

data analysis image databases↗

NASA NTRSDate not supplied

Applicability of Loads Estimation Techniques Using Sparse Acceleration Sensor Data to Spacecraft Structural Health Monitoring

The use of structural health monitoring systems on spacecraft structures can play a crucial role in ensuring the safety, reliability, and longevity of the structure by gathering and analyzing onboard sensor data. Of specific importance is monitoring for excessive loading at critical interfaces as any off-nominal structural excitations experienced by spacecraft structures can cause early unpredicted high structural life consumption or damage. The availability and cost of flight-certified sensors along with the size of spacecraft structures and allowable payload mass drives the need for a method to estimate loads using sparsely-located sensors. Numerous approaches such as physics-based, statistical learning, and physics-enhanced statistical learning algorithms have gained popularity among structural prognostics applications. However, developing noise-robust prediction models to assess loads and structural life predictions from a sparse multi-sensor data acquisition system can be a challenging task. This paper discusses the evaluation of physics-based versus machine-learning algorithms for predicting loads and structural life at mission critical locations on the spacecraft structure using a finite element loads analysis with the application of simulated noise and noise reduction techniques. To estimate the loads from accelerations, the physics-based algorithm leverages a loads transformation matrix from a Craig-Bampton reduced finite element model. A System Equivalent Reduction Expansion Process (SEREP) and a pseudo-inverse approach are considered to expand from the onboard sensor degrees of freedom to the Craig-Bampton model degrees of freedom. The machine learning algorithm provides a data driven solution/mapping of the sensor accelerations to the loads at the mission critical locations using a high dimensionality analysis. Although these strategies produce comparable loads prediction without noise, the limitations of these strategies with incorporating simulated noise and noise reduction techniques with low signal to noise ratio signals are evaluated. The study demonstrates the immense potential of statistical learning algorithms for sparse structural prognostic models and enhancing signal denoising techniques. These findings also highlight the need for noise-resilient prognostic models and low-noise data acquisition systems onboard spacecraft structures.

Spacecraft Structural Health Monitoring↗

DOE OSTI2018

Xi-cam : a versatile interface for data visualization and analysis

Xi-cam is an extensible platform for data management, analysis and visualization. Xi-cam aims to provide a flexible and extensible approach to synchrotron data treatment as a solution to rising demands for high-volume/high-throughput processing pipelines. The core of Xi-cam is an extensible plugin-based graphical user interface platform which provides users with an interactive interface to processing algorithms. Plugins are available for SAXS/WAXS/GISAXS/GIWAXS, tomography and NEXAFS data. With Xi-cam 's `advanced' mode, data processing steps are designed as a graph-based workflow, which can be executed live, locally or remotely. Remote execution utilizes high-performance computing or de-localized resources, allowing for the effective reduction of high-throughput data. Xi-cam 's plugin-based architecture targets cross-facility and cross-technique collaborative development, in support of multi-modal analysis. Xi-cam is open-source and cross-platform, and available for download on GitHub.

Pandolfi, Ronald J. (ORCID:0000000308248548)↗

NASA NTRS2001

Integrated Component-based Data Acquisition Systems for Aerospace Test Facilities

The Multi-Instrument Integrated Data Acquisition System (MIIDAS), developed by the NASA Langley Research Center, uses commercial off the shelf (COTS) products, integrated with custom software, to provide a broad range of capabilities at a low cost throughout the system s entire life cycle. MIIDAS combines data acquisition capabilities with online and post-test data reduction computations. COTS products lower purchase and maintenance costs by reducing the level of effort required to meet system requirements. Object-oriented methods are used to enhance modularity, encourage reusability, and to promote adaptability, reducing software development costs. Using only COTS products and custom software supported on multiple platforms reduces the cost of porting the system to other platforms. The post-test data reduction capabilities of MIIDAS have been installed at four aerospace testing facilities at NASA Langley Research Center. The systems installed at these facilities provide a common user interface, reducing the training time required for personnel that work across multiple facilities. The techniques employed by MIIDAS enable NASA to build a system with a lower initial purchase price and reduced sustaining maintenance costs. With MIIDAS, NASA has built a highly flexible next generation data acquisition and reduction system for aerospace test facilities that meets customer expectations.

Ross, Richard W.↗

NASA NTRS2017

Time and Measurement Days

Questions in data analysis involving the concepts of time and measurement are often pushed into the background or reserved for a philosophical discussion. Some examples are: a) Is causality a consequence of the laws of physics, or can the arrow of time be reversed? b) Can we determine the arrow of time of an event? c) Do we need the continuum hypothesis for the underlying function in any measurement process? d) Can we say anything about the analyticity of the underlying process of an event? e) Would it be valid to model a non-analytical process as function of time? f) What are the implications of all these questions for classical Fourier techniques? However, in the age of big data gathered either from space missions supplying ultra-precise long time series, or e.g. LIGO data from the ground, the moment to bring these questions to the foreground seems arrived. The limitations of our understanding of some fundamental processes is emphasized by the lack of solution for problems open for more than 2 decades, such as the non-detection of solar g-modes, or the modal identification of main sequence stellar pulsators like delta Scuti stars. Flicker noise or 1/f noise, for example, attributed in solar-like stars to granulation, is analyzed mostly only to apply noise reduction techniques, neither considering the classical problem of 1/f noise that was introduced a 100 years ago, nor taking into account ergodic or non-ergodic solutions that make inapplicable spectral analysis techniques in practice. This topic was discussed by Nicholas W. Watkins during the ITISE meeting held in Granada in 2016. There he presented preliminary results of his research on Mandelbrot's related work. We reproduce here his quotation of Mandelbrot (1999) "There is a sharp contrast between a highly anomalous ("non-white") noise that proceeds in ordinary clock time and a noise whose principal anomaly is that it is restricted to fractal time", suggesting a connection with the above proposed topics that could be phrased as the following additional questions:a) Is self-organized criticality (SOC) frequent in astrophysical phenomena? b) Could all fractals in nature be considered stochastic? c) Could we establish mathematical/physical relationships between chaotic and fractal behaviors in time series? d) Could the differences between fractals and chaos in terms of analyticity be used to understand the residuals of the fitting of stellar light curves? In this meeting we would like to approximate these problems from a holistic and multidisciplinary perspective, taking into account not only technical issues but also the deeper implications. In particular the concept of connectivity (introduced in Pascual-Granado et al. A&A, 2015) could be used to implement, within the framework of ARMA processes, an "arrow of time" (see attached document), and so studying the possible implications in the concept of time as envisaged by Watkins.

data analysis↗