Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data reduction methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
NASA NTRS1973

The figure of Mars and its effect on radar-ranging.

Formulae are given which indicate the errors made in computing the radar-range to a planet when one assumes the planet to have a spherical surface, centered at the center of mass. These formulae are evaluated for Mars, using an ellipsoidal fit of the martian surface obtained from the occultation measurements of the Mariner 9 spacecraft. The secularlike part of the errors amounts to about 4 km in range over the past 3 oppositions, due mainly to the flattening of 0.006 and to a 3 km southerly displacement of the center of figure from the center of mass. The method of solution is given in the appendix along with a method for applying the ellipsoidal model to the reduction of radar-range data.

Standish, E. M., Jr.↗
NASA NTRS2000

Initial OSSE Sky Survey Observation: Deep Survey in the Virgo Region

The OSSE Virgo survey covered about 1% of the sky, using eight weeks of observing time on the Compton Gamma Ray Observatory, and was performed in conjunction with the EGRET and COMPTEL instruments on CGRO. A fundamental aspect of the survey was the development of data reduction techniques for the scanning data from the OSSE detectors. A direct algebraic approach for spatially deconvolving the data was developed. For the survey work, no background observations isolated from the survey region were acquired, unlike normal discrete source observations. The survey, reduction method covers the mapped region with a grid of pixels, and estimates the flux from each pixel by least-squares agreement with the data, assuming positivity constraints. This technique has led to the detection of compact sources at hard X-ray energies within the Virgo region, and the OSSE survey studies have been extended to the south galactic pole and north Ecliptic pole, with similar analysis techniques.

Cameron, Robert↗
NASA NTRSDate not supplied

Mitigating the Impacts of Measurement Error in the Quesst Mission Community Noise Study

Beginning in 2025, the NASA Quesst mission will conduct a series of community response tests involving flyovers of the X-59 aircraft at select localities across the United States. Several waves of a longitudinal survey will be administered over approximately one month of testing in order to capture perceptual responses to low-amplitude sonic booms, or “sonic thumps”. Simultaneously, noise exposure levels will be estimated by fusing model-based predictions with measurements taken from a sparse network of monitors in the region. As one of the aims of the study is to produce a dose-response curve, a regression model relating perceptual response to noise exposure levels, it is important to acknowledge the potential attenuation bias that results from measurement error in the estimated noise exposure levels. In this presentation we review and compare several methods for dealing with measurement error in generalized linear mixed models. The methods are demonstrated on simulated data and real data collected during past NASA risk reduction studies.

measurement error↗
DOE OSTI2025

FedOSAA: Improving Federated Learning with One-Step Anderson Acceleration

Federated learning (FL) is a distributed machine learning approach that enables multiple local clients and a central server to collaboratively train a model while keeping the data on their own devices. First-order methods, particularly those incorporating variance reduction techniques, are the most widely used FL algorithms due to their simple implementation and stable performance. However, these methods tend to be slow and require a large number of communication rounds to reach the global minimizer. We propose FedOSAA, a novel approach that preserves the simplicity of first-order methods while achieving the rapid convergence typically associated with second-order methods. Our approach applies one Anderson acceleration (AA) step following classical local updates based on first-order methods with variance reduction, such as FedSVRG and SCAFFOLD, during local training. This AA step is able to leverage curvature information from the history points and gives a new update that approximates the Newton-GMRES direction, thereby significantly improving the convergence. We establish a local linear convergence rate to the global minimizer of FedOSAA for smooth and strongly convex loss functions. Numerical comparisons show that FedOSAA substantially improves the communication and computation efficiency of the original first-order methods, achieving performance comparable to second-order methods like GIANT.

Feng, Xue [University of California, Davis]↗
DOE OSTI2021

Geostatistical interpolation of streambed hydrologic attributes with addition of left censored data and anisotropy

Spatial geostatistical interpolation of point measurements of streambed attributes in the hyporheic zone may be constrained by the streambed anisotropy, and data density and spatial distribution may significantly impact the results. Spatial clustering and low spatial data density can be caused by bedrock outcropping at the streambed limiting installation of in-stream piezometers. This study examines parameter error variability of the geostatistical interpolation using anisotropic interpolation methods and increasing the data density by adding left censored values (i.e., data below measurement limit) to locations where measurements were limited by exposed bedrock lining the streambed. The reduction in relative standard error of the interpolation was determined for the spatial distributions of streambed attributes including hydraulic conductivity, seepage flux, and mercury solute flux measured in two different years along a study reach in East Fork Poplar Creek, Tennessee, USA. Here, two methods to impute the left censored values were compared including the conventional half the detection limit substitution method, and the Stochastic Approximation of Expectation-Maximization (SAEM) algorithm, which both had comparable results. Imputing left censored data increased the data density to recommended ranges, reduced data clustering, increased the spatial dependence for some attributes, and reduced the standard error for each of the three attributes. For the reach considered herein, addition of the left censored values resulted in a larger error reduction than the consideration of anisotropy within the interpolation, which confirms the benefit of data addition to increase data density within data-limited river corridors.

58 GEOSCIENCES↗
DOE OSTI2023

Unsupervised classification for region of interest in X-ray ptychography

X-ray ptychography offers high-resolution imaging of large areas at a high computational cost due to the large volume of data provided. To address the cost issue, we propose a physics-informed unsupervised classification algorithm that is performed prior to reconstruction and removes data outside the region of interest (RoI) based on the multimodal features present in the diffraction patterns. The preprocessing time for the proposed method is inconsequential in contrast to the resource-intensive reconstruction process, leading to an impressive reduction in the data workload to a mere 20% of the initial dataset. This capability consequently reduces computational time dramatically while preserving reconstruction quality. Through further segmentation of the diffraction patterns, our proposed approach can also detect features that are smaller than beam size and correctly classify them as within the RoI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 5: Utah FORGE Well 16B(78)-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 4: Utah FORGE Well 78B-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 3: Utah FORGE Well 56-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 1: Summary of Utah FORGE Wells 16A(78)-32, 56-32, 78B-32 and 16B(78)-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

A deep learning-based workflow for fast prediction of 3D state variables in geological carbon storage: A dimension reduction approach

Deep learning (DL) models are extensively used as surrogate models for high-fidelity simulations of multiphase fluid flow in porous media at large scales, enabling fast forecasts of the spatial–temporal evolution of three-dimensional (3D) state variables in geological carbon storage (GCS). However, training these models in high-dimensional space remains computationally demanding and prone to overfitting because of limited training data. This paper presents a novel workflow to address these challenges by integrating dimension reduction (DR) methods. Here, the proposed workflow employed pre-trained DR models to extract the latent variables of geological models and state variables and utilized the multi-layer perceptron (MLP) for constructing mapping functions between the input and output variables in latent spaces. Subsequently, the pre-trained reconstruction models converted the MLP-predicted latent state variables to their original high-dimensional form. Furthermore, we proposed a novel strategy for the DR and reconstruction of 3D saturation fields to account for the unique data characteristics of sparsity, nonuniformity, and discontinuity. The proposed strategy applied PCA and inverse PCA for 2D average saturation fields and developed a DL-based 3D reconstruction model, leveraging three 2D average saturation fields as input to produce a 3D saturation field as output. The pre-training of DR and reconstruction models and training of MLP models were conducted on 84 Gulf of Mexico (GoM) simulations and evaluated on 12 testing simulations. Each simulation contained 720 monthly time steps, with the first 360 months as the injection period and the rest as the post-injection period. The proposed workflow, incorporating DR and DL models, accurately predicts the normalized 3D pressure fields, achieving mean square error (MSE) of 2.92 × 10 -7 compared to the ground truth obtained from a full-physics simulator. Furthermore, the proposed strategy outperformed PCA and convolutional autoencoder (CAE) models on 3D saturation fields, resulting in minor workflow prediction errors with an MSE of 2.93 × 10 -5 . The results suggest the proposed workflow provides sufficient predictive fidelity across temporal and spatial scales, and enables a speedup of 160 times compared to the full-physics simulator, facilitating improved decision-making and risk assessment for large-scale GCS management in real-time scenarios.

3D reconstruction model↗
NASA NTRS1983

Carbon monoxide mixing ratio inference from gas filter radiometer data

A new algorithm has been developed which permits, for the first time, real time data reduction of nadir measurements taken with a gas filter correlation radiometer to determine tropospheric carbon monoxide concentrations. The algorithm significantly reduces the complexity of the equations to be solved while providing accuracy comparable to line-by-line calculations. The method is based on a regression analysis technique using a truncated power series representation of the primary instrument output signals to infer directly a weighted average of trace gas concentration. The results produced by a microcomputer-based implementation of this technique are compared with those produced by the more rigorous line-by-line methods. This algorithm has been used in the reduction of Measurement of Air Pollution from Satellites, Shuttle, and aircraft data.

Wallio, H. A.↗
DOE OSTI2020

A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy datasets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored dataset. Unfortunately, and this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as Automatic Relevance Determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial dataset, and estimates which K produces stable groups of latent features that reconstruct the initial dataset well. We then train a Multi-Layter Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solution, obtained from NMFk. In order to train the MLP classifier, a training set of 58,660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark datasets, the swimmer and MIT face data, NMFk/MLP correctly recovers the established number of hidden features. Finally, we compare the accuracy of our method to the ARD, AIC and Stability-based methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Efficient near-field ptychography reconstruction using the Hessian operator

X-ray ptychography is a powerful and robust coherent imaging method providing access to the complex object and probe (illumination). Ptychography reconstruction is typically performed using first-order methods due to their computational efficiency. Higher-order methods, while potentially more accurate, are often prohibitively expensive in terms of computation. In this study, we present a mathematical framework for reconstruction using second-order information derived from an efficient computation of the bilinear Hessian and Hessian operator. The formulation is provided for Gaussian-based models, enabling the simultaneous reconstruction of the object, probe, and object positions. Synthetic data tests, along with experimental near-field ptychography data processing, demonstrate a ten-fold reduction in computation time compared to first-order methods. The derived formulas for computing the Hessians, along with the strategies for incorporating them into optimization schemes, are well-structured and easily adaptable to various ptychography problem formulations.

Carlsson, Marcus [Lund Univ. (Sweden)] (ORCID:0000↗
NASA NTRS1992

Dynamic test/analysis correlation using reduced analytical models

Test/analysis correlation is an important aspect of the verification of analysis models which are used to predict on-orbit response characteristics of large space structures. This paper presents results of a study using reduced analysis models for performing dynamic test/analysis correlation. The reduced test-analysis model (TAM) has the same number and orientation of DOF as the test measurements. Two reduction methods, static (Guyan) reduction and the Improved Reduced System reduction, are applied to the test/analysis correlation of a laboratory truss structure. Simulated test results and modal test data are used to examine the performance of each method. It is shown that selection of DOF to be retained in the TAM is critical when large structural masses are involved. In addition, the use of modal test results may provide difficulties in TAM accuracy even if a large number of DOF are retained in the TAM.

Mcgowan, Paul E.↗
NASA NTRS1995

Reducing Truncation Error In Integer Processing

Improved method of rounding off (truncation of least-significant bits) in integer processing of data devised. Provides for reduction, to extremely low value, of numerical bias otherwise generated by accumulation of truncation errors from many arithmetic operations. Devised for use in integer signal processing, in which rescaling and truncation usually performed to reduce number of bits, which typically builds up in sequence of operations. Essence of method to alternate direction of roundoff (plus, then minus) on alternate occurrences of truncated values contributing to bias.

Thomas, J. Brooks↗