Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data reduction methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

QProR: An Efficient Framework for Quantity-of-Interest Based Progressive Retrieval with Guaranteed Error Control

Scientific applications generate an unprecedented volume of data, overwhelming the network and file systems’ bandwidth and posing challenges for efficient and scalable data retrieval and analysis. Progressive data compression offers a promising solution by enabling on-demand retrieval at reduced size. However, existing progressive methods either fail to bound the errors in essential quantities of interest (QoIs) derived from raw data or suffer from suboptimal retrieval efficiency. In this work, we propose QProR, an efficient QoI-based progressive framework that optimizes progressive retrieval for target QoIs. Our key contributions include: (1) a systematic framework that integrates error-controlled lossy compressors with bitplane encoding while decoupling the two processes for high flexibility and adaptability; (2) a novel weighted bitplane encoding method which incorperates QoI knowledge into data refactoring to enhance retrieval efficiency; (3) an optimized retrieval strategy that accounts for the varying impacts of different variables on multivariate QoIs; (4) comprehensive evaluations using six real-world datasets from multiple scientific applications and thorough comparisons against state of the arts. Experimental results demonstrate that QProR achieves up to 80.38% reduction in the retrieval size under the same requested QoI error tolerance, when compared with the best-performing existing methods. When transferring 384 GB of scientific data to remote sites, QProR delivers up to 1.68 × speedup in the end-to-end data transfer performance.

Li, Wenbo [University of Kentucky]↗
NASA NTRSDate not supplied

A Dynamic PCA and Machine Learning Tool for Automated Identification of Solar Wind Disturbances Impacting Earth’s Magnetosphere

Earth’s magnetosphere is continuously impacted by solar wind and interplanetary magnetic field (IMF) disturbances, such as shocks, discontinuities, magnetic clouds and more. Understanding how such disturbances propagate from the Sun and what is their impact on the different magnetospheric domains is key to understanding and forecasting energy transfer from the solar wind to Earth. The large number of overlapping solar wind and magnetospheric missions carrying magnetometers and the recent advances in communications and data storage technologies have enabled an unprecedented quantity of high-fidelity magnetic field data captured by in-situ spacecraft to be available at the click of a button. However, this massive quantity of available data can prove unwieldy for researchers, limiting the identification of interesting phenomena and disturbances to a relatively small percentage of the total dataset. Several techniques have been previously developed for automated identification of specific types of magnetic anomalies, but these methods are typically mission-specific and can be difficult to generalize. We present initial results for a generic method of automated anomaly detection in magnetic field measurements based on dimensionality reduction and unsupervised clustering via machine learning. The benefit of our technique is its high degree of generalizability and flexibility which make it a most useful data survey tool for a wide range of magnetic field datasets. This method can also be applied simultaneously to other observed time-series properties like plasma density, pressure, and velocity for more accurate event identification. Additionally, the application of this method to data captured by multiple spacecraft enables the simultaneous identification of disturbances and the determination of their propagation characteristics. Initial evaluation of this technique has been performed using data from Magnetospheric MultiScale (MMS) and THEMIS-ARTEMIS missions, providing a testbed scenario for the future Heliophysics Environmental and Radiation Measurement Experiment Suite (HERMES) platform instruments that will measure solar wind and IMF properties from lunar orbit onboard the Gateway station.

Miguel Martinez-Ledesma↗
NASA NTRSDate not supplied

A Dynamic PCA and Machine Learning Tool for Automated Identification of Solar Wind Disturbances Impacting Earth’s Magnetosphere

Earth’s magnetosphere is continuously impacted by solar wind and interplanetary magnetic field (IMF) disturbances, such as shocks, discontinuities, magnetic clouds and more. Understanding how such disturbances propagate from the Sun and what is their impact on the different magnetospheric domains is key to understanding and forecasting energy transfer from the solar wind to Earth. The large number of overlapping solar wind and magnetospheric missions carrying magnetometers and the recent advances in communications and data storage technologies have enabled an unprecedented quantity of high-fidelity magnetic field data captured by in-situ spacecraft to be available at the click of a button. However, this massive quantity of available data can prove unwieldy for researchers, limiting the identification of interesting phenomena and disturbances to a relatively small percentage of the total dataset. Several techniques have been previously developed for automated identification of specific types of magnetic anomalies, but these methods are typically mission-specific and can be difficult to generalize. We present initial results for a generic method of automated anomaly detection in magnetic field measurements based on dimensionality reduction and unsupervised clustering via machine learning. The benefit of our technique is its high degree of generalizability and flexibility which make it a most useful data survey tool for a wide range of magnetic field datasets. This method can also be applied simultaneously to other observed time-series properties like plasma density, pressure, and velocity for more accurate event identification. Additionally, the application of this method to data captured by multiple spacecraft enables the simultaneous identification of disturbances and the determination of their propagation characteristics. Initial evaluation of this technique has been performed using data from Magnetospheric MultiScale (MMS) and THEMIS-ARTEMIS missions, providing a testbed scenario for the future Heliophysics Environmental and Radiation Measurement Experiment Suite (HERMES) platform instruments that will measure solar wind and IMF properties from lunar orbit onboard the Gateway station.

Miguel Martinez-Ledesma↗
DOE OSTI2025

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

This work presents a data-driven method for learning low-dimensional time-dependent physics-based surrogate models whose predictions are endowed with uncertainty estimates. We use the operator inference approach to model reduction that poses the problem of learning low-dimensional model terms as a regression of state space data and corresponding time derivatives by minimizing the residual of reduced system equations. Standard operator inference models perform well with accurate training data that are dense in time, but producing stable and accurate models when the state data are noisy and/or sparse in time remains a challenge. Another challenge is the lack of uncertainty estimation for the predictions from the operator inference models. Our approach addresses these challenges by incorporating Gaussian process surrogates into the operator inference framework to (1) probabilistically describe uncertainties in the state predictions and (2) procure analytical time derivative estimates with quantified uncertainties. The formulation leads to a generalized least-squares regression and, ultimately, reduced-order models that are described probabilistically with a closed-form expression for the posterior distribution of the operators. The resulting probabilistic surrogate model propagates uncertainties from the observed state data to reduced-order predictions. Furthermore, we demonstrate the method is effective for constructing low-dimensional models of two nonlinear partial differential equations representing a compressible flow and a nonlinear diffusion–reaction process, as well as for estimating the parameters of a low-dimensional system of nonlinear ordinary differential equations representing compartmental models in epidemiology.

Data-driven model reduction↗
DOE OSTI2020

Learning to Solve Large-Scale Security-Constrained Unit Commitment Problems

Security-constrained unit commitment (SCUC) is a fundamental problem in power systems and electricity markets. In practical settings, SCUC is repeatedly solved via mixed-integer linear programming (MIP), sometimes multiple times per day, with only minor changes in input data. In this work, we propose a number of machine learning techniques to effectively extract information from previously solved instances in order to significantly improve the computational performance of MIP solvers when solving similar instances in the future. Based on statistical data, we predict redundant constraints in the formulation, good initial feasible solutions, and affine subspaces where the optimal solution is likely to lie, leading to a significant reduction in problem size. Computational results on a diverse set of realistic and large-scale instances show that using the proposed techniques, SCUC can be solved on average 4.3 times faster with optimality guarantees and 10.2 times faster without optimality guarantees, with no observed reduction in solution quality. Out-of-distribution experiments provide evidence that the method is somewhat robust against data-set shift. Summary of Contribution. The paper describes a novel computational method, based on a combination of mixed-integer linear programming (MILP) and machine learning (ML), to solve a challenging and fundamental optimization problem in the energy sector. The method advances the state-of-the-art, not only for this particular problem, but also, more generally, in solving discrete optimization problems via ML. We expect that the techniques presented can be readily used by practitioners in the energy sector and adapted, by researchers in other fields, to other challenging operations research problems that are solved routinely.

Machine Learning↗
DOE OSTI2020

Sampling Low-Dimensional Markovian Dynamics for Preasymptotically Recovering Reduced Models from Data with Operator Inference

This work introduces a method for learning low-dimensional models from data of high-dimensional black-box dynamical systems. The novelty is that the learned models are exactly the reduced models that are traditionally constructed with classical projection-based model reduction techniques. Thus, the proposed approach learns models that are guaranteed to have the well-studied properties of reduced models known from model reduction, without requiring full knowledge of the governing equations and without requiring the operators of the high-dimensional systems. The key ingredient is a new data sampling scheme to obtain re-projected trajectories of high-dimensional systems that correspond to Markovian dynamics in low-dimensional subspaces. The exact recovery of reduced models from these re-projected trajectories is guaranteed pre-asymptotically under certain conditions for finite amounts of data and for a large class of systems with polynomial nonlinear terms. Numerical results demonstrate that the low-dimensional models learned with the proposed approach match reduced models from traditional model reduction up to numerical errors in practice. In conclusion, the numerical results further indicate that low-dimensional models fitted to re-projected trajectories are predictive even in situations where models fitted to trajectories without re-projection are inaccurate and unstable.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1999

An ecologic study of dietary links to prostate cancer

BACKGROUND: The etiology of prostate cancer has not been fully resolved in the scientific and medical literature, although the non-fat portion of milk and calcium are emerging as leading dietary risk factors, with lycopene (found in tomatoes) and vitamin D apparently being risk reduction factors. METHODS: The ecologic (multi-country statistical) approach is used to study dietary links to prostate cancer. Mortality data from 1986 for various age groups in 41 countries are compared with national consumer macronutrient supply values for 1983 and tomato supply values for 1985. RESULTS: For 28 countries with more than five Kcal/day of tomatoes in the consumer supply, a linear combination of non-fat milk (risk factor) and tomatoes (risk reduction factor) was found to have the highest statistical association with prostate cancer mortality rates for men over the age of 35, with the Pearson regression coefficient (R2) for those aged 65-74 years = 0.67 and p < 0.001. For the 13 countries with fewer than six Kcal/day of tomatoes, non-fat milk had the highest association (R2 = 0.92, p < 0.001 for men aged 65-74 years). For 41 countries combined, the non-fat portion of milk had the highest association with prostate cancer mortality rates (R2 = 0.73, p < 0.001 for men aged 65-74 years). CONCLUSIONS: These results support the results of several cohort studies which found the non-fat portion of milk to have the highest association with prostate cancer, likely due to the calcium, and tomatoes to reduce the risk of prostate cancer, most likely due to lycopene.

Diet/adverse effects↗
DOE OSTI2022

Industrial battery operation and utilization in the presence of electrical load uncertainty using Bayesian decision theory

Behind the meter battery storage is becoming increasing popular in all sectors, though enthusiasm has recently lagged in the industrial sector. Even though there may be many factors contributing to this including lack of innovation, prohibitive costs, and undesirable rate structures, a difficulty arises in accounting for uncertainty of electrical load in industrial facilities while still attempting to utilize battery storage as much as possible all while trying to achieve fiscal profitability. Here this study utilizes Gaussian process regression and Bayesian decision theory to organize load data and quantify electrical load uncertainty to properly and effectively discharge industrial battery storage. The study employs a simulation model to set battery load setpoints for the span of the utility billing period according to the degree of risk aversion. This combination of economic analysis according to utility billing period and utilization of degree of risk aversion to make decisions on the uncertainty of the data has not before been applied to battery storage. The method resulted in an annual average reduction of peak demand by 3.8 % at the lowest amount of savings and lowest risk aversion. The highest risk aversion resulted in an annual average reduction of peak demand of 7.5 %. The maximum reduction of peak load in any month was 13.8 % in the month of December with a relatively high risk aversion. With a the highest amount risk aversion tested, the model reduced demand ten of the twelve months of the year.

25 ENERGY STORAGE↗
NASA NTRS1976

The profile of the helium 584 A solar line - Preliminary results from rocket-borne resonance absorption spectrometer

A measurement of the line profile of the He I 584 A line from the sun has been made on board a rocket on 20 January 1975. Wavelength resolution is obtained by the resonance absorption method. The instrument is described, and its performance in flight is analysed. The principles of data reduction are given, and preliminary results are presented and compared with available data from other sources. The width of the solar He I resonance line is found to be larger than 110 mA.

Delaboudiniere, J. P.↗
NASA NTRS1993

A survey for rapid variability among early main-sequence A stars

A survey of nonpeculiar early (A0-A5) main-sequence A stars for rapid, low-amplitude variability is presented. The survey is also an independent test of the main-sequence mass-loss theory proposed by Willson et al. (1986). All observations were gathered with the University of Wisconsin Two-Star Photometer. Several period-searching methods are used to analyze time series of differential-photometric data. This instrument coupled with a computerized high-speed data collection system was used with small telescopes at Pine Bluff Observatory and Table Mountain Observatory. Several period-search methods are used to analyze time series of differential-photometric data. This instrument and data-reduction technique allow millimagnitude light variations to be detected. The survey also discovered several low-amplitude Delta Sct stars, all of which are in or blueward of the recognized instability strip.

Schutt, Randy L.↗
NASA NTRS2003

Vector-Ordering Filter Procedure for Data Reduction

The vector-ordering filter (VOF) technique involves a procedure for sampling a large population of data vectors to select a subset of data vectors that fully characterize the state space of the large population. The VOF technique enables a large reduction of the volume of data that must be handled in the automated monitoring system and method discussed in the two immediately preceding articles. In so doing, the VOF technique enables the development of data-driven mathematical models of a monitored asset from sets of data that would otherwise exceed the memory capacities of conventional engineering computers. Data-driven mathematical models have been shown to offer high fidelity for purposes of control and monitoring of assets. In practice, a collection of asset-operating observations is acquired with the intention that the collection contain observations characteristic of the full dynamic range of operation of the asset. Often, such a collection contains an extremely large number of observations, many of which are redundant. The VOF technique fills the need for a means to extract, from the original collection of observational data, a reduced data matrix that excludes redundant data while maintaining the full statistical character and dynamic range of the original data. The reduced data matrix can then be used as the input data for development of a mathematical model of the monitored asset, or as training data for a neural-network substitute for an explicit mathematical model of the asset. Alternatively, the reduced data matrix can, itself, be used directly as a mathematical model of the monitored asset, as is commonly done in multivariate state-estimation techniques. The original data are collected from the asset over a range of operating states and are put in matrix form. Each column vector in the original data matrix represents the signal values acquired at a particular operational state of the asset. Thus, the number of columns of the original data matrix equals the number of observed states and the number of rows in this matrix equals the number of signals acquired at each observation. In the VOF technique, one extracts the reduced data matrix from the original data matrix through the selection of a representative subset of the column (state) vectors.

Bickford, Randall L.↗
NASA NTRS2019

Orion Capsule Parachute Assembly System (CPAS) Main and Drogue Canopy Instrumentation

Reefing systems are often used in parachute systems to control loading and inflation of the canopies. These systems typically use pyrotechnic cutters to sever lines to progress through the staging. Existing cutters have been used for more than 50 years and have been proven adequate through countless demands. These cutters must survive and operate through severe environments that include a large snatch load at deployment and random vibration through the stage. In lieu of extensive drop test campaigns, future cutters will need to be ground tested to environments which are representative of the flight environments. The Capsule Parachute Assembly System (CPAS) developed sensors that capture accelerations and rates in three axes at approximately 1500 Hertz (Hz) during parachute deployment and operation. These sensors have mass properties, size, and an attachment method similar to existing reefing line cutter assemblies, maximizing the usability of the data. Various risk reduction steps were taken to minimize any negative effects to the qualification parachute system and to increase the likelihood of collecting usable data. 24 sensors were flown on 12 qualification parachutes, successfully collecting sufficient data to characterize the deployment and operational environment of the reefing line cutters.

Daum, Jared S.↗
NASA NTRS1994

A Space and Atmospheric Visualization Science System

SAVS (a Space and Atmospheric Visualization Science system) is an integrated system with user-friendly functionality that employs a 'push-button' software environment that mimics the logical scientific processes in data acquisition, reduction, analysis, and visualization. All of this is accomplished without requiring a detailed understanding of the methods, networks, and modules that link the tools and effectively execute the functions. This report describes SAVS and its components, followed by several applications based on generic research interests in interplanetary and magnetospheric physics (IMP/ISTP), active experiments in space (CRRES), and mission planning focused on the earth's thermospheric, ionospheric, and mesospheric domains (TIMED). The final chapters provide a user-oriented description of interface functionalities, hands-on operations, and customized modules, with details of the primary modules presented in the appendices. The overall intent of the report is to reflect the accomplishments of the three-year development effort and to introduce potential users to the power and utility of the integrated data acquisition, analysis, and visualization system.

Szuszczewicz, E. P.↗
NASA NTRS1977

Analyses of atmospheric extinction data obtained by astronomers. I - A time-trend analysis of data with internal accidental errors obtained at four observatories

Long-term time-trend analysis was performed on astronomical atmospheric extinction data in wideband UBV and various narrow-band systems recorded at Cerro Tololo, Kitt Peak, Lick, and McDonald observatories. All of the data had to be transformed into uniform monochromatic extinction data before trend analysis could be performed. The paper describes the various reduction techniques employed. The time-trend analysis was then carried out by the method of least squares. A special technique, called 'histogram shaping', was employed to adjust for the fact that the errors of the reduced monochromatic extinction data were not essentially Gaussian. On the assumption that there are no compensatory background and local extinction changes, the best values obtained for extinction trends due to background aerosol changes during the years 1960 to 1972 are 0.006 + or - 0.013 (rms) and 0.009 + or - 0.009 (rms) stellar magnitudes per air mass per decade in the blue and yellow wavelength regions, respectively.

Taylor, B. J.↗
DOE OSTI2023

SonicPy: a suite of programs for ultrasound pulse-echo data acquisition and analysis

Sound speed and elastic constants measurements in solids and liquids are commonly performed using the ultrasound pulse-echo technique. Recent advances have expanded the use of this technique at numerous high pressure synchrotron beamlines and offline laboratories. However, the increased experimental throughput has revealed many limitations in existing software for handling the rapid measurement and the subsequent data-reduction. Here, we report the development of a collection of computer programs for sound speed measurements using the ultrasound pulse-echo technique, compatible with stepped multi-frequency, as well as broadband-pulse, couplant-corrected methods. The programs provide a highly interactive graphical interface, enable efficient measurement, exploration and near real-time analysis of the ultrasound data, and contain features useful for working with samples under high pressure and/or high temperature. The included analysis programs can alleviate the time required for data reduction from hours to less than a minute, allowing users to make timely and informed decisions regarding the appropriate experimental parameters.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2005

Method for Estimating Thread Strength Reduction of Damaged Parent Holes with Inserts

During normal assembly and disassembly of bolted-joint components, thread damage and/or deformation may occur. If threads are overloaded, thread damage/deformation can also be anticipated. Typical inspection techniques (e.g. using GO-NO GO gages) may not provide adequate visibility of the extent of thread damage. More detailed inspection techniques have provided actual pitch-diameter profiles of damaged-hardware holes. A method to predict the reduction in thread shear-out capacity of damaged threaded holes has been developed. This method was based on testing and analytical modeling. Test samples were machined to simulate damaged holes in the hardware of interest. Test samples containing pristine parent-holes were also manufactured from the same bar-stock material to provide baseline results for comparison purposes. After the particular parent-hole thread profile was machined into each sample a helical insert was installed into the threaded hole. These samples were tested in a specially designed fixture to determine the maximum load required to shear out the parent threads. It was determined from the pristine-hole samples that, for the specific material tested, each individual thread could resist an average load of 3980 pounds. The shear-out loads of the holes having modified pitch diameters were compared to the ultimate loads of the specimens with pristine holes. An equivalent number of missing helical coil threads was then determined based on the ratio of shear-out loads for each thread configuration. These data were compared with the results from a finite element model (FEM). The model gave insights into the ability of the thread loads to redistribute for both pristine and simulated damage configurations. In this case, it was determined that the overall potential reduction in thread load-carrying capability in the hardware of interest was equal to having up to three fewer threads in the hole that bolt threads could engage. One- half of this potential reduction was due to local pitch-diameter variations and the other half was due to overall pitch-diameter enlargement beyond Class 2 fit. This result was important in that the thread shear capacity for this particular hardware design was the limiting structural capability. The details of the method development, including the supporting testing, data reduction and analytical model results comparison will be discussed hereafter.

Johnson, David L.↗
DOE OSTI2026

Tabulated data for Lignin-Derived Phenolic Compounds and Water Are Effective Cosolvents for Reductive Catalytic Fractionation

Reductive catalytic fractionation (RCF) is an effective lignin-first biorefining method to extract lignin as a stabilized oil from lignocellulosic biomass. To realize RCF at scale, process modeling has shown that minimizing the use of exogenous organic solvents is critical. To this end, here we investigate the ability of lignin-derived monomers to act as either solvents or cosolvents for RCF. We begin by examining the influence of lignin-derived aromatic compounds (4-propylguaiacol, 4-propylphenol, and propylbenzene) on RCF monomer yields and subsequently extend our analysis to mixtures of 4-propylguaiacol and either methanol or water. We demonstrate that 4-propylguaiacol is an effective solvent for lignin extraction and depolymerization during RCF, especially when used in combination with water as a cosolvent. Cosolvent mixtures of 4-propylguaiacol and water enable up to 81% lignin extraction, monomer yields up to 25 wt %, and postreaction phase separation. However, unlike methanol, water as a cosolvent fails to inhibit aromatic ring hydrogenation when conducted over Ru/C as a catalyst, potentially leading to excess hydrogen consumption in a process utilizing this approach. Nonetheless, these results suggest a promising strategy for eliminating external organic solvents from RCF by utilizing mixtures of lignin-derived compounds and water as alternative extraction solvents. This is the tabulated data for the paper

biorefining↗
DOE OSTI2025

The Effects of Compounded Model Size Reductions on Adversarial Robustness

Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimization methods offer considerable advantages, they potentially introduce AI-related security vulnerabilities, particularly concerning model robustness with respect to adversarial AI attacks. Prior research has extensively examined the impact of quantization on adversarial robustness; however, the effects of alternative reduction techniques and their combinations remain understudied. This paper investigates the impact of model size reduction techniques on adversarial robustness, when applied individually and combined. We utilized Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks to generate adversarial perturbations for both training and testing data, and then evaluated the models' accuracy under adversarial training conditions. Our findings revealed that reduction techniques generally diminished robustness; although, combining techniques was not found to make robustness any worse than when applied individually. Moreover, specific techniques can potentially enhance resistance to small size perturbations. This research provides insights into the trade-offs between model size reduction and security, establishing a foundation for future investigations into improving adversarial training techniques and methodologies for maintaining robustness while preserving memory footprint benefits.

Austria, Phillipe [ORNL] (ORCID:0000000236223973)↗