Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data reduction techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Maintaining Trust in Reduction: Preserving the Accuracy of Quantities of Interest for Lossy Compression

As the growth of data sizes continues to outpace computational resources, there is a pressing need for data reduction techniques that can significantly reduce the amount of data and quantify the error incurred in compression. Compressing scientific data presents many challenges for reduction techniques since it is often on non-uniform or unstructured meshes, is from a high-dimensional space, and has many Quantities of Interests (QoIs) that need to be preserved. To illustrate these challenges, we focus on data from a large scale fusion code, XGC. XGC uses a Particle-In-Cell (PIC) technique which generates hundreds of PetaBytes (PBs) of data a day, from thousands of timesteps. XGC uses an unstructured mesh, and needs to compute many QoIs from the raw data, f.One critical aspect of the reduction is that we need to ensure that QoIs derived from the data (density, temperature, flux surface averaged momentums, etc.) maintain a relative high accuracy. We show that by compressing XGC data on the high-dimensional, nonuniform grid on which the data is defined, and adaptively quantizing the decomposed coefficients based on the characteristics of the QoIs, the compression ratios at various error tolerances obtained using a multilevel compressor (MGARD) increases more than ten times. We then present how to mathematically guarantee that the accuracy of the QoIs computed from the reduced f is preserved during the compression. We show that the error in the XGC density can be kept under a user-specified tolerance over 1000 timesteps of simulation using the mathematical QoI error control theory of MGARD, whereas traditional error control on the data to be reduced does not guarantee the accuracy of the QoIs.

Gong, Qian↗
NASA NTRS1982

A technique for determining solar irradiance deficits

An analytic technique which determines the variation of solar irradiance from long term averages is presented. The technique involves computer-assisted data reduction techniques, and was designed to improve system reliability by determining the amount of storage capability required to supplement a baseline system. Variations in time intervals of up to 60 days can be determined, and 10 years of data collection are reviewed. The technique involves first calculating average monthly irradiance values, then examining the average irradiance deviation over time intervals. The calculation procedure is clarified by determining solar energy level probabilities and the long term solar energy deviation (achieved by repeatedly integrating actual irradiance figures). It is found that a 15% increase in collector area and the addition of energy storage or backup are essential contributions to achieving cost-effectiveness. In addition, one to seven no-sun day storage capacities are required to accommodate weather caused deficits.

Gonzalez, C. C.↗
NASA NTRS2013

Improved Data Reduction Algorithm for the Needle Probe Method Applied to In-Situ Thermal Conductivity Measurements of Lunar and Planetary Regoliths

The needle probe method (also known as the' hot wire' or 'line heat source' method) is widely used for in-situ thermal conductivity measurements on soils and marine sediments on the earth. Variants of this method have also been used (or planned) for measuring regolith on the surfaces of extra-terrestrial bodies (e.g., the Moon, Mars, and comets). In the near-vacuum condition on the lunar and planetary surfaces, the measurement method used on the earth cannot be simply duplicated, because thermal conductivity of the regolith can be approximately 2 orders of magnitude lower. In addition, the planetary probes have much greater diameters, due to engineering requirements associated with the robotic deployment on extra-terrestrial bodies. All of these factors contribute to the planetary probes requiring much longer time of measurement, several tens of (if not over a hundred) hours, while a conventional terrestrial needle probe needs only 1 to 2 minutes. The long measurement time complicates the surface operation logistics of the lander. It also negatively affects accuracy of the thermal conductivity measurement, because the cumulative heat loss along the probe is no longer negligible. The present study improves the data reduction algorithm of the needle probe method by shortening the measurement time on planetary surfaces by an order of magnitude. The main difference between the new scheme and the conventional one is that the former uses the exact mathematical solution to the thermal model on which the needle probe measurement theory is based, while the latter uses an approximate solution that is valid only for large times. The present study demonstrates the benefit of the new data reduction technique by applying it to data from a series of needle probe experiments carried out in a vacuum chamber on JSC-1A lunar regolith stimulant. The use of the exact solution has some disadvantage, however, in requiring three additional parameters, but two of them (the diameter and the volumetric heat capacity of the probe) can be measured and the other (the volumetric heat capacity of the regolith/stimulant) may be estimated from the surface geologic observation and temperature measurements. Therefore, overall, the new data reduction scheme would make in-situ thermal conductivity measurement more practical on planetary missions.

JSC-1A↗
DOE OSTI2023

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗
NASA NTRS1992

Laser absorption flow diagnostics for use in hypersonic ground-based and flight experiments

Consideration is given to laboratory ground-based scale systems for measuring OH and NO and diode laser systems for measuring O2 and H2O. Several diagnostic instruments which are currently being used in a scramjet combustor test rig of the Ames 16-inch shock tunnel are described. Data on mole fraction, temperature, density, pressure, and velocity is obtained using specific data reduction techniques. These measurements provide critical data for evaluation of the propulsion system performance which include mass capture for quantification of fuel equivalence ratio (O2), flow contamination and emission levels (NO), combustion progress (OH), and injector mixing and combustion efficiency (H2O).

Cavolowsky, John A.↗
NASA NTRS1984

Radar data smoothing filter study

The accuracy of the current Wallops Flight Facility (WFF) data smoothing techniques for a variety of radars and payloads is examined. Alternative data reduction techniques are given and recommendations are made for improving radar data processing at WFF. A data adaptive algorithm, based on Kalman filtering and smoothing techniques, is also developed for estimating payload trajectories above the atmosphere from noisy time varying radar data. This algorithm is tested and verified using radar tracking data from WFF.

White, J. V.↗
NASA NTRS1989

A vector scanning processing technique for pulsed laser velocimetry

Pulsed-laser-sheet velocimetry yields two-dimensional velocity vectors across an extended planar region of a flow. Current processing techniques offer high-precision (1-percent) velocity estimates, but can require hours of processing time on specialized array processors. Sometimes, however, a less accurate (about 5 percent) data-reduction technique which also gives unambiguous velocity vector information is acceptable. Here, a direct space-domain processing technique is described and shown to be far superior to previous methods in achieving these objectives. It uses a novel data coding and reduction technique and has no 180-deg directional ambiguity. A complex convection vortex flow was recorded and completely processed in under 2 min on an 80386-based PC, producing a two-dimensional velocity-vector map of the flowfield. Pulsed-laser velocimetry data can thus be reduced quickly and reasonably accurately, without specialized array processing hardware.

Wernet, Mark P.↗
NASA NTRS1989

A vector scanning processing technique for pulsed laser velocimetry

Pulsed laser sheet velocimetry yields nonintrusive measurements of two-dimensional velocity vectors across an extended planar region of a flow. Current processing techniques offer high precision (1 pct) velocity estimates, but can require several hours of processing time on specialized array processors. Under some circumstances, a simple, fast, less accurate (approx. 5 pct), data reduction technique which also gives unambiguous velocity vector information is acceptable. A direct space domain processing technique was examined. The direct space domain processing technique was found to be far superior to any other techniques known, in achieving the objectives listed above. It employs a new data coding and reduction technique, where the particle time history information is used directly. Further, it has no 180 deg directional ambiguity. A complex convection vortex flow was recorded and completely processed in under 2 minutes on an 80386 based PC, producing a 2-D velocity vector map of the flow field. Hence, using this new space domain vector scanning (VS) technique, pulsed laser velocimetry data can be reduced quickly and reasonably accurately, without specialized array processing hardware.

Wernet, Mark P.↗
DOE OSTI2019

Machine-Learning Analysis of Moisture Carryover in Boiling Water Reactors

Moisture carryover (MCO) is modeled in the General Electric Type-4 boiling water reactor (BWR) using machine-learning methods and data from operating plants. Understanding MCO and the conditions that give rise to an elevated value is important since excessive MCO can damage critical turbine components, can result in elevated dose levels to on-site personnel, and can interfere with late-cycle power management. The analysis of MCO takes into account simplifying reactor symmetries and important geometric dependencies. The plant data are taken from several reactors and were collected over multiple years and multiple fuel cycles. A brief description of the origin of MCO in U.S. BWR plants is given. A machine-learning model is constructed from the data using applicable algorithms and data-reduction techniques. Matching model complexity with available data is one of the more challenging machine-learning tasks. Too many features and too little data will lead to overfitting. The data for each fuel cycle included over 6876 original features, 9 for each fuel bundle. Two approaches are used to reduce the data set into a manageable number of features. The first was an engineering analysis that resulted in the selection of steam quality Q and steam liquid phase velocity V L as the main features driving MCO. Using a Q and a V L for each fuel bundle gives 1528 Q and a V L feature describing the reactor behavior. An analysis of different functional forms of these two variables led to the actual inputs to the neural network model. The second approach involved the use of statistical techniques such as Pearson’s correlation and k-means analysis. The identified groupings of bundles behaved similarly. Treating each grouping as a single feature further reduced the input variable set to a manageable number. Here, a model selection criterion is proposed, and results are presented along with a discussion of related issues.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Data reduction for low energy nuclear physics experiments using data frames

Low energy nuclear physics experiments are transitioning towards fully digital data acquisition systems. Realizing the gains in flexibility afforded by these systems relies on equally flexible data reduction techniques. In this paper, methods utilizing data frames and in-memory techniques to work with data, including data from self-triggering, digital data acquisition systems, are discussed within the context of a Python package, sauce. It is shown that data frame operations can encompass common analysis needs and allow interactive data analysis. Two event building techniques, dubbed referenced and referenceless event building, are shown to provide a means to transform raw list mode data into correlated multi-detector events. These techniques are demonstrated in the analysis of two example data sets.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Codebase release 2.0 for sauce

Low energy nuclear physics experiments are transitioning towards fully digital data acquisition systems. Realizing the gains in flexibility afforded by these systems relies on equally flexible data reduction techniques. In this paper, methods utilizing data frames and in-memory techniques to work with data, including data from self-triggering, digital data acquisition systems, are discussed within the context of a Python package, sauce. It is shown that data frame operations can encompass common analysis needs and allow interactive data analysis. Two event building techniques, dubbed referenced and referenceless event building, are shown to provide a means to transform raw list mode data into correlated multi-detector events. These techniques are demonstrated in the analysis of two example data sets.

Marshall, Caleb (ORCID:0000000211942920)↗
NASA NTRS1979

Voyager high gain antenna calibration and pointing

A mathematical description of the data reduction technique used in analyzing Voyager calibration data is presented. To achieve the required telecommunication link performance, highly accurate pointing of the Voyager high gain antenna boresight relative to earth is necessary. To provide the optimum pointing, in-flight calibrations of the high gain antenna pointing mechanism are regularly made, and the design of the calibration and the antenna error models is delineated. It is shown that due to the use of wide angle sun sensors for celestial attitude control, the Voyager antenna error model differs from those of previous missions. Results of the in-flight calibrations and their implementation in improving the antenna pointing are also presented.

Jahanshahi, M. H.↗
NASA NTRS1991

A relative-intensity two-color phosphor thermography system

The NASA LaRC has developed a relative-intensity two-color phosphor thermography system. This system has become a standard technique for acquiring aerothermodynamic data in LaRC Hypersonic Facilities Complex (HFC). The relative intensity theory and its application to the LaRC phosphor thermography system is discussed along with the investment casting technique which is critical to the utilization of the phosphor method for aerothermodynamic studies. Various approaches to obtaining quantitative heat transfer data using thermographic phosphors are addressed and comparisons between thin-film data and thermographic phosphor data on an orbiter-like configuration are presented. In general, data from these two techniques are in good agreement. A discussion is given on the application of phosphors to integration heat transfer data reduction techniques (the thin film method) and preliminary heat transfer data obtained on a calibration sphere using thin-film equations are presented. Finally, plans for a new phosphor system which uses target recognition software are discussed.

Merski, N. Ronald↗
DOE OSTI2021

Prospects for Galactic and stellar astrophysics with asteroseismology of giant stars in the TESS continuous viewing zones and beyond

ABSTRACT The NASA Transiting Exoplanet Survey Satellite (NASA-TESS) mission presents a treasure trove for understanding the stars it observes and the Milky Way, in which they reside. We present a first look at the prospects for Galactic and stellar astrophysics by performing initial asteroseismic analyses of bright (G < 11) red giant stars in the TESS southern continuous viewing zone (SCVZ). Using three independent pipelines, we detect νmax and Δν in 41 per cent of the 15 405 star parent sample (6388 stars), with consistency at a level of $\sim \! 2{{\ \rm per\ cent}}$ in νmax and $\sim \! 5{{\ \rm per\ cent}}$ in Δν. Based on this, we predict that seismology will be attainable for ∼3 × 105 giants across the whole sky and at least 104 giants with ≥1 yr of observations in the TESS-CVZs, subject to improvements in analysis and data reduction techniques. The best quality TESS-CVZ data, for 5574 stars where pipelines returned consistent results, provide high-quality power spectra across a number of stellar evolutionary states. This makes possible studies of, for example, the asymptotic giant branch bump. Furthermore, we demonstrate that mixed ℓ = 1 modes and rotational splitting are cleanly observed in the 1-yr data set. By combining TESS-CVZ data with TESS-HERMES, SkyMapper, APOGEE, and Gaia, we demonstrate its strong potential for Galactic archaeology studies, providing good age precision and accuracy that reproduces well the age of high [α/Fe] stars and relationships between mass and kinematics from previous studies based on e.g. Kepler. Better quality astrometry and simpler target selection than the Kepler sample makes this data ideal for studies of the local star formation history and evolution of the Galactic disc. These results provide a strong case for detailed spectroscopic follow-up in the CVZs to complement that which has been (or will be) collected by current surveys.

Mackereth, J. Ted↗
DOE OSTI2022

Information-Theoretic Exploration of Multivariate Time-Varying Image Databases

Modern scientific simulations produce very large datasets, making interactive exploration of such data computationally prohibitive. An increasingly common data reduction technique is to store visualizations and other data extracts in a database. The Cinema project is one such approach, storing visualizations in an image database for post hoc exploration and interactive image-based analysis. This work focuses on developing efficient algorithms that can quantify various types of multivariate dependencies existing within multi-variable datasets. It applies specific mutual information measures for the quantification of salient regions from multivariate image data. Here, using such information measures, the opacity of the images is modulated so that the salient regions are automatically highlighted and the domain scientists can interactively explore the most relevant regions for scientific discovery.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2003

Computer Simulation of a Digital Global Velocimeter

The objective is to continue the development of a Monte-Carlo simulation of a Doppler Global Velocimetry (DGV) optical system starting with the Mie scattering from the particles in the fluid to the collection optics, through the beamsplitter, Iodine absorption cell and finally to the CCD sensor. The simulation program will read in a configuration file that completely describes the attributes of a DGV system (optical, physical layout, location of detectors, laser properties, CCD sensor noise characteristics...). The simulation program will output a set of images that simulate the data collected from a DGV system. NASA GRC has the DGV software developed at NASA Langley and this simulation program will prove invaluable for determining the accuracy of the DGV technique and provide insight into optimizing the data reduction techniques. The DGV simulation software will also prove valuable in a DDF funded projected entitled Planar Particle Imaging Doppler Velocimetry (PPIDV), which was recently awarded in March 2002.

Cheung, H. Mike↗
DOE OSTI2023

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗