Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “exploratory data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Physics-Informed Machine Learning for Epidemiological Models

One challenge of using compartmental SEIR models for public health planning is the difficulty in manually tuning parameters to capture behavior reflected in the real-world data. This team conducted initial, exploratory analysis of a novel technique to use physics-informed machine learning tools to rapidly develop data-driven models for physical systems. This machine learning approach may be used to perform data assimilation of compartment models which account for unknown interactions between geospatial domains (i.e. diffusion processes coupling across neighborhoods/counties/states/etc.). Results presented here are early, proof-of-concept ideas that demonstrate initial success in using a physically informed neural network (PINN) model to assimilate data in a compartmental epidemiology model. The results demonstrate initial success and warrant further research and development.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Gaussian process analysis of electron energy loss spectroscopy data: multivariate reconstruction and kernel control

Abstract Advances in hyperspectral imaging including electron energy loss spectroscopy bring forth the challenges of exploratory and physics-based analysis of multidimensional data sets. The multivariate linear unmixing methods generally explore similarities in the energy dimension, but ignore correlations in the spatial domain. At the same time, Gaussian process (GP) explicitly incorporate spatial correlations in the form of kernel functions but is computationally intensive. Here, we implement a GP method operating on the full spatial domain and reduced representations in the energy domain. In this multivariate GP, the information between the components is shared via a common spatial kernel structure, while allowing for variability in the relative noise magnitude or image morphology. We explore the role of kernel constraints on the quality of the reconstruction, and suggest an approach for estimating them from the experimental data. We further show that spatial information contained in higher-order components can be reconstructed and spatially localized.

36 MATERIALS SCIENCE↗
NASA NTRS2015

Enhancements of Bayesian Blocks; Application to Large Light Curve Databases

Bayesian Blocks are optimal piecewise linear representations (step function fits) of light-curves. The simple algorithm implementing this idea, using dynamic programming, has been extended to include more data modes and fitness metrics, multivariate analysis, and data on the circle (Studies in Astronomical Time Series Analysis. VI. Bayesian Block Representations, Scargle, Norris, Jackson and Chiang 2013, ApJ, 764, 167), as well as new results on background subtraction and refinement of the procedure for precise timing of transient events in sparse data. Example demonstrations will include exploratory analysis of the Kepler light curve archive in a search for "star-tickling" signals from extraterrestrial civilizations. (The Cepheid Galactic Internet, Learned, Kudritzki, Pakvasa1, and Zee, 2008, arXiv: 0809.0339; Walkowicz et al., in progress).

Kepler light curve archive↗
DOE OSTI2024

Measurement of the Neutron Electromagnetic Form Factor Ratio at High Momentum Transfer

The inner structure of the nucleon (proton and neutron) remains a topic of great interest in nuclear and particle physics, after many decades of study. For example, understanding the quark-gluon dynamics inside the nucleon would shed light on how 99% of the nucleon mass is created. The neutron electromagnetic form factors, Gn E and Gn M , give important insights into the neutron structure. The Super BigBite Spectrometer (SBS) program at Jefferson Lab (JLab) seeks to extend the form factor measurements for both the proton and the neutron. The neutron electric form actor, Gn E , has been historically difficult to measure due to the short lifetime of the free neutron and the small value of Gn E . The GEn-II experiment is part of the SBS program and seeks to measure Gn E , significantly increasing the high momentum transfer coverage. A newly designed polarized 3He target increased the figure of merit by three times compared to previous measurements. The analysis of this data is especially challenging due to the unprecedented high-rate environment caused by the open nature of the spectrometer with a direct line of sight to the target. This required developing new Gas Electron Multiplier (GEM) particle trackers which can cover large areas demanded by this setup and handle particle rates up to 500 kHz/cm2. Rates this high over a large area is unprecedented in particle tracking systems and came with a number of challenges. Data taken in the SBS program was critical to understanding hardware and software solutions that improved the track reconstruction efficiency to be >97% with a position resolution of 70 ?m. In previous experiments the proton electromagnetic form factors, Gp E and Gp M were measured up to Q2 = 8.5 GeV2 and Q2 = 30 GeV2, respectively, while Gn E has only been measured up to Q2 = 3.4 GeV2. The GEn-II experiment has measured the neutron form factor ratio, Gn E/Gn M, at Q2 values of 2.90, 6.50, and 9.47 GeV2 by scattering a polarized electron beam with a polarized 3He target, used here as an effective polarized neutron target, and measuring the double spin asymmetry of the cross section. Previous Gn E measurements do not extend above Q2 = 3.4 GeV2, and therefore this analysis has extended the world data by almost three times. The background correction is especially difficult at the higher Q2 settings leading to large systematic errors. As very exploratory results from this early analysis of the data, we find for Q2 = 2.90 GeV2, Gn E = 0.0157 ±stat 0.0016 ±sys 0.0011, for Q2 = 6.50 GeV2, Gn E = 0.0067 ±stat 0.0019 ±sys 0.0005, and for Q2 = 9.46 GeV2, Gn E = 0.0046 ±stat 0.0023 ±sys 0.0005. These results are compared to predictions from the Dyson-Schwinger Equations (DSE) model and a Relativistic Constituent Quark Model (RCQM).

Jeffas, Sean↗
DOE OSTI2024

Data and scripts associated with a manuscript on a meta-analysis synthesizing stream biogeochemical response to wildfires across space and time (v2)

This data package is associated with the publication “Catchment characteristics modulate the influence of wildfires on nitrate and dissolved organic carbon in lotic systems across space and time: A meta-analysis” submitted to Global Biogeochemical Cycles (Cavaiani et al. 2025). This study uses meta-analytical techniques to evaluate the effect of wildfire on in-stream responses in burned and unburned watersheds. The study aims to provide additional insight into the range of responses and net influences that wildfires have on hydro-biogeochemistry across broad spatial scales, burn extents, and the persistence of water-quality change. This study compiles data and metadata from 18 total publications that includes 1) surface water geochemistry data (dissolved organic carbon; nitrate), 2) climate classifications, 3) year of the wildfire, 4) the time lag between when the fire occurred and when the sampling occurred, and 5) study design of the publication. In total, this meta-analysis draws data that spans 8 climate guilds, 3 biomes, 62 watersheds, and 20 unique wildfires. See Sites_meta_data.csv for citations of the papers used in this meta-analysis. All R scripts and the associated data can also be found on GitHub at This data package was originally published in March 2024. It was updated in April 2025 (v2; new and modified files). See the change history section in the readme for more details. This data package contains five primary folders that include the following: (1) inputs; (2) output for analysis; (3) initial plots; (4) R scripts; and (5) GIS data. The data package also contains a data dictionary (dd) that provides column header definitions and a file-level metadata (flmd) file that describes every file. The “inputs” folder contains a list of all publications identified during the formal web search and an indication of whether each publication was included in the final analysis. Additionally, it includes site-level metadata, catchment characteristics, and GIS data for all publications included in the final analysis. The “Output_for_analysis” folder contains all data frames and figures generated from each R script used for additional data analysis. The “initial_plots” folder includes all exploratory figures that will be included in a supplemental and figures that will be submitted with the manuscript for publication. The “R_scripts” folder contains the scripts that perform all the data manipulations, statistical analyses, and plots. The “gis_data” folder includes shape files for each fire included in this meta-analysis. This data package contains the following file types: csv, pdf, jpeg, cpg, dbf, prj, shp, shp.ea.iso.xml, shp.iso.xml, shx.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

A simple and fast algorithm for estimating the capacity credit of solar and storage

Energy storage is a leading option to enhance the resource adequacy contribution of solar energy. Detailed analysis of the capacity credit of solar energy and energy storage is limited in part due to the data intensive and computationally complex nature of probabilistic resource adequacy assessments. This paper presents a simple algorithm for calculating the capacity credit of energy-limited resources that, due to the low computational and data needs, is well suited to exploratory analysis. Validation against benchmarks based on probabilistic techniques shows that it can yield similar insights. The method is used to evaluate the impact of different solar and storage configurations, particularly with respect to the strategy for coupling storage and solar photovoltaic systems. Furthermore, application of the method to a case study of utilities in Florida, where solar is rapidly growing and demand peaks in the winter and summer, demonstrates that it can improve on rules of thumb used in practice by some utilities. If storage is required to charge only from solar, periods of high demand driven by cold weather events accompanied by lower solar production can result in a capacity credit of solar and storage that is less than the capacity credit of storage alone.

14 SOLAR ENERGY↗
DOE OSTI2020

Performance Prediction of Big Data Transfer Through Experimental Analysis and Machine Learning

Big data transfer in next-generation scientific applications is now commonly carried out over connections with guaranteed bandwidth provisioned in High-performance Networks (HPNs) through advance bandwidth reservation. To use HPN resources efficiently, provisioning agents need to carefully schedule data transfer requests and allocate appropriate bandwidths. Such reserved bandwidths, if not fully utilized by the requesting user, could be simply wasted or cause extra overhead and complexity in management due to exclusive access. This calls for the capability of performance prediction to reserve bandwidth resources that match actual needs. Towards this goal, we employ machine learning algorithms to predict big data transfer performance based on extensive performance measurements, which are collected over a span of several years from a large number of data transfer tests using different protocols and toolkits between various end sites on several real-life physical or emulated HPN testbeds. We first identify a comprehensive list of attributes involved in a typical big data transfer process, including end host system configurations, network connection properties, and control parameters of data transfer methods. We then conduct an in-depth exploratory analysis of their impacts on application-level throughput, which provides insights into big data transfer performance and motivates the use of machine learning. We also investigate the applicability of machine learning algorithms and derive their general performance bounds for performance prediction of big data transfer in HPNs. Experimental results show that, with appropriate data preprocessing, the proposed machine learning-based approach achieves 95% or higher prediction accuracy in up to 90% of the cases with very noisy real-life performance measurements.

Yun, Daqing↗
NASA NTRS1988

Image processing software for imaging spectrometry

The paper presents a software system, Spectral Analysis Manager (SPAM), which has been specifically designed and implemented to provide the exploratory analysis tools necessary for imaging spectrometer data, using only modest computational resources. The basic design objectives are described as well as the major algorithms designed or adapted for high-dimensional images. Included in a discussion of system implementation are interactive data display, statistical analysis, image segmentation and spectral matching, and mixture analysis.

Mazer, Alan S.↗
NASA NTRS1974

An analysis of pilot error-related aircraft accidents

A multidisciplinary team approach to pilot error-related U.S. air carrier jet aircraft accident investigation records successfully reclaimed hidden human error information not shown in statistical studies. New analytic techniques were developed and applied to the data to discover and identify multiple elements of commonality and shared characteristics within this group of accidents. Three techniques of analysis were used: Critical element analysis, which demonstrated the importance of a subjective qualitative approach to raw accident data and surfaced information heretofore unavailable. Cluster analysis, which was an exploratory research tool that will lead to increased understanding and improved organization of facts, the discovery of new meaning in large data sets, and the generation of explanatory hypotheses. Pattern recognition, by which accidents can be categorized by pattern conformity after critical element identification by cluster analysis.

Kowalsky, N. B.↗
DOE OSTI2020

Slycat Enables Synchronized 3D Comparison of Surface Mesh Ensembles [Brief]

In support of analyst requests for Mobile Guardian Transport studies, researchers at Sandia National Laboratories have expanded data types for the Slycat ensemble-analysis and visualization tool to include 3D surface meshes. This new capability represents a significant advance in our ability to perform detailed comparative analysis of simulation results. Analyzing mesh data rather than images provides greater flexibility for post-processing exploratory analysis.

36 MATERIALS SCIENCE↗
NASA NTRS2000

A 3-D Model Analysis of The Impact of Asian Anthropogenic Emissions on the Sulfur Cycle Over the Pacific Ocean

The impact of anthropogenic activities on the SO2 and sulfate aerosol levels over the Pacific region is examined in the Georgia Tech/Goddard Global Ozone Chemistry Aerosol Radiation and Transport (GOCART) model. We focus on the analysis of the data from the NASA Pacific Exploratory Missions (PEM) over the western North Pacific and the tropical Pacific. These missions include PEM-West A in September-October 1991, when the Asian outflow was at the minimum but the upper atmosphere was heavily influenced by the Pinatubo volcanic eruption, and PEM-West B in March-April 1994 when the Asian outflow was at the maximum, and PEM-Tropics A in August-September at a region relatively free of direct anthropogenic influences. Specifically, we will examine the relative importance of anthropogenic, volcanic and biogenic sources to the SO2 and sulfate concentrations over the Pacific, and quantify the processes controlling the distributions of SO2 and sulfate in both the boundary layer and the free troposphere. We will also assess the global impact of SO2 emission in Asia on the sulfate aerosol loading.

Chin, Mian↗
NASA NTRS1993

The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data

The Center for the Study of Earth from Space (CSES) at the University of Colorado, Boulder, has developed a prototype interactive software system called the Spectral Image Processing System (SIPS) using IDL (the Interactive Data Language) on UNIX-based workstations. SIPS is designed to take advantage of the combination of high spectral resolution and spatial data presentation unique to imaging spectrometers. It streamlines analysis of these data by allowing scientists to rapidly interact with entire datasets. SIPS provides visualization tools for rapid exploratory analysis and numerical tools for quantitative modeling. The user interface is X-Windows-based, user friendly, and provides 'point and click' operation. SIPS is being used for multidisciplinary research concentrating on use of physically based analysis methods to enhance scientific results from imaging spectrometer data. The objective of this continuing effort is to develop operational techniques for quantitative analysis of imaging spectrometer data and to make them available to the scientific community prior to the launch of imaging spectrometer satellite systems such as the Earth Observing System (EOS) High Resolution Imaging Spectrometer (HIRIS).

Kruse, F. A.↗
NASA NTRS2001

Rotation of EOFs by the Independent Component Analysis: Towards A Solution of the Mixing Problem in the Decomposition of Geophysical Time Series

The Independent Component Analysis is a recently developed technique for component extraction. This new method requires the statistical independence of the extracted components, a stronger constraint that uses higher-order statistics, instead of the classical decorrelation, a weaker constraint that uses only second-order statistics. This technique has been used recently for the analysis of geophysical time series with the goal of investigating the causes of variability in observed data (i.e. exploratory approach). We demonstrate with a data simulation experiment that, if initialized with a Principal Component Analysis, the Independent Component Analysis performs a rotation of the classical PCA (or EOF) solution. This rotation uses no localization criterion like other Rotation Techniques (RT), only the global generalization of decorrelation by statistical independence is used. This rotation of the PCA solution seems to be able to solve the tendency of PCA to mix several physical phenomena, even when the signal is just their linear sum.

Aires, Filipe↗
NASA NTRS2000

Rotation of EOFs by the Independent Component Analysis: Towards a Solution of the Mixing Problem in the Decomposition of Geophysical Time Series

The Independent Component Analysis is a recently developed technique for component extraction. This new method requires the statistical independence of the extracted components, a stronger constraint that uses higher-order statistics, instead of the classical decorrelation, a weaker constraint that uses only second-order statistics. This technique has been used recently for the analysis of geophysical time series with the goal of investigating the causes of variability in observed data (i.e. exploratory approach). We demonstrate with a data simulation experiment that, if initialized with a Principal Component Analysis, the Independent Component Analysis performs a rotation of the classical PCA (or EOF) solution. This rotation uses no localization criterion like other Rotation Techniques (RT), only the global generalization of decorrelation by statistical independence is used. This rotation of the PCA solution seems to be able to solve the tendency of PCA to mix several physical phenomena, even when the signal is just their linear sum.

Aires, Filipe↗
DOE OSTI2020

Theory and applications of parton pseudodistributions

We review the basic theory of the parton pseudodistributions approach and its applications to lattice extractions of parton distribution functions. The crucial idea of the approach is the realization that the correlator $M(z,p)$ of the parton fields is a function $M(ν,-z^2)$ of Lorentz invariants $ν=-(zp)$, the Ioffe time, and the invariant interval $z^2$. This observation allows to extract the Ioffe-time distribution $M(ν,-z^2)$ from Euclidean separations z accessible on the lattice. Another basic feature is the use of the ratio $M(ν,-z^2)≡M(ν,-z^2)/M(0,-z^2)$, that allows to eliminate artificial ultraviolet divergence generated by the gauge link for spacelike intervals. The remaining $z^2$-dependence of the reduced Ioffe-time distribution $M(ν,-z^2)$ corresponds to perturbative evolution, and can be converted into the scale-dependence of parton distributions $f(x,μ^2)$ using matching relations. The ν-dependence of $M(ν,-z^2)$ governs the x-dependence of parton densities $f(x,μ^2)$. The perturbative evolution was successfully observed in exploratory quenched lattice calculation. The analysis of its precise data provides a framework for extraction of parton densities using the pseudodistributions approach. It was used in the recently performed calculations of the nucleon and pion valence quark distributions. We also discuss matching conditions for the pion distribution amplitude and generalized parton distributions, the lattice studies of which are now in progress.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1988

Autonomous image data reduction by analysis and interpretation

Image data is a critical component of the scientific information acquired by space missions. Compression of image data is required due to the limited bandwidth of the data transmission channel and limited memory space on the acquisition vehicle. This need becomes more pressing when dealing with multispectral data where each pixel may comprise 300 or more bytes. An autonomous, real time, on-board image analysis system for an exploratory vehicle such as a Mars Rover is developed. The completed system will be capable of interpreting image data to produce reduced representations of the image, and of making decisions regarding the importance of data based on current scientific goals. Data from multiple sources, including stereo images, color images, and multispectral data, are fused into single image representations. Analysis techniques emphasize artificial neural networks. Clusters are described by their outlines and class values. These analysis and compression techniques are coupled with decision making capacity for determining importance of each image region. Areas determined to be noise or uninteresting can be discarded in favor of more important areas. Thus limited resources for data storage and transmission are allocated to the most significant images.

Eberlein, Susan↗
NASA NTRS1988

Autonomous image data reduction by analysis and interpretation

Image data is a critical component of the scientific information acquired by space missions. Compression of image data is required due to the limited bandwidth of the data transmission channel and limited memory space on the acquisition vehicle. This need becomes more pressing when dealing with multispectral data where each pixel may comprise 300 or more bytes. An autonomous, real time, on-board image analysis system for an exploratory vehicle such as a Mars Rover is developed. The completed system will be capable of interpreting image data to produce reduced representations of the image, and of making decisions regarding the importance of data based on current scientific goals. Data from multiple sources, including stereo images, color images, and multispectral data, are fused into single image representations. Analysis techniques emphasize artificial neural networks. Clusters are described by their outlines and class values. These analysis and compression techniques are coupled with decision-making capacity for determining importance of each image region. Areas determined to be noise or uninteresting can be discarded in favor of more important areas. Thus limited resources for data storage and transmission are allocated to the most significant images.

Eberlein, Susan↗