Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Exploratory analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra

Abstract Transit spectroscopy is a powerful tool for decoding the chemical compositions of the atmospheres of extrasolar planets. In this paper, we focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. After cleaning and validating the data, we demonstrate methods for: (i) initial exploratory data analysis, based on summary statistics (estimates of location and variability); (ii) exploring and quantifying the existing correlations in the data; (iii) preprocessing and linearly transforming the data to its principal components; (iv) dimensionality reduction and manifold learning; (v) clustering and anomaly detection; and (vi) visualization and interpretation of the data. To illustrate the proposed unsupervised methodology, we use a well-known public benchmark data set of synthetic transit spectra. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We explore a number of different techniques for such dimensionality reduction and identify several suitable options in terms of summary statistics, principal components, etc. We uncover interesting structures in the principal component basis, namely well-defined branches corresponding to different chemical regimes of the underlying atmospheres. We demonstrate that those branches can be successfully recovered with a K-means clustering algorithm in a fully unsupervised fashion. We advocate for lower-dimensional representations of the spectroscopic data in terms of the main principal components, in order to reveal the existing structure in the data and quickly characterize the chemical class of a planet.

Matchev, Konstantin T. (ORCID:0000000341829096)↗
DOE OSTI2025

Using feature importance as an exploratory data analysis tool on Earth system models

Abstract. Machine learning (ML) models are commonly used to generate predictions, but these models can also support the discovery of new science. Generating accurate predictions necessitates that a model captures the structure of the underlying data. If the structure is properly extracted, ML could be a useful exploratory and evidential tool. In this paper, we present a case study that demonstrates the use of ML for exploratory data analysis (EDA) in the climate space. We apply the ML explainability method of spatiotemporal zeroed feature importance (stZFI) to understand how climate-variable associations evolve over space and time. Our analyses focus on data from ensembles of Earth system models (ESMs) which provide data on different climate states and conditions. We elect to work with ESM ensembles since they allow us to compare feature importance across alternative scenarios not available with observed data. The ensembles also account for natural variability so that we can distinguish between signal and noise due to natural climate variability when computing feature importance. The use of perturbed initial condition ensembles introduces variability mimicking the natural variability in the atmosphere; thus the signals emerging using feature importance (FI) can be evaluated against the natural variability in the climate system. For our analyses, we consider the 1991 volcanic eruption of Mount Pinatubo, which was a large stratospheric aerosol injection. We explore the climate pathway associated with the eruption from aerosols to radiation to temperature at both the near-surface and stratospheric levels. In addition to applying the method to data generated from two different ESMs, we apply stZFI to reanalysis data to compare the associations identified by stZFI. We show how stZFI tracks the importance of aerosol optical depth over time on forecasting temperatures. This case study illustrates usefulness of an ML tool (stZFI) for EDA on a well-studied climate exemplar.

Ries, Daniel (ORCID:0000000250294647)↗
DOE OSTI2019

Exploratory Visual Analysis of Anomalous Runtime Behavior in Streaming High Performance Computing Applications

Online analysis of runtime behavior is essential for performance tuning in streaming scientific workflows. Integration of anomaly detection and visualization is necessary to support human-centered analysis, such as verification of candidate anomalies utilizing domain knowledge. In this work, we propose an efficient and scalable visual analytics system for online performance analysis of scientific workflows toward the exascale scenario. Here, our approach uses a call stack tree representation to encode the structural and temporal information of the function executions. Based on the call stack tree features (e.g., execution time of the root function or vector representation of the tree structure), we employ online anomaly detection approaches to identify candidate anomalous function executions. We also present a set of visualization tools for verification and exploration in a level-of-detailed manner. General information, such as distribution of execution times, are provided in an overview visualization. The detailed structure (e.g., function invocation relations) and the temporal information (e.g., message communication) of the execution call stack of interest are also visualized. The usability and efficiency of our methods are verified in a real-world HPC application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

ftmsRanalysis: An R package for exploratory data analysis and interactive visualization of FT-MS data

The high-resolution and mass accuracy of Fourier transform mass spectrometry (FT-MS) has made it an increasingly popular technique for discerning the composition of soil, plant and aquatic samples containing complex mixtures of proteins, carbohydrates, lipids, lignins, hydrocarbons, phytochemicals and other compounds. Thus, there is a growing demand for informatics tools to analyze FT-MS data that will aid investigators seeking to understand the availability of carbon compounds to biotic and abiotic oxidation and to compare fundamental chemical properties of complex samples across groups. We present ftmsRanalysis, an R package which provides an extensive collection of data formatting and processing, filtering, visualization, and sample and group comparison functionalities. The package provides a suite of plotting methods and enables expedient, flexible and interactive visualization of complex datasets through functions which link to a powerful and interactive visualization user interface, Trelliscope. Example analysis using FT-MS data from a soil microbiology study demonstrates the core functionality of the package and highlights the capabilities for producing interactive visualizations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Exploratory Data analysis ENvironment eXtreme scale (EDENx)

EDENx is a multivariate data visualization tool that allows interactive user driven analysis of large-scale data sets with high dimensionality. EDENx builds on our earlier system, called EDEN to enable analysis of more dimensions and larger scale data sets. EDENx provides an initial overview of summary statistics for each variable in the data set under investigation. EDENx allows the user to interact with graphical summary plots of the data to investigate subsets and their statistical associations. These plots include histograms, binned scatterplots, binned parallel coordinate plots, timeline plots, and graphical correlation indicators. From the EDENx interface, a user can select a subsample of interest and launch a more detailed data visualization via the EDEN system. EDENx is best suited for high-level, aggregate analysis tasks while EDEN is more appropriate for detail data investigations.

Steed, Chad Allen↗
DOE OSTI2024

Dynamic modeling studies of basin-scale pressure interference and CO 2 plume evolution in multi-well geologic CO 2 storage

This research employs numerical modeling to analyze how CO 2 plumes and pressure fronts evolve when CO 2 is injected into a single storage formation from multiple projects located in proximity. Here, this analysis also evaluates the extent to which injection well spacing alleviates pressure buildup in the absence of active pressure management tactics. The simulation approach was based on a single, homogenous saline aquifer in which CO 2 injection occurs under a one-injector baseline case and several multi-well cases where well spacing varies. Analysis results show that the extent of pressure buildup is in the range of tens or a few hundreds of kilometers and contingent upon the defining pressure buildup demarcating the front edge. For the geological setting evaluated in this paper, our analysis suggests that without active basin pressure management strategies, commercial-scale projects would likely need to be sited far apart to avoid pressure interference from one another. Analysis results show the radius of CO 2 plume varies approximately from 2 to 3 km from injection wells (each injecting 1 Mt/year for 30 years) depending on cases and modeling parameters assumed. Given the pressure interference, this paper thus draws attention to the importance of greater coordination among storage operators and regulatory stakeholders. Because this analysis assumes a very specific geologic setting, this exploratory analysis bears further investigations across other geologic scenarios.

42 ENGINEERING↗
DOE OSTI2019

Hyperspectral imaging of minerals in the longwave infrared: the use of laboratory directional-hemispherical reference measurements for field exploration data

Hyperspectral Imaging (HSI) continues to grow as a method for remote detection of vegetation, materials, minerals, and pure chemicals. We have used a longwave infrared (7.7 - 11.8 µm) imaging spectrometer in a static outdoor experiment to collect HSI data from 24 minerals and background materials to determine the efficacy with which HSI can remotely detect and distinguish both pure minerals and mineral mixtures at three solar angles (25°, 35°, and 45° relative to ground) as well as varying backgrounds and other sample effects. Measurements were obtained separately for the minerals and materials mounted directly both on a bare plywood board and a plywood board coated with aluminum foil: 19 powders (3 mixtures and 16 pure mineral powders) held in polyethylene bottle lids as well as 5 samples in rock form were taped directly to the boards. The primary goal of the experiment was to demonstrate that a longwave infrared (LWIR) library of solids and minerals collected as directional-hemispherical reflectance spectra in the laboratory can be used directly for HSI field identification. Prior to the experiment, all 24 mineral/inorganic samples were measured in the laboratory using a Fourier transform infrared spectrometer (FTIR) equipped with a gold-coated IR integrating sphere; the spectra were assimilated as part of a larger reference library of 21 pure minerals, 3 mixtures, and the polyethylene lid. Principal components analysis (PCA) with mean-centering was used in an exploratory analysis of the HSI images and showed that for the aluminum coated board, the first principal component captured the difference between signal that resembled a blackbody and the highly reflective aluminum background. In contrast, principal components PC 2, 3, and 4 were able to discriminate the materials including phosphates, sulfates, silicates, carbonates, and the mixtures. Results from generalized least squares target detection clearly showed that laboratory reference spectra of minerals could be utilized as targets with high fidelity for field detection, both for pure and mixed samples.

47 OTHER INSTRUMENTATION↗
DOE OSTI2015

Cinema image-based in situ analysis and visualization of MPAS-ocean simulations

We created an in situ exploration visualization of an MPAS-Ocean simulation.We leveraged compositing in Cinema to provide interactive exploration.We decreased the storage footprint of the analysis and visualization results. Due to power and I/O constraints associated with extreme scale scientific simulations, in situ analysis and visualization will become a critical component to scientific exploration and discovery. Current analysis and visualization options at extreme scale are presented in opposition: write files to disk for interactive, exploratory analysis, or perform in situ analysis to save data products about phenomena that a scientists knows about in advance. In this study, we demonstrate extreme scale visualization of MPAS-Ocean simulations leveraging a third option based on Cinema, which is a novel framework for highly interactive, image-based in situ analysis and visualization that promotes exploration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Slycat Enables Synchronized 3D Comparison of Surface Mesh Ensembles [Brief]

In support of analyst requests for Mobile Guardian Transport studies, researchers at Sandia National Laboratories have expanded data types for the Slycat ensemble-analysis and visualization tool to include 3D surface meshes. This new capability represents a significant advance in our ability to perform detailed comparative analysis of simulation results. Analyzing mesh data rather than images provides greater flexibility for post-processing exploratory analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2020

COVID-19 Data Curation Effort: An Initial Analysis of the Data

During the COVID-19 pandemic of 2020, major case reporting outlets quickly coalesced around two or three primary vendors. Johns Hopkins University and The New York Times were among the more prominent, and all were of great value to the nation, particularly during the uncertain early stages of the pandemic. They primarily focused on three major attributes: number of new cases, deaths, and recovery. Recognizing that many states were reporting very detailed data sets (e.g., hospital beds) at a county level or finer, the ORNL Pandemic Modeling team embarked on a major data curation effort from March to June 2020 for the purpose of capturing this wealth of detailed data. The challenge of curating this data was daunting. The number of attributes reported by the states grew on almost on a weekly basis. States were routinely shifting their web tool strategies away from easily parsable HTML-based formatting to new Tableau and ArcGIS content. This growth in the sheer number of attributes, combined with the unpredictable shifts in data format, meant an aggressive and agile combination of automated scripting and manual scraping was required to capture new daily streams. Further, the team had to scale up staff and widen its approach for capture and storage. As a result, the team collected more than 11 million data points. Following the close of this data collection effort on June 30 th , 2020, the team embarked on a major effort to appraise what had been collected, including an inventory list, spatial completeness, temporal completeness, scale and geographic characteristics, and a determination. A report on this matter was submitted on September 15 th , 2020, titled “DOE COVID-19 Data Curation Effort: Overview of Data Collection Coverage”. Over 2000 unique attributes had been netted over a wide range of spatial scales, including state, county, zip codes, health regions, and census blocks. Over 11 million individual data points were collected across these attributes, and spatial coverage (in total) included all 50 states and multiple territories. What became apparent in the process is that in the absence of any data standards, many states reported a wide variety of unique attributes that were not always compatible with attributes reported in other states. As time continued, states began adding new attributes and offering finer grain detail in some older attributes. This meant that not all data streams existed for the entire time period; in fact, the number tended to increase dramatically towards the end. Often, states would begin an attribute series and then stop altogether. These highly variable and uncertain conditions illuminated the need for harmonization approaches that would reconcile and conflate changing attribute names and detail over time. For example, grouping racial data reported as either Black or African American, depending on the state, into a single harmonized attribute. These choices would make a within-state analysis possible during the time period and lead to potential between-state analytics later on. This was almost entirely a manual decision process, requiring some subjective decision-making at times, to prevent a fragmented, short-lived collection of time series fragments that would offer few insights into trends, patterns, and correlates. This report imports harmonized data for state and county into the World Spatio-Temporal Analytics and Mapping Project (WSTAMP). WSTAMP is a major space-time analysis and visualization tool developed at ORNL for the National Geospatial-Intelligence Agency specifically for this kind of exploratory analysis. WSTAMP offers a rich analytical and graphical environment consisting of a wide range of analytics. These include time series plots, statistical summaries, data mining techniques, trend and pattern detection, and hypothesis generation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2013

Exploration of high-dimensional scalar function for nuclear reactor safety analysis and visualization

The next generation of methodologies for nuclear reactor Probabilistic Risk Assessment (PRA) explicitly accounts for the time element in modeling the probabilistic system evolution and uses numerical simulation tools to account for possible dependencies between failure events. The Monte-Carlo (MC) and the Dynamic Event Tree (DET) approaches belong to this new class of dynamic PRA methodologies. A challenge of dynamic PRA algorithms is the large amount of data they produce which may be difficult to visualize and analyze in order to extract useful information. We present a software tool that is designed to address these goals. We model a large-scale nuclear simulation dataset as a high-dimensional scalar function defined over a discrete sample of the domain. First, we provide structural analysis of such a function at multiple scales and provide insight into the relationship between the input parameters and the output. Second, we enable exploratory analysis for users, where we help the users to differentiate features from noise through multi-scale analysis on an interactive platform, based on domain knowledge and data characterization. Our analysis is performed by exploiting the topological and geometric properties of the domain, building statistical models based on its topological segmentations and providing interactive visual interfaces to facilitate such explorations. We provide a user's guide to our software tool by highlighting its analysis and visualization capabilities, along with a use case involving data from a nuclear reactor safety simulation. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Relationships between Habitual Polyphenol Consumption and Gut Microbiota in the INCLD Health Cohort

While polyphenol consumption is often associated with an increased abundance of beneficial microbes and decreased opportunistic pathogens, these relationships are not completely described for polyphenols consumed via habitual diet, including culinary herb and spice consumption. This analysis of the International Cohort on Lifestyle Determinants of Health (INCLD Health) cohort uses a dietary questionnaire and 16s microbiome data to examine relationships between habitual polyphenol consumption and gut microbiota in healthy adults (n = 96). In this exploratory analysis, microbial taxa, but not diversity measures, differed by levels of dietary polyphenol consumption. Taxa identified as exploratory biomarkers of daily polyphenol consumption (mg/day) included Lactobacillus, Bacteroides, Enterococcus, Eubacterium ventriosum group, Ruminococcus torques group, and Sutterella. Taxa identified as exploratory biomarkers of the frequency of polyphenol-weighted herb and spice use included Lachnospiraceae UCG-001, Lachnospiraceae UCG-004, Methanobrevibacter, Lachnoclostridium, and Lachnotalea. Several of the differentiating taxa carry out activities important for human health, although out of these taxa, those with previously described pro-inflammatory qualities in certain contexts displayed inverse relationships with polyphenol consumption. Our results suggest that higher quantities of habitual polyphenol consumption may support an intestinal environment where opportunistic and pro-inflammatory bacteria are represented in a lower relative abundance compared to those with less potentially virulent qualities.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Investigation of similarity metrics for simulation based scaling analysis

Scaling analysis is a technique for characterizing degree of similarity between processes to occur in a real prototypic system and those to occur in its reduced scale (experimental) analog. In the context of nuclear reactor thermal-hydraulics and safety, scaling analysis is instrumental in assessing the applicability of data obtained from a scaled test facility for calibration of models and validation of simulation codes. Quantification of scale-related distortions is the key element in scaling analysis. In this work, we examine an approach for scaling analysis using computer simulation codes. Similarity analysis using information-theoretic approach is performed by comparing performance of experiment and application to variation of system conditions. Three different information theoretic measures namely mutual information, Kullback-Leibler divergence and overlap coefficient are selected for their potential use as metric for similarity analysis. We test the metrics by comparing them against physics-based scaling ratios, which are well known in fundamental thermal-hydraulic processes. In this exploratory analysis, a case of pressure drop of a steady-state flow through a channel is employed to test the proposed approach. It was found that in the absence of model uncertainties, mutual information serve as an effective measure of the scale effects present in the experiment. However, model uncertainty can significantly impact the quality of similarity analysis. Our analysis indicates that model uncertainties can have significant impact on the information theoretic metrics used for scaling analysis. Furthermore, if the model uncertainties exceeds beyond a critical value, the scale effect present in the experiment cannot be adequately evaluated. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

A review on recent machine learning applications for imaging mass spectrometry studies

Imaging mass spectrometry (IMS) is a powerful analytical technique widely used in biology, chemistry, and materials science fields that continue to expand. IMS provides a qualitative compositional analysis and spatial mapping with high chemical specificity. The spatial mapping information can be 2D or 3D depending on the analysis technique employed. Due to the combination of complex mass spectra coupled with spatial information, large high-dimensional datasets (hyperspectral) are often produced. Therefore, the use of automated computational methods for an exploratory analysis is highly beneficial. The fast-paced development of artificial intelligence (AI) and machine learning (ML) tools has received significant attention in recent years. These tools, in principle, can enable the unification of data collection and analysis into a single pipeline to make sampling and analysis decisions on the go. There are various ML approaches that have been applied to IMS data over the last decade. Here, in this review, we discuss recent examples of the common unsupervised (principal component analysis, non-negative matrix factorization, k-means clustering, uniform manifold approximation and projection), supervised (random forest, logistic regression, XGboost, support vector machine), and other methods applied to various IMS datasets in the past five years. The information from this review will be useful for specialists from both IMS and ML fields since it summarizes current and representative studies of computational ML-based exploratory methods for IMS.

47 OTHER INSTRUMENTATION↗
DOE OSTI2019

Survival Outcomes of Two Phase 2 Studies of Adjuvant Chemotherapy with S-1 Plus Oxaliplatin or Capecitabine Plus Oxaliplatin for Patients with Gastric Cancer After D2 Gastrectomy

Background: Two phase 2 trials of oxaliplatin-containing adjuvant therapy for patients with gastric cancer (GC) after D2 gastrectomy were conducted in Japan. The SOXaGC trial evaluated the tolerability and safety of adjuvant therapy with S-1 plus oxaliplatin (SOX), whereas the J-CLASSIC trial evaluated the feasibility of adjuvant therapy with capecitabine plus oxaliplatin (CAPOX). Because both were studies that did not evaluate survival results as study end points, the authors evaluated the survival outcomes for the patients in the two trials. Methods: All 62 and 100 patients in the full analysis set of the SOXaGC and J-CLASSIC trials, respectively, were included in the current study. Their information about survival outcome was collected. The primary end point was relapse-free survival (RFS), and the secondary end point was overall survival (OS). Results: For the pathologic stage (pStage 2) patients treated with CAPOX, the 3-year RFS rate was 87.8% and the 3-year OS rate was 92.7%. For the pStage 3 patients treated with SOX and CAPOX, the 3-year RFS rates were respectively 70.9% and 67.8% (hazard ratio [HR], 0.93; 95% confidence interval [CI], 0.50–1.72), whereas the 3-year OS rates were respectively 75.7% and 79.3% (HR, 1.10; 95% CI, 0.54–2.26). Subgroup analysis showed significant interactions between the treatment (SOX vs. CAPOX) and both sex (male vs. female; P = 0.024) and histologic type (diffuse vs. other, P = 0.069). Conclusions: This exploratory analysis demonstrated that SOX and CAPOX are suggested to have similar efficacy for pStage 3 GC patients after D2 gastrectomy. Differences in the treatment effect according to sex and histologic type warrant further evaluation.

62 RADIOLOGY AND NUCLEAR MEDICINE↗