Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data discovery”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Increasing Discovery and Usability of Earth Science Satellite Data with My NASA Data

For 20 years, the My NASA Data project at NASA Langley Research Center has developed innovative approaches to increase the use of NASA’s satellite data by learners. My NASA Data offers a variety of authentic Earth Science datasets and a data visualization tool, eliminating the need for educators and/or learners to obtain specialized knowledge of GIS data formats and software to access and use authentic Earth Science data. While there is no shortage of available data, as federal government agencies such as NASA house petabytes of freely accessible Earth Science datasets, much of the data are only available for download and visualization in specialized formats and software, limiting their accessibility to educators and learners, especially those in primary and secondary school. Using the Google Earth Engine platform, the My NASA Data team has recently reinvented their data visualization tool, called the Earth System Data Explorer (ESDE). The ESDE gives users the capability to explore over 60 Earth Science satellite datasets in a multitude of formats such as maps, graphs, and data table Its new and improved user interface design was developed based on the preferences of educators, whom the My NASA Data project has over 20 years’ experience working with. Earth Science and GIS Subject Matter Experts (SMEs) structured the data in a professional and scientific manner. During Fiscal Year 2023, the My NASA Data website received over 1 million digital engagements, with over one-third being visitors to the data visualization tool. These metrics highlight the interest in a visualization tool that is simple and free to use with reliable and trusted datasets. The ESDE empowers users to readily relate and analyze NASA Earth Science data within their area of interest. The team used a user-centered design (UCD) framework to receive and incorporate feedback into the application’s design. Core requested features include the ability to create time series graphs, comparative analysis of maps, and download the data as CSV file. Responses indicate that advances in data visualization tools such as the ESDE make authentic Earth Science data more accessible. This presentation will cover how the My NASA Data project develops tools to enhance data discovery and accessibility, as well as how SME and user suggestions are incorporated.

Desiray Wilson↗

Easing the Discovery of NASA and International Near-Real-Time Data Using the Global Change Master Directory

The Global Change Master Directory (GCMD) provides an extensive directory of descriptive and spatial information about data sets and data-related services, which are relevant to Earth science research. The directory's data discovery components include controlled keywords, free-text searches, and map/date searches. The GCMD portal for NASA's Land Atmosphere Near-real-time Capability for EOS (LANCE) data products leverages these discovery features by providing users a direct route to NASA's Near-Real-Time (NRT) collections. This portal offers direct access to collection entries by instrument name, informing users of the availability of data. After a relevant collection entry is found through the GCMD's search components, the "Get Data" URL within the entry directs the user to the desired data. http://gcmd.nasa.gov/r/p/gcmd_lance_nrt.

Olsen, Lola↗

DATA-DRIVEN DISCOVERY OF DYNAMICS FROM TIME-RESOLVED COHERENT SCATTERING

This software implements a data-driven framework to uncover mechanistic models of dynamics directly from time-resolved coherent X-ray scattering measurements. It employs neural differential equations to parameterize unknown real-space dynamics and incorporates a computational scattering forward model to relate real-space predictions to reciprocal-space observations.

Zhou, Tao↗

Toward a Data Scalable Solution for Facilitating Discovery of Scientific Data Resources

Science is increasingly motivated by the need to process larger quantities of data. It is facing severe challenges in data collection, management, and processing, so much so that the computational demands of "data scaling" are competing with, and in many fields surpassing, the traditional objective of decreasing processing time. Example domains with large datasets include astronomy, biology, genomic, climate and weather, and material sciences. This paper presents a real-world use case in which we wish to answer queries provided by domain scientists in order to facilitate discovery of relevant science resources. The problem is that the metadata for these science resources is very large and is growing quickly, rapidly increasing the need for a data scaling solution. We propose the use of our SGEM stack -- a system designed for answering graph-based queries over large datasets on cluster architectures -- for answering complex queries over the metadata, and we report early results for our current capability.

Chappell, Alan R.↗

A Relevancy Algorithm for Curating Earth Science Data Around Phenomenon

Earth science data are being collected for various science needs and applications, processed using different algorithms at multiple resolutions and coverages, and then archived at different archiving centers for distribution and stewardship causing difficulty in data discovery. Curation, which typically occurs in museums, art galleries, and libraries, is traditionally defined as the process of collecting and organizing information around a common subject matter or a topic of interest. Curating data sets around topics or areas of interest addresses some of the data discovery needs in the field of Earth science, especially for unanticipated users of data. This paper describes a methodology to automate search and selection of data around specific phenomena. Different components of the methodology including the assumptions, the process, and the relevancy ranking algorithm are described. The paper makes two unique contributions to improving data search and discovery capabilities. First, the paper describes a novel methodology developed for automatically curating data around a topic using Earthscience metadata records. Second, the methodology has been implemented as a standalone web service that is utilized to augment search and usability of data in a variety of tools.

earth science phenomena↗

Faults Discovery By Using Mined Data

Fault discovery in the complex systems consist of model based reasoning, fault tree analysis, rule based inference methods, and other approaches. Model based reasoning builds models for the systems either by mathematic formulations or by experiment model. Fault Tree Analysis shows the possible causes of a system malfunction by enumerating the suspect components and their respective failure modes that may have induced the problem. The rule based inference build the model based on the expert knowledge. Those models and methods have one thing in common; they have presumed some prior-conditions. Complex systems often use fault trees to analyze the faults. Fault diagnosis, when error occurs, is performed by engineers and analysts performing extensive examination of all data gathered during the mission. International Space Station (ISS) control center operates on the data feedback from the system and decisions are made based on threshold values by using fault trees. Since those decision-making tasks are safety critical and must be done promptly, the engineers who manually analyze the data are facing time challenge. To automate this process, this paper present an approach that uses decision trees to discover fault from data in real-time and capture the contents of fault trees as the initial state of the trees.

Lee, Charles↗

Constellation: A science graph network for scalable data and knowledge discovery in extreme-scale scientific collaborations

Constellation's overarching goal is the federation of information from resources within an extreme-scale scientific collaboration to enable the scalable discovery of data and new knowledge pathways. The resource fabric is comprised of petascale supercomputers and storage systems, users, jobs, datasets and lifecycle artifacts. For an extreme-scale supercomputing center, normal operations can generate hundreds of millions of data products and metadata entries describing the resource fabric. Constellation federates the information extracted from the resources using a custom, transformative science graph network; constructs rich metadata indexes and higher-order derived metadata from the extracted information; and conducts scalable graph analytics to unravel hidden data pathways. Our implementation and deployment for a production, supercomputing facility shows that the graph can scale to more than 750 million vertices, its domain agnostic indexing can answer interesting science queries, and its analytics can aid in structural, topological and temporal analysis to identify usage hotspots.

Vazhkudai, Sudharshan S.↗

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗

Big Data Synchrophasor Monitoring and Analytics for Resiliency Tracking (BDSMART)

This report contains key findings from a project titled Big Data Synchrophasor Monitoring and Analytics for Resiliency Tracking (BDSMART), which was carried out through a collaborative effort of a team of researchers from Texas A&M Engineering Experiment Station, Temple University, and Quanta Technology, LLC. The in-kind support came from OSIsoft (acquired by AVEVA), which provided their PI Historian software to demonstrate the use case of streaming PMU data. The first section of the report describes the project goals and objectives related to the development of Machine Learning (ML) models capable of detecting and classifying events by processing phasor measurements captured in the field by Phasor Measurement Units (PMUs). The data for this study was contributed by the utilities/ISOs from the Western and Eastern interconnects and ERCOT, further referred to as Interconnect B (IC B), Interconnect A (IC A), and Interconnect C (IC C), respectively. The approach that the BDSMART Research Team proposed and the key research tasks defined by the team are outlined in this section. The next section describes the technical approach. We first discuss the data constraints related to the PMU measurements and data interpretation constraints imposed by the data contributors. They provided neither the topological information of the grid nor PMU placement locations and captured recorded data at very few locations in the system with the reporting rate of either 30 or 60 fps. The recordings are mostly positive sequence voltage, frequency, and ROCOF, and in some limited cases, three-phase voltages and currents. We then reflect on the bad data issues that stem from poor recording practices and vague definitions of the PMU status bits to supposedly be used for bad data identification. Finally, the data discovery points to imprecise time stamps with incomplete event start/end time, as well as inconsistent and incomplete event labeling, which combined make the implementation of the data models using supervising learning quite challenging. Following the data discovery study, we hypothesize that because the IC B data has the most complete labels, we should focus our model development on that data and then test it on data from other interconnects. We also define the common metrics used to evaluate the results from the ML algorithm tests. We concluded this section by summarizing the common ML models we used and explaining how we implemented and tested them. The issues from this section are expanded in the Training Dataset Report from this project. The final section of this report deals with the accomplishments and conclusions. As the accomplishments, we formulate the problem we are solving and what is achieved by solving the problem. We then reflect on each of the analytics tools we developed and point out the performance of each tool when applied to solving the mentioned problems. We reference this work for further details to the papers we published on each tool. In the conclusions, we give recommendations on how to improve future PMU recording practices to facilitate the ML algorithm implementation and guidance for the future standardization work aimed at clarifying the ambiguities associated with the PMU status bits. We finally list future tasks that can bring about further improvements in the proposed algorithms. The issues from this section are expanded in the Training, and Test Dataset Report filed at the project completion date.

97 MATHEMATICS AND COMPUTING↗

Data driven discovery and quantification of hyperspectral leaf reflectance phenotypes across a maize diversity panel

Abstract Estimates of plant traits derived from hyperspectral reflectance data have the potential to efficiently substitute for traits, which are time or labor intensive to manually score. Typical workflows for estimating plant traits from hyperspectral reflectance data employ supervised classification models that can require substantial ground truth datasets for training. We explore the potential of an unsupervised approach, autoencoders, to extract meaningful traits from plant hyperspectral reflectance data using measurements of the reflectance of 2151 individual wavelengths of light from the leaves of maize ( Zea mays ) plants harvested from 1658 field plots in a replicated field trial. A subset of autoencoder‐derived variables exhibited significant repeatability, indicating that a substantial proportion of the total variance in these variables was explained by difference between maize genotypes, while other autoencoder variables appear to capture variation resulting from changes in leaf reflectance between different batches of data collection. Several of the repeatable latent variables were significantly correlated with other traits scored from the same maize field experiment, including one autoencoder‐derived latent variable (LV8) that predicted plant chlorophyll content modestly better than a supervised model trained on the same data. In at least one case, genome‐wide association study hits for variation in autoencoder‐derived variables were proximal to genes with known or plausible links to leaf phenotypes expected to alter hyperspectral reflectance. In aggregate, these results suggest that an unsupervised, autoencoder‐based approach can identify meaningful and genetically controlled variation in high‐dimensional, high‐throughput phenotyping data and link identified variables back to known plant traits of interest.

Tross, Michael C.↗

Coupling Visualization and Data Analysis for Knowledge Discovery from Multi-dimensional Scientific Data

Knowledge discovery from large and complex scientific data is a challenging task. With the ability to measure and simulate more processes at increasingly finer spatial and temporal scales, the growing number of data dimensions and data objects presents tremendous challenges for effective data analysis and data exploration methods and tools. The combination and close integration of methods from scientific visualization, information visualization, automated data analysis, and other enabling technologies"such as efficient data management" supports knowledge discovery from multi-dimensional scientific data. This paper surveys two distinct applications in developmental biology and accelerator physics, illustrating the effectiveness of the described approach.

97 MATHEMATICS AND COMPUTING↗

From Data to Discovery: AI's Transformative Role in Thin Film Research

The advancement of thin film technologies is pivotal for progress in numerous fields, including energy, electronics, and quantum computing. However, the traditional trial-and-error approach to materials discovery is inherently slow and inefficient. This presentation will showcase how artificial intelligence (AI) is transforming thin film research by enabling a data-driven paradigm shift. We will highlight our past successes in applying AI to understand radiation damage in thin film oxides, demonstrating how graph analytics can unravel complex material behavior. Additionally, we will provide insights into our current work at the National Renewable Energy Laboratory, where we are leading the charge in autonomous materials science. Backed by a $14M investment in our characterization facility, we are developing AI-guided workflows that seamlessly integrate experimentation and AI-guided decision-making. By harnessing the power of AI, we aim to accelerate the discovery and design of high-performance thin films, propelling innovation across a multitude of industries.

36 MATERIALS SCIENCE↗

Data-driven discovery of a formation prediction rule on high-entropy ceramics

The interest in high entropy ceramics (HECs) has increased steadily due to their superior properties. However, the prediction of their formation still poses challenges for the discovery of new systems. Here, we discover a rational rule for designing single-phase high entropy metal diborides (HEBs) using data-driven approach. The machine learning (ML) model is trained on data collected via high-throughput experiments (HTEs). K nearest neighbor (KNN) model shows an experimental validation accuracy of 93.75%. By implementing interpretable ML method, we demonstrate that a mismatch of the bonds between boron and transition metals (δ B-TM ) dominates the formation of HEBs. We propose an empirical rule that HEBs favor forming a single phase when δ B-TM < 3.66; otherwise, multiphase. The rule has a high accuracy of 93.33% for new HEBs predictions. In addition, we contribute 165 high quality HEBs data in total, which can promote the development of materials informatics in HEBs. Furthermore, this data-driven strategy can be expanded to accelerate the search for new HECs, paving a pathway to design novel HECs with superior properties rapidly.

36 MATERIALS SCIENCE↗

Hybrid Data‐Driven Discovery of High‐Performance Silver Selenide‐Based Thermoelectric Composites

Optimizing material compositions often enhances thermoelectric performances. However, the large selection of possible base elements and dopants results in a vast composition design space that is too large to systematically search using solely domain knowledge. To address this challenge, a hybrid data-driven strategy that integrates Bayesian optimization (BO) and Gaussian process regression (GPR) is proposed to optimize the composition of five elements (Ag, Se, S, Cu, and Te) in AgSe-based thermoelectric materials. Data is collected from the literature to provide prior knowledge for the initial GPR model, which is updated by actively collected experimental data during the iteration between BO and experiments. Within seven iterations, the optimized AgSe-based materials prepared using a simple high-throughput ink mixing and blade coating method deliver a high power factor of 2100 µW m −1 K −2 , which is a 75% improvement from the baseline composite (nominal composition of Ag 2 Se 1 ). In conclusion, the success of this study provides opportunities to generalize the demonstrated active machine learning technique to accelerate the development and optimization of a wide range of material systems with reduced experimental trials.

36 MATERIALS SCIENCE↗

Data-Driven Discovery and Experimental Validation of Solvent Polarity Effects on Conjugated Polymer Solution-to-Film Assembly Pathways

Understanding how solvent properties influence the solution-to-film assembly of conjugated polymers remains a critical challenge due to the complex and intertwined nature of polymer–solvent interactions. In this study, we integrate a data-driven framework with experimental validation to identify key parameters influencing the assembly and performance of poly[2,5-(2-octyldodecyl)-3,6-diketopyrrolopyrrole-alt-5,5-(2,5-di(thien-2-yl)thieno[3,2-b]thiophene)] (DPP-DTT) in organic field-effect transistors (OFETs). A machine learning (ML) approach identified the normalized Reichardt polarity parameter (E T N ) as a significant descriptor correlated with DPP-DTT hole mobility (μ). Systematic DPP-DTT devices fabricated using solvents across a wide E T N range revealed that higher E T N solvents yield enhanced μ. To elucidate the structural origins of high μ, we conducted comprehensive analyses using UV–vis–NIR spectroscopy and grazing incidence wide angle X-ray scattering (GIWAXS) measurements. The results revealed that films processed from high E T N solvents exhibit reduced paracrystallinity. By analyzing the solution-state behavior using optical microscopy and solution WAXS, we revealed polymer solubility differences in the various solvents and associated distinct polymer assembly pathways, elucidating why the high E T N solvent produces long-range ordered films. Notably, the high E T N solvent shows a pronounced preference for liquid-crystal (LC)-mediated assembly, providing a mechanistic explanation for the enhanced structural order. Therefore, these results demonstrate that solvent polarity, as evaluated by E T N , serves as an important parameter that plays a significant role in the DPP-DTT assembly pathway and resultant solid-state morphology. This work provides a strategy for integrating data science with experiments to identify critical parameters associated with complex polymer systems and helps guide rational process design for high-performance organic electronics.

36 MATERIALS SCIENCE↗

Machine learning for data-driven discovery in solid Earth geoscience

Understanding the behavior of Earth through the diverse fields of the solid Earth geosciences is an increasingly important task. It is made challenging by the complex, interacting, and multiscale processes needed to understand Earth’s behavior and by the inaccessibility of nearly all of Earth’s subsurface to direct observation. Substantial increases in data availability and in the increasingly realistic character of computer simulations hold promise for accelerating progress, but developing a deeper understanding based on these capabilities is itself challenging. Machine learning will play a key role in this effort. We review the state of the field and make recommendations for how progress might be broadened and accelerated.

Bergen, Karianne J.↗