Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Report on Mowry Shale Thermal Maturity Mapping in the Powder River Basin, Wyoming

The Enhanced Oil Recovery Institute (EORI) has completed a map of thermal maturity of the Mowry Shale in the Powder River Basin (PRB). Thermal maturity, the degree to which the total organic carbon in a formation has been transformed from kerogen to producible hydrocarbons due to heat and pressure, is an important measure for not just the quality of a source rock, but also helps for delineating areas more favorable for unconventional drilling. There is a dearth of published Mowry maturity maps available for the PRB and this publication endeavors to provide a detailed map based on a large, public data set covering the Wyoming portion of the PRB. Additional data, currently held confidential by operators in the basin, would be of benefit to future iterations of this map.

58 GEOSCIENCES↗
DOE OSTI2023

Machine Learning Prediction of Fracture Toughness in Hydrogen-charged Stainless Steels

Austenitic stainless steels are structural materials utilized in tritium gas pressure boundaries since they are resistant to hydrogen isotope embrittlement [1-3]. However, exposure to tritium over long periods of time leads to tritium uptake which decays to result in helium ingrowth. This helium ingrowth results in further embrittlement effects which are synergistic with that from the hydrogen isotope [4]. Therefore, it is important for tritium facilities to understand the material limitations of stainless steel in this environment. The Savannah River National Laboratory (SRNL) has available a large experimental data set of austenitic stainless steels which have been exposed to tritium environments for various lengths of time. With the availability of this data set, machine learning (ML) algorithms provide an opportunity to model the embrittlement of stainless steel due to the algorithm’s ability to identify patterns in data sets that are difficult and costly to identify in other manners [5]. Ultimately, the amount and quality of the available data is one defining force in the ability of a ML model to accurately predict the desired outputs. The models developed herein will illustrate the ability for the various algorithms to predict the change in fracture toughness in stainless steels due to hydrogen-isotope embrittlement.

Hoar, Eric T.↗
DOE OSTI2021

Large Eddy Simulation of the Southern Ocean

The data set contains seven Large Eddy Simulations (LES) at the Southern Ocean Flux Site for studies of deep turbulent ocean boundary layers, with and without surface wave effects, and with both idealized and observed forcing by wind, surface buoyancy flux and Stokes drift profiles. There are 20 days of hourly statistics computed every half-hour of turbulence quantities; namely, the vertical fluxes of buoyancy (temperature) and momentum, buoyancy and velocity variances, and the turbulent kinetic energy, its production terms and its dissipation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Validation of Synthetic U.S. Electric Power Distribution System Data Sets

There is a strong need for synthetic yet realistic distribution system test data sets that are as diverse, large, and complex to solve as real systems. Such data sets can facilitate the development of advanced algorithms and the assessment of emerging distributed energy resources while avoiding the need to acquire proprietary critical infrastructure or private data. Such synthetic data sets, however, are useful only if they are realistic enough to look and behave similarly to actual systems. This paper presents a comprehensive framework for validating synthetic distribution data sets using a three-pronged statistical, operational, and expert validation approach. Furthermore, it also presents a set of statistical and operational metric targets for achieving realistic data sets based on detailed characterization of more than 10,000 real U.S. utility feeders. The paper demonstrates the use of the proposed validation approach to validate three large-scale synthetic data sets developed by the authors representing Santa Fe, New Mexico; Greensboro, North Carolina; and the San Francisco Bay Area, California.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A machine-learning-driven data labeling pipeline for scientific analysis in MLExchange

This study introduces a novel labeling pipeline to accelerate the labeling process of scientific data sets by using artificial intelligence (AI)-guided tagging techniques. This pipeline includes a set of interconnected web-based graphical user interfaces (GUIs), where Data Clinic and MLCoach enable the preparation of machine learning (ML) models for data reduction and classification, respectively, while Label Maker is used for label assignment. Throughout this pipeline, data can be accessed through a direct connection to a file system or through Tiled for access through Hypertext Transfer Protocol (HTTP). Our experimental results present three use cases where this labeling pipeline has been instrumental for the study of large X-ray scattering data sets in the area of pattern recognition, the remote analysis of resonant soft X-ray scattering data and the fine-tuning process of foundation models. These use cases highlight the labeling capabilities of this pipeline, including the ability to label large data sets in a short period of time, to perform remote data analysis while minimizing data movement and to enhance the fine-tuning process of complex ML models with human involvement.

Chavez, Tanny (ORCID:0000000193172896)↗
NASA NTRS2011

Large Terrain Modeling and Visualization for Planets

Physics-based simulations are actively used in the design, testing, and operations phases of surface and near-surface planetary space missions. One of the challenges in realtime simulations is the ability to handle large multi-resolution terrain data sets within models as well as for visualization. In this paper, we describe special techniques that we have developed for visualization, paging, and data storage for dealing with these large data sets. The visualization technique uses a real-time GPU-based continuous level-of-detail technique that delivers multiple frames a second performance even for planetary scale terrain model sizes.

digital elevation map↗
DOE OSTI2021

Identifying Light-Duty Vehicle Travel from Large-Scale Multimodal Wearable GPS Data with Novelty Detection Algorithms

Identifying travel mode within travel survey data sets, especially light-duty vehicle (LDV) travel, is foundational, though nontrivial, to travel behavior analysis and fuel consumption estimation. Current travel mode detection approaches require well-sampled and balanced data sets with ground truth travel mode labels. They are rarely applied and validated on large-scale, real-world data sets, which may not satisfy the data requirements. This paper proposes an LDV travel mode detection model as a supplement to current travel mode detection methods, for the case when the training set is highly (and/or completely) unbalanced, to the extent that classical machine-learning approaches become difficult or impossible to deploy. The proposed model uses a novelty detection technique-one-class support vector machines (OCSVMs)-and a novel exhaustive feature extraction (EFE) technique on continuous time series data (i.e., Global Positioning System [GPS] speed profiles) for single-mode trip trajectories. Training and validation of the model are conducted on a large-scale, real-world data set. The proposed method accurately identifies LDV trips from a broad set of multimodal trips by leveraging a wealth of preexisting in-vehicle GPS travel data. Additional sensitivity analysis sheds light on the optimal training size, which will benefit applications limited by highly imbalanced data. The paper also discusses performance comparison with regular machine-learning approaches, the model's robustness, and the potential to extend the proposed model to multimodal prediction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Correlating real-world incidents with vessel traffic off the coast of Hawaii, 2017–2020

Abstract Objectives Because of the high-risk nature of emergencies and illegal activities at sea, it is critical that algorithms designed to detect anomalies from maritime traffic data be robust. However, there exist no publicly available maritime traffic data sets with real-world expert-labeled anomalies. As a result, most anomaly detection algorithms for maritime traffic are validated without ground truth. Data description We introduce the HawaiiCoast_GT data set, the first ever publicly available automatic identification system (AIS) data set with a large corresponding set of true anomalous incidents. This data set—cleaned and curated from raw Bureau of Ocean Energy Management (BOEM) and National Oceanic and Atmospheric Administration (NOAA) automatic identification system (AIS) data—covers Hawaii’s coastal waters for four years (2017–2020) and contains 88,749,176 AIS points for a total of 2622 unique vessels. This includes 208 labeled tracks corresponding to 154 rigorously documented real-world incidents.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The ongoing growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Through access to a large number of time series data sets from different power plants, we have found common issues that impede the modeling process. Namely, the time series data are hard to transfer between groups due to differences in variable nomenclature, and the quality of the data sets can vary. We address the issue of variable nomenclature by FAIRifying spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. We have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. With access to large time series data sets across many power plants, we can utilize spatiotemporal coherence between the sites in order to improve the quality of our data. To solve the issue of data missingness, we propose the use of Spatiotemporal-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗
DOE OSTI2022

Electron Tomography and Machine Learning for Understanding the Highly Ordered Structure of Leafhopper Brochosomes

Insects known as leafhoppers (Hemiptera: Cicadellidae) produce hierarchically structured nanoparticles known as brochosomes that are exuded and applied to the insect cuticle, thereby providing camouflage and anti-wetting properties to aid insect survival. Although the physical properties of brochosomes are thought to depend on the leafhopper species, the structure–function relationships governing brochosome behavior are not fully understood. Brochosomes have complex hierarchical structures and morphological heterogeneity across species, due to which a multimodal characterization approach is required to effectively elucidate their nanoscale structure and properties. In this work, we study the structural and mechanical properties of brochosomes using a combination of atomic force microscopy (AFM), electron microscopy (EM), electron tomography, and machine learning (ML)-based quantification of large and complex scanning electron microscopy (SEM) image data sets. This suite of techniques allows for the characterization of internal and external brochosome structures, and ML-based image analysis methods of large data sets reveal correlations in the structure across several leafhopper species. Our results show that brochosomes are relatively rigid hollow spheres with characteristic dimensions and morphologies that depend on leafhopper species. Nanomechanical mapping AFM is used to determine a characteristic compression modulus for brochosomes on the order of 1–3 GPa, which is consistent with crystalline proteins. Altogether, this work provides an improved understanding of the structural and mechanical properties of leafhopper brochosomes using a new set of ML-based image classification tools that can be broadly applied to nanostructured biological materials.

Chemical structure↗
NASA NTRS2012

Investigation into Cloud Computing for More Robust Automated Bulk Image Geoprocessing

Geospatial resource assessments frequently require timely geospatial data processing that involves large multivariate remote sensing data sets. In particular, for disasters, response requires rapid access to large data volumes, substantial storage space and high performance processing capability. The processing and distribution of this data into usable information products requires a processing pipeline that can efficiently manage the required storage, computing utilities, and data handling requirements. In recent years, with the availability of cloud computing technology, cloud processing platforms have made available a powerful new computing infrastructure resource that can meet this need. To assess the utility of this resource, this project investigates cloud computing platforms for bulk, automated geoprocessing capabilities with respect to data handling and application development requirements. This presentation is of work being conducted by Applied Sciences Program Office at NASA-Stennis Space Center. A prototypical set of image manipulation and transformation processes that incorporate sample Unmanned Airborne System data were developed to create value-added products and tested for implementation on the "cloud". This project outlines the steps involved in creating and testing of open source software developed process code on a local prototype platform, and then transitioning this code with associated environment requirements into an analogous, but memory and processor enhanced cloud platform. A data processing cloud was used to store both standard digital camera panchromatic and multi-band image data, which were subsequently subjected to standard image processing functions such as NDVI (Normalized Difference Vegetation Index), NDMI (Normalized Difference Moisture Index), band stacking, reprojection, and other similar type data processes. Cloud infrastructure service providers were evaluated by taking these locally tested processing functions, and then applying them to a given cloud-enabled infrastructure to assesses and compare environment setup options and enabled technologies. This project reviews findings that were observed when cloud platforms were evaluated for bulk geoprocessing capabilities based on data handling and application development requirements.

Brown, Richard B.↗
NASA NTRS2001

Estimation of comprehensive forest variable sets from multiparameter SAR data over a large area with diverse species

Polarimetric and multifrequency data from the NASA/JPL airborne synthetic aperture radar (AIRSAR) have been used in a multi-tier estimation algorithm to calculate a comprehensive set of forest canopy properties including branch layer moisture and thickness, trunk density, trunk water content and diameter, trunk height, and subcanapy soil moisture. The estimation algorithm takes advantage of species-specific allometric relations, and is applied to a 100Km x 100Km area in the Canadian boreal region containing many different vegetation species types. The results show very good agreement with ground measurements taken at several focused and auxiliary study sites. This paper expands on the results reported in [1] and applies the algorithm on the regional scale.

variable estimation forestry multiparameter SAR↗
NASA NTRS2004

Solving Large Problems Quickly: Progress in 2001-2003

This document describes the progress we have made and the lessons we have learned in 2001 through 2003 under the NASA grant entitled "Solving Important Problems Faster". The long-term goal of this research is to accelerate large, irregular scientific applications which have enormous data sets and which are difficult to parallelize. To accomplish this goal, we are exploring two complementary techniques: (i) using compiler-inserted prefetching to automatically hide the I/O latency of accessing these large data sets from disk; and (ii) using thread-level data speculation to enable the optimistic parallelization of applications despite uncertainty as to whether data dependences exist between the resulting threads which would normally make them unsafe to execute in parallel. Overall, we made significant progress in 2001 through 2003, and the project has gone well.

Mowry, Todd C.↗
NASA NTRS1990

Studying marine stratus with large eddy simulation

Data sets from field experiments over the stratocumulus regime may include complications from larger scale variations, decoupled cloud layers, diurnal cycle, or entrainment instability, etc. On top of the already complicated turbulence-radiation-condensation processes within the cloud-topped boundary layer (CTBL), these complexities may sometimes make interpretation of the data sets difficult. To study these processes, a better understanding is needed of the basic processes involved in the prototype CTBL. For example, is cloud top radiative cooling the primary source of the turbulent kinetic energy (TKE) within the CTBL. Historically, laboratory measurements have played an important role in addressing the turbulence problems. The CTBL is a turbulent field which is probably impossible to generate in laboratories. Large eddy simulation (LES) is an alternative way of 'measuring' the turbulent structure under controlled environments, which allows the systematic examination of the basic physical processes involved. However, there are problems with the LES approach for the CTBL. The LES data need to be consistent with the observed data. The LES approach is discussed, and results are given which provide some insights into the simulated turbulent flow field. Problems with this approach for the CTBL and information from the FIRE experiment needed to justify the LES results are discussed.

Moeng, Chin-Hoh↗
DOE OSTI2021

Validation of non-negative matrix factorization for rapid assessment of large sets of atomic pair distribution function data

The use of the non-negative matrix factorization (NMF) technique is validated for automatically extracting physically relevant components from atomic pair distribution function (PDF) data from time-series data such as in situ experiments. The use of two matrix-factorization techniques, principal component analysis and NMF, on PDF data is compared in the context of a chemical synthesis reaction taking place in a synchrotron beam, applying the approach to synthetic data where the correct composition is known and on measured PDFs from previously published experimental data. The NMF approach yields mathematical components that are very close to the PDFs of the chemical components of the system and a time evolution of the weights that closely follows the ground truth. Lastly, it is discussed how this would appear in a streaming context if the analysis were being carried out at the beamline as the experiment progressed.

36 MATERIALS SCIENCE↗
NASA NTRS2003

Program for User-Friendly Management of Input and Output Data Sets

A computer program manages large, hierarchical sets of input and output (I/O) parameters (typically, sequences of alphanumeric data) involved in computational simulations in a variety of technological disciplines. This program represents sets of parameters as structures coded in object-oriented but otherwise standard American National Standards Institute C language. Each structure contains a group of I/O parameters that make sense as a unit in the simulation program with which this program is used. The addition of options and/or elements to sets of parameters amounts to the addition of new elements to data structures. By association of child data generated in response to a particular user input, a hierarchical ordering of input parameters can be achieved. Associated with child data structures are the creation and description mechanisms within the parent data structures. Child data structures can spawn further child data structures. In this program, the creation and representation of a sequence of data structures is effected by one line of code that looks for children of a sequence of structures until there are no more children to be found. A linked list of structures is created dynamically and is completely represented in the data structures themselves. Such hierarchical data presentation can guide users through otherwise complex setup procedures and it can be integrated within a variety of graphical representations.

Klimeck, Gerhard↗
DOE OSTI2021

Explore Spatio‐Temporal Learning of Large Sample Hydrology Using Graph Neural Networks

Abstract Streamflow forecasting over gauged and ungauged basins play a vital role in water resources planning, especially under the changing climate. Increased availability of large sample hydrology data sets, together with recent advances in deep learning techniques, has presented new opportunities to explore temporal and spatial patterns in hydrological signatures for improving streamflow forecasting. The purpose of this study is to adapt and benchmark several state‐of‐the‐art graph neural network (GNN) architectures, including ChebNet, Graph Convolutional Network (GCN), and GraphWaveNet, for end‐to‐end graph learning. We explicitly represent river basins as nodes in a graph, learn the spatiotemporal nodal dependencies, and then use the learned relations to predict streamflow simultaneously across all nodes in the graph. The efficacy of the developed GNN models is investigated using the Catchment Attributes and MEteorology for Large‐sample Studies (CAMELS) data set under two settings, fixed graph topology (transductive learning), and variable graph topology (inductive learning), with the latter applicable to prediction in ungauged basins (PUB). Results indicate that GNNs are generally robust and computationally efficient, achieving similar or better performance than a baseline model trained using the long short‐term memory (LSTM) network. Further analyses are conducted to interpret the graph learning process at the edge and node levels and to investigate the effect of different model configurations. We conclude that graph learning constitutes a viable machine learning‐based method for aggregating spatiotemporal information from a multitude of sources for streamflow forecasting

Sun, Alexander Y.↗
NASA NTRS2024

SOFIA/HAWC+ Far-Infrared Polarimetric Large-area CMZ Exploration Survey. III. Full Survey Data Set

We present the second data release (DR2) of the Far-InfraREd Polarimetric Large-Area CMZ Exploration (FIREPLACE) survey. This survey utilized the Stratospheric Observatory for Infrared Astronomy High-resolution Airborne Wideband Camera plus instrument at 214 μm (E band) at a resolution of 19"6 to observe thermal polarized dust emission throughout the Central Molecular Zone (CMZ). DR2 consists of observations obtained in 2022 covering the region of the CMZ extending from the Brick to the Sgr C molecular clouds (corresponding to a 1° × 0°.75 region of the sky). We combine DR2 with the first FIREPLACE data release (DR1) to obtain full coverage of the CMZ (a 1°.5 × 0°.75 region of the sky). After applying total and polarized intensity significance cuts on the full FIREPLACE data set, we obtain ∼64,000 Nyquist-sampled polarization pseudovectors. The distribution of polarization pseudovectors confirms a bimodal distribution in the CMZ magnetic field orientations, recovering field components that are oriented predominantly parallel or perpendicular to the Galactic plane. This distribution of orientations is similar to what was observed in DR1 and other studies. We also inspect the magnetic fields toward a set of prominent CMZ molecular clouds (the Brick, Three Little Pigs, 50 and 20 km s −1 clouds, circumnuclear disk, CO 0.02-0.02, and Sgr C), revealing spatially varying magnetic fields having orientations that generally trace the total intensity morphologies of the clouds. We find evidence that compression from stellar winds and shear from tidal forces are prominent mechanisms influencing the structure of the magnetic fields.

Molecular clouds↗