Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83

Macrophysical and optical properties of midlatitude cirrus clouds from four ground-based lidars and collocated CALIOP observations

Ground-based lidar and CALIOP datasets gathered over four mid-latitude sites, two US and two French sites, are used to evaluate the consistency of cloud macrophysical and optical property climatologies that can be derived by such datasets. The consistency in average cloud height (both base and top height) between the CALIOP and ground datasets ranges from -0.4km to +0.5km. The cloud geometrical thickness distributions vary significantly between the different datasets, due in part to the original vertical resolutions of the lidar profiles. Average cloud geometrical thicknesses vary from 1.2 to 1.9km, i.e. by more than 50%. Cloud optical thickness distributions in subvisible, semi-transparent and moderate intervals differ by more than 50% between ground and space-based datasets. The cirrus clouds with 2 optical thickness below 0.1 (not included in historical cloud climatologies) represent 30-50% of the non-opaque cirrus class. The differences in average cloud base altitude between ground and CALIOP datasets of 0.0-0.1 km, 0.0-0.2 km and 0.0-0.2 km can be attributed to irregular sampling of seasonal variations in the ground-based data, to day-night differences in detection capabilities by CALIOP, and to the restriction to situations without low-level clouds in ground-based data, respectively. The cloud geometrical thicknesses are not affected by irregular sampling of seasonal variations in the ground-based data, while up to 0.0-0.2 km and 0.1-0.3 km differences can be attributed to day-night differences in detection capabilities by CALIOP, and to the restriction to situations without lowlevel clouds in ground-based data, respectively.

54 ENVIRONMENTAL SCIENCES↗

Use of Electrical Imaging and Distributed Temperature Sensing Methods to Characterize Surface Water-Groundwater Exchange Regulating Uranium Transport at the Hanford 300 Area, Washington

A critical challenge in advancing prediction of solute transport between contaminated aquifers and rivers is improving understanding of how fluctuations in river stage, combined with subsurface heterogeneity, impart spatiotemporal complexity to solute exchange along river corridors. Here, we explored the use of waterborne geoelectrical imaging, in conjunction with fiber-optic distributed temperature sensor (DTS) monitoring, to improve the conceptual model for uranium transport within the hyporheic corridor at the Hanford 300 Area. We first inverted waterborne geoelectrical (resistivity and induced polarization) datasets for distributions of electrical resistivity and polarizability, from which the spatial complexity of the primary hydrogeologic units was reconstructed. Variations in the depth to the interface between the overlying coarse-grained, high permeability Hanford formation and the underlying finer-grained, less permeable Ringold formation, an important contact that limits vertical migration of contaminants, were resolved along ~3 km of the river corridor centered on the 300 Area. Polarizability images were translated into lithologic images using established relationships between polarizability and surface area normalized to pore volume (Spor). The spatial variability captured in the geoelectrical datasets indicates that previous studies based on borehole projections and point probing overestimate the contributing area for uranium exchange within the Columbia River at the Hanford 300 Area. The DTS data recorded on 1. 5 km of cable with a 1 m spatial resolution and 5 minute sampling interval revealed sub-reaches showing (1) high temperature anomalies and, (2) a strong negative correlation between temperature and river stage, both indicative of groundwater influxes during winter months. The DTS datasets confirm the hydrologic significance of the variability identified in the geoelectrical imaging and reveal a pattern of highly focused hyporheic exchange, with exchange concentrated at springs where the Hanford formation is thick, and coinciding with a paleochannel identified in ground penetrating radar surveys at one location. No evidence for focused hyporheic exchange is observed in the DTS data where the Ringold unit is in contact with the riverbed. Our findings illustrate how the combination of waterborne geoelectrical imaging and DTS technologies can characterize hyporheic exchange in a complex, coupled river-aquifer system.

300 area↗

Global land use for 2015–2100 at 0.05° resolution under diverse socioeconomic and climate scenarios

Abstract Global future land use (LU) is an important input for Earth system models for projecting Earth system dynamics and is critical for many modeling studies on future global change. Here we generated a new global gridded LU dataset using the Global Change Analysis Model (GCAM) and a land use spatial downscaling model, named Demeter, under the five Shared Socioeconomic Pathways (SSPs) and four Representative Concentration Pathways (RCPs) scenarios. Compared to existing similar datasets, the presented dataset has a higher spatial resolution (0.05° × 0.05°) and spreads under a more comprehensive set of SSP-RCP scenarios (in total 15 scenarios), and considers uncertainties from the forcing climates. We compared our dataset with the Land Use Harmonization version 2 (LUH2) dataset and found our results are in general spatially consistent with LUH2. The presented dataset will be useful for global Earth system modeling studies, especially for the analysis of the impacts of land use and land cover change and socioeconomics, as well as the characterizing the uncertainties associated with these impacts.

54 ENVIRONMENTAL SCIENCES↗

Automated ICRF heating surrogate modeling via machine learning

This work introduces automated machine learning workflows that address critical bottlenecks in surrogate model development for Ion Cyclotron Range of Frequencies (ICRF) heating applications. The automated framework includes data analysis tools that transform raw datasets into actionable insights in seconds, replacing weeks of manual exploratory effort and ensuring consistent, reproducible dataset characterization. By integrating advanced hyperparameter optimization (HPO) methods including Bayesian optimization via BoTorch and Tree-structured Parzen Estimators (TPE), the framework significantly reduces model development time from weeks to hours, decreasing computational cost and required expertise, while enabling high-accuracy surrogate models. Compared to traditional hyperparameter scanning (HPS) techniques such as methodical, randomized, and grid searches, HPO methods achieve superior convergence and predictive performance, even when compared to already well-tuned reference models. On NSTX High Harmonic Fast Wave (HHFW) heating datasets, both Random Forest Regressor (RFR) and neural network surrogates demonstrate improved accuracy, achieving R 2 values beyond 0.97 and 0.98, respectively. The results show that while HPO gains are modest for robust architectures like RFR, they become essential for more sensitive models such as neural networks, highlighting the trade-offs across optimization strategies. Through automated workflows that eliminate manual hyperparameter tuning and require minimal ML expertise, this work enables widespread adoption of high-fidelity surrogate models across the fusion community for real-time plasma control, uncertainty quantification, rapid experimental scenario development, and integrated system optimization.

Sanchez-Villar, Alvaro [Princeton Plasma Physics L↗

Design and early experience with promoting user-created data in CMS

The Computing Model of the CMS experiment [1] does not address transfering user-created data between different Grid sites. Due to the limited resources of a single site, distribution of individual user-created datasets between sites is crucial to ensure accessibility. In contrast to official datasets, there are no special requirements for user datasets (e.g. concerning data quality). The StoreResults service provides a mechanism to elevate user-created datasets to central bookkeeping ensuring the data quality is the same as an official dataset. This is a prerequisite for further distribution within the CMS dataset infrastructure.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The medical science DMZ: a network design pattern for data-intensive medical science

Abstract Objective We describe a detailed solution for maintaining high-capacity, data-intensive network flows (eg, 10, 40, 100 Gbps+) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet-filter firewalls, network intrusion-detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive datasets between research institutions over national research networks, and give 3 detailed descriptions of implemented Medical Science DMZs. Discussion The exponentially increasing amounts of “omics” data, high-quality imaging, and other rapidly growing clinical datasets have resulted in the rise of biomedical research “Big Data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large datasets. Maintaining data-intensive flows that comply with the Health Insurance Portability and Accountability Act (HIPAA) and other regulations presents a new challenge for biomedical research. We describe a strategy that marries performance and security by borrowing from and redefining the concept of a Science DMZ, a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

Peisert, Sean↗

Detecting damage in composites using volume decomposition analysis of tomographic data

Detection of damage in a single tow ceramic matrix composite specimen has been achieved using orthogonal decomposition of volumetric tomographic datasets collected at four tensile loads. This decomposition approach has been applied at two different length scales: (i) individual fibres and (ii) bulk volumes containing fibres and matrix material. Volumes were first decomposed to feature vectors, orders of magnitude smaller than the original volume they describe, and then comparisons between datasets at different load levels were made in feature vector space. The results show quantitative measurements of damage location, damage morphology and the relative growth of this damage with increased load when compared with a dataset with less or no damage. No prior knowledge of the dataset or training of algorithms is required for damage to be detected, it is only necessary that at least two datasets are available for comparison, e.g. from in situ or repeated scanning measurements. Results are generated on significantly shorter timescales when compared with previous automated approaches to tomography data processing. This approach has the potential to be applied to damage detection in a range of materials through comparisons of volumetric datasets from a range of measurement or computational techniques.

Middleton, Ceri A.↗

Constraints on Λ CDM extensions from the SPT-3G 2018 $EE$ and $TE$ power spectra

Here, we present constraints on extensions to the Λ CDM cosmological model from measurements of the E-mode polarization autopower spectrum and the temperature-E-mode cross-power spectrum of the cosmic microwave background (CMB) made using 2018 SPT-3G data. The extensions considered vary the primordial helium abundance, the effective number of relativistic degrees of freedom, the sum of neutrino masses, the relativistic energy density and mass of a sterile neutrino, and the mean spatial curvature. We do not find clear evidence for any of these extensions, from either the SPT-3G 2018 dataset alone or in combination with baryon acoustic oscillation and Planck data. None of these model extensions significantly relax the tension between Hubble-constant, H 0 , constraints from the CMB and from distance-ladder measurements using Cepheids and supernovae. The addition of the SPT-3G 2018 data to Planck reduces the square-root of the determinants of the parameter covariance matrices by factors of 1.3–2.0 across these models, signaling a substantial reduction in the allowed parameter volume. We also explore CMB-based constraints on H 0 from combined SPT, Planck, and ACT DR4 datasets. While individual experiments see some indications of different H 0 values between the TT, TE, and EE spectra, the combined H 0 constraints are consistent between the three spectra. For the full combined datasets, we report H 0 = 67.49 ± 0.53 km s -1 Mpc -1 , which is the tightest constraint on H0 from CMB power spectra to date and in 4.1σ tension with the most precise distance-ladder-based measurement of H 0 . The SPT-3G survey is planned to continue through at least 2023, with existing maps of combined 2019 and 2020 data already having ~ 3.5 x lower noise than the maps used in this analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

Pseudonymization at Scale: OLCF’s Summit Usage Data Case Study

The analysis of vast amounts of data and the processing of complex computational jobs have traditionally relied upon high performance computing (HPC) systems, which offer reliable and efficient management of large-scale computational and data resources. Understanding these analyses’ needs is paramount for designing solutions that can lead to better science, and similarly, understanding the characteristics of the user behavior on those systems is important for improving user experiences on HPC systems. A common approach to gathering data about user behavior is to extract workload characteristics from system log data available only to system administrators. Recently at Oak Ridge Leadership Computing Facility (OLCF), however, we unveiled user behavior about the Summit supercomputer by collecting data from a user’s point of view with ordinary Unix commands.In this paper, we discuss the process, challenges, and lessons learned while preparing this dataset for publication and submission to an open data challenge. The original dataset contains personal identifiable information (PII) about the users of OLCF which needed be masked prior to publication, and we determined that anonymization, which scrubs PII completely, destroyed too much of the structure of the data to be interesting for the data challenge. We instead chose to pseudonymize the dataset, which reduced the linkability of the dataset to the users’ identities. Pseudonymization is significantly more computationally expensive than anonymization, and the size of our dataset, which is approximately 175 million lines of raw text, necessitated the development of a parallelized workflow that could be reused on different HPC machines. We demonstrate the scaling behavior of the workflow on two leadership class HPC systems at OLCF, and we show that we were able to bring the overall makespan time from an impractical 20+ hours on a single node down to around 2 hours. As a result of this work, we release the entire pseudonymized dataset and make the workflows and source code publicly available.

Maheshwari, Ketan↗

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling

Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabytescale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling.

Li, Chaojian [ORNL] (ORCID:0000000340309777)↗

ScienceSearch: Enabling Search through Automatic Metadata Generation

© 2018 IEEE. Scientific facilities are increasingly generating and handling large amounts of data from experiments and simulations. Next-generation scientific discoveries rely on insights derived from data, especially across domain boundaries. Search capabilities are critical to enable scientists to discover datasets of interest. However, scientific datasets often lack the signals or metadata required for effective searches. Thus, we need formalized methods and systems to automatically annotate scientific datasets from the data and its surrounding context. Additionally, a search infrastructure needs to account for the scale and rate of application data volumes. In this paper, we present ScienceSearch, a system infrastructure that uses machine learning techniques to capture and learn the knowledge, context, and surrounding artifacts from data to generate metadata and enable search. Our current implementation is focused on a dataset from the National Center for Electron Microscopy (NCEM), an electron microscopy facility at Lawrence Berkeley National Laboratory sponsored by the Department of Energy which supports hundreds of users and stores millions of micrographs. In this paper, we describe a) our search infrastructure and model, b) methods for generating metadata using machine learning techniques, and c) optimizations to improve search latency, and deployment on an HPC system. We demonstrate that ScienceSearch is capable of producing valid metadata for NCEM's dataset and providing low-latency good quality search results over a scientific dataset.

Rodrigo, Gonzalo P.↗

Deep Learning Based Frequency Stability Assessment in Power Grid with High Renewables

Frequency stability assessment is one critical aspect of power system security assessment. Traditional N-1 screening method is based on the simulations of a few typical daily and seasonal operation scenarios. However, the increasing integration of inverter-based renewables and the retirement of conventional synchronous generators result in decreasing system inertia and growing complexity of system operating conditions. Selecting a few typical operation scenarios cannot cover all operating conditions, and the time-domain simulation of all operation conditions requires tremendous time. This paper proposes a more efficient frequency stability assessment method based on deep learning. The affinity propagation clustering algorithm is used to divide the dataset into different clusters, so the selected dataset for training can cover the diversified operating conditions as much as possible. Also, feature normalization is applied to both the training dataset and testing dataset in order to remove any unnecessary bias. Especially, trained model based on full dataset normalization has bounded error in the prediction. The case study on the reduced 240-bus WECC system demonstrates that the proposed method can predict accurate frequency nadir with limited training dataset. The deep learning model using the revised feature normalization can predict more accurate frequency nadir than that using the traditional feature normalization and has very small maximum prediction error.

affinity propagation↗

A Novel Spatial-Temporal Variational Quantum Circuit to Enable Deep Learning on NISQ Devices

Quantum computing presents a promising approach for machine learning with its capability for extremely parallel computation in high-dimension through superposition and entanglement. Despite its potential, existing quantum learning algorithms, such as Variational Quantum Circuits (VQCs), face challenges in handling more complex datasets, particularly those that are not linearly separable. What’s more, it encounters the deployability issue, making the learning models suffer a drastic accuracy drop after deploying them to the actual quantum devices. To overcome these limitations, this paper proposes a novel spatial-temporal design, namely “ST-VQC”, to integrate nonlinearity in quantum learning and improve the robustness of the learning model to noise. Specifically, ST-VQC can extract spatial features via a novel block-based encoding quantum sub-circuit coupled with a layer-wise computation quantum sub-circuit to enable temporal-wise deep learning. Additionally, a SWAP-Free physical circuit design is devised to improve robustness. These designs bring a number of hyperparameters. After a systematic analysis of the design space for each design component, an automated optimization framework is proposed to generate the ST-VQC quantum circuit. The proposed ST-VQC has been evaluated on two IBM quantum processors, ibm-cairo with 27 qubits and ibmq-lima with 7 qubits to assess its effectiveness. The results of the evaluation on the standard dataset for binary classification show that ST-VQC can achieve over 30% accuracy improvement compared with existing VQCs on actual quantum computers. Moreover, on a non-linear synthetic dataset, the STVQC outperforms a linear classifier by 27.9%, while the linear classifier using classical computing outperforms the existing VQC by 15.58%.

Li, Jinyang↗

Using Synchrophasor Status Word as Data Quality Indicator: What to Expect in the Field?

Data quality plays a crucial role in successful applications of synchrophasor data in power system operation and control. This paper presents the results of a data quality analysis of a multi-year field-recorded synchrophasor dataset. The analysis has identified several typical data quality issues encountered in the field data. An examination of the PMU status words included with the dataset has revealed several inconsistent implementations and the lack of correlation between the PMU data quality and the status word, which impacts the usefulness of such information. Our investigation has concluded that the status word alone as found in the recorded field dataset could not be used as a reliable indicator of data quality for field-recorded data. Several recommendations are proposed to improve the usefulness of the PMU status word.

Cheng, Zheyuan↗

Machine Learning-Driven Reliability Estimation of PV Inverters Considering Alert-Ambient Variability

Weather-induced spatio-temporal degradation limits outdoor PV inverter lifetime and reliability, necessitating advanced data analysis. This study employs a top-down, data-driven approach utilizing multiple machine learning (ML) algorithms to estimate inverter reliability in a 1.4 MW PV power plant, considering factors such as irradiance, humidity, temperature, time of day, and weather conditions. An extensive alert dataset from 17 identical inverters, including alert types, propagation, and frequency, reveals significant correlations with environmental factors and inverter output power, enabling the construction of a performance reliability model. Dual-stage supervised-ML models are evaluated for accuracy, with the ‘classification-regression’ model by an artificial neural network (ANN) tested on the averaged “Alert-Ambient” dataset, which is outperformed by ‘clustering-regression’ models using random forest (RF) and K-Nearest Neighbors (KNN) on individual inverter datasets. K-means clustering applies principal component analysis to reduce dimensions, achieving improved accuracy beyond the 80% achieved by ANN on the averaged dataset. Second-stage regression estimates inverter reliability with a mean square error of 0.0195 on the averaged dataset and as low as 0.002 on individual inverter datasets using RF. Furthermore, these findings highlight the method's suitability for estimating PV inverter output reliability under ambient conditions, essential for digital twin development and related applications.

14 SOLAR ENERGY↗

Factor analysis in automated face detection: gender, occlusion, eyewear, brightness, contrast, and focus measure

Computer-based facial recognition algorithms exploit the unique characteristics of faces in images. However, in non-cooperative situations these unique characteristics are often disturbed. In this study, we examine the effect of six different factors on face detection in an unconstrained imaging environment: image brightness, image contrast, focus measure, eyewear, gender, and occlusion. The aim of this study is twofold: first, to quantify detection rates of conventional Haar cascade algorithms across these six factors; and second, to propose methods for automatically labeling datasets whose size prohibits manual labeling. First, we manually classify a uniquely challenging dataset comprising 9,688 images of passengers in vehicles acquired from a roadside camera system. Next, we quantify how each of the aforementioned factors affect face detection on this dataset. Of the six factors studied, occlusion had the most significant impact, resulting in a 54% decrease in detection rate between unoccluded and severely occluded faces in our unique dataset. Finally, we provide a methodology for data analytics of large datasets where manual labeling of the whole dataset is not possible.

Jafek, Benjamin↗

ARM Climate Modeling Best Estimate Data - A new data product for climate modelers

This paper provides an overview of a new data product, named the Climate Modeling Best Estimate (CMBE) dataset, developed by the U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) Program in order to better serve the need of climate model developers and encourage greater use of ARM data by modelers. The CMBE dataset contains those quantities that are often used in model evaluation and reflect unique ARM measurements of clouds and radiation (e.g., cloud occurrence, liquid water path, and surface radiative fluxes) from the highest quality data that ARM has for many years. The data are averaged over one hour period, which is comparable to a typical temporal resolution used in climate model output. They are currently available at five ARM Climate Research Facility (ACRF) sites located at the Southern Great Plains, North Slope of Alaska, and Tropic Western Pacific, and can be obtained from the ACRF data archive. The long-term continuous ARM data provide invaluable information to improve our understanding of the interaction between clouds and radiation and a solid observational basis for model validation and improvement. This paper shows some examples to demonstrate its unique values in studies of cloud processes, climate variability and change, and climate modeling. Plans for future enhancements of the CMBE product are also discussed.

54 ENVIRONMENTAL SCIENCES↗

MJO Signals in Latent Heating: Results from TRMM Retrievals

Four Tropical Rainfall Measuring Mission (TRMM) datasets of latent heating were diagnosed for signals in the Madden-Julian Oscillation (MJO). In all four datasets, vertical structures of latent heating are dominated by two components, one deep with its peak above the melting level and one shallow with its peak below. Profiles of the two components are nearly ubiquitous in longitude, allowing a separation of the vertical and zonal/temporal variations when the latitudinal dependence is not considered. All four datasets exhibit robust MJO spectral signals in the deep component as eastward propagating spectral peaks centered at period of 50 days and zonal wavenumber 1, well distinguished from lower- and higher-frequency power and much stronger than the corresponding westward power. The shallow component shows similar but slightly less robust MJO spectral peaks. MJO signals were further extracted from a combination of band-pass (30 – 90 day) filtered deep and shallow components. Largest amplitudes of both deep and shallow components of the MJO are confined to the Indian and western Pacific Oceans. There is a local minimum in the deep components over the Maritime Continent. The shallow components of the MJO differ substantially among the four TRMM datasets in their detailed zonal distributions in the eastern hemisphere. In composites of the heating evolution through the life cycle of the MJO, the shallow components lead the deep ones in some datasets and at certain longitudes. In many respects, the four TRMM datasets agree well in their deep components, but not in their shallow components and the phase relations between the deep and shallow components. These results indicate that caution must be exercised in applications of these latent heating data.

54 ENVIRONMENTAL SCIENCES↗