Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Chemical order-disorder nanodomains in Fe 3 Pt bulk alloy

Chemical ordering is a common phenomenon and highly correlated with the properties of solid materials. By means of the redistribution of atoms and chemical bonds, it invokes an effective lattice adjustment and tailors corresponding physical properties. To date, however, directly probing the 3D interfacial interactions of chemical ordering remains a big challenge. In this work, we deciphered the interlaced distribution of nanosized domains with chemical order/disorder in Fe 3 Pt bulk alloy. HAADF-STEM images evidence the existence of such nanodomains. The reverse Monte Carlo method with the X-ray pair distribution function data reveal the 3D distribution of local structures and the tensile effect in the disordered domains at the single-atomic level. The chemical bonding around the domain boundary changes the bonding feature in the disordered side and reduces the local magnetic moment of Fe atoms. This results in a suppressed negative thermal expansion and extended temperature range in Fe 3 Pt bulk alloy with nanodomains. Our study demonstrates a local revelation for the chemical order/disorder nanodomains in bulk alloy. The understanding gained from atomic short-range interactions within the domain boundaries provides useful insights with regard to designing new functional compounds.

36 MATERIALS SCIENCE↗
DOE OSTI2026

RC-SFA Data Management Templates and Guidance for Standardized, Reusable AI-Ready Data Packages

This data package provides templates and supporting documentation developed by the River Corridor Science Focus Area (RC-SFA; https://www.pnnl.gov/projects/river-corridor) to communicate its approach to managing and publishing AI-ready data. The package is intended to help data users and data producers understand the structures, metadata practices, and quality-control approaches that support consistent, reusable, and machine-actionable data products across RC-SFA studies. Rather than focusing on a single experimental dataset, this package documents the data management framework used to make RC-SFA data easier to find, ingest, navigate, and interpret. The materials in this package reflect RC-SFA practices for standardized data package organization, including the use of a human- and machine-readable README, file-level metadata, data dictionaries, descriptive file naming, method identifiers, and automated and review-based quality assurance procedures. Together, these components illustrate how RC-SFA extends FAIR data principles toward AI-readiness by prioritizing deep metadata, consistency across data packages, and support for informed downstream reuse by both humans and computational tools. This dataset is comprised of (1) readme; (2) presentation slides with an overview of RC-SFA approach and guidance; (3) document of RC-SFA best practices; (4) data dictionary (dd); (5) file level metadata (flmd); and a subfolder containing templates for dd and flmd. All files are .csv and .pdf. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About.

AI-readiness↗
DOE OSTI2024

3D Multiresolution Velocity Model Fusion with Probability Graphical Models

ABSTRACT The variability in spatial resolution of seismic velocity models obtained via tomographic methodologies is attributed to many factors, including inversion strategies, ray-path coverage, and data integrity. Integration of such models, with distinct resolutions, is crucial during the refinement of community models, thereby enhancing the precision of ground-motion simulations. Toward this goal, we introduce the probability graphical model (PGM), combining velocity models with heterogeneous resolutions and nonuniform data point distributions. The PGM integrates data relations across varying resolution subdomains, enhancing detail within low-resolution (LR) domains by utilizing information and prior knowledge from high-resolution (HR) subdomains through a maximum posterior problem. Assessment of efficacy, utilizing both 2D and 3D velocity models—consisting of synthetic checkerboard models and a fault-zone model from Ridgecrest, California—demonstrates noteworthy improvements in accuracy, compared to state-of-the-art fusion techniques. Specifically, we find reductions of 30% and 44% in computed travel-time residuals for 2D and 3D models, respectively, as compared to conventional smoothing techniques. Unlike conventional methods, the PGM’s adaptive weight selection facilitates preserving and learning details from complex, nonuniform HR models and applies the enhancements to the LR background domain.

Geochemistry & Geophysics↗
DOE OSTI2021

Distributed directory of named data elements in coordination namespace

An approach is described that provides a distributed directory structure within a storage of an information handling system (a local node). A request is received with the request corresponding to a shared virtual address. The shared virtual address that is shared amongst a number of nodes that includes the local node and some remote nodes. A Global Address Space Directory (GASD) is retrieved that corresponds to a global virtual address space. The GASD is stored in a Coordination Namespace that is stored in a memory that is distributed amongst the nodes. A mapping that is included in the GASD is used to determine the node where the shared virtual address currently resides. The shared virtual address is then accessed from the node where it currently resides.

Johns, Charles Ray↗
DOE OSTI2023

Uncertainty quantification of machine learning models to improve streamflow prediction under changing climate and environmental conditions

Machine learning (ML) models, and Long Short-Term Memory (LSTM) networks in particular, have demonstrated remarkable performance in streamflow prediction and are increasingly being used by the hydrological research community. However, most of these applications do not include uncertainty quantification (UQ). ML models are data driven and can suffer from large extrapolation errors when applied to changing climate/environmental conditions. UQ is required to quantify the influence of data noises on model predictions and avoid overconfident projections in extrapolation. In this work, we integrate a novel UQ method, called PI3NN, with LSTM networks for streamflow prediction. PI3NN calculates Prediction Intervals by training 3 Neural Networks. It can precisely quantify the predictive uncertainty caused by the data noise and identify out-of-distribution (OOD) data in a non-stationary condition to avoid overconfident predictions. We apply the PI3NN-LSTM method in the snow-dominant East River Watershed in the western US and in the rain-driven Walker Branch Watershed in the southeastern US. Results indicate that for the prediction data which have similar features as the training data, PI3NN precisely quantifies the predictive uncertainty with the desired confidence level; and for the OOD data where the LSTM network fails to make accurate predictions, PI3NN produces a reasonably large uncertainty indicating that the results are not trustworthy and should avoid overconfidence. PI3NN is computationally efficient, robust in performance, and generalizable to various network structures and data with no distributional assumptions. It can be broadly applied in ML-based hydrological simulations for credible prediction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Data-Driven Generic Turbines for Distributed Wind Modeling, Optimization, and Economic Studies

As distributed energy resources (DER) become less expensive and more popular, utilities, project developers, and customers have an increasing need to model the performance of existing and proposed DER systems. Distributed wind has been shown to have widespread economic potential but is often represented by a simplified model in - or excluded from - DER modeling tools and studies. There is often no economic imperative to extend models and studies to give full consideration to distributed wind. We present a set of data-driven generic turbines derived from 16 years of annual distributed wind market survey data. The proposed methodology can be used to derive generic turbines from separate or updated data sets. Finally, a mixed-integer linear programming approach to optimal distributed wind project sizing is used to demonstrate the generic turbine models. Combined, these models and methods can reduce barriers to considering distributed wind in modeling tools and studies.

Reiman, Andrew P.↗
DOE OSTI2021

The environmental footprint of data centers in the United States

Much of the world's data are stored, managed, and distributed by data centers. Data centers require a tremendous amount of energy to operate, accounting for around 1.8% of electricity use in the United States. Large amounts of water are also required to operate data centers, both directly for liquid cooling and indirectly to produce electricity. For the first time, we calculate spatially-detailed carbon and water footprints of data centers operating within the United States, which is home to around one-quarter of all data center servers globally. Our bottom-up approach reveals one-fifth of data center servers direct water footprint comes from moderately to highly water stressed watersheds, while nearly half of servers are fully or partially powered by power plants located within water stressed regions. Approximately 0.5% of total US greenhouse gas emissions are attributed to data centers. We investigate tradeoffs and synergies between data center's water and energy utilization by strategically locating data centers in areas of the country that will minimize one or more environmental footprints. Our study quantifies the environmental implications behind our data creation and storage and shows a path to decrease the environmental footprint of our increasing digital footprint.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

CODAS Data from Oliktok Point, Beaufort Sea, Alaska

Cryosphere/Ocean Distributed Acoustic Sensing (CODAS) data collected from the Beaufort Sea, Alaska, using ~37.4 km of dark telecommunications fiber located at Oliktok Point, Alaska. Data were collected with a Silixa iDAS, using 10 m gauge length, 2 m spatial resolution, and 1000 Hz sample rate. Provided here are the DAS-recorded time series for the rapid refreeze event described in Baker & Abbott (2022) (see link below). This covers a date range of 2021-11-10 15:00 UTC to 2021-11-11 17:00 UTC. Data have been decimated to 100 Hz and 20 m (i.e., every 10th channel for 1831 channels, total), as used in Baker & Abbott (2022). Data have been extracted from raw format into 1-hour long .sac* files and organized into directories by channel number, spanning channels 100 to 18400. Time series units are nano-strainrate (nm/m/s). For distribution, data have been compressed into .zip files containing all time series files for 100 channels. *For information on the Seismic Analysis Code (SAC) file format: https://seiscode.iris.washington.edu/projects/sac

16 TIDAL AND WAVE POWER↗
DOE OSTI2024

Models, data, and scripts associated with “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning”

This data package is associated with the publication “Prediction of Distributed River Sediment Respiration Rates using Community-Generated Data and Machine Learning’’ submitted to the Journal of Geophysical Research: Machine Learning and Computation (Scheibe et al. 2024). River sediment respiration observations are expensive and labor intensive to obtain and there is no physical model for predicting this quantity. The Worldwide Hydrobiogeochemisty Observation Network for Dynamic River Systems (WHONDRS) observational data set (Goldman et al.; 2020) is used to train machine learning (ML) models to predict respiration rates at unsampled sites. This repository archives training data, ML models, predictions, and model evaluation results for the purposes of reproducibility of the results in the associated manuscript and community reuse of the ML models trained in this project. One of the key challenges in this work was to find an optimum configuration for machine learning models to work with this feature-rich (i.e. 100+ possible input variables) data set. Here, we used a two-tiered approach to managing the analysis of this complex data set: 1) a stacked ensemble of ML models that can automatically optimize hyperparameters to accelerate the process of model selection and tuning and 2) feature permutation importance to iteratively select the most important features (i.e. inputs) to the ML models. The major elements of this ML workflow are modular, portable, open, and cloud-based, thus making this implementation a potential template for other applications. This data package is associated with the GitHub repository found at Please see the file level metadata (flmd; “sl-archive-whondrs_flmd.csv”) for a list of all files contained in this data package and descriptions for each. Please see the data dictionary (dd; “sl-archive-whondrs_dd.csv”) for a list of all column headers contained within comma separated value (csv) files in this data package and descriptions for each. The GitHub repository is organized into five top-level directories: (1) “input_data” holds the training data for the ML models; (2) “ml_models” holds machine learning models trained on the data in “input_data”; (3) “scripts” contains data preprocessing and postprocessing scripts and intermediate results specific to this data set that bookend the ML workflow; (4) “examples” contains the visualization of the results in this repository including plotting scripts for the manuscript (e.g., model evaluation, FPI results) and scripts for running predictions with the ML models (i.e., reusing the trained ML models); (5) “output_data” holds the overall results of the ML model on that branch. Each trained ML model resides on its own branch in the repository; this means that inputs and outputs can be different branch-to-branch. Furthermore, depending on the number of features used to train the ML models, the preprocessing and postprocessing scripts, and their intermediate results, can also be different branch-to-branch. The “main-*” branches are meant to be starting points (i.e. trunks) for each model branch (i.e. sprouts). Please see the Branch Navigation section in the top-level README.md in the GitHub repository for more details. There is also one hidden directory “.github/workflows”. This hidden directory contains information for how to run the ML workflow as an end-to-end automated GitHub Action but it is not needed for reusing the ML models archived here. Please the top-level README.md in the GitHub repository for more details on the automation.

13C↗
DOE OSTI2022

Detection of Stealthy False Data Injection Attacks in Unobservable Distribution Networks

In this paper, a composite scheme is proposed for detecting stealthy data manipulation attacks on distribution system which is unobservable with standard least squares based state estimators. This technique has three stages where the process of data imputation, voltage phasor estimation and the bad data detection are carried out in a systematic manner. The proposed approach is then integrated with moving target defense strategies which perturbs the network parameters to reveal stealthy false data injection attacks. The proposed approach is tested is validated on a three-phase, unbalanced 37-node distribution system and its results are presented. It is shown that the proposed approach has the ability to accurately detect the presence of FDI attacks using limited measurements (i.e., the test system is unobservable).

Rajasekaran, James K.↗
DOE OSTI2025

Quantifying Distribution System Resilience From Utility Data: Large Event Risk and Benefits of Investments

We focus on blackouts in electric distribution systems that have a large cost to customers. To quantify resilience to these events, we show how to calculate risk metrics from the historical outage data routinely collected by utilities' outage management systems. Risk is defined using a customer cost exceedance curve. The exceedance curve has a heavy tail that implies large fluctuations in large blackout costs, and this makes estimating the mean large cost in the usual way impractical. To avoid this problem, we use new resilience metrics describing the large event risk; these metrics are the probability of a large cost event, the annual log cost resilience index, and the average of the logarithm of the cost of large-cost events or the slope magnitude of the tail on a log–log exceedance curve. Resilience can be improved by planned investments to upgrade system components or speed up restoration. The benefits that these investments would have had if they had been made in the past can be quantified by “rerunning history” with the effects of the investment included, and then recalculating the large event risk to find the improvement in resilience. An example using utility data shows a 2% reduction in the probability of a large cost event due to 10% wind hardening and 6%–7% reduction due to 10% faster restoration in two different areas of a distribution utility. This new data-driven approach to quantify resilience and resilience investments is realistic and much easier to apply than complicated approaches based on modeling all the phases of resilience. Moreover, an appeal to improvements to past lived experience may well be persuasive to customers and regulators in making the case for resilience investments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Scale tests of the new DUNE data pipeline

In preparation for the second runs of the ProtoDUNE detectors at CERN (NP02 and NP04)[1], DUNE has established a new data pipeline for bringing the data from the EHN-1 experimental hall at CERN to primary tape storage at Fermilab and CERN, and then spreading it out to a distributed disk data store at many locations around the world. This system includes a new Ingest Daemon and a new Declaration Daemon. The Rucio[2] replica catalog, and FTS3 transport are used to transport all files. All file metadata is declared to the new MetaCat[3] metadata service. All of these new components have been successfully tested at a scale equal to the expected output of the detector data acquisition system (~2-4 GB/s), and the expected network bandwidth out of the experimental hall. We present the procedure that was used to test and the results of the test.

Timm, Steven↗
DOE OSTI2023

Robust Real-Time Modeling of Distribution Systems with Data-Driven Grid-Wise Observability (Final Technical Report)

The overall objective of this project is to leverage existing and emerging sensor measurements to develop data-driven observability enhancement algorithms as well as robust state estimation and parameter identification techniques to enable real-time grid-wise monitoring and modeling of loads and distributed energy resources (DERs). The project resulted in a holistic framework with the following key components: 1) data-driven and machine learning-based grid-edge visibility enhancement, 2) robust branch-current-based state estimation (BCSE), and 3) robust real-time steady-state and dynamic-state modeling of loads and DERs. The research outcomes have provided utility companies better network visibility, higher-fidelity load/DER models, and more accurate assessment of DERs’ impacts, thus, facilitating the integration of renewable energy sources. The synergistic collaborative project among Iowa State University (ISU), Argonne National Laboratory (ANL), Electric Power Research Center (EPRC), SIEMENS Industry, Alliant Energy, Cedar Falls Utilities (CFU), and Maquoketa Valley Electric Cooperative (MVEC) to leverage the team’s extensive expertise and experience in power distribution systems, state estimation, online modeling and identification, and DER integrations. The proposed frameworks have been verified with industry adopted software and attempted integrated with existing tool wherever possible

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Semi-supervised Hybrid Machine Learning Framework for the Qualification of Resistance Spot Welds

• Industries requiring high structural integrity, including automotive, aerospace, and construction, place considerable significance on weld quality classification. • The inspection normally involves human expertise through predefined quality metrics that are subjective, error-prone, and time-intensive • The challenge to classification model development is the scarcity of labeled data and imbalanced distributions in the data that are labeled. • This work develops a new hybrid methodology that achieves clustering using KMeans++ together with supervised classification to overcome these challenges. • The ensemble-based classifiers were identified as optimal, with accuracy enhancements of up to 8% using the pseudo-labeled dataset. • The work provides practical insight into feature engineering and machine learning integration in industrial quality assurance applications.

Rogers, Jeremy K. [Savannah River National Laborat↗
DOE OSTI2020

Realistic Synthetic Distribution Grids: Summary of Validation Results

The presentation slides will walk us through the validation process and results of the synthetic distribution grid data. The synthetic grids for the distribution systems were developed as part of the ARPA-E funded GRIDDATA program project, SMART-DS. Three large scale datasets for US style distribution system were developed: 1) SAF: Santa Fe urban region, 2) GSO: Greensboro urban, semi-urban and industrial region, and 3) SFO: San Francisco Bay Area urban, semi-urban, industrial and rural areas. These datasets were validated using a three-pronged validation process, including statistical, operational and expert validation phases; the results of which have been summarized in this deck.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Gluon helicity from global analysis of experimental data and lattice QCD Ioffe time distributions

We perform a new global analysis of spin-dependent parton distribution functions with the inclusion of Ioffe time pseudodistributions computed in lattice QCD (LQCD), which are directly sensitive to the gluon helicity distribution, Δ g . These lattice data have an analogous relationship to parton distributions as do experimental cross sections, and can be readily included in global analyses. We focus in particular on the constraining capability of current LQCD data on the sign of Δ g at intermediate parton momentum fractions x , which was recently brought into question by analysis of data in the absence of parton positivity constraints. We find that present LQCD data cannot discriminate between positive and negative Δ g solutions, although significant changes in the solutions for both the gluon and quark sectors are observed. Published by the American Physical Society 2024

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Beyond traditional diagnostics: Identifying active galactic nuclei using spectral energy distribution fitting in DESI data

Active galactic nuclei (AGN) are typically identified through their distinctive X-ray or radio emissions, mid-infrared (MIR) colors, or emission lines. However, each method captures different subsets of AGN due to signal-to-noise (S/N) limitations, redshift coverage, and extinction effects, underscoring the necessity for a multiwavelength approach for comprehensive AGN samples. This study explores the effectiveness of spectral energy distribution (SED) fitting as a robust method for AGN identification. Using CIGALE optical-MIR SED fits on DESI Early Data Release galaxies, we compare SED-based AGN selection (AGNFRAC ≥ 0.1) with traditional methods including BPT diagrams, WISE colors, X-ray, and radio diagnostics. The SED fitting identifies ∼70% of narrow- and broad-line AGN and 87% of WISE-selected AGN. Incorporating high S/N WISE photometry reduces star-forming galaxy contamination from 62% to 15%. Initially, ∼50% of SED-AGN candidates are undetected by standard methods, but additional diagnostics classify ∼85% of these sources, revealing low-ionization nuclear emission-line regions and retired galaxies potentially representing evolved systems with weak AGN activity. Further spectroscopic and multiwavelength analysis will be essential to determine the true AGN nature of these sources. SED fitting provides complementary AGN identification, unifying multiwavelength AGN selections. This approach enables more complete – albeit somewhat contaminated – AGN samples, which are essential for upcoming large-scale surveys where spectroscopic diagnostics may be limited.

Seyfert↗
DOE OSTI2021

Dynamic Distribution of High-Rate Data Processing from CERN to Remote HPC Data Centers

The prompt reconstruction of the data recorded from the Large Hadron Collider (LHC) detectors has always been addressed by dedicated resources at the CERN Tier-0. Such workloads come in spikes due to the nature of the operation of the accelerator and in special high load occasions experiments have commissioned methods to distribute (spill-over) a fraction of the load to sites outside CERN. The present work demonstrates a new way of supporting the Tier-0 environment by provisioning resources elastically for such spilled-over workflows onto the Piz Daint Supercomputer at CSCS. Furthermore, this is implemented using containers, tuning the existing batch scheduler and reinforcing the scratch file system, while still using standard Grid middleware. ATLAS, CMS and CSCS have jointly run selected prompt data reconstruction on up to several thousand cores on Piz Daint into a shared environment, thereby probing the viability of the CSCS high performance computer site as on demand extension of the CERN Tier-0, which could play a role in addressing the future LHC computing challenges for the high luminosity LHC.

97 MATHEMATICS AND COMPUTING↗