Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Solar-to-Grid Public Data File for Utility-scale (UPV) and Distributed Photovoltaics (DPV) Generation, Capacity Credit, and Value

Lawrence Berkeley National Laboratory (Berkeley Lab) estimates hourly project-level generation data for utility-scale solar projects and hourly county-level generation data for residential and non-residential distributed photovoltaic (PV) systems in the seven organized wholesale markets and 10 additional Balancing Areas. To encourage its broader use, Berkeley Lab has made this data file public here at OEDI. The public project-level dataset is updated annually with data from the previous calendar year. For more information about the research project, including a technical report, briefing material, visualizations, and additional data, please visit the project homepage linked in this submission. A newer version of the data exists and can be found linked in the resources of this submission under "Solar-to-Grid Public Data File Updated 2021".

annual solar value↗
DOE OSTI2024

Generalization Across Experimental Parameters in Neural Network Analysis of High-Resolution Transmission Electron Microscopy Datasets

Neural networks are promising tools for high-throughput and accurate transmission electron microscopy (TEM) analysis of nanomaterials, but are known to generalize poorly on data that is “out-of-distribution” from their training data. Given the limited set of image features typically seen in high-resolution TEM imaging, it is unclear which images are considered out-of-distribution from others. Here, we investigate how the choice of metadata features in the training dataset influences neural network performance, focusing on the example task of nanoparticle segmentation. We train and validate neural networks across curated, experimentally collected high-resolution TEM image datasets of nanoparticles under various imaging and material parameters, including magnification, dosage, nanoparticle diameter, and nanoparticle material. Overall, we find that our neural networks are not robust across microscope parameters, but do generalize across certain sample parameters. Additionally, data preprocessing can have unintended consequences on neural network generalization. Our results highlight the need to understand how dataset features affect deployment of data-driven algorithms.

42 ENGINEERING↗
DOE OSTI2022

A Hybrid Data-Driven and Model-Based Anomaly Detection Scheme for DER Operation: Preprint

This paper proposes a hybrid data and model-based anomaly detection for securing the operation of distributed energy resources (DERs) in distribution grids. Data-driven autoencoders (AE) are set up at the edge level by taking local DER data and detect anomalous operations by leveraging the reconstruction ability. In parallel, model-based state estimation (SE) is running at the system level by taking system models and measurements, the anomalies are identified by analyzing the measurements residual. The hybrid scheme preserves the benefits of both data-driven and model-based analysis and thus improves the robustness and accuracy of anomaly detection. It can be established by getting full use of the existing infrastructures in distribution grids. Numerical tests on a realistic distribution feeder in Southern California highlight the effectiveness as well as benefits of the proposed scheme.

anomaly detection↗
DOE OSTI2024

Perspectives on AI Architectures and Codesign for Earth System Predictability

Abstract Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, laboratory, modeling, and analysis activities, called model experimentation (ModEx). BER’s ModEx is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the AI Architectures and Codesign session and associated outcomes. The AI Architectures and Codesign session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including 1) DOE high-performance computing (HPC) systems, 2) cloud HPC systems, and 3) edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this codesign area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as 1) reimagining codesign, 2) data acquisition to distribution, 3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with Earth system modeling and simulation, and 4) AI-enabled sensor integration into Earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper. Significance Statement This study aims to provide perspectives on AI architectures and codesign approaches for Earth system predictability. Such visionary perspectives are essential because AI-enabled model-data integration has shown promise in improving predictions associated with climate change, perturbations, and extreme events. Our forward-looking ideas guide what is next in codesign to enhance Earth system models, observations, and theory using state-of-the-art and futuristic computational infrastructure.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Perspectives on AI Architectures and Co-design for Earth System Predictability

Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, laboratory, modeling, and analysis activities, called model experimentation (ModEx). BER’s ModEx is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the AI Architectures and Codesign session and associated outcomes. The AI Architectures and Codesign session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including 1) DOE high-performance computing (HPC) systems, 2) cloud HPC systems, and 3) edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this codesign area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as 1) reimagining codesign, 2) data acquisition to distribution, 3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with Earth system modeling and simulation, and 4) AI-enabled sensor integration into Earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper.

58 GEOSCIENCES↗
DOE OSTI2024

Enabling DER visibility using a distributed dissemination network

The electrical grid is currently undergoing a series of rapid transformational changes that have resulted in the introduction of new actors and operational schemes that have fragmented the data and control planes. To help address the issue, this paper describes the implementation of a sensor-oriented, distributed data dissemination network that seeks to eliminate data silos. The implementation is based on the DGSS architecture previously described in [1]. The developed product seeks to facilitate the seamless integration of multi-operator, multi-origin, multi-domain sensor data by using a distributed systems approach. The proposed solution decouples the sensor’s data streams from the application-specific infrastructure and migrates them into a software-defined databus that can be configured to suit the end application’s demands. To further validate DGSS capabilities, a DER oriented use case has been developed.

Sensor Dissemination Networks, Enhanced DER visibi↗
DOE OSTI2021

Solar-to-Grid Public Data File for Utility-scale (UPV) and Distributed Photovoltaics (DPV) Generation, Capacity Credit, and Value for 2012-2020

Lawrence Berkeley National Laboratory (Berkeley Lab) estimates hourly project-level generation data for utility-scale solar projects and hourly county-level generation data for residential and non-residential distributed photovoltaic (PV) systems in the seven organized wholesale markets and 10 additional Balancing Areas. To encourage its broader use, Berkeley Lab has made this data file public here at OEDI, covering the years 2012-2020. The public project-level dataset is updated annually with data from the previous calendar year. For more information about the research project, including a technical report, briefing material, visualizations, and additional data, please visit the project homepage linked in this submission.

annual solar value↗
DOE OSTI2023

Non-parametric projections of national income distribution consistent with the Shared Socioeconomic Pathways

Understanding and projecting income distributions within countries and regions is important to understanding consumption trends and the distributional consequences of climate impacts and responses. Several global, country-level projections of income distribution are available but most project only the Gini coefficient (a summary statistic of the distribution) or utilize the Gini along with the assumption of a lognormal distribution. We test the lognormal assumption and find that it typically underestimates income in the highest deciles and over-estimates it in others. We find that a new model based on two principal components of national time series data for income distribution provides a better fit to the data for all deciles, especially for the highest and lowest. We also construct a projection model in which the first principal component is driven by the Gini coefficient and the second captures deviations from this relationship. We use the model to project income distribution by decile for all countries for the five shared socioeconomic pathways. We find that inequality is consistently higher than projections based on the Gini and the lognormal functional form, with some countries reaching ratios of the highest to lowest income deciles that are almost three times their value using the lognormal assumption.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

A Hybrid Data-Driven and Model-Based Anomaly Detection Scheme for DER Operation

This paper proposes a hybrid data and model-based anomaly detection scheme to secure the operation of distributed energy resources (DERs) in distribution grids. Data-driven autoencoders are set up at the edge device level and they use local DER operational data as inputs. The abnormal statuses are detected by analyzing reconstruction errors. In parallel, modelbased state estimation (SE) is set up at the central level and it uses system-wide models and measurements as data inputs. The anomalies are identified by analyzing measurement residuals. The hybrid scheme preserves the benefits of both data-driven and model-based analyses and thus improves the robustness and the accuracy of anomaly detection. Numerical tests based on the model of a real distribution feeder in Southern California highlight the proposed scheme's effectiveness and benefits.

anomaly detection↗
DOE OSTI2021

Subfield crop yields and temporal stability in thousands of US Midwest fields

Understanding subfield crop yields and temporal stability is critical to better manage crops. Several algorithms have proposed to study within-field temporal variability but they were mostly limited to few fields. In this study, a large dataset composed of 5520 yield maps from 768 fields provided by farmers was used to investigate the influence of subfield yield distribution skewness on temporal variability. The data are used to test two intuitive algorithms for mapping stability: one based on standard deviation and the second based on pixel ranking and percentiles. The analysis of yield monitor data indicates that yield distribution is asymmetric, and it tends to be negatively skewed (p < 0.05) for all of the four crops analyzed, meaning that low yielding areas are lower in frequency but cover a larger range of low values. The mean yield difference between the pixels classified as high-and-stable and the pixels classified as low-and-stable was 1.04 Mg ha –1 for maize, 0.39 Mg ha –1 for cotton, 0.34 Mg ha –1 for soybean, and 0.59 Mg ha —1 for wheat. The yield of the unstable zones was similar to the pixels classified as low-and-stable by the standard deviation algorithm, whereas the two-way outlier algorithm did not exhibit this bias. Furthermore, the increase in the number years of yield maps available induced a modest but significant increase in the certainty of stability classifications, and the proportion of unstable pixels increased with the precipitation heterogeneity between the years comprising the yield maps.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Orientation microscopy–assisted grain boundary analysis for protonic ceramic cell electrolytes

Abstract Grain boundaries in protonic ceramic cell (PCC) electrolytes hinder proton transport, reducing interfacial conductivity. In multicomponent PCC electrolytes, the inclusion of sintering aids further accentuates the complexity of grain boundaries. In this study, we synthesize nanocrystalline BaCe 0.4 Zr 0.4 Y 0.1 Yb 0.1 O 3− δ thin films via pulsed laser deposition and analyze their grain boundary character distributions using orientation data collected by precession electron diffraction technique. The results reveal an anisotropic distribution of grain boundary characters, with notably high populations of 180°‐tilt and twist grain boundaries. These findings provide critical insights into identifying the predominant grain boundaries in this PCC electrolyte material, assessing the vast five‐dimensional grain boundary space.

Patel, Sooraj [School of Aerospace and Mechanical ↗
DOE OSTI2024

GDM (Grid Data Models) [SWR-24-31]

Grid Data Models (GDM) is a collection of pydantic data models for power distribution and transmission system components. These data models can be easily serialized and deserialized. GDM provides a standard interface for accessing power system component data with built in validation. See also: https://pypi.org/project/grid-data-models/

Duwadi, Kapil↗
DOE OSTI2020

Detecting Rain–Snow-Transition Elevations in Mountain Basins Using Wireless Sensor Networks

Here, to provide complementary information on the hydrologically important rain–snow-transition elevation in mountain basins, this study provides two estimation methods using ground measurements from basin-scale wireless sensor networks: one based on wet-bulb temperature T wet and the other based on snow-depth measurements of accumulation and ablation. With data from 17 spatially distributed clusters (178 nodes) from two networks, in the American and Feather River basins of California’s Sierra Nevada, we analyzed transition elevation during 76 storm events in 2014–18. A T wet threshold of 0.5°C best matched the transition elevation defined by snow depth. Transition elevations using T wet in upper elevations of the basins generally agreed with atmospheric snow level from radars located at lower elevations, while radar snow level was ~100 m higher due to snow-level lowering on windward mountainsides during orographic lifting. Diurnal patterns of the difference between transition elevation and radar snow level were observed in the American basin, related to diurnal ground-temperature variations. However, these patterns were not found in the Feather basin due to complex terrain and higher uncertainties in transition-elevation estimates. The American basin tends to exhibit 100-m-higher transition elevations than does the Feather basin, consistent with the Feather basin being about 1° latitude farther north. Transition elevation averaged 155 m higher in intense atmospheric river events than in other events; meanwhile, snow-level lowering was enhanced with a 90-m-larger difference between radar snow level and transition elevation. On-the-ground continuous observations from distributed sensor networks can complement radar data and provide important ground truth and spatially resolved information on transition elevations in mountain basins.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Geophysical and Environmental Monitoring Data, Lower Watershed, Teller Road Mile Marker 27, Seward Peninsula, Alaska, 2017-2019

This data set contains geophysical and environmental monitoring data acquired between September 2017 and 2019 at the lower Teller watershed, Seward Peninsula, Alaska. Geophysical data comprises processed resistivity data, acquired daily between Spring and Fall of 2018 and 2019, and a baseline measurement of Fall 2017. The environmental monitoring data comprises depth resolved, distributed soil moisture and soil temperature data. These measurements were obtained used Decagon 5TE sensors placed at 0.1 m, 0.2 m , 0.3 m, and 0.4 m depth. In addition, temperature data at 0.5 m, 1.0 m, and 1.5 m were acquired using Hobo temperature sensors. Data were collected to improve our understanding of the hydrological response of discontinuous permafrost systems, in particular focusing on multi-annual dynamics and short term disturbances, such as snowmelt or precipitation events. The electrical resistivity tomography (ERT) monitoring data are included as processed resistivity models, with model cells x dimension equal to the distance along the profile, and model z dimension being elevation. The start and end point of the transect are (UTM Zone 3N): E 454881.66 m, N 7178949.45 m, and E 454772.60, N 7178885.81. Locations of the soil moisture and temperature sensors are provided in the data package. This dataset is discussed in detail in the Uhlemann, S. et al 2021 paper listed in the references. This dataset includes one *.pdf user guide and 315 *.csv data files included within four zipped files.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗