Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Albedo Data Sets for Bifacial PV Systems: Preprint

For use by the PV and financial communities to better estimate the performance and to reduce the risk of bifacial PV systems, data sets of ground albedo and associated meteorological data were developed by using existing measurement network data and data contributed by the PV industry. The data sets include time-series data as well as summary information of tabular monthly and yearly data and plots of monthly and hourly albedo values. Complete information is presented in a user’s guide and data are available for download from NREL’s DuraMAT website.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗
DOE OSTI2021

Measured and satellite-derived albedo data for estimating bifacial photovoltaic system performance

The albedo of the ground surface is an important factor in the cost-effectiveness of a bifacial photovoltaic (PV) system. To improve the availability of reliable albedo data, datasets of ground albedo and associated meteorological data were developed by using existing measurement network data and data measured by the PV industry. The measured datasets include time-series data as well as summary information of tabular monthly and yearly data and plots of monthly and hourly albedo values. Satellite-derived values of albedo are available from the National Solar Radiation Data Base (NSRDB). The NSRDB albedos were compared to the measured albedos for Surface Radiation budget (SURFRAD) network locations for the period 2001–2017, and the mean bias difference results were from -0.044 to +0.056. Overall, these differences are greater than the albedo measurement uncertainty of ±0.02; consequently, the NSRDB albedos should be used with caution for estimating the performance of bifacial PV systems. Differences between SURFRAD and NSRDB albedos are attributed to the NSRDB method for determining albedo and to the ground surfaces within the NSRDB 4 km spatial resolution pixel consisting of a mixture of surface types rather than just the single surface types viewed by the albedometers at the SURFRAD stations.

14 SOLAR ENERGY↗
DOE OSTI2021

Adaptive neural network management system

A method and computer system for managing a neural network. Data is sent into an input layer in a portion of layers of nodes in the neural network. The data moves on an encode path through the portion such that an output layer in the portion outputs encoded data. The encoded data is sent into the output layer on a decode path through the portion back to the input layer to obtain a reconstruction of the data by the input layer. A determination is made as to whether an undesired amount of error has occurred in the output layer based on the data sent into the input layer and the reconstruction of the data. A number of new nodes is added to the output layer when a determination is present that the undesired amount of the error occurred, enabling reducing the error using the number of the new nodes.

Draelos, Timothy J.↗
DOE OSTI2021

Review of Intrusion Detection Methods and Tools for Distributed Energy Resources

Recent trends in the growth of distributed energy resources (DER) in the electric grid and newfound malware frameworks that target internet of things (IoT) devices is driving an urgent need for more reliable and effective methods for intrusion detection and prevention. Cybersecurity intrusion detection systems (IDSs) are responsible for detecting threats by monitoring and analyzing network data, which can originate either from networking equipment or end-devices. Creating intrusion detection systems for PV/DER networks is a challenging undertaking because of the diversity of the attack types and intermittency and variability in the data. Distinguishing malicious events from other sources of anomalies or system faults is particularly difficult. New approaches are needed that not only sense anomalies in the power system but also determine causational factors for the detected events. In this report, a range of IDS approaches were summarized along with their pros and cons. Using the review of IDS approaches and subsequent gap analysis for application to DER systems, a preliminary hybrid IDS approach to protect PV/DER communications is formed in the conclusion of this report to inform ongoing and future research regarding the cybersecurity and resilience enhancement of DER systems.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2020

On Performance Prediction of Big Data Transfer in High-performance Networks

Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.

Liu, Wuji↗
DOE OSTI2021

ADMS Test Bed Updates

This webinar will present results from a joint project with utility partner Xcel Energy in which we evaluated their ADMS application for volt-var optimization using different levels of model quality. We were able to help Xcel Energy understand the trade-offs of telemetry measurements and model quality when managing a feeder's voltage profile to maximize energy conservation. We simulated scenarios with varying levels of model quality and measurement density to evaluate Xcel's options for best using its ADMS. The results help Xcel and other utilities understand how network data affects voltage management as grid operations see continued growth in solar photovoltaic (PV) systems and electric vehicles (EVs).

ADMS↗
DOE OSTI2022

ThunderSecure: deploying real-time intrusion detection for 100G research networks by leveraging stream-based features and one-class classification network

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks (40/100G) across distributed sites for processing, storage, and analysis. Like general purpose networks, research networks experience intrusions. However, monitoring anomalies in such high-speed network traffics is challenging given current cyber-infrastructure. Moreover, traditional network intrusion detection systems (NIDS) are signature based. However, anomaly patterns are difficult to define and that rulesets are often not updated frequently enough to reflect the changes of attack behaviors. We present ThunderSecure, a high-throughput, unsupervised learning-based intrusions detection system for 100G research networks. ThunderSecure implements an efficient packet processing and detection pipeline using multi-cores and GPUs. It extracts statistical and temporal features from real-time network data streams and feeds them to a one-class anomaly detection network. A baseline of normal distribution will be created based on the training observation. Testing traffic deviated from the learned profile will be marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets mirrored from two 100G network connections at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. Results show that ThunderSecure can recognize science data traffic captured long after the training and made nearly certain detection on the segment of the streams where anomalous flows were injected.

100G research network↗
DOE OSTI2022

Game-Theoretic Strategies for Cyber-Physical Infrastructures Under Component Disruptions

In this work, networked infrastructures of recursively defined systems composed of discrete cyber and physical components are considered. The components of basic systems at the finest levels can be disrupted by cyber or physical means, and can be reinforced to survive at certain costs. A problem of ensuring the infrastructure performance is formulated as a game between a provider and an attacker, who probabilistically choose components to reinforce and attack, respectively. The disruptions of this infrastructure are characterized using the aggregate failure correlation function that specifies the conditional failure probability of the infrastructure given the failure of an individual system at that level. The survival probabilities of basic systems satisfy simple product-form, first-order differential equations expressed in terms of the multiplier functions. The utility functions of the provider and attacker are composed of the reward and cost terms, both expressed in terms of the component reinforcement and attack probabilities. The Nash equilibrium of this game is characterized, along with the sensitivity functions of the survival probabilities of basic systems that highlight their dependence individually on the cost-benefit terms, the correlation functions, and the multiplier functions. These results are illustrated using simplified models of a distributed cloud servers infrastructure, a 5G data network infrastructure, a high performance computing federation, and a smart energy grid infrastructure.

42 ENGINEERING↗
DOE OSTI2025

Adaptive continuity-preserving simplification of street networks

Street network data is widely used to study human-based activities and urban structure. Often, these data are geared towards transportation applications, which require highly granular, directed graphs that capture the complex relationships of potential traffic patterns. While this level of network detail is critical for certain fine-grained mobility models, it represents a hindrance for studies concerned with the morphology of the street network. For the latter case, street network simplification — the process of converting a highly granular input network into its most simple morphological form — is a necessary, but highly tedious preprocessing step, especially when conducted manually. In this manuscript, we develop and present a novel adaptive algorithm for simplifying street networks that is both fully automated and able to mimic results obtained through a manual simplification routine. The algorithm — available in the neatnet Python package — outperforms current state-of-the-art procedures when comparing those methods to manually, human-simplified data, while preserving network continuity.

Python↗
DOE OSTI2021

Modeling Data Flows with Network Calculus in Cyber-Physical Systems: Enabling Feature Analysis for Anomaly Detection Applications

The electric grid is becoming increasingly cyber-physical with the addition of smart technologies, new communication interfaces, and automated grid-support functions. Because of this, it is no longer sufficient to only study the physical system dynamics, but the cyber system must also be monitored as well to examine cyber-physical interactions and effects on the overall system. To address this gap for both operational and security needs, cyber-physical situational awareness is needed to monitor the system to detect any faults or malicious activity. Techniques and models to understand the physical system (the power system operation) exist, but methods to study the cyber system are needed, which can assist in understanding how the network traffic and changes to network conditions affect applications such as data analysis, intrusion detection systems (IDS), and anomaly detection. In this paper, we examine and develop models of data flows in communication networks of cyber-physical systems (CPSs) and explore how network calculus can be utilized to develop those models for CPSs, with a focus on anomaly and intrusion detection. This provides a foundation for methods to examine how changes to behavior in the CPS can be modeled and for investigating cyber effects in CPSs in anomaly detection applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

ESnet Requirements Review Program Through the IRI Lens: A Meta-Analysis of Workflow Patterns Across DOE Office of Science Programs (Final Report)

The Department of Energy (DOE) ensures America’s security and prosperity by addressing its energy, environmental, and nuclear challenges through transformative science and technology solutions. The DOE’s Office of Science (SC) delivers groundbreaking scientific discoveries and major scientific tools that transform our understanding of nature and advance the energy, economic, and national security of the United States. The SC’s programs advance DOE mission science across a wide range of disciplines and have developed the research infrastructure needed to remain at the forefront of scientific discovery. The DOE SC’s world-class research infrastructure — exemplified by the 28 SC scientific user facilities — provides the research community with premier observational, experimental, computational, and network capabilities. Each user facility is designed to provide unique capabilities to advance core DOE mission science for its sponsor SC program and to stimulate a rich discovery and innovation ecosystem. Research communities gather and flourish around each user facility, bringing together diverse perspectives. A hallmark of many facilities is the large population of students, postdoctoral researchers, and early-career scientists who contribute as full-fledged users. These facility staff and users collaborate over years to devise new approaches to utilizing the user facility’s core capabilities. The history of the SC user facilities has many examples of wildly inventive researchers challenging operational orthodoxy to pioneer new vistas of discovery; for example, the use of the synchrotron X-ray light sources for study of proteins and other large biological molecules. This continual reinvention of the practice of science — as users and staff forge novel approaches expressed in research workflows — unlocks new discoveries and propels scientific progress. Within this research ecosystem, the high-performance computing (HPC) and networking user facilities stewarded by SC’s Advanced Scientific Computing Research (ASCR) program play a dynamic cross-cutting role, enabling complex workflows demanding high performance data, networking, and computing solutions. The DOE SC’s three HPC user facilities and the Energy Sciences Network (ESnet) high-performance research network serve all of the SC’s programs as well as the global research community. Argonne Leadership Computing Facility (ALCF), the National Energy Research Scientific Computing Center (NERSC), and Oak Ridge Leadership Computing Facility (OLCF) conceive, build, and provide access to a range of supercomputing, advanced computing, and large-scale data-infrastructure platforms, while ESnet interconnects DOE SC research infrastructure and enables seamless exchange of scientific data. All four facilities operate testbeds to expand the frontiers of computing and networking research. Together, the ASCR facilities enterprise seeks to understand and meet the needs and requirements across SC and DOE domain science programs and priority efforts, highlighted by the formal requirements reviews (RRs) methodology. In recent years, the research communities around the SC user facilities have begun experimenting with and demanding solutions integrated with HPC and data infrastructure. This rise of integrated-science approaches is documented in many community and high-level government reports. At the dawn of the era of exascale science and the acceleration of artificial intelligence (AI) innovation, there is a broad need for integrated computational, data, and networking solutions. In response to these drivers, DOE has developed a vision for an Integrated Research Infrastructure (IRI): To empower researchers to meld DOE’s world-class research tools, infrastructure, and user facilities seamlessly and securely in novel ways to radically accelerate discovery and innovation.

42 ENGINEERING↗
DOE OSTI2022

Data-Driven Chance-Constrained Design of Voltage Droop Control for Distribution Networks: Preprint

This paper addresses the design of local control methods for voltage control in distribution networks with high level of distributed energy resources (DERs). The designed control methods adapt the active and reactive power output of distributed energy resources proportional to the deviation of the local measured voltage magnitudes from a reference voltage, which is referred to as droop control. Thus, the design focuses on determining the droop characteristics which satisfy network-wide voltage magnitude constraints. The uncertainty and variability of DERs renders the design of optimal droop controls very challenging. Hence, this paper proposes chance constraints to limit the risk from intermittent DERs, by designing droop control coefficients that guarantee the satisfaction of network operational constraints with a specific probability. In addition, the proposed approach relies entirely on historical data rather than assuming knowledge of the probability distributions that characterize the uncertainty of DERs. The efficacy of the proposed method is demonstrated on a 37-bus distribution feeder.

chance-constrained optimization↗
DOE OSTI2026

Mauka Energy FEVER Tool Dataset

Mauka Energy’s dataset, developed under the Forestry Electric Vehicle Energy Routing (FEVER) project and funded by the U.S. Department of Energy’s Small Business Innovation Research program, is a high-resolution geospatial resource designed to support energy modeling for electric log trucks in complex forestry environments. The dataset integrates detailed spatial and road network data to enable accurate simulation of vehicle performance across varied terrain. At its core, the dataset incorporates lidar-derived elevation models, road alignments, and surface classifications from Oregon State University’s McDonald-Dunn Research Forest. These data capture fine-scale variations in slope, curvature, and surface conditions across forest road systems, allowing for vehicle-level analysis of energy consumption and recovery. The dataset also includes data collected on the surrounding public and private road networks in Benton County, Oregon, used in real-world haul routes. These connecting segments provide critical context for modeling transitions between forest operations and regional transportation infrastructure, incorporating attributes such as grade profiles, elevation change, and speed constraints. This combined dataset underpins the development of Mauka Energy’s rolldown tool, which quantifies energy use and regenerative braking potential on downhill and variable-grade segments. By leveraging high-resolution terrain and road data, the FEVER project enables more accurate assessment of electric vehicle feasibility and performance in forestry applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Achieving Cyber-Resilience for Power Systems using a Learning, Model-Assisted Blockchain Framework

The secure integration and management of distributed energy resources (DER) and power aggregators in the electric grid requires secure communications and a physics-aware Command and Control (C2) strategy. A Blockchain (BC)-based overlay network was developed to provide a security layer for the existing power grid network that mitigates risks in current and legacy network and C2 protocols. By integrating a Model-Assisted Machine Learning (MAML) framework with a Secure Blockchain Overlay Network (SBON) a defense-in-depth strategy was achieved. In our approach, the MAML framework leveraged a smart contract framework to gather network data and learn the dynamics of DER to develop detection strategies for attacks targeting sensors and actuators used by DER. The MAML framework learned dynamical systems models for individual DERs to detect sensor attacks. For DER we utilized a Digital Twin (DT) to accelerate the learning process for a model resistant to stealthy attacks. The project created DT for PV inverters and BESS. The DTs were coupled with a model-assisted, data-driven learning of DER behavior. Specifically, we evaluated architectures for model-based learning with model-free fine-tuning. Additionally, differential privacy techniques were used to obfuscate data, while still allowing the computation of attack detection results based on obfuscated data. The SBON developed leverages a private permissioned blockchain network orchestrated with the Hyperledger Fabric framework. To connect the cyber world, which orchestrates the blockchain fabric, and the physical world where the power network resides, we developed a system implementation to enable the secure interaction of the physical world and the abstracted blockchain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Efficient mapping between void shapes and stress fields using Deep Convolutional Neural Networks with sparse data

Establishing fast and accurate structure-to-property relationships is an important component in the design and discovery of advanced materials. Physics-based simulation models like the finite element method (FEM) are often used to predict deformation, stress, and strain fields as a function of material microstructure in material and structural systems. Such models may be computationally expensive and time intensive if the underlying physics of the system is complex. This limits their application to solve inverse design problems and identify structures that maximize performance. In such scenarios, surrogate models are employed to make the forward mapping computationally efficient to evaluate. However, the high dimensionality of the input microstructure and the output field of interest often renders such surrogate models inefficient, especially when dealing with sparse data. Deep convolutional neural network (CNN) based surrogate models have shown great promise in handling such high-dimensional problems. In this paper, a single ellipsoidal void structure under a uniaxial tensile load represented by a linear elastic, high-dimensional and expensive-to-query, FEM model. We consider two deep CNN architectures, a modified convolutional autoencoder framework with a fully connected bottleneck and a UNet CNN, and compare their accuracy in predicting the von Mises stress field for any given input void shape in the FEM model. Additionally, a sensitivity analysis study is performed using the two approaches, where the variation in the prediction accuracy on unseen test data is studied through numerical experiments by varying the number of training samples from 20 to 100.

surrogate modeling; convolutional neural networks;↗
DOE OSTI2023

GNATFinder

SAND2023-05510O GNATFinder is a software tool for extracting graphical neural activity threads (GNATs) from spiking neural networks. GNATFinder efficiently extracts GNATs from spiking neural network data. A GNAT is a graph that represents the causal relationships between individual spikes in a spiking network using a combination of the network connectivity structure and the timing of individual spikes. The software allows for identifying similar, repeating threads that often reappear in a dataset. Using quadtree data structures to recursively subdivide the time interval over which spikes were produced by the spiking network, GNATFinder allows for efficient pairwise comparisons of spike times to determine the degree of causal relatedness among all pairs of spikes in a dataset. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2025

Using Neural Networks to Identify Mixture Components in Hyperspectral Reflectance Data

Neural networks have been employed to identify materials of interest from hyperspectral data (generally imagery) based on their unique spectral signatures. This approach assumes that there is a single material that is standing out from the rest of the spectrum to be identified. However, pixels often contain more than one material, or a material of interest may itself be a mixture of multiple materials. Neural networks are only as good as the data used to train them, and it takes a great deal of work in the laboratory to identify, make, and measure all potential mixtures of interest. Thus, researchers often calculate synthetic spectra using algorithms with varying degrees of fidelity to the physics that govern the interactions between light and multiple materials. In this work, we have (1) adapted a neural network designed to identify mixture components from Raman spectroscopy to work with visible to near‐infrared reflectance data and (2) tested three common mixture algorithms to determine the most accurate and least computationally expensive method to build synthetic training datasets. With our initial test dataset, we have achieved accuracies of > 90% and found that the synthetic training dataset produced using the Hapke mixture model provides the best results.

99 GENERAL AND MISCELLANEOUS↗