SEARCH · Engineering Papers

Results for “Network data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Detecting anomalous packets in network transfers: investigations using PCA, autoencoder and isolation forest in TCP

Large-scale scientific workflows rely heavily on high-performance file transfers. These transfers require strict quality parameters such as guaranteed bandwidth, no packet loss or data duplication. To have successful file transfers, methods such as predetermined thresholds and statistical analysis need to be done to determine abnormal patterns. Network administrators routinely monitor and analyze network data for diagnosing and alleviating these, making decisions based on their experience. However, as networks grow and become complex, monitoring large data files and quickly processing them, makes it improbable to identify errors and rectify these. Abnormal file transfers have been classified by simply setting alert thresholds, via tools such as PerfSonar and TCP statistics (Tstat). This paper investigates the feasibility of unsupervised feature extraction methods for identifying network anomaly patterns with three unsupervised classification methods—principal component analysis, autoencoder and isolation forest. Here, we collect file transfer statistics from two experiment sets—synthetic iPerf generated traffic and 1000 Genome workflow runs, with synthetically introduced anomalies. Our results show that while PCA and a simple autoencoder finds it difficult to detect clusters, the tree-variant isolation forest is able to identify anomalous packets by breaking down TCP traces into tree classes early.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Eliater: a Python package for estimating outcomes of perturbations in biomolecular networks

We introduce Eliater, a Python package for estimating the effect of perturbation of an upstream molecule on a downstream molecule in a biomolecular network. The estimation takes as input a biomolecular network, observational biomolecular data, and a perturbation of interest, and outputs an estimated quantitative effect of the perturbation. We showcase the functionalities of Eliater in a case study of Escherichia coli transcriptional regulatory network.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Electric Utility Industry Standards Landscape

The electric utility industry relies on robust communication protocols to manage complex electrical grid data. The inherent networked nature of electrical grids, coupled with the radial structure of the “last mile” portion delivering power to end-use customers, presents difficulties in describing electrical models using simple data constructs. The paper provides an overview of communication protocols that address electric utility data, including grid data, and classifies these protocols through identification key characteristics that make them suitable for different electric grid data domains. Because of their variety, grid edge devices and their associated dedicated protocols are assessed by groups: those primarily designed for energy production and storage, those related to flexible loads, and those related to electric vehicles. The intent of this report is to provide guidance for stakeholders to navigate the challenges posed by the numerous overlapping protocols available to address electric grid data. Although further industry review, refinement, and validation of the categorization of these protocols is recommended, the authors propose this categorization as a start to improve electric grid awareness and understanding. In addition, this report includes three recommended industry actions regarding protocols to improve communications on the electric grid.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

Reference Site Conditions for Floating Wind Arrays in the United States

17 WIND ENERGY↗

DOE OSTI2024

Collaborative Research: Enhancing Laser-Based Ion Sources with High Data Rate Techniques

This collaborative research project focuses on leveraging advanced machine learning techniques to analyze and optimize data from high-repetition-rate laser experiments. The main goal is to apply modern computing hardware, customized data acquisition firmware/software, and machine learning approaches to improve data analysis and experimental control. The project also explores how methodology can be developed on smaller-scale experimental setups and then translated to larger facilities within DOE's LaserNetUS network. With extensive data collection and modeling, the research aims to predict and optimize experimental parameters to enhance performance and efficiency.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Reference Site Condition Datasets for Floating Wind Arrays in the United States

Floating offshore wind farm design is highly site-specific, requiring detailed information about the specific conditions of a project area for realistic design studies. Unfortunately, publicly available site condition data for potential floating offshore wind project sites in the United States is scarce. To support U.S. offshore wind research, we developed reference site condition datasets, including metocean and seabed information, for four potential floating wind project areas in the U.S.: Humboldt Bay, Morro Bay, the Gulf of Maine, and the Gulf of Mexico. These datasets were compiled using publicly available data. Our metocean analysis, covering wind, waves, and surface currents, utilized measurement data from 2000 to 2020. Sources included the National Renewable Energy Laboratory’s National Offshore Wind Dataset for wind data, National Data Buoy Center buoys for wave data, and the High Frequency Radar Network for surface currents. These data were integrated into hourly time series used to compute extreme return periods up to 500 years, monthly statistics, and joint probability clusters for fatigue analysis. Soil conditions were evaluated using the usSEABED database and bathymetry grids were interpolated from the NCEI Digital Elevation Model Global Mosaic. Further information on the datasets and how they were created can be found in: Biglu, M., M. Hall, E. Lozon, S. Housner. 2024. Reference Site Conditions for Floating Wind Arrays in the United States. Golden, CO: National Renewable Energy Laboratory (NREL). NREL/TP-5000-89897. The data are also available at: https://github.com/FloatingArrayDesign/SiteConditions The content of each dataset is as follows: _NOW23_wind.txt: Hourly NOW-23 wind data up to a height of 400 meter. _metocean_1hr.txt: Hourly time series including wind, wave, surface current and temperature data. _Summary.xlsx: Metocean data, including extreme values, joint probability distributions and monthly statistics. _usSEABED_soil.csv: Extract of the usSEABED database for this specific site. _bathymetry_200m.txt (and 500m, 1000m): Gridded seabed depth data.

16 TIDAL AND WAVE POWER↗

DOE OSTI2022

Unraveling the effects of management and climate on carbon fluxes of U.S. croplands using the USDA Long-Term Agroecosystem (LTAR) network

Understanding the carbon fluxes and dynamics from a broad range of agricultural systems has the potential to improve our ability to increase carbon sequestration while maintaining crop yields. Short-term, single-location studies have limited applicability, but long-term data from a network of many locations can provide a broader understanding across gradients of climate and management choices. Here we examine eddy covariance measured carbon dioxide (CO 2 ) fluxes from cropland sites across the United States Department of Agriculture's Long-Term Agroecosystem Research (LTAR) network. The dataset was collected between 2001 and 2020, spanning 13 sites for a total of 182 site-years. Average seasonal patterns of net ecosystem CO 2 exchange (NEE), gross primary productivity (GPP), and ecosystem respiration (R eco ) were determined, and subsequent regression analysis on these “flux climatologies” was used to identify relationships to mean annual temperature (MAT), mean annual precipitation (MAP), cropping systems, and management practices. At rainfed sites, carbon fluxes were better correlated with MAP (r2 ≤ 0.5) than MAT (r2 ≤ 0.22). Net carbon balance was different among cropping systems (p < 0.001), with the greatest net carbon uptake occurring in sugarcane (Saccharum spp. hybrids) and the least in soybean (Glycine max) fields. Crop type had a greater effect on carbon balance than irrigation management at a Nebraska site. Across cropping systems, grain crops often had higher GPP and were more likely to have net uptake when compared to legume crops. This multi-site analysis highlights the potential of the LTAR network to further carbon flux research using eddy covariance measurements.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Distributed Energy Resource Cybersecurity Framework and Cyber Range Integration

Distributed energy resource (DER) systems feature complex, data-driven communications networks that require careful system coordination and constant vigilance to ensure that grid assets are secure. Because DERs are an important component of the decarbonization strategy, agencies need to secure energy data that could implicate issues of national security if compromised. To help federal energy managers assess, monitor, and manage cybersecurity while achieving decarbonization, the National Renewable Energy Laboratory's (NREL's) Distributed Energy Resource Cybersecurity Framework (DER-CF) offers a comprehensive, web-based assessment tool focusing on cyber governance or policies, technical management, and physical security. The DER-CF currently presents users with a series of pertinent cybersecurity questions that are used to generate a site-specific report and recommendations. This paper outlines a plan to integrate the DER-CF with another key asset-NREL's cyber range-to visualize cybersecurity resilience and compliance and to enhance the usability and accessibility of the DER-CF for federal facility energy managers and planners. This integration will result in a visualization environment to interpret and interact with compliance data. Its development will include regular conversations with stakeholders to assess the effectiveness of these efforts, refine the visualization capability, and ensure its value to our partners.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2023

Are System Baselines within OT Environments Feasible?

Critical infrastructure stakeholders need to baseline their systems to understand expected protocol communications.Baseline behaviors may vary based on operational context.Expected operations during a maintenance window, for example, may be different from normal operations.Furthermore, constructing system baselines for Industrial Control Systems (ICS) is difficult and time-consuming.ICS processes generate artifacts expressed across heterogeneous data sources such as network and device logs. There needs to be a corpus of data in order to develop and compare methods that evaluate the feasibility, performance, and generality of approaches to construct baselines for ICS events. Standalone repositories of network packet captures are insufficient to develop methods to classify or recognize operational events expressed across multiple data sources. Moreover, static data corpora do not enable researchers to compare the impact of changing the underlying system for which a baseline is being constructed and this limits the ability to evaluate the performance of system baselines given system changes (e.g. patches, configuration, maintenance events). In order to address these limitations within the community, this talk intends to promote discussion about the state of the practice of constructing baselines. In this manner, we can continue to understand requirements within industry that are not being met by current approaches to baseline construction. This talk builds on two previous talks on the topic of system baselines for OT environments. First, Weaver co-presented at the RSA Conference ICS Sandbox with Dan Gunter. The talk confirmed the need within industry to construct baselines across multiple types of data sources relative to the semantics of specific business processes. Second, Weaver presented at IEEE Security and Privacy Workshop on Language-Theoretic Security.

02 PETROLEUM↗

DOE OSTI2020

Data-driven predictive probability density function control of fiber length stochastic distribution shaping in refining process

Pulp is the most important raw material in paper in paper industries, whose Fiber Length Stochastic Distribution (FLSD) shaping directly determines the energy consumption and paper quality of the subsequent papermaking processes. However, the mean and variance are insufficient to describe the output FLSD shaping, which display non-Gaussian distribution properties. Therefore, the traditional control, method based on the mean and variance of fiber length is difficult to control the FLSD shaping effectively. In this paper, a novel data-driven predictive probability density function (PDF) control method is proposed for output FLSD shaping in refining process. Primarily, in order to improve the approximation accuracy of the PDF employing RBF neural network, the parameters (center value and width) of each RBF basis functions are tuned via utilizing iterative learning control (ILC) law, and the corresponding estimations of weights law can be obtained. Secondly, considering that the conventional linear model of weights vector has drawbacks of low accuracy and weak generalization ability, random vector functional link networks (RVFLNs) based data driven nonlinear modelling method is employed to characterize the prediction model between the input variables and weights vector. Finally, in order to reduce the randomness of the output FLSD, a minimum entropy control method under mean constraint is employed to design the predictive PDF controller for the output FLSD shaping with the help of the established nonlinear stochastic distribution model. Both simulations and experiments are given to illustrate the effectiveness and practicability of the proposed method.

Probability debsity function, Fiber length distrib↗

DOE OSTI2022

Simurgh: A Framework for Cad-Driven Deep Learning Based X-Ray CT Reconstruction

High-resolution X-ray computed tomography (XCT) is an important technique for the inspection of additively manufactured (AM) parts. While XCT is typically used off-line to inspect a subset of manufactured parts, significantly accelerating measurement speed while retaining accuracy would enable use of XCT for in-line inspection to rapidly identify defects in each part as it is manufactured. Here, we propose a deep learning (DL) based approach that uses computer aided design (CAD) models of the AM parts and physics-based information to rapidly produce high-quality reconstructions from sparse XCT measurements without high quality ground truth data. Our approach uses a generative adversarial neural network (GAN) to produced realistic training data from the CAD-based simulations and a deep neural network that is trained using data from the first stage to produce accurate 3D reconstructions. Using experimental XCT data of metal parts, we demonstrate enhanced defect detection capabilities while dramatically reducing the scan time.

Ziabari, Amir↗

DOE OSTI2020

Policy Innovation and Governance for Irrigation Sustainability in the Arid, Saline San Joaquin River Basin

This paper provides a chronology and overview of events and policy initiatives aimed at addressing irrigation sustainability issues in the San Joaquin River Basin (SJRB) of California. Although the SJRB was selected in this case study, many of the same resource management issues are being played out in arid, agricultural regions around the world. The first part of this paper provides an introduction to some of the early issues impacting the expansion of irrigated agriculture primarily on the west side of the San Joaquin Valley and the policy and capital investments that were used to address salinity impairments to the use of the San Joaquin River (SJR) as an irrigation water supply. Irrigated agriculture requires large quantities of water if it is to be sustained, as well as supply water of adequate quality for the crop being grown. The second part of the paper addresses these supply issues and a period of excessive groundwater pumping that resulted in widespread land subsidence. A joint federal and state policy response that resulted in the facilities to import Delta water provided a remedy that lasted almost 50 years until the Sustainable Groundwater Management Act of 2014 was passed in the legislature to address a recurrence of the same issue. The paper describes the current state of basin-scale simulation modeling that many areas, including California, are using to craft a future sustainable groundwater resource management policy. The third section of the paper deals with unique water quality issues that arose in connection with the selenium crisis at Kesterson Reservoir and the significant threats to irrigation sustainability on the west side of the San Joaquin Valley that followed. The eventual policy response to this crisis was incremental, spanning two decades of University of California-led research programs focused on finding permanent solutions to the salt and selenium contamination problems constraining irrigated agriculture, primarily on the west side. Arid-zone agricultural drainage-induced water quality problems are becoming more ubiquitous worldwide. One policy approach that found traction in California is an innovative variant on the traditional Total Maximum Daily Load (TMDL) approach to salinity regulation, which has features in common with a scheme in Australia’s Hunter River Basin. The paper describes the real-time salinity management (RTSM) concept, which is geared to improving coordination of west side agricultural and wetland exports of salt load with east side tributary reservoir release flows to improve compliance with river salinity objectives. RTSM is a concept that requires access to continuous flow and electrical conductivity data from sensor networks located along the San Joaquin River and its major tributaries and a simulation model-based decision support designed to make salt load assimilative capacity forecasts. Web-based information dissemination and data sharing innovations are described with an emphasis on experience with stakeholder engagement and participation. The last decade has seen wide-scale, global deployment of similar technologies for enhancing irrigation agriculture productivity and protecting environmental resources.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Discovery of Signatures, Anomalies, and Precursors in Synchrophasor Data with Matrix Profile and Deep Recurrent Neural Networks (Final Project Report)

The widespread deployment of phasor measurement unit (PMU) across the U.S. together with the burgeoning machine learning technology made it possible to develop data-driven PMU data analytics to improve grid security and reliability in a more insightful and effective manner. Although PMU applications have been explored for over a decade, the representative PMU usage is limited to the bulk power system monitoring mainly due to the data integrity issues associated with PMUs (typically missing, fragmented, and wrongly amplified data). To forge a breakthrough on this stalemate and embrace PMUs for power system control and protection as well, we applied various advanced machine learning and big data analysis technology to the power system event detection and classification as the first step toward the power system control and protection pertaining to grid security enhancement.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Rocket Launch Detection with Smartphone Audio and Transfer Learning

Rocket launches generate infrasound signatures that have been detected at great distances. Due to the sparsity of the networks that have made these detections, however, most signals are detected tens of minutes to hours after the rocket launch. In this work, a method of near-real-time detection of rocket launches using data from a network of smartphones located 10–70 km from launch sites is presented. A machine learning model is trained and tested on the open-access Aggregated Smartphone Timeseries of Rocket-generated Acoustics (ASTRA), Smartphone High-explosive Audio Recordings Dataset (SHAReD), and ESC-50 datasets, resulting in a final accuracy of 97% and a false positive rate of <1%. The performance and behavior of the model are summarized, and its suitability for persistent monitoring applications is discussed.

acoustics↗

DOE OSTI2025

Anomaly Detection in Electronic Health Records Across Hospital Networks: Integrating Machine Learning With Graph Algorithms

In a large hospital system, a network of hospitals relies on electronic health records (EHRs) to make informed decisions regarding their patients in various clinical domains. Consequently, the dependability of the health information technology (HIT) systems responsible for collecting EHR data is of utmost importance for patient safety. Recently, novel methods and tools aimed at identifying anomalies in EHR data to bolster the reliability of HIT systems have been introduced. However, these existing methods and tools primarily concentrate on individual hospitals, which limits our understanding of system-wide anomalous events and their potential impact on patient safety across multiple hospitals. In this article, we introduce a new approach to detecting anomalies in EHR data within a network of hospitals. This is achieved by combining advanced machine learning techniques with graph algorithms to create a tool capable of swiftly identifying and responding to deviations. Our proposed approach employs a combination of five machine learning models, harnessing the unique strengths of each model to provide a more robust detection system. The detected anomalies are then represented as graphs, allowing us to recognize patterns across the hospital network. This aids in identifying anomalies that span multiple medical facilities, potentially indicating broader system-level risks. Extensive real-world testing of our approach demonstrated its ability to offer actionable insights compared to existing methods. Additionally, its scalable design ensures seamless integration into existing HIT infrastructures.

Niu, Haoran [Oak Ridge National Laboratory (ORNL),↗

DOE OSTI2023

Revealing the Milky Way’s most recent major merger with a Gaia EDR3 catalogue of machine-learned line-of-sight velocities

Machine learning can play a powerful role in inferring missing line-of-sight velocities from astrometry in surveys such as Gaia. In this paper, we apply a neural network to Gaia Early Data Release 3 (EDR3) and obtain line-of-sight velocities and associated uncertainties for ~92 million stars. The network, which takes as input a star’s parallax, angular coordinates, and proper motions, is trained and validated on ~6.4 million stars in Gaia with complete phase-space information. The network’s uncertainty on its velocity prediction is a key aspect of its design; by properly convolving these uncertainties with the inferred velocities, we obtain accurate stellar kinematic distributions. As a first science application, we use the new network-completed catalogue to identify candidate stars that belong to the Milky Way’s most recent major merger, Gaia-Sausage-Enceladus (GSE). We present the kinematic, energy, angular momentum, and spatial distributions of the ~450 000 GSE candidates in this sample, and also study the chemical abundances of those with cross matches to GALAH and APOGEE. Furthermore, the network’s predictive power will only continue to improve with future Gaia data releases as the training set of stars with complete phase-space information grows. This work provides a first demonstration of how to use machine learning to exploit high-dimensional correlations on data to infer line-of-sight velocities, and offers a template for how to train, validate, and apply such a neural network when complete observational data is not available.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

On the integration of molecular dynamics, data science, and experiments for studying solvent effects on catalysis

Computational workflows that combine molecular dynamics (MD) simulations and emerging data-centric (DC) methods can accelerate the screening and analysis of solvent systems experimentally and computationally. Here, MD simulations provide atomic positions and velocities of reactant, solvent, and catalyst materials that can be manipulated into data representations that in turn can be used by DC techniques to conduct predictive modeling, feature extraction, and experimental design. For liquid-phase catalytic applications, emerging DC techniques such as Convolutional and Graph Neural Networks (CNN/GNN), Topological Data Analysis (TDA), and Active Learning (AL) can leverage MD and experimental data to quickly predict solvent effects on reaction outcomes. For instance, in recent studies, 3D solvent environments obtained with MD have been exploited by CNNs to predict experimental reaction rates for homogeneous acid-catalyzed lignocellulosic processes. In this perspective, we discuss basic principles of DC methods and how these can be combined with MD to enable high-throughput screening of solvent selection for diverse catalysis applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

Comparability of Liquid Chromatography Tandem Mass Spectrometry Analysis of Dissolved Organic Matter across Laboratories

Non-targeted liquid chromatography tandem highresolution mass spectrometry (LC−MS/MS) is increasingly applied for the structure-resolved chemical analysis of dissolved organic matter (DOM). With new developments in MS instrumentation and analysis software, the approach has gained substantial momentum over the past decade. However, achieving high-quality analytical data that is reproducible and comparable across laboratories can be a bottleneck in non-targeted metabolomics and organic matter chemical analysis, especially for data reuse in repository-scale analyses. Understanding the capabilities as well as challenges of comparing LC−MS/MS data from different laboratories is necessary for inferring global trends from public data sets. To illuminate instrumentation factors that drive differences and variability, we used a standardized data analysis pipeline, including classical (CMN) and featurebased molecular networking (FBMN), to analyze data from a ring trial by 24 laboratories on identical sample sets of algal and DOM extracts that were mixed in predefined concentrations and spiked with standards. Our results showed that data sets from similar mass spectrometer types with unified instrument parameters were qualitatively comparable, resolving the same general trends and shared mass spectral features. Interlaboratory comparability was best for high-intensity features, while low-intensity features showed greater detection variability. Our analysis also highlights challenges when comparing data from instruments with different acquisition rates or operating with less standardized methods. Lastly, we provide recommendations for data integration, public data sharing, standardization, and best practices for standardized LC−MS/MS data acquisition, which will be critical for long-term time series and intercomparability of DOM chemical analyses.

DOM↗