Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

A network of soil moisture, soil temperature, air temperature, net radiation, ground heat flux and ground water for Chicago, Illinois

This dataset contains environmental monitoring data collected using solar-powered Multi-Function Research (MFR) Long Range Wide Area (LoRaWAN)-enabled nodes at 11 sites in Chicago, Illinois, as part of the DOE Urban Integrated Field Lab CROCUS project. The MFR node system consists of an Input/Output Digital Input Module (IB8) interface box (ICT International) providing wired connections for environmental sensors and an MFR-Node-L data logger that manages power, data processing, and LoRaWAN communication. The wireless data are ingested via Sage network (https://sagecontinuum.org/) nodes that contain LoRaWAN antennae. Measurements were collected from 11 MFR nodes deployed across Chicago State University (CSU), Northeastern Illinois University (NEIU), Northwestern University (NU), University of Illinois Chicago (UIC), West Woodlawn "Blacks in Green" (BIG), and Indian Boundary Prairies (IBP). Each MFR node supports a consistent suite of sensors measuring atmospheric, soil, and hydrological variables. Atmospheric measurements include 2m air temperature (°C), 2m vapor pressure deficit (kPa), and 2m shortwave/longwave radiation (incoming and outgoing, W/m²) measured using ATH-VPD and Apogee SN500 sensors. Soil measurements include volumetric water content (VWC, %) and temperature (°C) at four depths (15, 30, 45, and 60 cm below surface) using Meter Teros54 sensors, and heat flux (W/m²) at 10 cm depth using Huske HFP01-05 sensors. At selected locations, Meter Hydros21 sensors measure groundwater depth (mm), specific conductivity (dS/m), and temperature (°C). The dataset includes timestamps, site identifiers with location names, device IDs, Global Positioning System (GPS) coordinates, variable names with units, measurement depths, values, sensor names, and Sage node identifiers. All timestamps are in local Chicago time (CDT/CST). Quality control flags are provided using a 6-bit binary system indicating physical range violations, step spikes, 24-hour flat-line conditions, 6-hour jitter, 7-day ultra-low variance, and persistent high offset. Data is provided in CSV and CF-compliant NetCDF formats. This dataset is part of a larger collection of CROCUS environmental monitoring data, including linked datasets from Air Quality Transmitter (AQT) sensors, Weather Transmitter (WXT) sensors, and Sap Flow Meter (SFM1x) sensors.

Chicago↗
DOE OSTI2022

Processing Particle Data Flows with SmartNICs

Many distributed applications implement complex data flows and need a flexible mechanism for routing data between producers and consumers. Recent advances in programmable network interface cards, or SmartNICs, represent an opportunity to offload data-flow tasks into the network fabric, thereby freeing the hosts to perform other work. System architects in this space face multiple questions about the best way to leverage SmartNICs as processing elements in data flows. In this paper, we advocate the use of Apache Arrow as a foundation for implementing data-flow tasks on SmartNICs. We report on our experiences adapting a partitioning algorithm for particle data to Apache Arrow and measure the on-card processing performance for the BlueField-2 SmartNIC. Our experiments confirm that the BlueField-2’s (de)compression hardware can have a significant impact on in-transit workflows where data must be unpacked, processed, and repacked.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Barrier-Certificated Reinforcement Learning Approach for Enhancing Power System Transient Stability

Increasing integration of renewable resources brings more flexibility and poses new challenges to modern power systems, leading to highly nonlinear and complex dynamics. Here, this paper aims to provide a general solution framework to traditional control problems, such as frequency control and voltage control, which attempt to maintain the stability of either synchronous generators-governed or inverter-governed systems when subjected to a disturbance and simultaneously guarantee operational constraints, providing a complete complement to existing works on control design. Building on reinforcement learning (RL) and control barrier functions, the framework includes two subsystems, i.e., a model-free controller and a barrier-certification system, which discover RL-based control actions and sequentially filter them using a barrier certificate to satisfy operational constraints. Calculating a barrier function is generally challenging for a complex power system. This is addressed by representing the barrier function using neural networks (NNs) and data-based approaches. An adaptive method is introduced to certify the neural barrier function that perseveres barrier conditions, which is more compatible with online implementation. The proposed framework synthesizes a stabilizing controller that satisfies predefined safety regions. The effectiveness of the proposed framework is demonstrated via several comparative case studies.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Smart-DS: Synthetic Models for Advanced, Realistic Testing: Distribution Systems and Scenarios

The explosion of new ideas for distributed energy resources and advanced distribution grid algorithms faces a critical challenge: how can one effectively develop and evaluate these breakthrough technologies without an adequate set of power systems network, load, and resource data to test against? Today's open test systems, particularly for distribution systems, are tiny and incomplete; and though with enough effort, obtaining utility data under NDA can overcome scale challenges, the results can't be openly shared, preventing fair comparison. The Smart-DS project-led by NREL in partnership with MIT, Universidad Pontificia Comillas, CYME, and EDD-is working to overcome this gap by generating multiple large-scale, open, synthetic, distribution systems that are realistic but not real. These datasets go beyond a single medium voltage feeder (~1000 customers) to cover entire metropolitan areas and their surrounds with up to hundreds of feeders and millions of customers, complete with low voltage customer connections and high voltage sub-transmission. This scale allows adequate testing and analysis of not just local controls and advanced DERs, but also multiple substation interactions, optimized switching, full-scale distribution-OPF, and more. To accompany these datasets, the project is also building a rich set of scenario generation tools that can be used for distribution and transmission systems alike. This includes automated, spatially aware access to world-class, high-resolution solar, wind, and weather data: forward looking generation mixes: and highly configurable DER, load, climate, outage, control-scheme, and other scenarios. We will also highlight DiTTo, an exciting open-source, multi-way distribution dataset transformation tool for many-to-many format translation with scenario, merge/split, and many other manipulation capabilities.

DERs↗
DOE OSTI2020

Interaction networks for the identification of boosted H → $b\overline b$ decays

We develop an algorithm based on an interaction network to identify high-transverse-momentum Higgs bosons decaying to bottom quark-antiquark pairs and distinguish them from ordinary jets that reflect the configurations of quarks and gluons at short distances. The algorithm's inputs are features of the reconstructed charged particles in a jet and the secondary vertices associated with them. Describing the jet shower as a combination of particle-to-particle and particle-to-vertex interactions, the model is trained to learn a jet representation on which the classification problem is optimized. The algorithm is trained on simulated samples of realistic LHC collisions, released by the CMS Collaboration on the CERN Open Data Portal. The interaction network achieves a drastic improvement in the identification performance with respect to state-of-the-art algorithms.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Cutting the Deployment Costs of Physics-Based MPC in Buildings by Simulation-Based Imitation Learning

It has been shown that model predictive control (MPC) is a promising solution for energy-efficient building operations. However, the deployment of MPC in a large portion of the building stock has not been possible partially because of high installation costs. Every building is unique and requires a tailored MPC solution. The best performing solutions are often based on physics-based modeling, which is, however, computationally expensive and requires dedicated software. A promising direction that tackles this problem is to train a neural network-based optimal control policy to imitate the behavior of physics-based MPC from the simulation data generated offline. The neural networks give control actions that closely approximate those produced by physics-based MPC, but with a fraction of the computational and memory requirements and without the need for licensed software. The main advantage of the proposed approach stems from simple evaluation at execution time, leading to low computational footprints and easy deployment on embedded HW platforms. In the case study, we present the energy savings potential of physics-based MPC applied to an office building in Belgium. We demonstrate how neural network approximators can be used to cut the implementation and maintenance costs of MPC deployment without compromising performance. We also critically assess the presented approach by pointing out the remaining challenges and open research questions.

Drgona, Jan↗
DOE OSTI2024

A Bayesian Multi-fidelity Neural Network to Predict Nonlinear Frequency Backbone Curves

The use of structural mechanics models during the design process often leads to the development of models of varying fidelity. Often low-fidelity models are efficient to simulate but lack accuracy, while the high-fidelity counterparts are accurate with less efficiency. Here, this paper presents a multi-fidelity surrogate modeling approach that combines the accuracy of a high-fidelity finite element model with the efficiency of a low-fidelity model to train an even faster surrogate model that parameterizes the design space of interest. The objective of these models is to predict the nonlinear frequency backbone curves of the Tribomechadynamics Research Challenge benchmark structure which exhibits simultaneous nonlinearities from frictional contact and geometric nonlinearity. The surrogate model consists of an ensemble of neural networks that learn the mapping between low and high-fidelity data through nonlinear transformations. Bayesian neural networks are used to assess the surrogate model's uncertainty. Once trained, the multi-fidelity neural network is used to perform sensitivity analysis to assess the influence of the design parameters on the predicted backbone curves. Additionally, Bayesian calibration is performed to update the input parameter distributions to correlate the model parameters to the collection of experimentally measured backbone curves.

42 ENGINEERING↗
DOE OSTI2025

ESnet Data and AI Workshop Report

In February 2025, the DOE user facility Energy Sciences Network (ESnet) held a three-day Data and AI Workshop in Berkeley, California. The objective of the workshop was to identify challenges within ESnet that could be addressed through data-driven methods, to help define ESnet’s data-analysis requirements, and to shape its AI strategy, guiding data-stewardship efforts and the direction of AI research and AIOps exploration for ESnet7, the next iteration of ESnet’s network. This report summarizes the multi-faceted discussions and findings and presents a set of recommendations for next steps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Multi-Modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimation

As data collection and simulation capabilities advance, multi-modal learning, the task of learning from multiple modalities and sources of data, is becoming an increasingly important area of research. Surrogate models that learn from data of multiple auxiliary modalities to support the modeling of a highly expensive quantity of interest have the potential to aid outer loop applications such as optimization, inverse problems, or sensitivity analyses when multi-modal data are available. We develop two multi-modal Bayesian neural network surrogate models and leverage conditionally conjugate distributions in the last layer to estimate model parameters using stochastic variational inference (SVI). We provide a method to perform this conjugate SVI estimation in the presence of partially missing observations. Here, we demonstrate improved prediction accuracy and uncertainty quantification compared to unimodal surrogate models for both scalar and time series data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Communication Network Layer State Estimation Measurement Model for a Cyber-Secure Smart Grid

Network communication has been proven to be a very important tool and a key factor in the recent development and progress of the power grid operation. It is also considered as the foundation for the smart grid because information and communication are integrated into electricity distribution to achieve reliable and accurate knowledge of the power grid. In previous years, absorbing energy from substations and delivering it to customers was the only type of interaction we knew between utility companies and customers. Presently, the growing connections of small distributed generation units caused by the cost reduction of most of the technologies used in generation and storage of electrical energy, along with the potential benefits of renewable energy have pushed many researchers to look into the improvement of information and communication technologies (ICT) in order to ensure a bidirectional flow of power and data. Moreover, the evolution of information and communication technologies and its applications to smart grid have converted the smart grid into a cyber-physical system where vulnerabilities and additional security challenges such as cyber-threats and cyber-attacks have emerged. Previously, we have demonstrated that using machine learning-based processing on data gathered from communication networks and the power grid was a promising solution for detecting cyber threats by implementing a co-simulation of cyber-security for cross-layer strategy. Since the majority of the challenges observed can only be solved in the network communication layer, we present in this work a physics-based state estimation model of the communication network system towards enhanced cyber-physical security of the smart grid. Information integration with the previously developed machine learning model is developed, providing a enhanced cyber-physical security application for the smart grid. Easy-to-implement model, without hard-to-derive parameters, highlight potential aspects of the model for real-life applications.

Mathieu, Reynold↗
DOE OSTI2023

When Spectral Modeling Meets Convolutional Networks: A Method for Discovering Reionization-era Lensed Quasars in Multiband Imaging Data

Over the last two decades, around 300 quasars have been discovered at z ≳ 6, yet only one has been identified as being strongly gravitationally lensed. We explore a new approach—enlarging the permitted spectral parameter space, while introducing a new spatial geometry veto criterion—which is implemented via image-based deep learning. We first apply this approach to a systematic search for reionization-era lensed quasars, using data from the Dark Energy Survey, the Visible and Infrared Survey Telescope for Astronomy Hemisphere Survey, and the Wide-field Infrared Survey Explorer. Our search method consists of two main parts: (i) the preselection of the candidates, based on their spectral energy distributions (SEDs), using catalog-level photometry; and (ii) relative probability calculations of the candidates being a lens or some contaminant, utilizing a convolutional neural network (CNN) classification. The training data sets are constructed by painting deflected point-source lights over actual galaxy images, to generate realistic galaxy–quasar lens models, optimized to find systems with small image separations, i.e., Einstein radii of θ E ≤ 1''. Visual inspection is then performed for sources with CNN scores of P lens > 0.1, which leads us to obtain 36 newly selected lens candidates, which are awaiting spectroscopic confirmation. These findings show that automated SED modeling and deep learning pipelines, supported by modest human input, are a promising route for detecting strong lenses from large catalogs, which can overcome the veto limitations of primarily dropout-based SED selection approaches.

High-redshift galaxies↗
DOE OSTI2020

Using Deep Learning to Derive Shear-Wave Velocity Models from Surface-Wave Dispersion Data

We present a new algorithm for derivations of 1D shear-wave velocity models from surface-wave dispersion data using convolutional neural networks (CNNs). The technique is applied for continental China and the plate boundary region in southern California. Different CNNs are designed for these two regions and are trained using theoretical Rayleigh-wave phase and group velocity images computed from reference 1D VS models. The methodology is tested with 3260 phase–group images for continental China and 4160 phase–group images for southern California. The conversions of these images to velocity profiles take ~23 s for continental China and ~30 s for southern California on a personal laptop with the NVIDIA GeForce GTX 1060 core and a memory of 6 GB. The results obtained by the CNNs show high correlation with previous studies using conventional methods. The effectiveness of the CNN technique makes this fast method an important alternative for deriving shear-wave velocity models from large datasets of surface-wave dispersion data.

Geochemistry & Geophysics↗
DOE OSTI2020

Deep Learning Prediction of Interspecies Interactions from Self-organized Spatiotemporal Patterns of Co-evolving Organisms

Microorganisms colonizing natural habits such as soils co-evolve to form specific spatial patterns through interspecies interactions. These self-organized patterns are a key ecological phenotype, which provides critical information on their interaction mechanisms. However, conventional network inference techniques that analyze species population data in bulk have yet to be extended to account for such spatial heterogeneity. Here we proposed supervised deep learning as a new network inference tool for predicting interspecies interactions from spatiotemporal patterns of microbial evolution. Due to lack of biological imaging data that can be used for training deep learning networks, we used in silico data generated from high-fidelity agent-based models to determine model structure and parameters. Even though networks were trained under simple configurations where interaction coefficients are assumed to be spatially invariant, we demonstrated that the resulting model can be utilized to successfully predict spatial variation of interactions in more complex domains (i.e., configured with a context-dependent mixture of interaction coefficients) as well as in simple domains without further training. In the further test against real biological data obtained through imaging experiments of a binary consortium (Pseudomonas fluorescens and a mutant of Escherichia coli), our model also predicted the dramatic shifts in interactions of the two organisms across different environmental contexts. Through various successful demonstrations in this work, the combined use of the agent-based model and machine learning algorithm provides a means to use new type of data - microscopic images - for extracting microbial interactions, therefore presenting itself as a useful tool for the analysis of more complex microbial community interactions.

Lee, Joon-Yong↗
DOE OSTI2024

A Science Gateway for the Repeatable Analysis of Machine Learning Predicted Gravity Anomalies

In recent years, deep learning has become an increasingly popular alternative for modeling in geoscience applications due to its scalability and efficiency. However, the interpretability, compute, data volume, and hyperparameter tuning requirements of deep learning models make development and monitoring difficult. Furthermore, model explainability and communicating results obtained by these models to users or domain experts is a challenge, as domain experts in geoscience also need to have a deep understanding of how those models function in order to support their scientific works. Here, we describe a science gateway and machine learning pipeline for predicting gravity anomalies from geophysical data. The gateway, built on open-source technologies, provides a holistic view of the pipeline through interactive visualizations aimed at enabling efficient exploratory data analysis. The repeatability, reproducibility, and monitoring capabilities of this overall system allow us to iterate and analyze at scale. Using this pipeline and gateway, we can repeatedly produce accurate high-resolution gravity anomaly datasets. By describing the underlying technologies, implementation, and results, here we provide a foundation for the broader adoption of science gateways into cross-cutting geoscience and machine learning research projects as a means to improve the scientific discovery and collaboration in the geophysics and computational sciences community.

58 GEOSCIENCES↗
DOE OSTI2025

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗
DOE OSTI2023

Advanced data analysis in inertial confinement fusion and high energy density physics

Bayesian analysis enables flexible and rigorous definition of statistical model assumptions with well-characterized propagation of uncertainties and resulting inferences for single-shot, repeated, or even cross-platform data. This approach has a strong history of application to a variety of problems in physical sciences ranging from inference of particle mass from multi-source high-energy particle data to analysis of black-hole characteristics from gravitational wave observations. The recent adoption of Bayesian statistics for analysis and design of high-energy density physics (HEDP) and inertial confinement fusion (ICF) experiments has provided invaluable gains in expert understanding and experiment performance. In this Review, we discuss the basic theory and practical application of the Bayesian statistics framework. We highlight a variety of studies from the HEDP and ICF literature, demonstrating the power of this technique. Due to the computational complexity of multi-physics models needed to analyze HEDP and ICF experiments, Bayesian inference is often not computationally tractable. Two sections are devoted to a review of statistical approximations, efficient inference algorithms, and data-driven methods, such as deep-learning and dimensionality reduction, which play a significant role in enabling use of the Bayesian framework. We provide additional discussion of various applications of Bayesian and machine learning methods that appear to be sparse in the HEDP and ICF literature constituting possible next steps for the community. We conclude by highlighting community needs, the resolution of which will improve trust in data-driven methods that have proven critical for accelerating the design and discovery cycle in many application areas.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Reconfigurable Network Slicing Orchestration in Network Function Virtualization Compatible Operational Technology Environment

The ongoing transition to Industry 4.0, which is characterized by increased inter-connectivity of cyber-physical systems, requires having time-sensitive, high throughput, and secure transfer of critical data in industrial sites. In this context, network slicing emerges as a critical tool to ensure timely data delivery by provisioning the network resources to cater to specific applications’ requirements and mitigating potential cyber attacks. To address these challenges, this paper aims to tackle two key questions essential for the successful implementation of network slicing in industrial environments. First, it investigates architectural considerations for developing a network infrastructure capable of supporting network slicing functionalities effectively. The proposed approach significantly improves deployment efficiency over traditional manual configurations. Second, it delves into the automated orchestration process, elucidating the steps and components involved in transitioning from a static network management approach to dynamically leverage network function virtualization schemes for creating network slices in ad-hoc manner. The system demonstrates high throughput suitable for production-level solutions and maintains exceptionally low latency, making it ideal for ultra-reliable low-latency communications. Even with increased network demands, the system remains stable, with effective Quality of Service (QoS) management, ensuring reliable performance under varying conditions. The proposed architecture outlines the necessary components, services, and communication protocols required for a production-level orchestrator for network segmentation in SCADA environments.

Rodiles Delgado, Brian G.↗